A personal knowledge base for your agent without vector databases: incremental connectors, cheap-model summaries and "atoms", BM25 search, recency weighting

Why not just stuff everything into context

• Everything looks equally true. An old price and a new price sit side by side with no signal about which is current, so the model confidently returns whichever it finds first.
• Context rot: retrieval accuracy drops as the context grows, and a huge window is also slow and expensive on every call.
• The fix is retrieval: keep the data outside the context, search it first, and pass in only the best few matches.

The pipeline, step by step

1. List your sources. Email, website, lead conversations, Slack, call transcripts, and your agent sessions.
2. Write one small connector script per source. Each run pulls only what's new since the last run and stamps every record with the current date and time. His Gmail connector uses the Gmail API with read-only access, which he insists on: never give the pipeline write access. The first run backfilled a year of email from three accounts. Later runs only fetch the gap. He runs it every 15 minutes.
3. Index your Claude sessions. He points out they are already stored locally as plain text, so they are easy to index, but they are deleted after a while. In Claude Code the retention period is the cleanupPeriodDays setting (settings reference); he says it's 30 days by default. Copying them into the knowledge base keeps months-old decisions searchable.
4. Have a cheap model write the metadata for every record. For each incoming item, generate:
   • a short summary, including the question someone would type to find it;
   • a list of notes, rephrased several ways;
   • atoms: exact error messages, the exact fixes, commands that worked, and the decisions made and why.
   He uses Claude Haiku 5.5 for this, or whatever is the cheapest model from a provider you trust with the data. The agent searches the summaries and atoms, then follows a pointer to the full raw record.
5. Search with BM25, not embeddings. Plain-text BM25 ranking (around since the 1990s) instead of a vector database. He reports that inference time and running cost both fell by roughly 90% versus his earlier RAG systems, with results he describes as "not even close" in BM25's favour.
6. Apply recency weighting. Boost recently stamped records and down-weight old ones, so the agent prefers this month's price over one from two years ago. Periodically sweep out records that are no longer useful.
7. Query flow. For any question, a fast model picks the search terms, the search returns a handful of best matches (he says in under half a second), and those are added to the prompt before Claude answers.
8. Keep it fresh. Run a background check that every connector is still returning data. API keys rotate, permissions change and routes break. A silent connector means a silently stale knowledge base.

Gotchas & Caveats

• The record counts, the 90% figures and the claim that plain search beat every other approach are his own reports. He also attributes a similar view to Boris Cherny (who created Claude Code) and says the ingestion design was inspired by Cohere. We haven't verified either.
• The pipeline indexes email, Slack and call transcripts. Decide where it lives and who can query it before you point it at client data.
• No code is shown on screen. You build the connectors and the BM25 index yourself, or ask your agent to build them from this outline. Any BM25 library or a SQLite full-text index will do.
• The recency boost is a tuning knob, not a fixed formula. Set it by how fast your facts go stale.

Key Takeaways

• Incremental, read-only, date-stamped connectors, plus a health check that they keep running.
• Index cheap-model summaries, likely questions and "atoms" (errors, fixes, commands, decisions), not just raw text.
• BM25 plus recency weighting is his whole retrieval stack. No vector database needed.
• Copy your agent's session transcripts out before local cleanup deletes them.