The 5 Levels of a Claude Code Second Brain (Memory & Context Engineering)

Step-by-Step Breakdown

1. Level 1 — CLAUDE.md as a router + markdown folders

   Start with a CLAUDE.md (or agents.md for Codex) that loads automatically and acts as a router: it states the agent's role plus explicit routing rules ('for personal info look here, for Q1 priorities look there'). Back it with simple folders — a context/ folder (about-me, stack, a decisions log) and a projects/ folder. The agent won't search your whole project on its own; if it doesn't know where something lives, it can't find it.

2. Level 2 — LLM wiki + automemory

   Add an indexed 'wiki' of linked markdown (e.g. one wiki for YouTube transcripts, one for meeting notes) that Claude builds when you tell it to ingest a source. Turn on Claude Code's automemory with /memory so it writes and updates memory.md on its own. To stay tool-agnostic, keep a copy of CLAUDE.md as agents.md for Codex and point Codex at memory.md. Nate runs his entire main project at this level.

3. Level 3 — Semantic / vector search

   When keyword routing isn't enough, vectorize one specific unit (e.g. transcripts) so you can search by meaning, not exact words: chunk the text, run each chunk through an embeddings model, store in a vector DB (Pinecone, Supabase, Qdrant). It's great for needle-in-haystack lookups ('remind me what rule 17 was') but bad for whole-document questions, because it only returns matching chunks.

4. Level 4 — Knowledge / relationship graphs

   Build entity-relationship graphs (LightRAG, GraphRAG) when your data is relational — clients, companies, who-works-where, who-endorses-whom. The graph software embeds the relationships for you; your job is feeding it enough data. Nate uses a 'grill me' skill that interviews him relentlessly about a topic and writes a rich brainstorm file to populate the graph.

5. Level 5 — Full autonomy

   The system maintains itself so you barely think about it. Nate explicitly does NOT recommend always climbing to level 5 — higher isn't better. Find the lowest level that removes your current pain; if there's no pain in your current setup, don't add complexity for its own sake.

Commands & Code Shown

/memory

/memory

Purpose: Toggle Claude Code automemory on/off and view the memory file

When to use: Level 2 and up — let the agent write and update memory.md automatically instead of you maintaining it by hand

Common Errors & Fixes Covered

Level 1 CLAUDE.md router (structure shown in the video)

Template shown in the video. Adapt to your project — do not copy verbatim without reviewing each line.

# CLAUDE.md
# This file loads automatically every time you open Claude Code in this folder.
# It tells the AI who you are, how you work, and where things live.

## Role
[who you are / how you work — your system prompt for this project]

## Where things live (routing rules)
- /context/about-me.md   -> always-true background about you and how you work. Read this first.
- /context/stack.md      -> tools and stack you use
- /context/decisions.md  -> decision log (append new decisions + dates)
- /projects/             -> one file or folder per ongoing project / client

## Rules
- When you make a big decision, append it (with the date) to /context/decisions.md

Gotchas & Caveats

• Higher level is not better. Nate runs his entire main project at Level 2 — only climb a level when you feel a real retrieval pain.
• Anything you process through Claude goes to Anthropic. For private client data, use local / open-source models instead of putting everything into a cloud second brain.
• Vector databases are not magic — they retrieve similar chunks, not whole documents. Don't rely on them for summaries or aggregate questions.
• There is no single 'correct' folder structure. What matters is that the routing makes sense to both you and your agent.

Key Takeaways

• A second brain is just markdown files and folders organized so both you and your agent can find things again — it's harness-agnostic (works with Claude Code, Codex, Hermes).
• Work backwards from the question: how you'll retrieve the data determines how you should store it in the first place.
• The five levels: CLAUDE.md router + folders → LLM wiki + automemory → semantic/vector search → knowledge graphs → full autonomy.
• Mix storage styles inside one second brain — markdown for full-context items, vectors for needle-in-haystack lookups, graphs for relational data.
• Before blaming the AI for bad retrieval, check whether you actually got everything out of your head and into the system.