Build your own agentic harness in Python: the loop, tool calling, MCP servers, memory and model switching

Setup

uv init .
uv add mcp openai pydantic

He uses the uv project manager, deletes the generated main.py and README, and installs three packages: mcp (to connect to MCP servers), openai (a client for any OpenAI-compatible endpoint, including a local Ollama server) and pydantic (typing). He says none of these is an agent framework: the loop, tools and memory are written by hand. The code for every stage is linked from the description; he pastes it in module by module rather than typing it.

The build, stage by stage

1. v0: talk to the model. A chat function sends messages to a chat completion endpoint. To use a local model, point the OpenAI client's base URL at Ollama; to use OpenAI, remove the base URL and set an API key from the platform's API keys page (normally via an environment variable). He advises skipping local models on a phone or low-end hardware. There is a line for Windows consoles so emoji don't crash the terminal.
2. The idea of a harness. An LLM only turns text into text. It can say it wants a tool, but your code must run it and return the result; the model is the engine and the harness is the car.
3. v1: tool calling. Create an empty workspace folder, write the tools (read file, write file, list files) and a tool schema describing each tool and its arguments. Pass the schemas, not the functions, to the model. When the response contains tool calls, run them and send the results back.
4. v2: the loop. A while True loop keeps calling the model, executing any tool calls and appending results, and breaks only when the model answers with no tool calls. Without the loop the model can't use a result in the same turn.
5. v3: MCP servers. Connect to servers over stdio (a script on your own machine, started for you through uv) using the async mcp client: loop over the configured servers, connect, wait for the session to initialise, list each server's tools and convert them into the same schema format. His two example servers answered 'what time is it' and fetched a web page. He notes this is roughly what Claude Code does behind the scenes.
6. v4: persistent memory. A memory.md file in the directory, a save-memory tool the model can call, and the file's contents injected into the system prompt on every start, so earlier facts are known without being repeated.
7. Full harness. Servers move into an mcp_servers.json config loaded at runtime, and a model config lists several models (local small and large models, GPT and Claude entries that need keys you set). Slash commands /models, /model, /tools, /memory and /quit let you switch models, list tools and view memory. He tested it by asking it to summarise every document in the workspace into one file.

Gotchas & Caveats

• Model routing in the intro is, in practice, a manual switch via /model and a config list; there is no automatic routing logic.
• His harness had no delete-file tool, so it couldn't clean up files; a deliberately small tool set.
• A very small local model (he used one for the final test) may struggle with multi-step tasks; he suggests an API key when quality matters.
• The code is not on this page. Use the link in his description, and review any MCP server you connect before running it.
• The video includes a sponsor segment for a free HubSpot resource.

Key Takeaways

• A harness is just: a model call, a tool schema, a loop, an execution layer for tool calls, and state you inject back in.
• Pass schemas to the model and keep the real functions on your side.
• MCP tools are discovered at runtime and converted into the same schema as local tools.
• Memory can be a markdown file plus one save tool and a system-prompt injection.