Agent Sandboxes: Running Your Whole Software Factory Inside a VM

Step-by-Step Breakdown

1. Name the actual bottleneck

   The framing for the whole video: "if you are inside the loop, you are the bottleneck." Most engineers allocate a small slice of their own machine to agents, or lean on CI/CD and containers. That keeps a human in the path of every run. The stated exception is worth keeping — if you are building the system that builds the system, get in the loop deliberately and stay hands-on.

2. Understand why a container isn't enough

   A container buys isolation only. The claim is that an agent sandbox buys three things: true isolation, scale (spin up as many as you want), and agency — the agent owns a whole machine and everything it needs to finish the job without interfering with your work or you interfering with its. Durability matters too: many sandbox providers expire an environment after 24 hours or a few days, whereas the pattern here assumes a VM you own and can keep up or tear down at will.

3. Assemble the stack

   Four pieces, and the video is explicit that the pattern matters more than the tools: Herder as the terminal multiplexer for watching agents and kicking off jobs; Claude Code as the top-level orchestrator sitting outside every sandbox; Pi as the agent SDK / harness running inside each factory; and exe.dev as the sandbox provider ("computers for developers and agents"). Substitute freely — the architecture is the point.

4. Build the three-tier orchestration

   This is the part most viewers will skim past and shouldn't. There are three layers: an out-of-sandbox orchestrator (kicks everything off, then stops — it does no work itself), an in-sandbox orchestrator (one per sandbox, drives that box's pipeline), and the ADW agents inside the software factory doing the actual development work. Putting a second orchestrator inside the sandbox is what makes each box self-sufficient rather than remote-controlled.

5. Run best-of-N across five model configurations

   One prompt — a UI redesign brief for a mock writing app called Inkwell, with "the quiet room" as the design principle — is passed into five sandboxes at once. The configurations are named Default, Frontier, Deepest Seek (all DeepSeek V4 Flash), Open Weights (no closed-lab models at all), and Top Speed. Each sandbox exposes its own URLs for both the running app and its factory dashboard, so all five can be compared side by side in the browser.

6. Pick a model stack, not a model

   Models are sorted into three tiers — state-of-the-art, workhorse, and lightweight (defined as "runnable on a previous-gen GPU node or on your Mac if you have the unified memory"). The repeated instruction is combine compute, don't select compute. Concrete assignments shown: the Default set mixes Gemini Flash, DeepSeek V4 Flash, GLM 5.2 and GPT Luna; Top Speed uses Gemini 3.6 Flash, DeepSeek V4 Flash 0731 and GPT-5.6 Luna.

7. Watch the runs and read the failures

   Top Speed finished a full software-development-lifecycle run in 14 minutes and roughly 2 million tokens. Claude Opus 5 spent 13 minutes on the plan phase alone at high thinking — noted as expected, and as a setting he'd have dialled down for a demo. The Open Weights run failed outright: Kimi K3 could not produce the required JSON format. That failure is presented as the argument for best-of-N rather than a mark against it.

8. Keep a way back into the box

   Two access workflows are demonstrated. First, the orchestrator uses Herder to open five panes and establish SSH into every sandbox, so you can check status and logs by hand. Second — the more interesting one — an interactive Claude Code session is booted on top of the boxes so you can talk to the in-sandbox orchestrator directly and ask it to summarise what it changed. The stated principle: "if you can't touch your boxes, you can't actually get anything done."

9. Gate the non-deterministic parts

   The factory runs deterministic gate checks between phases — the observability view shows a JSON-format error being caught and repaired automatically before the pipeline moves on. This is what separates the pattern from "fire off an agent and hope": every phase declares a purpose, a prompt (system and user), a model, a thinking level, and a tool set, all visible in the dashboard.

Gotchas & Caveats

• A container is not a sandbox for this purpose. It gives you isolation without scale or autonomy — the video treats "I already use Docker" as the most common way engineers talk themselves out of the pattern.
• The demo app is a mock. Inkwell exists to make the five runs comparable on camera; no production code is shown. Treat the factory as a template to adapt, not a result to copy.
• An open-weights run failed completely. Kimi K3 bombed on JSON formatting. If you run best-of-N, budget for a configuration returning nothing — that is the cost of the pattern working.
• High thinking is expensive in wall-clock time. 13 minutes on a plan phase is a real number. Match the thinking level to the phase rather than leaving it at maximum.
• Bypass/"dangerous" permission mode appears inside the sandboxes. The video's justification is explicit and worth repeating: this is acceptable because the blast radius is the box. That reasoning does not transfer to running the same flags on your own machine. See our security center before adopting it.
• Don't outsource your thinking. A recurring warning about handing engineering judgement to AI labs and third-party tools — the distinction drawn is that "vibe coding is not knowing what your system does and not looking; agentic engineering is knowing what your system does so well you don't have to look."

Key Takeaways

• Give each agent a whole machine, not a corner of yours — isolation, scale and autonomy come as a set, and containers only deliver the first.
• The three-tier split (outside orchestrator → in-sandbox orchestrator → factory agents) is what lets the top-level agent stop working once the run begins.
• Best-of-N across differently-configured sandboxes turns a single model failure into a non-event.
• Budget by tier: state-of-the-art for planning, workhorse models for the 90% of daily work, lightweight for what runs on your own hardware.
• Keep SSH and an interactive agent session into every box — observability you can't step into isn't observability.
• Provision API keys at the factory level (an OpenRouter-provisioned key is used here) so no long-lived credential lives in the sandbox image.