Code mode: give the agent a sandbox instead of a tool catalog, with Arize's Harbor results (43% fewer tokens, 31% lower cost)

Why code mode

• Less context bloat. A big tool catalogue and bulky tool results fill the window and cause turns, compaction and context rot. In code mode the agent searches for the tool it needs, runs it in code, and returns only what it needs.
• Composition. Tools can run in sequence, in parallel, with conditionals, fan-out and chaining, many in one turn.
• Progressive disclosure of tools, similar to the tool search now built into model providers.
• Emergent tools. Asked to remove personal data from records, a tool-calling agent without a suitable tool may do it in context and leak card numbers into the window. With a sandbox it can write its own tool.

What they put in the sandbox

1. Bash, to compose built-ins like sed and grep with their own CLI and offload output to files for later.
2. A Python sandbox.
3. Their REST create, read, update and delete endpoints hoisted into the coding environment (such as fetching traces or changing datasets) so the agent can compose them.
4. SQL. Not everything is CRUD; an aggregate query such as "LLM spans over a P95 latency" should not need paging through every trace.
5. A UI programming paradigm: a web worker with two-way binding to the browser so the agent can change the time range or read data in the front end.

How they tested it

1. Run the agent through variations in sandboxed Harbor environments with network access turned off and clear instructions.
2. Run each task several times as trials to remove flake, since the system is non-deterministic.
3. Write evals as Harbor verifiers that score each task 1 or 0, and track pass@k per task.
4. Compare before and after across the surfaces (Claude Code, MCP and CLI). The reported result: tokens down 43%, cost down 31%, latency roughly constant despite sandbox overhead.
5. Keep the trajectories and evals, then use them to tell a coding agent which instructions steer the agent wrong and hill-climb. Introducing a new tool can reduce efficiency at first.

Caveats: what code mode costs

• Resources and egress: hosted sandboxes need CPU, memory and lifetime limits, and you must decide what network access to allow.
• Secrets must not return to the context window; vaulting strategies differ by sandbox provider.
• Permissions are often over-broad, and can be correct for the first operations and wrong for a later one or a composition.
• Harder to debug: you debug code rather than individual tools, tool parameters are now code, and a failure in the middle of a big script cannot be resumed.
• Double sandboxing when code mode exists both in MCP and in the harness, with results serialised to JSON between them.
• Binary outputs such as images should not be serialised back into context; a signed hand-off may be needed.
• Language and model choice: JavaScript suits models but may not suit maths-heavy work; frontier models are strong at code mode, and a smaller, cheaper model may not be.