Harness engineering for teams: making agents work across a shared codebase

Step-by-Step Breakdown

1. Reframe it as a leadership problem, not an IC problem

   The "figure it out for yourself" paradigm produces a split team: some people ship 10 PRs a day, others ship one or two — and the one-to-two group inherits the review burden for everyone else's agent output. That group then sees bad code all day, blames the agents, and never crosses over. Khandelwal's point is structural: the most impactful setup changes require changing how the codebase itself is organised, which no individual contributor has the authority to do.

2. Do the basics — make progressive disclosure real

   Find the ICs whose setups actually work, extract their practices, and ship them across the team as a shared setup rather than letting everyone live in a private configuration. He is explicit that this is hard for engineers because it means accepting that your own setup is imperfect.

3. Keep the agent instruction file a thin index

   Don't let CLAUDE.md or AGENTS.md grow into one big document. It should be a thin index that points to the right files, because that file is what gets loaded into the agent's first prompt. Keep SKILL.md files under a hard limit of 100 lines — a skill is really a folder, so the detail belongs in the folder, not the entry file.

4. Put runbooks in the code comments, not just in docs

   The mechanism that makes progressive disclosure work: if a piece of code needs a runbook, reference it from the comments in that file. Then when the agent greps its way into the code, it reads the comment and knows which document describes how that code is relevant. The whole codebase becomes the retrieval index — what he calls smart prompt injection, giving the model the right context at the right time without you doing it manually.

5. Invest in one high-value skill and let it prove itself

   His team built a single skill called ship it, covering everything from "code is done" to "PR ready for review": opening the PR, writing the description, handling review comments and merge comments, and working through CI failures. It frequently ran for over an hour, which initially scared people — but it was the thing that convinced skeptics the agent could be trusted without babysitting. One skill that visibly works beats a dozen half-adopted ones.

6. Close the loop so the system self-heals

   Slop is inevitable, so build the pipeline that detects and removes it. They wired issues and boards into the repo, added CI/CD and automated reviews, and ran a "code gardener" nightly that reads through the codebase and flags anything not organised correctly — which means the team has to define what correct organisation means.

7. Win over the skeptics by making the setup editable

   The test of real buy-in is not whether skeptics use the shared setup but whether they edit and play with it. Treating a skeptic as "just someone who's scared" is the easy read and the wrong one; getting them to contribute is the signal that the setup is genuinely shared.

8. Budget IC time for continuous iteration

   This is not a one-month project. Models change, harnesses change underneath you, and the setup has to keep moving. Accept that some percentage of IC time goes into harness work that produces no shippable PRs up front.

Diagnostic Signals Your Setup Is Broken

The talk's most directly usable section: a checklist of symptoms, each with the underlying cause.

Gotchas & Caveats

• Hard-cap SKILL.md at 100 lines. A skill is a folder — push the detail into the folder and keep the entry file scannable.
• Long-running agents are a feature, not a bug. Their ship it skill routinely ran over an hour. In a reasoning paradigm, the longer the agent thinks the better the output; the expectation to manage is the team's, not the agent's.
• Give experiments an opt-out lane. Prototype code that will never ship should be exempted from the rigorous standards applied to the rest of the codebase, and labelled as such — otherwise the standards get diluted for everything.
• Merge hell is real and unsolved. He names it as a cost of parallel agent work rather than offering a fix.
• People move on the fear/confidence spectrum day to day. Adoption is a human problem; a mandate ("token max everything") is what produced the slop and the sev-2s in the first place.
• Don't expect to reach a finished setup. The iteration cost is permanent, and engineers find this harder to accept than the technical work.

Key Takeaways

• Almost all agent-setup advice assumes one developer and one repo; it breaks on a shared production codebase, and fixing that requires authority an IC doesn't have.
• Progressive disclosure is the central technique: a thin index file, runbooks referenced from code comments, and a codebase organised so the agent can find things at the moment it needs them.
• Context consumption is a measurable health metric — ~20–25K at session start is normal, 40–50K means your disclosure is broken.
• One skill that reliably takes work from "code done" to "PR ready" does more for adoption than a broad library of partially-trusted ones.
• Closing the loop (issues and boards in the repo, CI, automated review, a nightly code-organisation pass) is what keeps inevitable slop from accumulating.
• The real measure of team buy-in is whether skeptics edit the shared setup, not whether they use it.