The 1-Minute Test: Is a Task a Chat, Single-Agent, Multi-Agent, or No-AI Job?

The four-factor test

Estimate four things about any task in about a minute. Together they tell you the right tool.

1. Size

   Is the task bigger than what one agent can hold at full quality? A calendar or a quarter of email usually fits in a context window. A pile of a thousand documents does not.

2. Independence

   Can the parts be done without knowing what the other parts did? Reading 100 documents splits perfectly — each reader agent works alone. Coding sometimes splits, sometimes doesn't, depending on how you organize files.

3. Separation of concerns

   Do any parts need to be done by different minds? A critic who didn't write the draft, an auditor who didn't keep the books. Agents give you fresh eyes on demand for the first time — a mind that has never seen the thing.

4. Checkability

   Is verifying an answer far cheaper than producing one? A test suite, an exit code, a source document you can point at. If checking is almost free, extra attempts pay off. If checking is expensive, multi-agent value tops out fast.

The verdicts

• Small → a chat. Just a back-and-forth.
• Fits one context, self-checks → a single agent with a goal. It works alone and gets the job done (e.g. "find a gym slot around my meetings").
• Too big, or needs separate perspectives → a team of agents. Piles of documents, contract review, project handoffs.
• A pure judgment call → no AI at all. Some hiring, naming, and direction calls still need you.

The research behind it

• Stanford (2024): a cheap coding model given one attempt per bug fixed 15.9%; given 250 attempts it hit 56% — beating the best single-shot frontier model (43%) without changing the model or harness. Improvement followed a smooth scaling law across four orders of magnitude of attempts.
• The catch nobody quotes: at 10,000 attempts, a correct answer existed in the pile >95% of the time — but you can only find it with a mechanical checker. Without external validation, majority voting and reward models stalled around 100 attempts. Every dollar past that line buys answers you can't identify.
• Anthropic: token spend explained ~80% of the difference between a good run and a bad one; a team of agents is how you spend more tokens than a single context can usefully hold. Multi-agent runs can cost 10–30x a single agent.

The "Ringer" harness (cost control)

His multi-agent setup encodes both limits as design:

• Every task gets a spec written once by the strongest model, which then never touches the work again.
• Every finished task gets a mechanical check — the source has to be attached and match the task or the entry is rejected. "The agent's opinion of its own work is not evidence."
• A failed task retries with the failure included; every result feeds a running scorecard you watch stream.
• Letting Fable 5 act as the brains/orchestrator while cheap worker agents burn the execution tokens cut costs ~10x while keeping Fable's judgment. Setup took under an hour.

Where multi-agent pays off

• Piles of documents: contract folders, inbox quarters, a tool-renewal + usage audit across dozens of SaaS subscriptions.
• Project handoffs — briefing a new hire from scattered notes and threads.
• Research archives, financial statements, medical records.
• Privacy note from the video: for sensitive piles, land exports in a folder you control, give agents read-only access to just that folder, and run it on a machine you own so financial/health data doesn't leak.

Key Takeaways

• The scarce new skill isn't prompting — it's knowing when a task needs one agent, many agents, or none.
• More tokens reliably help only when you can mechanically check each attempt; otherwise multi-agent gains stall.
• Two things justify a multi-agent team: a memory constraint (too big for one context) or a separation-of-concerns constraint (parts that must be done by different minds).
• Orchestrate with an expensive model and execute with cheap workers to keep multi-agent runs affordable.
• Run sensitive multi-agent work locally with tightly scoped, read-only folder access.