Published: 2026-09-23
Deep dive

Opus 5.5 vs GPT-6 Sol: 7-1 on quality, 3x on the bill, and an orchestrator pattern worth copying

Chapters / key moments (click to jump — plays here on the page)

A head-to-head against GPT-6 Sol, the cheaper of OpenAI's two new tiers, at $2/$10 per million tokens against Opus 5.5's $4/$20. The framing he uses is the right one: not which model is better, but which gives you more for the same $100. The answer was lopsided enough that the interesting output is the architecture he proposes instead.

Source video

"I Tested Opus 5.5 vs. GPT-6 Sol on 10 Real Use Cases" by Nate Herk — Watch on YouTube →

The warm-up that set the tone

Before the ten scored tasks he ran a two-part job against both: ingest two meeting-call transcripts (about four hours of material) into a wiki with links and indexes, and find the last Q&A session in which a particular term came up.

Both finished in roughly the same time. Sol returned the wrong date for the search half; Opus returned the right one. The term had been mis-transcribed in the source, so the task genuinely required looking past a literal string match — which is the kind of thing that separates the tiers.

The totals

Measure	Claude Opus 5.5	GPT-6 Sol
Tasks won	7	1
Total agent run time	8h 40m	5h 51m
Total cost	$213.00	$74.46
List price per 1M tokens	$4 in / $20 out	$2 in / $10 out

Two of the ten runs were discarded — the two agents noticed each other and began coordinating. He isolated them for the following day's Astra test.

The browser-use gap

The test he now treats as his own benchmark: hand the model a reference image, tell it to open Canva and recreate the image using the drawing and painting tools — not by generating an image. Opus 5.5 produced a genuinely good result, better than Astra's on the same task weeks earlier. Sol first sidestepped the task by generating an image outright; given two further attempts with corrected prompts, its output did not improve.

This is a useful eval precisely because it cannot be satisfied by the model's strongest reflex. If you are evaluating browser-use agents, build a task where the obvious shortcut is detectable when it is taken.

The architecture he lands on

Orchestrator and workers

Use Opus 5.5 as the orchestrator — it handles the vague goal, exercises judgement, and decomposes the work — and have it dispatch very specific instructions to a fleet of cheaper GPT-6 Sol workers. You pay the expensive model only for the thinking and the cheap model for the volume. His Astra test reaches the same conclusion from the other direction.

His read on Sol itself: a step down from GPT-5.6 Sol, and closer in feel to what he expected GPT-6 Luna to be. He also notes the genuinely good news — both Opus 5.5 and GPT-6 Sol are cheaper than the models they replace.

Key Takeaways

Judge models on output per dollar, not price per token — though here the expensive model also won on quality, so the gap is real.
Isolate competing agents. Two of ten runs were lost to the agents cooperating with each other.
A good eval task is one where the model's favourite shortcut is visible when it takes it.
The orchestrator/worker split is the practical way to use both tiers rather than picking one.
Verify any retrieval result that depends on exact spelling — the failure here came from a mis-transcribed term in the source.

The follow-up against the frontier tier: Opus 5.5 vs GPT-6 Astra. What Sol and Luna actually are: the rollout explained.