Published: 2026-07-09
Summary

GPT-5.6 Soul on Codex Ultra Autonomously Produced This Video

Chapters / key moments (click to jump — plays here on the page)

Nate Herk gave one prompt to GPT-5.6 Soul running inside Codex on the new "Ultra" tier and let it produce an entire narrated, avatar-led video end to end — research, scripting in his voice, triggering paid APIs (ElevenLabs, HeyGen, Hyperframes), rendering, and self-checking every frame. He walks through OpenAI's benchmark claims (91.9% on Terminal Bench 2.1, 92.2% on BrowseComp for Ultra) and then digs into the real cost: Ultra spun up nine sub-agents and burned about 450M tokens (~$300), which he argues was Ultra over-delegating — the same prompt on "high" would likely match the result at roughly half the cost.

Source video

"GPT 5.6 Sol Made This Entire Video" by Nate HerkWatch on YouTube →

Key Takeaways

  • Ultra coordinates a team, not a single answer. GPT-5.6 Soul on Codex "Ultra" ran a whole production — here a main session plus nine sub-agents — instead of responding to one question.
  • OpenAI's cited benchmarks: 91.9% on Terminal Bench 2.1 (up from 85.6% for GPT-5.5) and 92.2% on BrowseComp for agentic browsing. In Nate's own 13-task test, Soul earned 97% of objective points (7 wins, 5 ties, 1 loss) — strongest on coding and structured execution.
  • It chained tools it doesn't own: ElevenLabs for the cloned voice (script broken into sub-60-second sections to keep the voice consistent), HeyGen for the avatar (driven via browser automation when the API wouldn't lock the newest motion engine), and Hyperframes for the edit — with Soul planning and operating the chain.
  • It self-verified. Separate agents inspected rendered frames, checked every entrance and exit, looked for text outside the frame, confirmed the avatar never disappeared, and compared factual claims against OpenAI's release notes — each failed frame forced another fix, render, and review.
  • Cost reality check. Ultra reported 3M tokens, but inspecting the logs showed ~450M total across the main agent (~86M) and nine sub-agents — roughly $300 at input/output pricing. Nate blames Ultra "over-delegating" and recommends running such models on "high," not max, to avoid overthinking.
  • Billing note: GPT-5.6 Soul's API pricing is roughly half of Fable 5's — similar to Opus 4.8 — and most of his all-day runs came out cheaper than Fable.
  • The workflow lesson: give a capable model a vague, ambiguous goal plus explicit delegation and verification, then get out of its way — and iterate from there with skills and feedback.

Weekly Digest — In Your Inbox

Get the week's top AI agent news, updates, and guides — every Friday.