Muse Glimmer 30B: Running Meta's Open Agentic Model Locally on vLLM

Step-by-Step Breakdown

1. Check what you're actually getting

   30B parameters, distilled from Muse Spark, with a perception encoder for image input and a 128K context window. Released under Apache 2.0 — free for commercial use — and the release is complete rather than partial: full BF16 weights, both quantized builds, the vision encoder and the Deep Flash drafter. Meta has also signalled Muse Spark 1.2 in open weights soon.

2. Size your hardware before you download

   Quantized down, it fits in roughly 24GB of VRAM — the number that decides whether this runs on a consumer card. Mirza's own setup is Ubuntu with a single Nvidia A100 (80GB) on DGX, which is far more headroom than the model needs.

3. Serve it with vLLM

   He downloads the model and starts it under vLLM, exposing a local endpoint that the later tests point at. Note that the vLLM install itself is not covered in this video — he explicitly defers it to his earlier vLLM videos, so budget for that setup separately if you have not run vLLM before.

4. Understand Deep Flash before you judge the speed claims

   Deep Flash is speculative decoding, and it is the feature that makes local agents tolerable. Normally a model emits one token at a time — fine for chat, painful when an agent is reasoning through five steps and firing tool calls. Deep Flash runs a tiny companion model that guesses a whole block of tokens ahead, and the big model verifies the entire block in one pass, keeping what is right and correcting what is not. Output quality is unchanged; only the speed changes. On a 5090 that is roughly a 3× jump; on Apple Silicon the gain is smaller but real, with M5 ahead of M4.

5. Read the benchmark split, not the headline

   Where Glimmer dominates is agentic work: MCP tool orchestration, deep search, banking workflows and long-context recall — clearly what it was trained for, and it beats its rivals convincingly there. Qwen still takes a lot of cells on the other side of the table, specifically computer use, terminal work and general SWE-bench. Mirza's read: this is a model with a sharp specialty rather than a broad-board winner, and if your use case is desktop automation or heavy coding, Qwen is still very much in the fight.

6. Check VRAM once it's serving — and turn down KV cache if you need to

   With the model served on the A100 he shows consumption at just under 70GB of VRAM. His stated lever for bringing that down is to turn down the KV cache a bit — with the caveat that he thinks the model deserves at least that much memory, so expect a trade against context handling if you squeeze it.

7. Test 1 — one prompt doing three jobs (vision + extraction + long-form code)

   A single prompt against the local endpoint: read a dense technical image, pull the numbers and labels out of it, and write a complete working web app in one shot. The result is a responsive seven-tab app — overview, test setup, instrumentation with correct numbers, simulation, conclusion — with animated streamlines and SVG charts and no external dependencies. He notes it did not add the sliders he was hoping for.

8. Test 2 — a six-stage reasoning chain, run through Hermes

   This is the agent test. He drives the model through an already-configured Hermes agent with a six-stage financial chain where each step feeds the next, so one early slip corrupts the final number. It is deliberately trapped: a holiday roll that adds days, and a fee that changes one leg of the trade but not the other. He also makes it harder than designed by removing the 187-day hint. The model gets every step right, including the 187 days from the holiday roll and dealing on the bid rather than the mid — and it surfaced a rounding-convention issue he had not built into the answer key, flagging it as a question rather than silently picking one. That last behaviour is the interesting part for agent work: it escalated an ambiguity instead of guessing.

9. Test 3 — 78 languages in one pass

   A script asks the model to render one blessing in 78 languages. It holds up well, including on several low-resource and regional languages, with some renderings a bit literal. Notably it also declared where it was least confident rather than emitting plausible nonsense for those languages — Mirza calls it the first time a model has refused that way for him.

Configuration Notes

No install commands are dictated in this video — the concrete numbers below are what it does give you, and they are the ones that determine whether this model fits your box.

• ~24GB VRAM — quantized footprint. The threshold for consumer hardware.
• ~70GB VRAM — observed consumption serving under vLLM without tuning, on an 80GB A100.
• KV cache — the single lever named for reducing VRAM. Turn it down if you are tight; expect a cost to long-context behaviour.
• 128K context — the window you are working inside.
• Apache 2.0 — commercial use permitted, full BF16 weights plus quantized builds, vision encoder and Deep Flash drafter all included.
• Serving stack — vLLM exposing a local endpoint; a Hermes agent pointed at that endpoint for the agentic tests. See our Hermes free models guide and local GPU guide.

Gotchas & Caveats

• Prompt-injection resistance is only middling. Mirza flags that Glimmer sits in the middle of the safety rows on prompt injection — which matters more than usual precisely because the pitch is handing this model access to your own machine. Read our security center before wiring it to real tools.
• Treat the 3× Deep Flash figure as a best case. The error bars are wide: predictable text gets guessed well, unusual text does not. The multiplier depends on hardware and prompt mix.
• It is not a general winner. Qwen still leads on computer use, terminal work and general SWE-bench. Pick Glimmer for tool orchestration and long-context agent work, not for desktop automation.
• vLLM setup is out of scope here. The video assumes vLLM is already installed and working.
• Mirza notes on camera that he is recording with a throat illness, so audio quality is rough — the chapter jump links below help if you are skimming.

Key Takeaways

• Meta released a genuinely complete open-weights package under Apache 2.0 — weights, quantized builds, vision encoder and speculative-decoding drafter — which is the standard worth holding other "open" releases to.
• ~24GB quantized is the number that matters: this is an agent-capable model that fits on a single consumer card.
• Deep Flash speculative decoding is what makes multi-step local agents usable, because the pain of one-token-at-a-time compounds across tool calls.
• The model is a specialist: strong on MCP tool orchestration, deep search and long-context recall; behind Qwen on computer use and terminal work.
• In the six-stage chain it held every dependent step and escalated an ambiguity rather than guessing — the behaviour you actually want from an agent model.
• Middling prompt-injection resistance is the caveat to plan around when the whole point is local machine access.