Two model swaps on one RTX 5090 — and why only one happened

Within five days two models appeared aimed at the same class of hardware: Meta's Muse Glimmer 30B on 10 August 2026 and Alibaba's Qwen3.8-27B on 14 August — both under Apache 2.0, both built for 24 to 32 GB cards.

We tested both against the same question: do you replace the model that reads our documents? The answer differed — and the interesting part is why.

The candidate that did not arrive

Muse Glimmer is a strong model. The vendor positions it as an agentic model for always-on local operation, and the published benchmarks support that: on tool use and multi-step workflows it leads clearly.

That is simply not our task. Our lane reads documents — delivery notes, invoices, orders. On the benchmarks measuring document understanding, Muse Glimmer lands level or slightly behind. Its clear wins are agentic, and agentic work runs on a different, considerably larger machine here.

An architectural argument came on top: Muse Glimmer is dense, with nearly 30 billion active parameters plus a vision encoder. Our model at the time was a mixture-of-experts with roughly three billion active parameters per token. Under parallel load on a single card that is the difference between "flowing" and "queueing".

Result: no swap. Not because the model is weak, but because it was built for another job.

The rule that was wrong

Four days later Qwen3.8-27B arrived — and with it a problem: this model is dense too. By the logic above it should have been ruled out. We even had a fixed rule in our own runbook: "a dense 27B does not fit this card."

That rule was wrong. It came from a single failed start days earlier whose logs nobody had kept. The vision-encoder peak of roughly 10.9 GiB it claimed does not exist — measured, it is 3.0 GiB.

In fact the dense model fits better than its MoE predecessor:

	dense (new)	MoE (old)
Weights	19.41 GiB	22.74 GiB
KV cache	6.47 GiB	3.41 GiB
KV tokens	184,320	158,222

The lesson is uncomfortable and general: do not turn a single failure into an architectural rule — and keep the logs. We came within a hair of discarding the new model unexamined, on the strength of a measurement that never existed.

The lever nobody had pulled

One switch makes it possible: --kv-cache-dtype fp8_e4m3.

The KV cache holds the context so far during generation. Store it in 8 bits rather than 16 and roughly twice as much fits in the same memory. That switch had never been tried — and it is the reason the arithmetic works at all.

Something else became clear in the process: lowering context length or memory utilisation, the usual first reflex, does not help here. Both had been tested without success beforehand.

Three flags against three silent failures

The real practical value of this swap sits in three startup parameters. Each prevents a failure that does not present as a failure — which is exactly what makes them expensive:

--max-num-seqs 64 — the architecture consumes one internal block per parallel sequence. Set too high, startup aborts after memory has already been reserved. It looks like a memory problem and is not.
--tool-call-parser qwen3_xml rather than the obvious default — the model emits tool calls as XML. With the wrong parser the calls come back empty. No error, nothing in the log, just a result with no content.
--default-chat-template-kwargs '{"enable_thinking": false}' — the template opens a reasoning section by itself. Without this switch the reasoning text lands in the answer instead of its own field, and any downstream JSON parsing breaks on it.

All three share a trait: the system reports success and delivers something unusable. Watch only startup messages and status codes and you will miss it.

The price: slower alone, strong under load

Honesty belongs here: the dense model is markedly slower on single requests — 27 billion active parameters against roughly three. It gets interesting under parallel load:

Our own measurement on an RTX 5090. The predecessor reached 209.2 alone and 1,230.8 at 8 parallel — it was never measured above 8, so any comparison in that range would be unevidenced. (Quelle: senn-tech — own measurement, 08/2026)

For a lane processing documents in batches, the right-hand side of that chart is the relevant one. For interactive chat it would be the left — same hardware, same models, opposite recommendation.

What this means in our own operations

The swap ran on 15 August, one day after the model was published. What helped was not courage but preparation: a rollback that takes a minute because the old weights sit in cache, and a test script that exercises every dependent service after the change.

The latter proved more necessary than expected. In the process we discovered that one of our test scripts had been broken since day one: a flaw in its structure meant every run sent an empty document to the endpoint — and dutifully received an empty answer, in a fraction of a second. Green, fast, worthless. The production path was never affected, but the test had been testing nothing for months.

That is the real story of this swap: the model was not the risk, the assumptions around it were. An invented memory measurement, a test that tested nothing, and three switches whose absence disguises itself as success.

Still open, and we say so

The quality comparison between the two models is still outstanding. We have measured fit and speed, not the accuracy of extraction on real documents. Until that comparison exists, the swap is technically done but not professionally settled.

We state this because the opposite is common in technical writing: measure throughput, assert quality. A tokens-per-second figure says nothing about whether the invoice number was read correctly.

Conclusion

Two models in one week, one swap. Muse Glimmer is strong — for agentic work, not for our documents. Qwen3.8 fits, contrary to a rule of our own that turned out to be an invention.

The transferable part has nothing to do with either model: examine the rules you carry around about your own infrastructure. Some of them date from a single bad day on which nobody kept the logs.