Skip to content
senn-techsenn-tech
Back to blog
AI Hardware
Local AI and GPU hardware2026-11-15· By Franz Senn

DSpine for Qwen3.8-27B: What the New Draft Model Would Change on Our Fallback Host

Tencent published the weights of a drafter for Qwen3.8-27B on 7 November 2026, and over the following days the pieces arrived that make it usable outside a fixed test setup: commits on 12 and 13 November extended batching, sampling and the tuning of dynamic verification. We run Qwen3.8-27B on our fallback host and wanted to know whether this matters to us. The short answer: Tencent's numbers are impressive, but the drafter does not run on our hardware, and the reasons say something about future decisions.

DSpine is a draft model for speculative decoding. A small model proposes a block of 16 tokens in one pass, the large target model checks the block in a single step and keeps the prefix that matches its own output. With exact verification the result equals normal decoding, with fewer steps of the large model. DSpine passes the predicted features of earlier positions on to later ones in every layer, so the 16 candidates depend on each other. It also uses dynamic verification modelled on DSpark, whose effect on the output we did not check. The basic principle is in our post on speculative decoding and dFlash.

Tencent wordmark
Tencent published the weights and the figures for DSpine. We did not measure any of it. (Quelle: Tencent / Wikimedia Commons, public domain (PD textlogo), trademark: Tencent)
How a draft model works: one block of 16 tokens per passDraft modelproposes 16 tokens in onepassTarget modelchecks the whole block inone stepAcceptthe matching prefix is kept
Our own diagram based on the Tencent model card. (Quelle: Tencent, model card)

What Tencent reports

The model card names SGLang 0.5.22, BF16, FlashAttention 3 and concurrency 8. It does not name a GPU. The comparison is plain autoregressive decoding (AR) and DFlash2 from z-lab, the previous draft model for the same target. The values are total throughput over all requests (tokens per second) while replaying agent traces, reasoning level xhigh.

BenchmarkARDFlash2DSpineAcceptance length DFlash2 / DSpine
WorkBuddy Code193.00550.40 (2.85×)613.55 (3.18×)4.06 / 4.92
WorkBuddy Web252.26734.04 (2.91×)840.14 (3.33×)3.95 / 5.01
WorkBuddy Office215.05662.34 (3.08×)763.83 (3.55×)4.20 / 5.49
SWE-bench Pro248.17711.60 (2.87×)791.30 (3.19×)4.00 / 4.77
τ²-bench340.39871.98 (2.56×)978.39 (2.87×)4.29 / 4.95

Source: model card tencent/Qwen3.8-27B-DSpine-b16, retrieved 15 November 2026, vendor figures. Against DFlash2 that is 11 to 15 percent more throughput and 15 to 31 percent longer accepted blocks (our own arithmetic on the table).

DSpine versus autoregressive decoding, throughput factor (vendor figures, xhigh)WorkBuddy Office3.55WorkBuddy Web3.33SWE-bench Pro3.19WorkBuddy Code3.18tau2-bench2.8704
Tencent measures total throughput at concurrency 8 with SGLang 0.5.22 in BF16. The card names no GPU, and we did not measure any of these numbers. (Quelle: Tencent, model card Qwen3.8-27B-DSpine-b16)

The paper (arXiv 2609.36173, 28 September 2026) backs the method with Qwen3-4B and Qwen3-8B results. Its abstract says nothing on Qwen3.8-27B; the 27B figures come from the model card alone.

What already runs here

Since 11 November our one-RTX-5090 fallback host runs llama.cpp instead of vLLM, with a Q4_K_M build of a Qwen3.8-27B variant (a finetune called "TURBO" according to a comment in the compose file; the base model it is not), using multi-token prediction as the draft with at most two draft tokens per step. Older posts of ours still describe the vLLM setup. On 15 November 2026 we evaluated the draft acceptance lines in the container's docker logs and read memory use with nvidia-smi. Both are read-only; we changed nothing on the host.

MeasureValueKind
Window11 to 15 November 2026, container started 11 Novlog extract
Requests with speculation334counted
Drafted tokens accepted77,625 of 91,081 (85.2 percent)counted
Tokens per verification step, volume-weighted2.70 (maximum with two drafts: 3)derived
Per request, median2.93derived
Decode speed, median39.7 tokens/s (10th percentile 9.4, 90th percentile 81.3)55 replies of 300+ tokens

The decode speed comes from mixed load with up to two parallel slots and varying context lengths, hence the spread. It cannot be compared to Tencent's figures: different engine, different quantization, different target, different load mix. The acceptance lengths line up only roughly as well. We assume that Tencent's acceptance length counts the bonus token of a verification step the way llama.cpp's mean len does, but we did not check it.

One point holds anyway: our draft reaches 2.70 of a maximum 3.0, close to its own ceiling, while a 16-token block design like DSpine has a far higher ceiling. Tencent's table sits at just under 5.

The fit calculation

Following the rules of our hardware assessment: real bytes from the Hugging Face API, the card read via nvidia-smi.

ItemValueSource
RTX 5090 card32,607 MiB (31.84 GiB)nvidia-smi, 15 Nov 2026
Used / free today30,882 / 1,269 MiBnvidia-smi, 15 Nov 2026
Our Q4_K_M weights17.23 GiB; mmproj 0.86 GiBfile sizes in the container
DSpine draft, model.safetensors4,051,831,880 B = 3.77 GiB (3,864 MiB)HF API
Qwen3.8-27B BF16 (target in Tencent's instructions)27,781,427,952 parameters, 51.7 GiBHF API, derived ×2

Result: the drafter alone needs 3,864 MiB and 1,269 MiB are free. At least 2,595 MiB are missing, before the drafter's own KV cache (five layers, 8,192-token window) and activations. We could free that space, for instance with smaller contexts or a smaller vision part, but the example in the instructions uses the BF16 target with --tp-size 1, so the weights alone take 51.7 GiB, 1.6 times one 5090.

Then the runtime: Tencent provides an SGLang plugin for 0.5.22 and an MLX variant for Apple Silicon. A third party mentions a DSpine fork of llama.cpp in a discussion thread, but as far as we can see nothing of it is merged upstream in llama.cpp or vLLM. Running a third runtime next to vLLM and llama.cpp for a single fallback lane is out of proportion to the gain.

What we take from it

For the fallback host we recommend changing nothing. Three things are worth watching: whether DSpine or DFlash2 lands in vLLM or llama.cpp, whether a drafter appears for the quantized, finetuned target we run on, and whether 16-token blocks keep their acceptance against Q4 targets. Until then the cheapest step is to make the existing draft visible: pull the acceptance length from the log into monitoring, so a later change of drafter is judged on a number of our own. If you want to assess a decision like this for your own operation, our offer is under Corporate LLM; the broader picture on local hardware is in the topic hub Local AI hardware, and our earlier 27B setup on four 5090s is in Qwen3.8-27B self-hosted (August state, since changed).

Open points. Not verified: Tencent's throughput figures and the GPU behind them (the card names none); whether the acceptance length there includes the bonus token; how the drafter behaves against a quantized or finetuned target; whether vLLM PR 52816, which the DFlash2 card names for the predecessor drafter, is merged; whether a DSpine fork of llama.cpp lands upstream (only a mention in a discussion thread, we read no code). GitHub's license detector shows "Other" because License.txt combines Apache 2.0 with an exception list for third-party components; we did not go through that list. Our own values are a four-day log extract under varying load, without a with-and-without-draft comparison run.

Read next: Inkling 2 Small against our VRAM limit · Huawei MACaQ for GGUF, measured · memory as the limit of local AI agents · running Qwen3.8-27B ourselves · speeding up self-hosted AI inference

Further Reading

Questions?
What is DSpine and how does it differ from DFlash?+

DSpine is a draft model for speculative decoding that proposes a block of 16 tokens in one pass. In every layer it passes the predicted features of each position on to its successor, so the candidates depend on each other. According to Tencent's model card (14 November 2026), acceptance length on agent benchmarks for Qwen3.8-27B is 15 to 31 percent above DFlash2 and throughput 11 to 15 percent above. Both are vendor figures that we did not re-measure.

Does DSpine run on a 32 GB RTX 5090?+

Not without rebuilding the setup, by our calculation of 15 November 2026. The draft model weighs 3.77 GiB and our card has 1.24 GiB free. Tencent's instructions also assume the BF16 target model, which takes 51.7 GiB, about 1.6 times one 5090, and SGLang 0.5.22 instead of vLLM or llama.cpp.

What acceptance length does our current draft reach?+

From the llama-server logs of our fallback host (11 to 15 November 2026, 334 requests with speculation), with at most two draft tokens per step, the mean is 2.70 tokens per verification step, and 85 percent of drafted tokens were accepted. That is a log extract, not a controlled benchmark.