vLLM 0.33.0 Drops CUDA 12.9: What Self-Hosters Need to Check Now

On 2 November vLLM published version 0.33.0: 927 commits from 389 contributors, 122 of them new. For operators of mixed machine rooms, one line in the release notes matters more than any throughput headline. Release wheels and images are CUDA 13.0 only, and the CUDA 12.9 wheels and the -cu129 image tags are no longer published (#62575). In late September cu129 was still a stopgap tag next to the CUDA 13 default build, as we described in our post on vLLM 0.30 and llama.cpp 0.5. The stopgap is gone now. When the tag, image and release notes of 0.33.0 appeared, we measured in vLLM v0.33.0 and llama.cpp b12033. This post asks something else: what dropping cu129 means for drivers and operations.

What is cu129? A cu129 image is a vLLM container image built against CUDA 12.9 that needs, by NVIDIA's table, a driver from R525 up. From vLLM 0.33.0 the project publishes only CUDA 13.0 artifacts, and according to NVIDIA, CUDA 13.x needs driver R580 or newer. Anyone who has been running a driver between 525 and 579 has to change it before updating the image.

What disappears with cu129

Pull request #62575 was merged on 20 October. It removes the CUDA 12.9 wheels and images, including the Ubuntu 24.04 variant, from the release pipeline. Other items from the release notes: PyTorch 2.14.0 with torchvision 0.29.0 and Triton 3.8.0 (#52183), a CUDA wheel that is ABI-stable with libtorch, and ROCm 10.0 as the default ROCm image with ROCm 7.2 kept as -rocm72 (#58761). Every release image is also published under the shorter name vllm/vllm, for example vllm/vllm:v0.33.0.

Which driver is enough

You can answer this in a minute: nvidia-smi shows the driver version. NVIDIA's minor version compatibility docs list a driver of 580 or newer for CUDA 13.x and 525 or newer for CUDA 12.x in their table. Drivers 525 to 579 therefore serve CUDA 12 images but no CUDA 13 applications. An Ubuntu 24.04 host with the nvidia-driver-550 package could start the cu129 images. From 0.33.0 it needs a newer driver or the forward compatibility packages. NVIDIA describes those in its forward compatibility docs for data-center GPUs, selected server-ready RTX models and Jetson. We have not tried whether they work on cards like the RTX 5090.

What else changes in the release

The release notes list under "Breaking changes":

1. Medusa and the MLP speculator are removed (#61701). EAGLE, MTP and the DFlash/DSpark drafters stay, according to the PR description.
2. The legacy Ray compiled-graph executor is removed; RayExecutorV2 is the only Ray executor (#61712).
3. The NIXL connector needs NIXL 1.5 or newer (#61032). We have not tested how an older NIXL behaves at startup.
4. Requests that arrive while generation is paused get HTTP 503. --mamba-cache-mode, LM Format Enforcer and --ir-op-priority are marked deprecated.

On the Rust front end (vllm-rs), long conversations failed with HTTP 500 although the prompt sat far below the context length. The cause was a hard-coded MiniJinja budget of 500,000 instructions per render. According to the pull request description (which cites bug report #62598), a GLM-5.3 agent session with 1,730 messages and 21,000 tokens, about 2 percent of its one-million-token context, was rejected (#62374). Since 0.33.0 the server renders its own template without a budget, like the Python front end. The budget now applies only to templates supplied in the request, which need --trust-request-chat-template, and can be changed via VLLM_CHAT_TEMPLATE_FUEL. If you run vllm-rs even experimentally, test both paths with long sessions before a production swap.

On speed, the notes give figures from the pull requests that we have not re-measured. DFlash and DSpark no longer reserve draft slots in the token budget; in the PR's benchmark, throughput for DeepSeek-V4.1-Flash with DSpark on a GB300 at --max-num-batched-tokens 512 goes from 1,373 to 3,390 output tokens per second (#59468). The lossy FLy verifier is opt-in and reaches 1.04 to 1.31 times decode throughput according to the notes (MI355X and B300, #53987). According to the release notes, newly on by default are lookahead block hashes, so EAGLE and MTP prefix-cache hits keep their last block (#61589). On the cache itself, we covered two vLLM security advisories on the prefix cache.

Among the new models, the notes list Reflection Beam-1.0 (501B MoE, 23B active) with a DFlash2 drafter (#62074), Yandex AliceAI-Foundation-80B-A3B, NVIDIA Alpamayo 2 Super and Recursive Flint (40B MoE, 2.5B active) with BF16 and NVFP4 GGUF checkpoints (#62938). We have measured none of them.

Our lane

The table shows the state of the production lane on .180.3 (4x RTX 5090, EPYC 8024P). These are values from our measurement logs, not a new measurement for this post.

The cu129 change therefore does not hit this lane: driver and image base are already on CUDA 13. What is open is the rebuild. The image carries our port of the NVFP4 KV cache from a still-open pull request (#54846), described in our post on Flash-Next on vLLM 0.32. Whether the image carries further patches is not something this post documents. The P2P-patched NVIDIA kernel module is silently reverted by a kernel update, after which nvidia-smi topo -p2p r shows GNS again. Whether and when we move to 0.33.0 is not decided.

What operators should do now

1. Run nvidia-smi on every vLLM host: a driver below 580 means a driver change in the update window, and only then pulling the image.
2. If you use your own distribution or vendor images instead of the vLLM wheels, check their CUDA base. It is not automatically 13.0.
3. If you run Medusa drafters, the old Ray executor or an older NIXL, read the list above first.
4. For a first overview of speculative decoding, see speeding up self-hosted AI inference. More hardware posts are in the local AI hardware hub. If you are planning your own inference environment, our offer is under Corporate LLM.