vLLM introduces Humming: new quantization kernels, checked against our own production image

On 21 October 2026, Ant Group and the vLLM community released Humming, a kernel library for quantized inference. The post on the vLLM blog explicitly names the workstation line SM120, which includes our RTX 5090. So we checked two things: what the project itself documents, and how much of it reaches our running production image. The library is already in that image, although we never ordered it.

Humming is an open-source library (Apache-2.0) from Ant Group and the vLLM community. It generates GPU kernels for matrix multiplication with quantized weights at run time, through just-in-time compilation. It supports NVIDIA cards from Turing onward, integer and floating-point formats from INT1 to FP8, and both dense and mixture-of-experts layers. In vLLM 0.29.0 and later you select it with the options --linear-backend and --moe-backend.

Status as of 5 November 2026

This post was substantively revised on 5 November, which is why the header carries an updated date. The first version of 21 October contained a throughput table (13.1 to 33.4 tokens per second, "vLLM 0.29.1 versus 0.33.0") attributed to the vLLM blog. We cannot find those numbers on the blog page as it reads today. The end-to-end table there compares two MoE backends and has different values; it is shown below. The first version also called 0.33.0 an unreleased development build. That is out of date: vLLM 0.33.0 shipped on 2 November and pins humming-kernels==0.1.19. Further corrections: the 19 and 9 percent lead applies to SM121 (GB10, DGX Spark), not to the RTX 5090 line SM120. Three statements without a source (a configuration count, a Windows pipeline, a Hacker News discussion) have been removed.

What the project documents, and what it does not

All measurements come from the project, that is, from the team promoting the library. We found no independent re-measurement.

Unit: output tokens per second and milliseconds. Measurement date: publication on 21 Oct 2026. Test rig: eight H20 cards, DeepSeek-V4.1-Flash, tensor parallelism 8. Vendor numbers, not re-measured by us.

Three caveats belong with it. First, the table ran on data-center cards without Blackwell, with a model we do not operate. Second, the lead is largest under many concurrent requests: a single request gains 14 percent, 64 concurrent ones gain 43. Someone running a personal assistant on one card is closer to the first row, someone serving a company is closer to the last. Third, the table compares two backends for the same weights and says nothing about a different quantization format.

For the SM120 and SM121 lines, the post says Humming leads FlashInfer CUTLASS at most measured points on MoE layers with W4A8 (MXFP4 × MXFP8) and W4A4 (NVFP4 × NVFP4). The peaks of about 19 percent (W4A8) and 9 percent (W4A4) at small batches apply to SM121. According to the chart labels that is a GB10 (DGX Spark); for SM120 an RTX PRO 6000 was measured. The axis of the SM120 chart for W4A4 only spans 0.99 to 1.04, so the lead there is in the low single-digit percent range. The charts are image files, and we only take numbers from them where the text states them. No chart measures an RTX 5090.

On compile time: on an H200 server with CUDA 13.4, a W4A16 kernel takes 1.82 seconds with Humming against 23.35 seconds with CUTLASS and NVCC, with caches disabled, according to the post. The project says Humming compiles the kernel set for one layer in about 2 to 4 seconds in parallel during warmup. The post also mentions two previously unexposed Blackwell formats (float4e0m3 and float8e3m4) that Humming addresses. Rubin does not support them, so building on them means building on a single card generation.

The library is young. Its first PyPI release dates from 13 May 2026. On 5 November the GitHub repo stood at 349 stars, and PyPI listed version 0.1.19 from 30 October. Five items are open in the issue tracker. The oldest (#31) reports that the indexed StreamK path for MXFP4A8 loses precision through a BF16 output. One (#72) cites only about 43 percent of memory bandwidth for INT3 on SM80 at a single request, another (#69) an FP8 model that fails to load at tensor parallelism 16. #190 was added on 4 November: NVRTC drops a reduction in the NVFP4 MoE kernel while NVCC keeps it. A usage signal beyond the kernel set comes from the vLLM blog of 15 September: Novita AI presents its Chord kernel for Kimi K2.x models with a Humming-compatible indexed path.

Checked in our own image

On the morning of 21 October 2026 we looked into our running production image, read-only, and changed nothing. The test rig is host 192.168.180.3 with four RTX 5090, container fn-port, image vllm-flashnext-hybrid:v032-nvfp4port.

These are our own measurements from the first version; we did not repeat them on 5 November. Where the library came from can be shown publicly. On 5 November we read the requirements files of the vLLM tags:

Unit: version pin in requirements/cuda.txt (extra cu13). Measurement date: 5 Nov 2026. Source: the files in the vLLM repository, release dates per PyPI.

For us this means: the library has been in the image since 0.29.0, but it does not compute where the charts show gains. The expert layers run through the FlashInfer CUTLASS backend we pinned. In the linear layers auto decides, and which kernel wins there could only be shown by a measurement run, so it stays open.

What operators can take from this

Anyone running vLLM 0.29.0 or later with CUDA already has the library installed. Anyone who set backend options by hand, as we did, should check the --moe-backend line, because it decides whether Humming is even a candidate for that layer type. For owners of an RTX 5090 machine the advice stays: in a maintenance window, run the same requests once with the current backend and once with Humming, and switch only for a measurable gain with identical answers. That is how we already compared two vLLM builds for Qwen3.8-Flash-Next; the version question around 0.33.0 is covered in vLLM 0.33.0 without CUDA 12.9. More posts on operating self-hosted models and their limits are collected under AI in production. If you want to know whether an in-house language model pays off for your needs, see our corporate LLM consulting.

Where we stand: library installed, switch available, benefit on our configuration unmeasured, MoE path deliberately wired elsewhere. The throughput run against Humming is pending. No date has been decided.

Further sources

• vLLM blog: Announcing Humming, 21 Oct 2026, vendor statement with all measurement tables and charts
• GitHub repo vllm-project/humming, Apache-2.0, as of 5 Nov 2026: 349 stars
• Humming issue tracker, as of 5 Nov 2026: five open issues
• humming-kernels on PyPI, version history
• vLLM on PyPI, version and release dates, and the requirements files v0.29.0, v0.32.0 and v0.33.0