Huawei MACaQ Checked: Supposedly Better GGUFs, Old Version Pins
On 10 November 2026 Huawei's Systems Lab put MACaQ on GitHub, three days after the paper appeared on arXiv. The quantizer targets the two runtimes our inference runs on: GGUF for llama.cpp and compact checkpoints for vLLM. Licence Apache-2.0. On 15 November 2026 we read the repository, the paper, the requirements files and the published weights read-only, and measured the surrounding conditions on our GPU host. We have not installed or started MACaQ. More measurements from this field are in the hub Local AI and GPU hardware.
W4A4 means that both the weights and the activations of a language model are computed at four bits per value. The usual 4-bit setup quantizes only the weights and keeps activations wider. With W4A4 the compute kernels can also run the multiplication at low precision, which raises throughput. The price is custom kernels and a quality loss that depends on the model and has to be measured on your own workload.
According to the paper, MACaQ is an MXFP4 W4A4 method: each linear layer is treated on its own, with a per-channel scale and a block-Hadamard transform applied to the activations at run time.
What MACaQ promises
Every figure in this table is vendor-claimed, taken from the paper and the README. We did not reproduce them. Two can be recomputed; see below.
| Vendor claim | Value | Context |
|---|---|---|
| Quantizing a 122B model | 64 min | 32.5 GB device memory, per the paper |
| Quantization time against MR-GPTQ and DuQuant++ | 11× shorter | same perplexity, Llama-3-8B, per the paper |
| Overhead against plain MXFP4 | 2.3 % | Qwen3-32B in vLLM, per the paper |
| Qwen3.8-27B, W4A4 against BF16 | 2.33× prefill, 2.58× decode | perplexity 2.8994 against 2.8463, per the README |
| Qwen3.6-35B-A3B, MACaQ-R Q4_K_XL | 22.36 GB | KL divergence to FP16: 0.013367 |
| Same model, Unsloth UD-Q4_K_XL | 22.36 GB | KL divergence to FP16: 0.015731 |
Recomputed: the KL divergence of the MACaQ-R file is 15.0 percent below that of the equally sized Unsloth file. And 2.8994 against 2.8463 is 1.87 percent more perplexity.
What we measured on 15 Nov 2026
Method: the GitHub API and raw repository files, the Hugging Face API, PyPI data for vLLM, and read-only queries on our four-card GPU host (Docker containers and nvidia-smi). The values sit in a table so the measuring platform is clear.
| Finding | Value | Method |
|---|---|---|
| Repository created | 10 Nov 2026 08:52 UTC | GitHub API |
| Last push | 12 Nov 2026 09:57 UTC | GitHub API |
| Stars, forks, open issues | 307, 21, 6 | GitHub API, 15 Nov |
| Licence | Apache-2.0 | GitHub API |
| Published GGUF, Qwen3.6-35B-A3B | 22,360,455,968 bytes (22.36 GB, 20.82 GiB) | Hugging Face API |
| Downloads (30 days), likes | 1,933, 13 | Hugging Face API, 15 Nov |
| Bundled llama.cpp, state | 28 Aug 2026 | file llamacpp/UPSTREAM |
| That state's distance behind llama.cpp master | 1,744 commits | GitHub compare API |
| vLLM pin of the standard guide | 0.20.2 of 10 May 2026 | requirements/vllm-base.txt, PyPI |
| vLLM pin of the Flash-Next guide | 0.30.0 of 22 Sep 2026 | requirements/vllm-flash.txt, PyPI |
| Newest vLLM version | 0.33.0 of 2 Nov 2026 | PyPI |
| Our vLLM lane | 0.32.0, image of 18 Oct 2026 | import inside the container on the four-card host |
| Cards on that host | 4× RTX 5090, 32,607 MiB each, compute capability 12.0 | nvidia-smi |
File size causes frequent mix-ups. The Hugging Face repository reports 22,360,455,968 bytes: 22.36 GB in powers of ten, 20.82 GiB in powers of two. A card with 31.84 GiB keeps 11.02 GiB after the weights for key-value cache and compute graph. Anyone who enters 22.36 GB into a GiB budget plans 1.5 GiB too much.
The narrow path and the wide path
MACaQ has two paths, and they do not cost the same.
The wide path is the full MACaQ route with its own llama.cpp copy, which you build yourself against the CUDA toolkit, with Python 3.12 and a separate environment per backend. The copy's state is in llamacpp/UPSTREAM: a commit of 28 Aug 2026, which the GitHub comparison puts 1,744 commits behind current llama.cpp. Whoever takes this path maintains a branch of their own.
The narrow path is MACaQ-R. It refines the weights of an existing GGUF and leaves file format and block scales alone. By the vendor's account the file runs on a standard llama.cpp install. The published example is 20.82 GiB, a class that fits our cards. For a test run this is the path we would take: download the file, start a standard llama.cpp, compare against reference runs.
The vLLM path registers itself as a plugin in the worker processes. The standard guide (requirements/vllm.txt, and the variant for Qwen3.8-27B too) pulls in vLLM 0.20.2 of 10 May 2026 through vllm-base.txt. The guide for Qwen3.8-Flash-Next pins 0.30.0 of 22 Sep 2026. Our lane runs 0.32.0, and the newest release is 0.33.0. The README also says the current CLI processes one sequence at a time. A production lane serves several sequences at once. So in this form the path is not meant for production serving. For Flash-Next the README gives 360 GB for the original source and 178 GB for the serving checkpoint. Whether the recipe fits our Flash-Next model we did not check.
A kernel that exists only in its own image is tied to its own build chain. We described the same pattern for the quant kernels in vLLM.
What the documentation leaves open
First, in the README files we read we found no list of supported GPU generations for the W4A4 kernels. The paper names MXFP4. Whether our Blackwell cards (compute capability 12.0) run the kernels is known only after a real build. Second, for Ascend accelerators the README links two pull requests in the CANN repositories; we did not check their state. Third, the project is five days old, with six open issues. We found no versioning other than the repository itself.
On download time: a cold download from Hugging Face ran at about 30 MiB/s on 12 Nov 2026 on our standby GPU host. On 15 Nov 2026 a 2 GB range request on the same host delivered about 106 MiB/s. At 30 MiB/s the 22.36 GB are on disk after 11.8 minutes, at 106 MiB/s after 3.4 minutes. The BF16 source weights of a 122B model would be 244 GB on paper (122 billion parameters times two bytes, derived): 2.15 hours at 30 MiB/s, 37 minutes at 106 MiB/s. The vendor's 64 minutes describe the compute work of quantization and not the road to a finished file. The rate varies by a factor of 3.5 here and is no constant.
What we do with it
A test run is worth it, but only in the cheap variant: start the MACaQ-R file for Qwen3.6-35B-A3B on a free card with an unmodified llama.cpp and run it against our reference answers. That is the yardstick we apply to every change of data type: replay of our own corpus first, verdict second. Rebuilding the production lane on W4A4 is not what this means.
Anyone who does not quantize but takes what model authors and GGUF vendors ship can use MACaQ-R to check, without any infrastructure change, whether a refinement really holds 15 percent lower KL divergence at the same file size, as the vendor states. For businesses without a free card for such tests, that is exactly the question we discuss under AI on your own hardware.
Open points
- All MACaQ quality and speed figures are vendor claims we did not reproduce.
- Whether the W4A4 kernels run on RTX 5090 cards: untested.
- State of the Ascend pull requests: not checked.
- Whether the Flash-Next recipe fits our Flash-Next model: not checked.
- The Hacker News thread serves only as a pointer to third-party reproduction attempts; we take no figures from it.
Read next: DSpine as a draft model for Qwen3.8-27B · Inkling 2 Small against our VRAM limit
Further Reading
- MACaQ on GitHub (vendor, primary): README, licence, tables; stars, forks and push date measured by us through the GitHub API.
- Paper on arXiv (vendor, primary): 64 minutes for 122B, 11-fold time saving, 2.3 percent overhead.
- MACaQ-R GGUF on Hugging Face (vendor, primary): file size, downloads, licence.
- vLLM requirements file and vLLM guide (vendor, primary): pin 0.20.2, one sequence per run, disk needs.
- vLLM on PyPI (primary): release dates of 0.20.2, 0.30.0, 0.32.0 and 0.33.0, read by us from the PyPI data.
- Hacker News thread (community): pointer to third-party reproductions.
Do I have to fork llama.cpp to use MACaQ?+
For the full path yes, for the narrow one no. The full path ships its own llama.cpp copy, whose state is a commit from 28 Aug 2026 according to the UPSTREAM file. MACaQ-R only refines the weights of an existing GGUF and produces a file that, by the vendor's account, runs on a standard llama.cpp install. Anyone who just wants supposedly better 4-bit files starts there.
Does the published MACaQ-R GGUF fit an RTX 5090?+
On paper, yes. The Hugging Face API gives the file as 22,360,455,968 bytes, which is 20.82 GiB. One of our cards reports 32,607 MiB, or 31.84 GiB. That leaves 11.02 GiB for key-value cache and compute graph. This is a space calculation; we did not start the file.
Isn't 4-bit automatically worse?+
It is measurably different; whether it is worse depends on the use. For Qwen3.8-27B the vendor reports perplexity 2.8994 in W4A4 against 2.8463 in BF16, which is 1.87 percent higher, at 2.33 times the prefill speed. Whether that shows in our agent answers only a replay of our own corpus can tell.
senn-tech