Skip to content
senn-techsenn-tech
Back to blog
AI Hardware
Local AI and GPU hardware2026-11-15· By Franz Senn

Huawei MACaQ Checked: Supposedly Better GGUFs, Old Version Pins

On 10 November 2026 Huawei's Systems Lab put MACaQ on GitHub, three days after the paper appeared on arXiv. The quantizer targets the two runtimes our inference runs on: GGUF for llama.cpp and compact checkpoints for vLLM. Licence Apache-2.0. On 15 November 2026 we read the repository, the paper, the requirements files and the published weights read-only, and measured the surrounding conditions on our GPU host. We have not installed or started MACaQ. More measurements from this field are in the hub Local AI and GPU hardware.

W4A4 means that both the weights and the activations of a language model are computed at four bits per value. The usual 4-bit setup quantizes only the weights and keeps activations wider. With W4A4 the compute kernels can also run the multiplication at low precision, which raises throughput. The price is custom kernels and a quality loss that depends on the model and has to be measured on your own workload.

According to the paper, MACaQ is an MXFP4 W4A4 method: each linear layer is treated on its own, with a per-channel scale and a block-Hadamard transform applied to the activations at run time.

Huawei logo
Huawei Systems Lab published MACaQ on 10 November 2026. Quality and speed figures are vendor statements. (Quelle: Simple Icons (Repo CC0), trademark: Huawei)

What MACaQ promises

Every figure in this table is vendor-claimed, taken from the paper and the README. We did not reproduce them. Two can be recomputed; see below.

Vendor claimValueContext
Quantizing a 122B model64 min32.5 GB device memory, per the paper
Quantization time against MR-GPTQ and DuQuant++11× shortersame perplexity, Llama-3-8B, per the paper
Overhead against plain MXFP42.3 %Qwen3-32B in vLLM, per the paper
Qwen3.8-27B, W4A4 against BF162.33× prefill, 2.58× decodeperplexity 2.8994 against 2.8463, per the README
Qwen3.6-35B-A3B, MACaQ-R Q4_K_XL22.36 GBKL divergence to FP16: 0.013367
Same model, Unsloth UD-Q4_K_XL22.36 GBKL divergence to FP16: 0.015731

Recomputed: the KL divergence of the MACaQ-R file is 15.0 percent below that of the equally sized Unsloth file. And 2.8994 against 2.8463 is 1.87 percent more perplexity.

What we measured on 15 Nov 2026

Method: the GitHub API and raw repository files, the Hugging Face API, PyPI data for vLLM, and read-only queries on our four-card GPU host (Docker containers and nvidia-smi). The values sit in a table so the measuring platform is clear.

FindingValueMethod
Repository created10 Nov 2026 08:52 UTCGitHub API
Last push12 Nov 2026 09:57 UTCGitHub API
Stars, forks, open issues307, 21, 6GitHub API, 15 Nov
LicenceApache-2.0GitHub API
Published GGUF, Qwen3.6-35B-A3B22,360,455,968 bytes (22.36 GB, 20.82 GiB)Hugging Face API
Downloads (30 days), likes1,933, 13Hugging Face API, 15 Nov
Bundled llama.cpp, state28 Aug 2026file llamacpp/UPSTREAM
That state's distance behind llama.cpp master1,744 commitsGitHub compare API
vLLM pin of the standard guide0.20.2 of 10 May 2026requirements/vllm-base.txt, PyPI
vLLM pin of the Flash-Next guide0.30.0 of 22 Sep 2026requirements/vllm-flash.txt, PyPI
Newest vLLM version0.33.0 of 2 Nov 2026PyPI
Our vLLM lane0.32.0, image of 18 Oct 2026import inside the container on the four-card host
Cards on that host4× RTX 5090, 32,607 MiB each, compute capability 12.0nvidia-smi

File size causes frequent mix-ups. The Hugging Face repository reports 22,360,455,968 bytes: 22.36 GB in powers of ten, 20.82 GiB in powers of two. A card with 31.84 GiB keeps 11.02 GiB after the weights for key-value cache and compute graph. Anyone who enters 22.36 GB into a GiB budget plans 1.5 GiB too much.

Age of the versions in days, as of 15 Nov 2026vLLM pin 0.20.2189 · standard guidebundled llama.cpp79 · commit of 28 AugvLLM pin 0.30.054 · Flash-Next guideour vLLM lane28 · image of 18 OctMACaQ repo5 · created 10 Nov0200
Age in days up to 15 Nov 2026: date of the release or commit against today. The standard guide's pin is the oldest value in this calculation. (Quelle: our own measurement: GitHub API, PyPI, Docker on the four-card host, 15 Nov 2026)

The narrow path and the wide path

MACaQ has two paths, and they do not cost the same.

The wide path is the full MACaQ route with its own llama.cpp copy, which you build yourself against the CUDA toolkit, with Python 3.12 and a separate environment per backend. The copy's state is in llamacpp/UPSTREAM: a commit of 28 Aug 2026, which the GitHub comparison puts 1,744 commits behind current llama.cpp. Whoever takes this path maintains a branch of their own.

The narrow path is MACaQ-R. It refines the weights of an existing GGUF and leaves file format and block scales alone. By the vendor's account the file runs on a standard llama.cpp install. The published example is 20.82 GiB, a class that fits our cards. For a test run this is the path we would take: download the file, start a standard llama.cpp, compare against reference runs.

The vLLM path registers itself as a plugin in the worker processes. The standard guide (requirements/vllm.txt, and the variant for Qwen3.8-27B too) pulls in vLLM 0.20.2 of 10 May 2026 through vllm-base.txt. The guide for Qwen3.8-Flash-Next pins 0.30.0 of 22 Sep 2026. Our lane runs 0.32.0, and the newest release is 0.33.0. The README also says the current CLI processes one sequence at a time. A production lane serves several sequences at once. So in this form the path is not meant for production serving. For Flash-Next the README gives 360 GB for the original source and 178 GB for the serving checkpoint. Whether the recipe fits our Flash-Next model we did not check.

A kernel that exists only in its own image is tied to its own build chain. We described the same pattern for the quant kernels in vLLM.

What the documentation leaves open

First, in the README files we read we found no list of supported GPU generations for the W4A4 kernels. The paper names MXFP4. Whether our Blackwell cards (compute capability 12.0) run the kernels is known only after a real build. Second, for Ascend accelerators the README links two pull requests in the CANN repositories; we did not check their state. Third, the project is five days old, with six open issues. We found no versioning other than the repository itself.

On download time: a cold download from Hugging Face ran at about 30 MiB/s on 12 Nov 2026 on our standby GPU host. On 15 Nov 2026 a 2 GB range request on the same host delivered about 106 MiB/s. At 30 MiB/s the 22.36 GB are on disk after 11.8 minutes, at 106 MiB/s after 3.4 minutes. The BF16 source weights of a 122B model would be 244 GB on paper (122 billion parameters times two bytes, derived): 2.15 hours at 30 MiB/s, 37 minutes at 106 MiB/s. The vendor's 64 minutes describe the compute work of quantization and not the road to a finished file. The rate varies by a factor of 3.5 here and is no constant.

What we do with it

A test run is worth it, but only in the cheap variant: start the MACaQ-R file for Qwen3.6-35B-A3B on a free card with an unmodified llama.cpp and run it against our reference answers. That is the yardstick we apply to every change of data type: replay of our own corpus first, verdict second. Rebuilding the production lane on W4A4 is not what this means.

Anyone who does not quantize but takes what model authors and GGUF vendors ship can use MACaQ-R to check, without any infrastructure change, whether a refinement really holds 15 percent lower KL divergence at the same file size, as the vendor states. For businesses without a free card for such tests, that is exactly the question we discuss under AI on your own hardware.

Open points

  • All MACaQ quality and speed figures are vendor claims we did not reproduce.
  • Whether the W4A4 kernels run on RTX 5090 cards: untested.
  • State of the Ascend pull requests: not checked.
  • Whether the Flash-Next recipe fits our Flash-Next model: not checked.
  • The Hacker News thread serves only as a pointer to third-party reproduction attempts; we take no figures from it.

Read next: DSpine as a draft model for Qwen3.8-27B · Inkling 2 Small against our VRAM limit

Further Reading

Questions?
Do I have to fork llama.cpp to use MACaQ?+

For the full path yes, for the narrow one no. The full path ships its own llama.cpp copy, whose state is a commit from 28 Aug 2026 according to the UPSTREAM file. MACaQ-R only refines the weights of an existing GGUF and produces a file that, by the vendor's account, runs on a standard llama.cpp install. Anyone who just wants supposedly better 4-bit files starts there.

Does the published MACaQ-R GGUF fit an RTX 5090?+

On paper, yes. The Hugging Face API gives the file as 22,360,455,968 bytes, which is 20.82 GiB. One of our cards reports 32,607 MiB, or 31.84 GiB. That leaves 11.02 GiB for key-value cache and compute graph. This is a space calculation; we did not start the file.

Isn't 4-bit automatically worse?+

It is measurably different; whether it is worse depends on the use. For Qwen3.8-27B the vendor reports perplexity 2.8994 in W4A4 against 2.8463 in BF16, which is 1.87 percent higher, at 2.33 times the prefill speed. Whether that shows in our agent answers only a replay of our own corpus can tell.