MACaQ: Huawei's internal quantizer for local inference, now open source
Huawei-CSL open-sourced MACaQ, an MXFP4 W4A4 post-training quantization method that treats every linear layer independently (per-channel scales + block-Hadamard transforms + Hessian-aware Flip-Refinement), shipping code for llama.cpp GGUF, vLLM, Diffusers, and PyTorch under Apache 2.0.
Why it matters
Quantize a 122B model in 64 minutes using 32.5 GB of GPU RAM, with lower perplexity than DuQuant++ and MR-GPTQ and 11x less quantization time — and it works unchanged across text, image, video, and audio models, not just LLMs.
What you could build with it
A developer shipping a local AI app could use MACaQ to quantize a 30B-class model into a GGUF that runs on a consumer laptop, then compare its KL divergence to FP16 against existing Unsloth quants before committing to it as the app's bundled model.
Does it hold up?
Too early to judge: repo created Nov 10 with 103 stars and a detailed README with per-model recipes, but all quality numbers are the authors' own and no independent quant reproductions have appeared yet.
Built with MACaQ
- MACaQ: Fast, Model Agnostic Calibrated Quantization for Low-Precision Weights and Activationshuggingface · HF paper page: the abstract plus the Huawei-CSL team's own comment that this is the quantizer they have used internally for months, with code linked.
- MACaQ: Fast, Model Agnostic Calibrated Quantization for Low-Precision Weights and Activationsarxiv · The paper itself (Nov 7): layer-local design, 2.3% end-to-end overhead over pure MXFP4 on Qwen3-32B in vLLM, and the 122B-in-64-minutes claim.
- huawei-csl/MACaQgithub · 103 ★ · The official repo: quantize and run text, image, video and audio models with llama.cpp, vLLM, Diffusers and PyTorch, including a MACaQ-R variant that runs on stock llama.cpp.
Learn more
First spotted on github: source.
More AI models
VoiceStudio: open-source, fully-local ElevenLabs alternative for voice cloning, dubbing, and audiobooksdebpalash/VoiceStudio is a desktop app that runs voice cloning, voice design, video dubbing, dictation, transcription…models · JEV 0.75Edge0-145B: open 145B MoE that runs on a single consumer GPUEdge0 open-sourced 145B and 120B latent-MoE models derived from Qwen3.8-Flash-Next, plus an inference framework that…models · JEV 0.74Strata: run a 125B model on a gaming PCFree, MIT-licensed local inference engine that runs Qwen3.8-Flash-Next (125B MoE, about 6B active per token) on…models · JEV 0.73Google AI Edge Gallery: offline speech recognition and synthesis on-deviceGoogle added fully offline speech recognition and text-to-speech to AI Edge Gallery, with ML Kit APIs on Android and…models · JEV 0.73Soup: fine-tune an 8B LLM on a 4 GB laptop GPU from one YAML fileAn open-source CLI that fine-tunes LLMs from a single YAML file, with layer streaming that trains an 8B model on a 4…models · JEV 0.73Tencent Hunyuan Hy3: open 295B MoE with 21B active under Apache 2.0Tencent open-sourced Hy3, a 295B-parameter mixture-of-experts model with only 21B parameters active per token, under…models · JEV 0.73
Get the week's best AI launches, plus 3 ideas worth building
One email every Saturday. Ranked by traction, not hype. Free.