Qwen3-8B GGUF: Alibaba's official ready-to-run local builds with dual reasoning modes
Qwen's team published official GGUF quantizations of Qwen3-8B (Q4_K_M through Q8_0) for llama.cpp, Ollama, and LM Studio, with thinking and non-thinking modes in one checkpoint.
Why it matters
Removes the conversion step for local runners: one 5 GB Q4_K_M file runs the 8.2B model anywhere llama.cpp runs, and /think and /no_think switch reasoning on and off per request.
What you could build with it
A researcher building an offline writing assistant for a low-power laptop could ship the 5 GB Q4_K_M build inside the app, switching between thinking mode for drafts and fast mode for short replies without downloading separate weights.
Does it hold up?
Too early to judge — the official GGUF quants landed 14 Nov; the 5 GB Q4_K_M build makes the 8B model runnable on consumer hardware via llama.cpp and Ollama, but no independent benchmarks of these quants exist yet.
Built with Qwen3-8B GGUF
- Alibaba Ships Qwen3-8B as Ready-to-Run Local AI With Dual Reasoning Modesarticle · Details the Q4_K_M (5.03 GB) and Q8_0 (8.71 GB) builds, the /think and /no_think switch, and the 32K native context extendable to 131K with YaRN.
Learn more
First spotted on hf: source.
More AI models
VoiceStudio: open-source, fully-local ElevenLabs alternative for voice cloning, dubbing, and audiobooksdebpalash/VoiceStudio is a desktop app that runs voice cloning, voice design, video dubbing, dictation, transcription…models · JEV 0.75Edge0-145B: open 145B MoE that runs on a single consumer GPUEdge0 open-sourced 145B and 120B latent-MoE models derived from Qwen3.8-Flash-Next, plus an inference framework that…models · JEV 0.74Strata: run a 125B model on a gaming PCFree, MIT-licensed local inference engine that runs Qwen3.8-Flash-Next (125B MoE, about 6B active per token) on…models · JEV 0.73Google AI Edge Gallery: offline speech recognition and synthesis on-deviceGoogle added fully offline speech recognition and text-to-speech to AI Edge Gallery, with ML Kit APIs on Android and…models · JEV 0.73Soup: fine-tune an 8B LLM on a 4 GB laptop GPU from one YAML fileAn open-source CLI that fine-tunes LLMs from a single YAML file, with layer streaming that trains an 8B model on a 4…models · JEV 0.73Tencent Hunyuan Hy3: open 295B MoE with 21B active under Apache 2.0Tencent open-sourced Hy3, a 295B-parameter mixture-of-experts model with only 21B parameters active per token, under…models · JEV 0.73
Get the week's best AI launches, plus 3 ideas worth building
One email every Saturday. Ranked by traction, not hype. Free.