Maple: sub-2-bit MoE pruning that runs Gemma 4 26B on a 6GB GPU
Manifold-Activated Pruning of Latent Experts compresses Gemma 4 26B-A4B from 9.92 GB to a 4.75 GB GGUF that the author reports at 45.4 tokens/s on a 6GB laptop GPU.
Why it matters
Excises 80 dormant experts to fit a 26B MoE on a 6GB laptop GPU at 1.48 bits per parameter, with all 30 layers resident instead of 16 offloaded.
What you could build with it
An indie developer with a 6GB laptop GPU could run the pruned Gemma 4 26B locally as a private coding assistant, keeping code on-device instead of paying for a hosted API.
Does it hold up?
Too early to judge: a single-developer benchmark with no independent verification yet; the 4.75 GB GGUF is downloadable from the repo for anyone to test.
Built with Maple
- MAPLE puts Gemma 4 26B-A4B on 6GB GPUs at 45.4 tokens/secarticle · Walks through the benchmark table: 4,180 MiB VRAM with all layers on GPU, versus 6.9 tok/s for the unpruned model on the same card.
- Maple Pruning Enables Gemma 4 26B on 6GB GPUsarticle · Show HN tracker for the 7-8 Nov debut, summarising the 1.48-bits-per-parameter compression and the 45.4 tok/s claim.
Learn more
First spotted on github: source.
More AI models
VoiceStudio: open-source, fully-local ElevenLabs alternative for voice cloning, dubbing, and audiobooksdebpalash/VoiceStudio is a desktop app that runs voice cloning, voice design, video dubbing, dictation, transcription…models · JEV 0.75Soup: fine-tune an 8B LLM on a 4 GB laptop GPU from one YAML fileAn open-source CLI that fine-tunes LLMs from a single YAML file, with layer streaming that trains an 8B model on a 4…models · JEV 0.73Tencent Hunyuan Hy3: open 295B MoE with 21B active under Apache 2.0Tencent open-sourced Hy3, a 295B-parameter mixture-of-experts model with only 21B parameters active per token, under…models · JEV 0.73NVIDIA Isaac Lab 3.0 GA + Isaac OS open runtime + Agile ONE SN humanoidMajor Isaac platform expansion at GTC Berlin Oct 21: Isaac Lab 3.0 is generally available as the open-source robot…models · JEV 0.73NVIDIA Nemotron 3.5 Retriever collection: Embed 8B/1B, Rerank 8B, Parse 631MNVIDIA's Oct 21 collection drop for agentic retrieval: Nemotron-3.5-Embed in 8B BF16, 1B BF16 and 1B NVFP4…models · JEV 0.72Kimi K3.1 Pro: Moonshot's 2.8T open-weight flagship officially launchesMoonshot AI officially launched Kimi K3.1 Pro on Oct 23, a mixture-of-experts model with 2.8 trillion total / 104…models · JEV 0.72
Get the week's best AI launches, plus 3 ideas worth building
One email every Saturday. Ranked by traction, not hype. Free.