llama-modchip: 284B DeepSeek-V4-Flash on 64GB of CPU-only RAM
A llama.cpp fork that runs the 284B DeepSeek-V4-Flash at 1.71-bit quantization in under 59 GiB of RAM with no GPU, plus 30B MoEs at 25-111 tok/s on a plain CPU.
Why it matters
Frontier-class inference on a GPU-free desktop: 284B total parameters in 64GB RAM at 256K context, via custom SwiGLU kernels and a Stockfish-inspired L3 cache.
What you could build with it
A researcher with a RAM-rich, GPU-poor workstation could try running a 284B-class model locally for long-context document analysis, measuring whether the 1.71-bit quant keeps enough quality for their task before committing to it.
Does it hold up?
Too early to judge: all speed figures are the author's own, and early Hacker News testers report install failures and results no better than stock llama.cpp on some hardware; no independent reproduction yet.
Built with llama-modchip
- Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU - HN discussionarticle · Hacker News thread (305 points) where early testers debate the self-reported numbers; some report setup failures and results no better than stock llama.cpp on their hardware.
- Llama.cpp fork reports a 284B model running on CPU with 64GB of RAMarticle · Notes the figures are developer-reported and unverified, and points readers to the repo to reproduce the result themselves.
- Llama-modchip runs 284B DeepSeek-V4-Flash on CPU with 64 GB RAMarticle · Repository review summarising the 44.4 tok/s prefill, 9.1-14.7 tok/s generation and 13 ms Claude Code state restore.
Learn more
First spotted on github: source.
More AI models
VoiceStudio: open-source, fully-local ElevenLabs alternative for voice cloning, dubbing, and audiobooksdebpalash/VoiceStudio is a desktop app that runs voice cloning, voice design, video dubbing, dictation, transcription…models · JEV 0.75Soup: fine-tune an 8B LLM on a 4 GB laptop GPU from one YAML fileAn open-source CLI that fine-tunes LLMs from a single YAML file, with layer streaming that trains an 8B model on a 4…models · JEV 0.73Tencent Hunyuan Hy3: open 295B MoE with 21B active under Apache 2.0Tencent open-sourced Hy3, a 295B-parameter mixture-of-experts model with only 21B parameters active per token, under…models · JEV 0.73NVIDIA Isaac Lab 3.0 GA + Isaac OS open runtime + Agile ONE SN humanoidMajor Isaac platform expansion at GTC Berlin Oct 21: Isaac Lab 3.0 is generally available as the open-source robot…models · JEV 0.73NVIDIA Nemotron 3.5 Retriever collection: Embed 8B/1B, Rerank 8B, Parse 631MNVIDIA's Oct 21 collection drop for agentic retrieval: Nemotron-3.5-Embed in 8B BF16, 1B BF16 and 1B NVFP4…models · JEV 0.72Kimi K3.1 Pro: Moonshot's 2.8T open-weight flagship officially launchesMoonshot AI officially launched Kimi K3.1 Pro on Oct 23, a mixture-of-experts model with 2.8 trillion total / 104…models · JEV 0.72
Get the week's best AI launches, plus 3 ideas worth building
One email every Saturday. Ranked by traction, not hype. Free.