DeepSeek V3 in ~200 lines of pure JAX
The full DeepSeek-V3 architecture — MLA, DeepSeekMoE, aux-loss-free balancing, multi-token prediction — in 190 lines of dependency-free JAX, trained from scratch on a MacBook Air.
Why it matters
A frontier architecture made readable: the entire model fits in one file you can read top to bottom, with a from-scratch training run (36.2M params, validation loss 5.02 to 3.51) and 39 tests proving each mechanism.
What you could build with it
A university lecturer teaching modern LLM design could assign students this single 190-line file, having each team swap one mechanism — the MoE router, the balancing bias, the MTP head — and measure the effect on validation loss in an afternoon.
Does it hold up?
Usable today as a learning reference — 39 tests pass, the full run reproduces on a MacBook Air or an AMD GPU, and a v1.0 checkpoint is downloadable; as a language model it is a 36M-parameter demo trained on detective fiction, not a usable assistant.
Built with DeepSeek V3 in ~200 lines of pure JAX
- Show HN: DeepSeek V3 in ~200 lines of pure Jaxarticle · Coverage of the 171-point Show HN post: a 190-line JAX implementation trained on public-domain detective fiction on a MacBook Air.
- Show HN: DeepSeek V3 in ~200 lines of pure Jaxarticle · Summary of the release: 190 lines of pure JAX, no Flax or PyTorch, 36.2M parameters, median training speed about 10.7k tokens/s on a MacBook Air.
- DeepSeek V3 implemented in ~200 lines of pure Jaxarticle · Launch listing describing the minimal implementation as a learning and experimentation reference for the architecture.
Learn more
First spotted on github: source.
More AI models
VoiceStudio: open-source, fully-local ElevenLabs alternative for voice cloning, dubbing, and audiobooksdebpalash/VoiceStudio is a desktop app that runs voice cloning, voice design, video dubbing, dictation, transcription…models · JEV 0.75Edge0-145B: open 145B MoE that runs on a single consumer GPUEdge0 open-sourced 145B and 120B latent-MoE models derived from Qwen3.8-Flash-Next, plus an inference framework that…models · JEV 0.74Strata: run a 125B model on a gaming PCFree, MIT-licensed local inference engine that runs Qwen3.8-Flash-Next (125B MoE, about 6B active per token) on…models · JEV 0.73Google AI Edge Gallery: offline speech recognition and synthesis on-deviceGoogle added fully offline speech recognition and text-to-speech to AI Edge Gallery, with ML Kit APIs on Android and…models · JEV 0.73Soup: fine-tune an 8B LLM on a 4 GB laptop GPU from one YAML fileAn open-source CLI that fine-tunes LLMs from a single YAML file, with layer streaming that trains an 8B model on a 4…models · JEV 0.73Tencent Hunyuan Hy3: open 295B MoE with 21B active under Apache 2.0Tencent open-sourced Hy3, a 295B-parameter mixture-of-experts model with only 21B parameters active per token, under…models · JEV 0.73
Get the week's best AI launches, plus 3 ideas worth building
One email every Saturday. Ranked by traction, not hype. Free.