smolmaxxing: run 16B LLMs on a 1 GB TV box or a 2013 iPad
An MIT-licensed llama.cpp patch set that runs language models up to a 16B MoE on devices with about 1 GB of usable RAM — a Xiaomi TV box and a 2013 iPad Air — with every number measured on the device itself.
Why it matters
First public, reproducible record of a 16B model running at usable speed on ~1 GB RAM hardware, with patched kernels and ready-made model files for anyone to reproduce.
What you could build with it
An indie developer could build a privacy-first voice assistant that runs entirely on a cheap TV box plugged into a living-room TV, answering questions and controlling smart-home devices with no cloud account and no monthly fee.
Does it hold up?
Usable today for tinkerers: ready-made GGUF files published 8 and 11 Nov run with the patched llama.cpp build, but stock llama.cpp cannot load them and speeds are modest (2.6-7.7 tok/s), so it is a hobbyist setup, not a product platform yet.
Built with smolmaxxing
- smolmaxxing: Run a 16B LLM on a 1 GB TV Box (llama.cpp)ai-tldr.dev · Measured speeds: Qwen3.5-2B at 7.7 tokens/s and the 16B Ling-mini-2.0 MoE at 5.1 tokens/s on the Xiaomi TV box.
- I'm making LLMs work on small and old deviceshacker news · The author describes 160+ experiments to find which models and kernels run on each device, and open-sources the patches and results.
- Getting the biggest possible models on the smallest hardwarevibingtalk.com · Author thread with the repo and Hugging Face links, and a call for harder target devices.
Learn more
First spotted on github: source.
More AI models
VoiceStudio: open-source, fully-local ElevenLabs alternative for voice cloning, dubbing, and audiobooksdebpalash/VoiceStudio is a desktop app that runs voice cloning, voice design, video dubbing, dictation, transcription…models · JEV 0.75Edge0-145B: open 145B MoE that runs on a single consumer GPUEdge0 open-sourced 145B and 120B latent-MoE models derived from Qwen3.8-Flash-Next, plus an inference framework that…models · JEV 0.74Strata: run a 125B model on a gaming PCFree, MIT-licensed local inference engine that runs Qwen3.8-Flash-Next (125B MoE, about 6B active per token) on…models · JEV 0.73Google AI Edge Gallery: offline speech recognition and synthesis on-deviceGoogle added fully offline speech recognition and text-to-speech to AI Edge Gallery, with ML Kit APIs on Android and…models · JEV 0.73Soup: fine-tune an 8B LLM on a 4 GB laptop GPU from one YAML fileAn open-source CLI that fine-tunes LLMs from a single YAML file, with layer streaming that trains an 8B model on a 4…models · JEV 0.73Tencent Hunyuan Hy3: open 295B MoE with 21B active under Apache 2.0Tencent open-sourced Hy3, a 295B-parameter mixture-of-experts model with only 21B parameters active per token, under…models · JEV 0.73
Get the week's best AI launches, plus 3 ideas worth building
One email every Saturday. Ranked by traction, not hype. Free.