Soup: fine-tune an 8B LLM on a 4 GB laptop GPU from one YAML file
An open-source CLI that fine-tunes LLMs from a single YAML file, with layer streaming that trains an 8B model on a 4 GB laptop GPU.
Why it matters
Layer streaming keeps the frozen base in RAM and feeds it to the GPU one layer at a time, so an 8B model fine-tunes at 119.6 tok/s in 3.32 GB peak, measured bit-exact against a resident run.
What you could build with it
An indie developer with a 4 GB laptop GPU could fine-tune a small open model on their own support tickets or notes with one YAML file, then export the adapter as GGUF and run it locally.
Does it hold up?
Usable with caveats: layer streaming is still beta, limited to plain LoRA on text models and slower in wall-clock time than resident training; the headline 119.6 tok/s figure predates a correctness fix and has not been re-measured on the same card.
Built with Soup
- Soup: Fine-Tuning an 8B Model on a 4 GB Laptop GPU, One Layer at a Timetoknow.ai · Independent explainer of layer streaming, the measured 119.6 tok/s figure, and the honesty caveats: beta status, slower wall-clock time, and a fixed correctness bug.
- Soup: Fine-Tune an 8B LLM on a 4 GB Laptop GPU with This One-Command CLItopdigg · Writeup of the one-command workflow, 23 training methods and 142 recipes, and the project's practice of publishing its benchmark failures as well as its wins.
- Training Setup Complete with a Single Command (Soup intro video)youtube · Short video walkthrough: install soup-cli, generate a config with soup init, train with soup train, then test and export the model.
Learn more
First spotted on github: source.
More AI models
VoiceStudio: open-source, fully-local ElevenLabs alternative for voice cloning, dubbing, and audiobooksdebpalash/VoiceStudio is a desktop app that runs voice cloning, voice design, video dubbing, dictation, transcription…models · JEV 0.75Tencent Hunyuan Hy3: open 295B MoE with 21B active under Apache 2.0Tencent open-sourced Hy3, a 295B-parameter mixture-of-experts model with only 21B parameters active per token, under…models · JEV 0.73NVIDIA Isaac Lab 3.0 GA + Isaac OS open runtime + Agile ONE SN humanoidMajor Isaac platform expansion at GTC Berlin Oct 21: Isaac Lab 3.0 is generally available as the open-source robot…models · JEV 0.73NVIDIA Nemotron 3.5 Retriever collection: Embed 8B/1B, Rerank 8B, Parse 631MNVIDIA's Oct 21 collection drop for agentic retrieval: Nemotron-3.5-Embed in 8B BF16, 1B BF16 and 1B NVFP4…models · JEV 0.72Kimi K3.1 Pro: Moonshot's 2.8T open-weight flagship officially launchesMoonshot AI officially launched Kimi K3.1 Pro on Oct 23, a mixture-of-experts model with 2.8 trillion total / 104…models · JEV 0.72NVIDIA Cosmos 3.5 world models: now openly availableNVIDIA's Cosmos 3.5 family of open world foundation models for robots, autonomous vehicles and vision agents is now…models · JEV 0.71
Get the week's best AI launches, plus 3 ideas worth building
One email every Saturday. Ranked by traction, not hype. Free.