MiniMind: train your own 64M LLM from scratch in about 2 hours
A complete, from-scratch LLM training pipeline at 64M parameters - pretraining, SFT, LoRA, DPO, PPO/GRPO, tool use and agentic RL - runnable on a single rented RTX 3090 for about 3 yuan.
Why it matters
The whole modern post-training stack as readable code you can step through: every stage is one script, with open datasets and weights that load in Transformers, llama.cpp, vLLM and Ollama.
What you could build with it
A university instructor or bootcamp could build a hands-on LLM course around MiniMind, where each student trains a tiny model from scratch on a rented GPU and inspects every training stage instead of only calling an API.
Does it hold up?
Proven as a teaching tool: multiple independent tutorials run the pipeline end to end on CPU or one GPU. As a model it is not production-usable - the 64M weights hallucinate freely and score near chance on most benchmarks.
Built with MiniMind
- MiniMind: the whole LLM training pipeline at 64M parametersnowucca.com · A careful read of the repo against the headline claims: what the 2-hour/3-yuan numbers actually cover, and where the data is distilled from bigger models.
- Train your own LLM from scratch: hands-on with MiniMind's 64M-parameter modelaifrontierpost.com · A hands-on tutorial that runs the released weights, then runs the real pretraining and SFT scripts on a small data slice on an ordinary CPU.
- Train Your Own LLMarjavjain.org · A beginner-friendly walkthrough of the full MiniMind workflow, from cloning the repo to chatting with a checkpoint trained on a rented GPU.
Learn more
First spotted on github: source.
More AI models
VoiceStudio: the open-source, fully-local ElevenLabs alternativeA desktop app bundling 20 local speech-generation engines and 10 transcription options behind one UI — voice cloning…models · JEV 0.76Edge0-145B: open 145B MoE that runs on a single consumer GPUEdge0 open-sourced 145B and 120B latent-MoE models derived from Qwen3.8-Flash-Next, plus an inference framework that…models · JEV 0.74Strata: run a 125B model on a gaming PCFree, MIT-licensed local inference engine that runs Qwen3.8-Flash-Next (125B MoE, about 6B active per token) on…models · JEV 0.73tiny-voca: a fully open 4.5M-parameter TTS trained for under $10 of computeA fully open 4.5M-parameter English TTS — code, weights and dataset under Apache 2.0 — trained from scratch in 7.7…models · JEV 0.67Qwen4-Max: Alibaba's new flagship model goes liveAlibaba's 2.4T-parameter flagship is now generally available via API, with a 1M-token context, native vision, and…models · JEV 0.67Apple Depth Pro 2: sharp, temporally consistent metric depth for images and videoApple released Depth Pro 2, a family of four open-weight models (24M-872M) that estimate metric depth in meters from a…models · JEV 0.66
Get the week's best AI launches, plus 3 ideas worth building
One email every Saturday. Ranked by traction, not hype. Free.