Strata: run a 125B model on a gaming PC
Free, MIT-licensed local inference engine that runs Qwen3.8-Flash-Next (125B MoE, about 6B active per token) on consumer GPUs with 12GB of VRAM, via a one-click installer and an OpenAI-compatible local API.
Why it matters
A 125B-class model at 50-94 tokens/s on a 12GB gaming GPU, with no account, no cloud, and no telemetry by default.
What you could build with it
An indie developer could ship a private, offline coding assistant that runs on a client's gaming PC, keeping proprietary code on the machine instead of sending it to a cloud API, and sell it as a one-time download to small studios.
Does it hold up?
Usable today as a resident local API server - reviewers measure 50-94 tok/s on RTX 5070-class cards - but cold starts take 3-5 minutes and it wants 64GB of system RAM, so it suits an always-on setup, not ad-hoc launches.
Built with Strata
- Strata Inference Engine: Models, Hardware, Local APIithub.directory · Hands-on guide to installing Strata and which models, GPUs and context sizes it supports.
- Strata Review 2026: Running 125B Models on Consumer GPUssortwell.net · Independent review measuring 50-94 tok/s on consumer GPUs, with honest notes on 3-5 minute cold starts and 64GB RAM needs.
- Strata 0.2.2 Released With Memory-Light Gyro Variantlinuxcompatible.org · Covers the 0.2.2 release bringing the memory-light Gyro variant to Intel Arc and Apple Silicon.
Learn more
First spotted on github: source.
More AI models
VoiceStudio: open-source, fully-local ElevenLabs alternative for voice cloning, dubbing, and audiobooksdebpalash/VoiceStudio is a desktop app that runs voice cloning, voice design, video dubbing, dictation, transcription…models · JEV 0.75Edge0-145B: open 145B MoE that runs on a single consumer GPUEdge0 open-sourced 145B and 120B latent-MoE models derived from Qwen3.8-Flash-Next, plus an inference framework that…models · JEV 0.74Google AI Edge Gallery: offline speech recognition and synthesis on-deviceGoogle added fully offline speech recognition and text-to-speech to AI Edge Gallery, with ML Kit APIs on Android and…models · JEV 0.73Soup: fine-tune an 8B LLM on a 4 GB laptop GPU from one YAML fileAn open-source CLI that fine-tunes LLMs from a single YAML file, with layer streaming that trains an 8B model on a 4…models · JEV 0.73Tencent Hunyuan Hy3: open 295B MoE with 21B active under Apache 2.0Tencent open-sourced Hy3, a 295B-parameter mixture-of-experts model with only 21B parameters active per token, under…models · JEV 0.73NVIDIA Isaac Lab 3.0 GA + Isaac OS open runtime + Agile ONE SN humanoidMajor Isaac platform expansion at GTC Berlin Oct 21: Isaac Lab 3.0 is generally available as the open-source robot…models · JEV 0.73
Get the week's best AI launches, plus 3 ideas worth building
One email every Saturday. Ranked by traction, not hype. Free.