Radar / AI infrastructure / colibri
colibri: run 744B-parameter MoE models on consumer hardware
In today's Spotlight · Picked by 3 of 5 modelsLets an ordinary computer run enormous AI models by moving parts between graphics memory, RAM and disk as needed.Like a librarian keeping popular books at the desk, rare ones in storage.
Pure-C inference engine with zero dependencies that streams MoE experts from NVMe disk, keeping hot experts on GPU, warm ones in RAM and cold ones on disk — running GLM-5.2 (744B) on a machine with about 25GB of RAM; v2.1.0 landed October 31, about 47,000 stars, #4 on GitHub daily trending on November 4.
Why it matters
Turns VRAM, RAM and storage into one managed memory hierarchy without silently changing precision or router semantics, so frontier-scale MoE models run on hardware people already own.
What you could build with it
A researcher with a single workstation could run a 744B-parameter open model locally through colibri to test prompts against a frontier-scale model without renting GPU servers, accepting slower tokens in exchange for zero cloud cost.
Does it hold up?
Usable today for patient local experimentation — prebuilt releases and a web dashboard exist — but a 744B model still needs about 372GB of disk and runs well under one token per second on laptop-class hardware, so it suits testing, not production serving.
Built with colibri
- Open-source C engine streams massive MoE models from your diskearlysignal · Trending note: about 825 stars in a day, 46.9K total, aimed at developers who want frontier models without expensive GPUs.
- From an 8K-star Rust desktop agent to a 92K-star web scraping enginecoddykit · Trending roundup placing colibri at 46,263 stars as the local-first inference entry in the execution-infrastructure wave.
- JustVugg/colibri repository overviewtheresanaiforthat · README mirror documenting the GLM-5.2 int4 container at about 372GB and the quality-preserving streaming policy.
Learn more
First spotted on github: source.
More AI infrastructure
claude-mem: persistent memory plugin that lets coding agents pick up where the last session left offPlugin that hooks five Claude Code lifecycle events, compresses session activity into a local SQLite + Chroma store…infra · JEV 0.74FreeToken: datacenter-scale MoE serving on desktop GPUsAn edge-native mixture-of-experts serving engine from UC Berkeley researchers that runs 290B+ parameter MoE models on…infra · JEV 0.74TorchTPU: Google's native PyTorch stack for TPUs, now open sourceAnnounced at PyTorch Conference 2026 on October 20: Google open-sourced TorchTPU, its native PyTorch backend for Cloud…infra · JEV 0.66Mesh-LLM: pool GPUs across machines behind one OpenAI-compatible endpointRust project that splits large models across machines into layer stages, fetches model shards by package, and serves…infra · JEV 0.64DeepInfer: DeepSeek's open-source production inference engineDeepSeek open-sourced DeepInfer, the high-performance inference framework powering its production services (155…infra · JEV 0.6Message Threads: Anthropic's public beta that stops agents resending the whole conversation every turnMessage Threads lets Messages API apps send only new messages after an initial full-conversation request, cutting…infra · JEV 0.57
Get the week's best AI launches, plus 3 ideas worth building
One email every Saturday. Ranked by traction, not hype. Free.