Radar / AI infrastructure / Janas
Janas: run LLMs bigger than your RAM from SSD
A dependency-free C engine that streams mixture-of-experts weights from an NVMe SSD, running a 48 GB model on a 32 GB laptop at a claimed 37 tokens per second.
Why it matters
Breaks the RAM wall for local inference: only the experts each token needs are streamed from disk, with hot experts cached in memory.
What you could build with it
An indie developer could build a laptop app that runs a large open MoE model entirely offline for private document Q&A, using Janas to keep the model on disk instead of requiring a GPU workstation.
Does it hold up?
Too early to judge: the author calls it an experiment, not a product, tested on one machine, and the 37 tok/s figure is self-reported with no independent reproduction yet.
Built with Janas
- Run a 48 GB AI Model on a 32 GB Laptop: Janas Streams Model Experts From Your SSD at 37 tok/sarticle · Walks through Janas's claimed benchmarks against llama.cpp and lists what to verify before trusting them.
Learn more
First spotted on github: source.
More AI infrastructure
DeepGEMM: DeepSeek's open GPU BLAS kernel libraryDeepSeek's DeepGEMM, a clean and efficient open-source BLAS kernel library for NVIDIA GPUs (up to 1550 TFLOPS on…infra · JEV 0.77OpenViking — self-evolving context database for AI agentsByteDance Volcengine's open-source (AGPL-3.0) context database that unifies agent memory, knowledge RAG, and skills…infra · JEV 0.76Modular open-sources MAX inference server, Mojo stdlib and accelerator kernelsThe core of Modular's unified AI deployment stack is now open: the MAX inference server (OpenAI-compatible endpoints)…infra · JEV 0.76OmniRouteFree MIT AI gateway: one endpoint, 290+ providers and 500+ models with quota-aware auto-fallback.infra · JEV 0.75Context Mode: context-window optimization for AI coding agentsMCP server for AI coding agents that sandboxes verbose tool output and persists session memory, cutting context use by…infra · JEV 0.74StrataOpen-source inference engine that runs the 125B-parameter Qwen3.8-Flash-Next MoE on a single consumer GPU with 12GB+…infra · JEV 0.74
Get the week's best AI launches, plus 3 ideas worth building
One email every Saturday. Ranked by traction, not hype. Free.