Radar / Categories / AI infrastructure

New AI infrastructure, ranked by traction

New routers, memory layers, tokenizers and inference infrastructure. Every launch from the last few weeks, ranked by JEV traction and GitHub stars. Updated four times a day.

Router · 26

vLLM v0.34.0: extensible KV cache by default, Model Runner V2 only, and 10 new modelsThe open inference engine's latest release spans 1,061 commits from 412 contributors, adds 10 models including Mistral…infra · JEV 0.86llama.cpp v1.0.0: the local-AI engine's first stable releasellama.cpp and ggml hit v1.0.0, promising semantic versioning so breaking C API changes only arrive with major bumps.infra · JEV 0.84colibri: run 744B-parameter MoE models on consumer hardwarePure-C inference engine with zero dependencies that streams MoE experts from NVMe disk, keeping hot experts on GPU…infra · JEV 0.78FreeToken: datacenter-scale MoE serving on desktop GPUsAn edge-native mixture-of-experts serving engine from UC Berkeley researchers that runs 290B+ parameter MoE models on…infra · JEV 0.74AWS Emulator: a local copy of AWS, written entirely by AI agentsAWS's free, open-source local emulator covering 200+ services from 438 API models, written in Rust, running offline on…infra · JEV 0.67DeepSeek App API: the whole consumer app, now as developer APIsDeepSeek opened its app's capabilities through its API: web search, file parsing, URL reading, safety checks, voice…infra · JEV 0.67XLA Serving Engine: Google open-sources its C++ inference runtimeGoogle's general-purpose model export and inference runtime for the OpenXLA ecosystem, now open source under Apache 2.0.infra · JEV 0.67TorchTPU: Google's native PyTorch stack for TPUs, now open sourceAnnounced at PyTorch Conference 2026 on October 20: Google open-sourced TorchTPU, its native PyTorch backend for Cloud…infra · JEV 0.66Mesh-LLM: pool GPUs across machines behind one OpenAI-compatible endpointRust project that splits large models across machines into layer stages, fetches model shards by package, and serves…infra · JEV 0.64NVKE: NVIDIA's open-source management plane for GPU Kubernetes on bare metalNVIDIA open-sourced NVKE, a preview management plane that runs isolated Kubernetes clusters with their own control…infra · JEV 0.63Google TPU 8tx and TPU 8i Max: confidential inference and double-chip long contextGoogle's eighth-gen TPU family grows to four systems in one year: 8tx for confidential inference, 8i Max for massive…infra · JEV 0.61DeepInfer: DeepSeek's open-source production inference engineDeepSeek open-sourced DeepInfer, the high-performance inference framework powering its production services (155…infra · JEV 0.6Floci: a free local AWS emulator that starts in 24 msFree, open-source, MIT-licensed local AWS emulator that runs 100+ services on localhost:4566 with no account, no auth…infra · JEV 0.54Together Compiler: AI agents that continuously retune inferenceTogether AI launched Together Compiler, a system in which teams of agents continually rewrite and measure GPU code for…infra · JEV 0.51OpenRouter Auto Router: next-gen model routing with a dedicated coding routerOpenRouter's rebuilt Auto Router picks the model and reasoning effort per request, alongside a new Auto Code router…infra · JEV 0.51Render Sandboxes: a computer for every agent, in public betaRender launched Sandboxes alongside Workflows and Shared Filesystems, giving every AI agent its own isolated Linux…infra · JEV 0.47Sarvam Token Factory: frontier open-weight models on Indian GPUsSarvam opens Token Factory to everyone: eight open-weight models - Sarvam 105B, GLM 5.3, DeepSeek V4.1 Flash, Kimi…infra · JEV 0.46Flide: decision-model inference up to 9.5x faster than vLLMNUS and Oxford researchers built an exact inference engine for Jev-style decision models that flattens a decision tree…infra · JEV 0.41inferd: a Mac inference engine built for 64 to 128 agents at onceIncept5's Apache 2.0 inference engine for Apple Silicon, written from scratch in C++/Metal, that trades single-stream…infra · JEV 0.37GPUWorks: a 40MB open-source platform to manage a thousand GPU serversZStack open-sourced GPUWorks under Apache 2.0, a 40MB+ platform that deploys in about 60 seconds and manages GPU…infra · JEV 0.35Rival.io GraphX: run DeepSeek V4 Flash on eight consumer-grade NVIDIA L4 GPUsProprietary kernel-optimization and silicon-placement technology that ran DeepSeek V4 Flash on eight NVIDIA L4 GPUs…infra · JEV 0.35You.com Inference RouterOne OpenAI-compatible API for 12 open-weight models with a 99.9% uptime SLA, zero data retention, and built-in…infra · JEV 0.31Prefixwise: name why each LLM request missed the prompt cacheAn open-source Python proxy and CLI that sits between an agent and the model API and names the cause of every…infra · JEV 0.3EAGLE-4: one flexible speculative-decoding drafter for all batch sizes and hardwareEagle's newest speculative-decoding drafter adapts its draft tree to batch size, hardware, and workload, speeding up…infra · JEV 0.28

Memory · 9

claude-mem: persistent memory plugin that lets coding agents pick up where the last session left offPlugin that hooks five Claude Code lifecycle events, compresses session activity into a local SQLite + Chroma store…infra · JEV 0.74GBrain: Garry Tan's self-wiring memory layer for AI agentsY Combinator CEO Garry Tan's open-source, markdown-first knowledge layer that gives OpenClaw and Hermes agents…infra · JEV 0.71billion-context: month-long agent sessions in a 100K context windowA local proxy that sits between a coding agent and its model API, letting the model fold old messages into searchable…infra · JEV 0.65Atuin Threads: all your coding-agent sessions, searchable and synced across machinesAtuin 18.24 adds Threads, a searchable, end-to-end-encrypted history for coding-agent sessions that follows a…infra · JEV 0.64Message Threads: Anthropic's public beta that stops agents resending the whole conversation every turnMessage Threads lets Messages API apps send only new messages after an initial full-conversation request, cutting…infra · JEV 0.57Memorilla: latent semantic memory for LLMsStanford and Arc Institute memory system that compresses an entire document collection into 16 query-dependent memory…infra · JEV 0.53Beacon: open-source, self-improving memory for AI coding agentsAsymptote Labs launched Beacon, an open-source, cross-harness memory layer that lets AI coding agents remember project…infra · JEV 0.38Hyper — a shared company brain for AI agentsYC-backed shared memory layer that plugs into a company's tools and conversations so AI agents and automations act on…infra · JEV 0.29NVIDIA Auto OntologyApache-2.0 framework that builds a semantic, graph-based layer over enterprise databases from metadata, then serves…infra · JEV 0.18

Get the week's best AI launches, plus 3 ideas worth building

One email every Saturday. Ranked by traction, not hype. Free.