Radar / AI infrastructure / ModelSphere
ModelSphere — open-source production-grade AI deployment platform
One-stop, Apache-2.0-licensed platform for deploying and operating LLMs in production, with unified mixed scheduling across heterogeneous chips (Huawei Ascend, Iluvatar CoreX, Moore Threads, Hygon DCU) so teams skip per-chip low-level rewrites.
Why it matters
Operates at cluster-architecture level on top of vLLM/SGLang: load- and cache-aware request routing, unified KV cache pool, QoS-based autoscaling, automatic stream resumption after failures; claims up to 285% peak-throughput uplift.
What you could build with it
A SaaS company serving models across mixed GPU inventories could use ModelSphere to unify deployment across vendors' chips, routing inference through one QoS-aware control plane instead of maintaining separate deployment stacks per accelerator.
Does it hold up?
Too early to judge — press-release claims only (285% throughput) with no independent production deployments yet; the Apache 2.0 code drop is what to watch.
Learn more
First spotted on article: source.
More AI infrastructure
DeepGEMM: DeepSeek's open GPU BLAS kernel libraryDeepSeek's DeepGEMM, a clean and efficient open-source BLAS kernel library for NVIDIA GPUs (up to 1550 TFLOPS on…infra · JEV 0.77OpenViking — self-evolving context database for AI agentsByteDance Volcengine's open-source (AGPL-3.0) context database that unifies agent memory, knowledge RAG, and skills…infra · JEV 0.76Modular open-sources MAX inference server, Mojo stdlib and accelerator kernelsThe core of Modular's unified AI deployment stack is now open: the MAX inference server (OpenAI-compatible endpoints)…infra · JEV 0.76OmniRouteFree MIT AI gateway: one endpoint, 290+ providers and 500+ models with quota-aware auto-fallback.infra · JEV 0.75Context Mode: context-window optimization for AI coding agentsMCP server for AI coding agents that sandboxes verbose tool output and persists session memory, cutting context use by…infra · JEV 0.74StrataOpen-source inference engine that runs the 125B-parameter Qwen3.8-Flash-Next MoE on a single consumer GPU with 12GB+…infra · JEV 0.74
Get the week's best AI launches, plus 3 ideas worth building
One email every Saturday. Ranked by traction, not hype. Free.