Modular open-sources MAX inference server, Mojo stdlib and accelerator kernels

The core of Modular's unified AI deployment stack is now open: the MAX inference server (OpenAI-compatible endpoints), the Mojo standard library, and the MAX accelerator kernels, all in the modular/modular repo.

Why it matters

What you could build with it

Does it hold up?

Real and buildable today: the monorepo is huge but genuinely builds, and community forks already extend it with custom GPU kernels and local MAX inference paths. Mojo toolchain edges (contributions to the compiler not yet accepted) are the main friction.

Built with Modular open-sources MAX inference server…

• Oss-Atlas adoption analysis of the Modular stackgithub · Independent critical evaluation of betting on MAX + Mojo: when the vertically-integrated stack wins, and when vLLM/TGI/TensorRT-LLM are safer choices.
• h4d3zs/vscodium-rustgithub · 255 ★ · Maintains a maintained fork of modular/modular (checked out at ~1.2GB) with custom kortex retrieval kernels and an exploratory MAX inference path serving local models via MAX's OpenAI-compatible server and Mojo GPU kernels.
• wendylabsinc/templatesgithub · Hand-ported YOLOv8n to the MAX graph API (13.5 ms/frame on arm64 CPU) and logged every missing op/feature -- a real-world stress test of serving robotics perception models on the newly open MAX stack.

Learn more

official docs →

First spotted on engineering blog: source.

More AI infrastructure