Radar / AI infrastructure / inferd
inferd: a Mac inference engine built for 64 to 128 agents at once
Incept5's Apache 2.0 inference engine for Apple Silicon, written from scratch in C++/Metal, that trades single-stream speed for fleet throughput: 849 combined tokens/s from 64 agents running Qwen3.8-27B on one Mac Studio.
Why it matters
First Mac engine designed around the agent-fleet case — agents arriving in crowds, sharing context, with per-agent budgets — instead of optimizing for one prompt at a time.
What you could build with it
A small team could run a fleet of coding agents on a single Mac Studio in the office, each agent with its own context, sharing one warm model instead of paying per-token cloud API calls for every agent.
Does it hold up?
Too early to judge - v0.4.1 runs on Mac Studio hardware with published numbers, but only 34 stars, no independent reproduction beyond one user's issue report, and the authors admit other Mac engines are faster for a single stream.
Built with inferd
- 64 Agents, One Mac Studio: How inferd Trades Single-Stream Speed for Fleet Throughputmedium · Independent walkthrough of the v0.4.1 numbers, the shared-context design, and where inferd loses to single-stream Mac engines.
- Incept5 AI Releases: inferdthursdai.news · Launched live on ThursdAI on 5 Nov 2026 by Incept5 CTO John Davies; concurrency went from 64 to 128 agents on the day of the show.
- External result: Mac Studio M5 Max 128 GB, v0.4.1 prebuiltgithub · An independent user measured the v0.4.1 prebuilt on a Mac Studio M5 Max with 128 GB and published the full harness results.
Learn more
First spotted on github: source.
More AI infrastructure
vLLM v0.34.0: extensible KV cache by default, Model Runner V2 only, and 10 new modelsThe open inference engine's latest release spans 1,061 commits from 412 contributors, adds 10 models including Mistral…infra · JEV 0.86llama.cpp v1.0.0: the local-AI engine's first stable releasellama.cpp and ggml hit v1.0.0, promising semantic versioning so breaking C API changes only arrive with major bumps.infra · JEV 0.84colibri: run 744B-parameter MoE models on consumer hardwarePure-C inference engine with zero dependencies that streams MoE experts from NVMe disk, keeping hot experts on GPU…infra · JEV 0.78claude-mem: persistent memory plugin that lets coding agents pick up where the last session left offPlugin that hooks five Claude Code lifecycle events, compresses session activity into a local SQLite + Chroma store…infra · JEV 0.74FreeToken: datacenter-scale MoE serving on desktop GPUsAn edge-native mixture-of-experts serving engine from UC Berkeley researchers that runs 290B+ parameter MoE models on…infra · JEV 0.74GBrain: Garry Tan's self-wiring memory layer for AI agentsY Combinator CEO Garry Tan's open-source, markdown-first knowledge layer that gives OpenClaw and Hermes agents…infra · JEV 0.71
Get the week's best AI launches, plus 3 ideas worth building
One email every Saturday. Ranked by traction, not hype. Free.