Radar / AI infrastructure / inferd

inferd: a Mac inference engine built for 64 to 128 agents at once

Incept5's Apache 2.0 inference engine for Apple Silicon, written from scratch in C++/Metal, that trades single-stream speed for fleet throughput: 849 combined tokens/s from 64 agents running Qwen3.8-27B on one Mac Studio.

trendinginfra · routerJEV traction 0.37Incept5-AI/inferd · 35 ★added 2026-11-14
Open inferd →View on the radar

Why it matters

First Mac engine designed around the agent-fleet case — agents arriving in crowds, sharing context, with per-agent budgets — instead of optimizing for one prompt at a time.

What you could build with it

A small team could run a fleet of coding agents on a single Mac Studio in the office, each agent with its own context, sharing one warm model instead of paying per-token cloud API calls for every agent.

Does it hold up?

Too early to judge - v0.4.1 runs on Mac Studio hardware with published numbers, but only 34 stars, no independent reproduction beyond one user's issue report, and the authors admit other Mac engines are faster for a single stream.

Built with inferd

Learn more

technical deep dive →

First spotted on github: source.

More AI infrastructure

vLLM v0.34.0: extensible KV cache by default, Model Runner V2 only, and 10 new modelsThe open inference engine's latest release spans 1,061 commits from 412 contributors, adds 10 models including Mistral…infra · JEV 0.86llama.cpp v1.0.0: the local-AI engine's first stable releasellama.cpp and ggml hit v1.0.0, promising semantic versioning so breaking C API changes only arrive with major bumps.infra · JEV 0.84colibri: run 744B-parameter MoE models on consumer hardwarePure-C inference engine with zero dependencies that streams MoE experts from NVMe disk, keeping hot experts on GPU…infra · JEV 0.78claude-mem: persistent memory plugin that lets coding agents pick up where the last session left offPlugin that hooks five Claude Code lifecycle events, compresses session activity into a local SQLite + Chroma store…infra · JEV 0.74FreeToken: datacenter-scale MoE serving on desktop GPUsAn edge-native mixture-of-experts serving engine from UC Berkeley researchers that runs 290B+ parameter MoE models on…infra · JEV 0.74GBrain: Garry Tan's self-wiring memory layer for AI agentsY Combinator CEO Garry Tan's open-source, markdown-first knowledge layer that gives OpenClaw and Hermes agents…infra · JEV 0.71

Get the week's best AI launches, plus 3 ideas worth building

One email every Saturday. Ranked by traction, not hype. Free.