Radar / AI infrastructure / EAGLE-4
EAGLE-4: one flexible speculative-decoding drafter for all batch sizes and hardware
Eagle's newest speculative-decoding drafter adapts its draft tree to batch size, hardware, and workload, speeding up Qwen3.8-27B inference up to 3.4x in SGLang on 4x B300 GPUs.
Why it matters
A single pip-installable eagle-harness package that plugs into SGLang, vLLM, and TokenSpeed without engine patches, with day-one AMD MI355X support validated by AMD itself.
What you could build with it
An indie developer or small team serving open-weight models could drop the eagle-harness package into an existing SGLang or vLLM deployment to cut inference costs on Qwen3.8-27B-class models — the drafter adapts its speculation tree to batch size and hardware automatically, so a 4-GPU serving bill can shrink to a fraction of the GPUs for the same throughput.
Does it hold up?
Too early to judge for production use: the repo launched Oct 20 with 653 stars and the speed figures are the project's own (unverified), though AMD's independent MI355X measurements lend credibility. Note the code is source-available under Eagle's commercial license, not OSI open source.
Built with EAGLE-4
- AMD validates EAGLE-4 on Instinct MI355X from day onearticle · AMD's own technical article measured EAGLE-4 on MI355X: 3.5x plain decoding and 1.6x built-in MTP for Qwen3.8-27B in SGLang, with tree verification added for Qwen3.8's linear-attention layers.
- Eagle-4 is faster than DFlash 2 and DSpark across batch sizesarticle · Independent writeup of the v0.1.0 release: 2.99x plain decoding with vLLM on 4x B300 at concurrency 1, and 28,340 aggregate tokens/s at 512 concurrent requests vs 16,641 for DFlash 2.
- AMD confirms EAGLE-4 runs on Instinct MI355X GPUs from day onearticle · Coverage of AMD's day-one support announcement: same checkpoint and launch flags as NVIDIA hardware, with EAGLE-Harness cost tables measured on the MI355X GPU.
Learn more
First spotted on github: source.
More AI infrastructure
DeepGEMM: DeepSeek's open GPU BLAS kernel libraryDeepSeek's DeepGEMM, a clean and efficient open-source BLAS kernel library for NVIDIA GPUs (up to 1550 TFLOPS on…infra · JEV 0.77OpenViking — self-evolving context database for AI agentsByteDance Volcengine's open-source (AGPL-3.0) context database that unifies agent memory, knowledge RAG, and skills…infra · JEV 0.76Modular open-sources MAX inference server, Mojo stdlib and accelerator kernelsThe core of Modular's unified AI deployment stack is now open: the MAX inference server (OpenAI-compatible endpoints)…infra · JEV 0.76ai-memory - git-backed long-term memory shared across coding agentsRust server and CLI that captures agent sessions via lifecycle hooks, compiles them into a git-backed markdown wiki…infra · JEV 0.76OmniRouteFree MIT AI gateway: one endpoint, 290+ providers and 500+ models with quota-aware auto-fallback.infra · JEV 0.75Context Mode: context-window optimization for AI coding agentsMCP server for AI coding agents that sandboxes verbose tool output and persists session memory, cutting context use by…infra · JEV 0.74
Get the week's best AI launches, plus 3 ideas worth building
One email every Saturday. Ranked by traction, not hype. Free.