Radar / AI infrastructure / Flide
Flide: decision-model inference up to 9.5x faster than vLLM
NUS and Oxford researchers built an exact inference engine for Jev-style decision models that flattens a decision tree into one token stream with ForkAttention.
Why it matters
Makes categorical decision models up to 9.5x faster than vLLM and SGLang while keeping every branch exactly as accurate as an isolated forward pass.
What you could build with it
A team running an agent fleet could serve its decision models through Flide so every routing, ranking or approval call answers in milliseconds instead of waiting on a text-generation engine.
Does it hold up?
Too early to judge for production: pip-installable (flide 0.1.10) and serving OneJev, Open-Jev and Perplexity's decider, but the 9.5x speedup is a self-reported best case with no independent reproduction yet, and the repo has only 125 stars.
Built with Flide
- Flide runs AI decision models up to 9.5x faster than vLLMarticle · Plain-English explainer of ForkAttention and the exactness guarantee, with the caveat that the 9.5x figure is a best case and unreproduced.
- Flide project pagearticle · Official project page with the model table, quick start, and per-workload speedup numbers on an H200.
- OmniJev/OneJevgithub · 252 ★ · Multimodal System One decision model served by Flide; the paper reports about 6.8x faster answers on it versus its previous engine.
Learn more
First spotted on github: source.
More AI infrastructure
colibri: run 744B-parameter MoE models on consumer hardwarePure-C inference engine with zero dependencies that streams MoE experts from NVMe disk, keeping hot experts on GPU…infra · JEV 0.78claude-mem: persistent memory plugin that lets coding agents pick up where the last session left offPlugin that hooks five Claude Code lifecycle events, compresses session activity into a local SQLite + Chroma store…infra · JEV 0.74FreeToken: datacenter-scale MoE serving on desktop GPUsAn edge-native mixture-of-experts serving engine from UC Berkeley researchers that runs 290B+ parameter MoE models on…infra · JEV 0.74XLA Serving Engine: Google open-sources its C++ inference runtimeGoogle's general-purpose model export and inference runtime for the OpenXLA ecosystem, now open source under Apache 2.0.infra · JEV 0.67TorchTPU: Google's native PyTorch stack for TPUs, now open sourceAnnounced at PyTorch Conference 2026 on October 20: Google open-sourced TorchTPU, its native PyTorch backend for Cloud…infra · JEV 0.66billion-context: month-long agent sessions in a 100K context windowA local proxy that sits between a coding agent and its model API, letting the model fold old messages into searchable…infra · JEV 0.65
Get the week's best AI launches, plus 3 ideas worth building
One email every Saturday. Ranked by traction, not hype. Free.