Radar / AI infrastructure / Prefixwise
Prefixwise: name why each LLM request missed the prompt cache
An open-source Python proxy and CLI that sits between an agent and the model API and names the cause of every prompt-cache miss - a timestamp in the system prompt, a reordered tool list, an expired TTL - prices it in tokens, and tells you the fix.
Why it matters
Prompt caching can cut LLM bills by up to 90%, but providers only report how many tokens missed, never why; Prefixwise names 19 possible causes per request and reports tokens per successful task.
What you could build with it
A small team running a fleet of coding agents could put Prefixwise in front of its model API for a week, find which prompt patterns waste the most cached tokens, and cut its monthly LLM bill without changing any agent behaviour.
Does it hold up?
Too early to judge - 22 stars and two days old, with fixes verified only on scripted sessions and small models, not weeks of live traffic; the live measurements against real models are published with run records, which is a good sign.
Built with Prefixwise
- Prefixwise - names why each LLM request missed the prompt cacherubric.hrih.si · Three-judge review of the Show HN launch, rating it highest on new information and technical depth.
- Prefixwise Tool Explains Why LLM Requests Miss Prompt Cacheemergent.sh · News summary of the November 11 launch and what the tool diagnoses.
- Why did my vLLM / SGLang prefix cache miss?vibingtalk.com · Full breakdown of the tool's 19 miss causes, live-model measurements, and CI checks.
Learn more
First spotted on github: source.
More AI infrastructure
vLLM v0.34.0: extensible KV cache by default, Model Runner V2 only, and 10 new modelsThe open inference engine's latest release spans 1,061 commits from 412 contributors, adds 10 models including Mistral…infra · JEV 0.86llama.cpp v1.0.0: the local-AI engine's first stable releasellama.cpp and ggml hit v1.0.0, promising semantic versioning so breaking C API changes only arrive with major bumps.infra · JEV 0.84colibri: run 744B-parameter MoE models on consumer hardwarePure-C inference engine with zero dependencies that streams MoE experts from NVMe disk, keeping hot experts on GPU…infra · JEV 0.78claude-mem: persistent memory plugin that lets coding agents pick up where the last session left offPlugin that hooks five Claude Code lifecycle events, compresses session activity into a local SQLite + Chroma store…infra · JEV 0.74FreeToken: datacenter-scale MoE serving on desktop GPUsAn edge-native mixture-of-experts serving engine from UC Berkeley researchers that runs 290B+ parameter MoE models on…infra · JEV 0.74GBrain: Garry Tan's self-wiring memory layer for AI agentsY Combinator CEO Garry Tan's open-source, markdown-first knowledge layer that gives OpenClaw and Hermes agents…infra · JEV 0.71
Get the week's best AI launches, plus 3 ideas worth building
One email every Saturday. Ranked by traction, not hype. Free.