Radar / AI infrastructure / Prefixwise

Prefixwise: name why each LLM request missed the prompt cache

An open-source Python proxy and CLI that sits between an agent and the model API and names the cause of every prompt-cache miss - a timestamp in the system prompt, a reordered tool list, an expired TTL - prices it in tokens, and tells you the fix.

new launchinfra · routerJEV traction 0.3tejachavali/prefixwise · 27 ★added 2026-11-13
Open Prefixwise →View on the radar

Why it matters

Prompt caching can cut LLM bills by up to 90%, but providers only report how many tokens missed, never why; Prefixwise names 19 possible causes per request and reports tokens per successful task.

What you could build with it

A small team running a fleet of coding agents could put Prefixwise in front of its model API for a week, find which prompt patterns waste the most cached tokens, and cut its monthly LLM bill without changing any agent behaviour.

Does it hold up?

Too early to judge - 22 stars and two days old, with fixes verified only on scripted sessions and small models, not weeks of live traffic; the live measurements against real models are published with run records, which is a good sign.

Built with Prefixwise

Learn more

official docs →

First spotted on github: source.

More AI infrastructure

vLLM v0.34.0: extensible KV cache by default, Model Runner V2 only, and 10 new modelsThe open inference engine's latest release spans 1,061 commits from 412 contributors, adds 10 models including Mistral…infra · JEV 0.86llama.cpp v1.0.0: the local-AI engine's first stable releasellama.cpp and ggml hit v1.0.0, promising semantic versioning so breaking C API changes only arrive with major bumps.infra · JEV 0.84colibri: run 744B-parameter MoE models on consumer hardwarePure-C inference engine with zero dependencies that streams MoE experts from NVMe disk, keeping hot experts on GPU…infra · JEV 0.78claude-mem: persistent memory plugin that lets coding agents pick up where the last session left offPlugin that hooks five Claude Code lifecycle events, compresses session activity into a local SQLite + Chroma store…infra · JEV 0.74FreeToken: datacenter-scale MoE serving on desktop GPUsAn edge-native mixture-of-experts serving engine from UC Berkeley researchers that runs 290B+ parameter MoE models on…infra · JEV 0.74GBrain: Garry Tan's self-wiring memory layer for AI agentsY Combinator CEO Garry Tan's open-source, markdown-first knowledge layer that gives OpenClaw and Hermes agents…infra · JEV 0.71

Get the week's best AI launches, plus 3 ideas worth building

One email every Saturday. Ranked by traction, not hype. Free.