FDE-Gym: the online judge for the LLM era
Agent evaluation platform where each case is a real customer delivery: a coding agent works a live system and is scored on the customer's own metric once it goes live, not on hidden tests.
Why it matters
Reframes agent eval from code correctness to delivery outcomes: a live site (fde-gym.com), HF dataset, Docker self-hosting, and an arXiv paper, launched together.
What you could build with it
A startup hiring AI-forward engineers could use FDE Gym cases as a practical interview loop, grading candidates on how they direct agents to deliver a fix to a live customer system.
Does it hold up?
Too early to judge: the platform, cases, dataset and paper launched together, but no independent agent scores have been published yet. (Repo opened 2026-10-06; paper/site launch 2026-10-16.)
Learn more
First spotted on github: source.
More AI agents
Agent ReachOpen-source CLI and SKILL.md kit that gives AI agents read/search access to 15+ internet platforms (Twitter, Reddit…agents · JEV 0.8AI Job Search — the job hunt that runs on your machineOpen-source Claude Code framework that turns job hunting into an agentic pipeline: scrape portals, score fit, tailor…agents · JEV 0.79PentAGI — fully autonomous AI agents for penetration testingPentAGI is a self-hosted, open-source system where a team of AI agents plans and executes authorized penetration tests…agents · JEV 0.77PPT Master: AI skill that generates native, editable PowerPoint decks from any documentOpen-source AI agent workflow that turns documents or topics into real, native PowerPoint decks: native shapes…agents · JEV 0.76Docker Agent: AI agent builder and runtime by Docker EngineeringA Docker CLI plugin for building, running and sharing AI agents with declarative YAML config, multi-agent…agents · JEV 0.75K-Dense-AI/scientific-agent-skills: turn any AI agent into an AI scientistK-Dense AI's open collection of 177 validated, lazily-loaded skills that turn any agent (Cursor, Claude Code, Codex…agents · JEV 0.74
Get the week's best AI launches, plus 3 ideas worth building
One email every Saturday. Ranked by traction, not hype. Free.