GDPx: SemiAnalysis's 206-task agentic benchmark where the best model scores just 12%
SemiAnalysis released GDPx v1.0 on Nov 2: 206 long tasks across 11 professional domains, built with 14 specialist data partners. Claude Opus 5.5 leads at 12.0%, ahead of Gemini 4 Carbon at 10.9% and GPT-6 Astra at 9.6%; every model scored zero on finance and legal under binary grading.
Why it matters
It tests long, realistic expert work with 8-hour tasks in each model's native harness, replacing saturated short benchmarks and showing how far agents still are from professional-grade autonomy.
What you could build with it
An engineering leadership team could adopt GDPx-style long, domain-specific tasks as its internal agent acceptance test, running candidate coding agents against realistic multi-hour jobs from its own backlog instead of short benchmarks, and only rolling out agents that clear a set bar on the team's hardest task types.
Does it hold up?
Too early to judge as a purchasing yardstick: GDPx is a paid SemiAnalysis product whose partner-built tasks have not been independently audited yet, but its public per-domain leaderboard already gives teams a realistic way to compare agents on expert work.
Built with GDPx
- The Best AI Agent Scores Just 12% on a Tough New Test of Expert Worksubstack · Charts SemiAnalysis's published GDPx v1.0 scores and notes the results have not been independently re-run.
- GDPx Benchmark: Opus 5.5 Tops at Only 12%explainx · Walks through the 206 tasks, 11 domains, scoring method, and caveats including the 98% cyber-alignment rule-break rate.
- SemiAnalysis launches GDPx: the best AI agent completes just 12% of its high-value tasksdealroom · Reports per-model domain strengths, the $16.29 vs $51.41 per-task cost gap, and that the results are SemiAnalysis's own runs, not an independent test.
Learn more
First spotted on article: source.
More AI agents
CowAgent: open-source super AI assistant and agent harness with memory that compoundsAn open-source personal AI assistant and agent harness by zhayujie, back on Hacker News' front page this week: it…agents · JEV 0.78Agent Desktop: Vercel Labs' CLI that gives agents reliable desktop control via the accessibility treeVercel Labs' Rust CLI and C library lets AI agents read and operate desktop apps through OS accessibility trees: a…agents · JEV 0.76Open Notebook: self-hosted, multi-model NotebookLM alternative trending againOpen-source research notebook that turns documents, web pages, audio and video into searchable notebooks with chat…agents · JEV 0.75OpenBot: self-hosted AI coworkers, each with its own computer and audit trailCopilotKit's MIT-licensed alpha gives each AI coworker its own container with a browser, files and logins, behind a…agents · JEV 0.73DeepTutor: open-source, agent-native personalized tutor trending againHKUDS/DeepTutor, the University of Hong Kong's open-source personalized learning assistant, is trending on GitHub…agents · JEV 0.72claude-seo: universal SEO skill for Claude Code, trending on GitHubAgrici Daniel's MIT-licensed Claude Code plugin bundles 26 sub-skills and 19 specialist agents that run full SEO…agents · JEV 0.72
Get the week's best AI launches, plus 3 ideas worth building
One email every Saturday. Ranked by traction, not hype. Free.