SkillEvaluator: NVIDIA's multi-tier framework for evaluating AI agent skills
NVIDIA's open framework grades AI agent skills and plugins with quality gates, semantic overlap detection, synthetic evaluation datasets, and live agent evaluation that measures how skills change agent behavior.
Why it matters
As agent skills proliferate, nobody was measuring whether a skill actually helps: NVIDIA's framework scores the skill itself, including whether it overlaps or conflicts with others.
What you could build with it
A team shipping a library of agent skills could run SkillEvaluator's quality gates in CI, so a new skill that overlaps an existing one or degrades agent behavior fails the build before it reaches production.
Does it hold up?
Too early to judge: the framework is documented and open, but public evidence of teams using its live agent evaluation in production is thin so far.
Built with SkillEvaluator
- SkillEvaluator surfaces as a trending daily pick for agent-skill evaluationgitdiscover · GitDiscover's Nov 17 daily picks flagged SkillEvaluator for its approach to evaluating agent skills and plugins with quality gates and live agent evaluation.
Learn more
First spotted on github: source.
More AI agents
Agent Browser: Vercel Labs' browser automation CLI for AI agentsVercel Labs' open-source CLI that lets AI agents drive a real browser: click, fill forms, and complete web tasks, with…agents · JEV 0.78CowAgent: open-source super AI assistant and agent harness with memory that compoundsAn open-source personal AI assistant and agent harness by zhayujie, back on Hacker News' front page this week: it…agents · JEV 0.78google/skills: Google's own agent skills for its products and Cloudgoogle/skills is Google's official collection of agent skills for Google products and Google Cloud, giving coding…agents · JEV 0.78HorizonSelf-hosted AI news radar in Python: fetches RSS, Hacker News, Reddit, Telegram, X and GitHub concurrently, AI-scores…agents · JEV 0.78anthropics/skills: Anthropic's official collection of 19 Agent SkillsAnthropic's official, curated collection of 19 Agent Skills covering design, documents, coding, and enterprise…agents · JEV 0.77Orca: run a fleet of coding agents side by sideStably AI's open-source desktop orchestrator fans one prompt across 5-30 CLI coding agents, each in its own git…agents · JEV 0.77
Get the week's best AI launches, plus 3 ideas worth building
One email every Saturday. Ranked by traction, not hype. Free.