Radar / Ideas / CallRehearse: voice agents that pass a…

CallRehearse: voice agents that pass a phone audition before they take real calls

daily ideamoderateJEV confidence 0.68JEV monetization 0.62026-10-29

OutcomeA voice agent ships only after passing a scored phone audition: zero unhandled-failure calls in its first week live.

The problem

Voice agents fail in ways text evals cannot see. A caller with a strong regional accent gets misheard, an interruption derails the booking, background noise breaks the transcript, and the failure surfaces as an angry customer, not a failed test. Teams building multilingual agents feel this worst: a Hinglish QA checklist circulating this month insists on native code-mixers, regional speakers and noisy-setting testers precisely because builders keep discovering these failures after launch.

The idea

A self-hosted audition kit for voice agents. A team writes scenarios (book, reschedule, complain, pay, escalate) and caller profiles (accent, language, pace, interruption style, background noise), then the kit places real calls against the agent over WebRTC or telephony, transcribes both channels with speaker attribution, and scores every call on outcome criteria: task resolved, policy held, escalation correct, latency acceptable. Failures become regression tests automatically, and a CI gate blocks deploys while the suite is red. The kit ships with an Indian-language caller matrix first: Hindi, Hinglish, Tamil and other Indian English varieties, regional accents, and noisy environments like streets and kitchens, because that is where imported QA matrices fail hardest.

Why now

Voice agents just got cheap to build: Vaani's self-serve platform launched October 29, speech-to-speech models like ElevenLabs Realtime are production-ready, and open transcription with speaker attribution landed days earlier. The bottleneck has moved from building the agent to proving it works on a real phone line, and the QA category is still dominated by closed, per-minute SaaS while Indian-language coverage remains an afterthought.

What it combines

Three capabilities combine: Vaani-style self-serve voice platforms that let anyone deploy a voice agent in an afternoon, Nemotron Transcribe's speaker-attributed transcription that turns a messy call into two clean channels of evidence, and a small decision model that scores each call against outcome criteria. The mix matters because voice agents fail in audio, not in text: accents, interruptions, background noise and multi-turn tool calls are invisible to transcript-only evals, and only the combination of a real call, a real transcript and a scored verdict catches them before customers do.

MVP

Weekend scope: a CLI that places calls over WebRTC to one voice agent, runs 10 scripted scenarios across 3 caller profiles, transcribes both channels, scores pass/fail on 5 outcome criteria, and prints a report. Deliberately skipped: PSTN telephony, a hosted dashboard, production-call monitoring, and auto-generated scenarios.

How it makes money

Budget owner: the Head of Contact Center Operations or QA lead who already owns voice-agent quality and pays for human QA reviewers. Pricing anchor: Cekura's published voice-testing rate of $0.25 per tested minute and $0.05 per monitored call, cited in Hamming's build-vs-buy analysis (https://hamming.ai/resources/build-vs-buy-voice-agent-testing). Willingness-to-pay evidence: Hamming AI raised a $3.8M seed round for voice-agent QA (https://runtimewire.com/article/hamming-ai-founder-sumanyu-sharma-voice-agent-qa), and TestMu AI was named a Leader in Gartner's 2026 Magic Quadrant for Agentic Software Quality Assurance Platforms (https://www.testmuai.com/blog/ai-voice-agent-testing-tools/), showing enterprises already fund this budget line. Replacement budget: it replaces per-minute SaaS QA fees plus human QA hours, and runs on the team's own infrastructure at near-zero marginal cost per call.

The moat

The structural advantage is data plus deployment shape. Incumbent voice-QA vendors are closed SaaS products that meter per-minute usage; they cannot open-source their harness without destroying their pricing model, and they cannot run inside a bank or telecom that refuses to send call audio to a third-party cloud. An open, self-hosted kit with a growing corpus of caller profiles and failure regressions is adoptable exactly where the incumbents are not allowed in, and that corpus compounds with every team that contributes scenarios.

Distribution

Sold direct to customer-experience and QA leads at firms running voice agents, with a B2B2C wedge through voice-agent platforms and telephony providers that embed the audition kit as a pre-launch check. First wedge: Indian contact centers and voice-agent studios shipping multilingual agents, where US-built caller matrices are thinnest.

Why it wins

Hamming, Evalgent and Audient are closed SaaS products with US-centered caller matrices, and Retell's QA only covers agents built on Retell. CallRehearse is self-hosted and open, works with any voice stack, and leads with an Indian-language caller matrix that the incumbents treat as an afterthought.

Risks

Biggest risk: synthetic callers diverge from real callers, so the gate certifies agents that still fail in production. The MVP de-risks this by calibrating scores against 50 real recorded calls before any gate is allowed to block a deploy.

Build it with

Vaani AI Research: self-serve platform to build and deploy voice AI agentsThe agent under test is built and deployed on a self-serve voice platform like this one, so the audition kit plugs into the same stack teams already use.
NVIDIA Nemotron 3.5 Transcribe: open-weight speech model that transcribes, translates and attributes speakers in one passSpeaker-attributed transcription of both call channels turns every audition call into structured evidence: who said what, when, and whether the agent interrupted or talked over the caller.
ElevenLabs Eleven Realtime: speech-to-speech models for voice agentsSpeech-to-speech agents are the hardest to test with text-only evals, so the kit must exercise real audio paths including latency, interruption and overlap.
Mouse-1: decision model for agent harnessesThe scored pass/fail verdict per call criterion is a structured decision over call evidence, which is exactly the job of a small decision model sitting behind the gate.

Repo to start from

callrehearse — an open, self-hosted voice-agent audition kit: scenario runner, caller-profile matrix, speaker-attributed transcripts, scored verdicts, and a CI gate.

Evidence

Get the week's best AI launches, plus 3 ideas worth building

One email every Saturday. Ranked by traction, not hype. Free.