TurnTaker
The problem
Voice agents still run on fixed silence timers: wait 400–500 ms of quiet, then reply. A caller who pauses to think, recall a date, or breathe gets cut off mid-sentence; a caller who finishes gets 1–5 seconds of dead air while the pipeline catches up. The result is repeated answers, wrong dates and policy numbers, and callers who hang up — and the builder has no metric for it, only a silence-timeout slider to guess with.
The idea
Why now
The pieces for a learned turn-taking decision just got cheap enough to run on every turn: a real-time decision model built for streaming turn-taking at 12.5 decisions per second, an open 0.5B ASR model at ~€10 per 1,000 hours that can run locally for near-free transcription, a GA micro-decision model 50x faster and 79x cheaper than a frontier model for routing each turn, and a live speech-to-speech model with 880 ms time-to-first-audio. Turn-taking stops being a timer setting and becomes a cheap model call.
What it combines
uRun Sense (streaming turn-taking decisions at 12.5/sec) + Rumble-2-Flash (open 0.5B ASR, 40+ languages, 6.96% WER, ~€10/1000h — near-free local transcription feeding the decision) + Microsoft Decision 1.1 (micro-decision routing per turn at 50x faster / 79x cheaper than GPT-6.1 Sol) + MAI Realtime 1 (live speech-to-speech at 880 ms time-to-first-audio, $0.01/min — the fast response path once the decision says SPEAK). The mix matters because turn-taking is a decision problem, not a timer problem: cheap streaming ASR plus a cheap per-turn decision model makes a learned decision affordable on every turn, where previously only a fixed silence threshold was cheap enough.
MVP
Build first: (1) a streaming decision service that takes live transcript + audio features and emits SPEAK / HOLD / YIELD, wired to one stack (Pipecat + a local ASR); (2) a 20-call benchmark harness with scripted thinking-pauses, interruptions, and backchannels that reports interruption rate and dead-air seconds before/after; (3) a one-page dashboard showing those two numbers. Deliberately skip: telephony integration, multi-language support, the Vapi/Retell plugins, and any hosted multi-tenant service — run it locally against recorded calls first.
How it makes money
The moat
The incumbents monetize per billable minute and own a closed pipeline, so a portable turn-taking layer works against their business model: fewer interruptions and less dead air mean shorter calls and fewer wasted LLM/TTS tokens — fewer billable minutes. A Vapi or Retell turn-taking improvement also only ever works inside its own stack, while TurnTaker's dataset — interruption labels across many stacks, accents, and call types — is a cross-platform asset no single vendor can collect from inside its own walls. The moat is the cross-stack measurement dataset plus a pricing model (tiny per-minute fee) that incumbents cannot copy without cannibalising their own per-minute revenue.
Distribution
Wedge is a drop-in SDK for the platforms builders already use — a Pipecat/LiveKit plugin and a Vapi/Retell webhook that replaces their endpointing config in an afternoon, with a free tier measured on the builder's own calls. Paid tier is usage-based per call-minute, sold to voice-agent builders and agencies; the measurement dashboard is the viral loop (builders share their before/after interruption rates).
Why it wins
Every incumbent bakes turn-taking into its own closed pipeline as a fixed silence timer or a model default; none of them sell turn-taking as a portable, measurable layer that works across any STT/LLM/TTS stack and reports interruption rate and dead-air seconds as first-class metrics.
Risks
Biggest risk is that a single platform (e.g. LiveKit's turn-detector model or OpenAI's semantic VAD) becomes 'good enough' for most builders, shrinking the market to a niche. The MVP de-risks it by benchmarking against those defaults on the same 20-call harness: if TurnTaker cannot beat LiveKit's default or OpenAI server VAD by a clear margin on interruption rate, the idea fails fast before any platform plugins are built.
Build it with
- uRunStreaming turn-taking decisions at 12.5/sec.
- Rumble-2-FlashCheap local ASR feeding the decision model.
- Microsoft Decision 1.1: a GA decision model for high-frequency agent micro-decisionsMicro-decision routing per turn, 50x faster / 79x cheaper.
- MAI Realtime 1: Microsoft's first live speech-to-speech modelFast speech-to-speech response once SPEAK fires.
Repo to start from
turntaker — vendor-neutral streaming turn-taking engine for voice agents, with interruption-rate and dead-air benchmarks
Evidence
- LiveKit Agents: turn detection
- Voice agent latency benchmark: server VAD vs semantic VAD
- Callers interrupted mid-answer when they pause
- Hacker News: voice agents train us to worsen our speech
- Retell AI pricing
Get the week's best AI launches, plus 3 ideas worth building
One email every Saturday. Ranked by traction, not hype. Free.