A New Approach to Turn-Taking in Voice AI: Turn Prediction v3 and Interruption Prediction v1

Max 16 min read

The natural rhythm of conversation depends on knowing when to start speaking and when to stop. Humans handle this effortlessly: we sense the end of a turn, we recognize a quick “uh-huh” as encouragement rather than an interruption, and we can stop mid-sentence when someone clearly needs to break in. Voice AI agents have struggled with all three.

Last year we introduced Krisp Turn-Taking, the first audio-based end-of-turn prediction model designed for Voice AI. Then we shipped v2, with substantial accuracy gains and integration into the Krisp VIVA SDK. With this release, we’re expanding what Turn-Taking means at Krisp.

Krisp Turn-Taking: Turn Prediction v3 and Interruption Prediction v1

Krisp Turn-Taking now comprises two complementary models:

• Turn Prediction v3 — our end-of-turn prediction model, substantially faster and more accurate than v2, and now multilingual.
• Interruption Prediction v1 — a brand-new model that distinguishes between backchannels (short acknowledgments like “yeah” or “uh-huh”) and genuine interruptions when the user wants to take the turn and interrupt the speaking AI agent.

Together, they cover both halves of the turn-taking problem: knowing when to speak, and knowing when to stop. The naming reflects the maturity of each component: Turn Prediction has reached v3, while Interruption Prediction is introduced as v1.

Headline gains in this release:

• Fast responses (under 200 ms) jumped from 47% to 69% compared to v2, without increasing the risk of interrupting the user mid-sentence.
• Better latency-accuracy curve for end-of-turn prediction
• Introduction of Interruption Prediction v1 — a game-changing approach to interruption handling that significantly outperforms VAD- and word-count-based methods
• The test dataset used in our end-of-turn prediction evaluation is now public on HuggingFace: Krisp-AI/turn-taking-test-v1

Architecture overview

Turn Prediction v3

Turn Prediction v3 listens to conversational audio and outputs a probability between 0 and 1 that the speaker has finished their turn. The probability is progressively refined during the silence period that follows speech — so the model can react quickly when the end of the turn is clear, and hold longer when it isn’t yet.

The model:

• Operates on audio frames of configurable duration (e.g., 40 ms segments)
• Returns a probability per frame, with the binary end-of-turn decision controlled by a configurable threshold (default 0.5)
• Is multilingual: supports English, German, French, Spanish, Hindi, Finnish, Italian, Portuguese, Chinese, Japanese, Korean, Russian, and additional languages

Interruption Prediction v1 — a new capability

Interruption Prediction v1 addresses a problem most voice agents handle poorly: the difference between a backchannel (short acknowledgments like ‘yeah’ or ‘uh-huh’) and an interruption.

When a human says “yeah” while the agent is speaking, they’re encouraging the agent to continue — not interrupting. When they say “wait, hold on”, they want the agent to stop. Most current systems use crude heuristics: VAD on any sound (which fires on every “uh-huh”), fixed timing thresholds, minimum word counts, or stop-word lists. The result is bots that interrupt themselves on backchannels or fail to stop when the user genuinely needs to speak.

Interruption Prediction v1 takes a different approach. It produces a probability/confidence rate between 0 and 1 that the user genuinely intends to interrupt. The probability is progressively refined during the user’s speech segment, distinguishing intent from acknowledgment without waiting for the user to complete a full sentence.

The model:

• Operates on audio frames of 40 ms duration
• Returns a probability per frame, with the binary interrupt decision controlled by a configurable threshold (recommended default 0.4)
• Supports English-only at v1

Symmetry by design

The two core models share an evaluation philosophy and a mirrored mechanism for turn-taking decisions: Turn Prediction refines its probability during the silence after speech, while Interruption Prediction refines its probability during the user’s overlapping speech.

Specs at a glance

Both models run efficiently on the CPU and are included in the Krisp VIVA SDK.

Evaluation methodology

Turn-taking models trade off two things: how accurately they predict turn boundaries, and how quickly they react. A model that gives accurate predictions but waits too long to respond will feel sluggish; a model that fires after 100 ms will feel snappy but will cut users off mid-thought. Single-number metrics like accuracy or F1 hide this trade-off.

In our v1 blog post we introduced the latency–accuracy curve as the right way to think about end-of-turn prediction. With this release, we extend the same philosophy to interruption prediction.

Mean Shift Time vs False Positive Rate

For Turn Prediction we use two coupled metrics.

Mean Shift Time (MST) measures latency. Given a set of true turn-shift cases S and a model that outputs probability P i (t) at time t after the end of speech, MST at threshold τ is the average time the model takes to declare a shift across all true-shift cases:

If a silence is shorter than the time needed to cross the threshold, the silence is conceptually extended — we measure when the model would have fired.

False Positive Rate (FPR) measures accuracy on the negative class. Given a set of true-hold cases H (where the speaker pauses but does not finish their turn), FPR is the fraction of holds where the model erroneously crosses the threshold at any point during the silence:

Sweeping the threshold from 0 to 1 traces out a curve in the MST–FPR plane. Lower curves are better — they mean faster reaction at any given accuracy level.

Mean Interruption Time vs False Positive Rate (Interruption Prediction)

The same logic extends to interruption prediction. The latency metric is Mean Interruption Time (MIT) — the average duration between the moment the user begins speaking over the bot and the moment the model classifies that speech as an interruption rather than a backchannel.

Higher thresholds mean fewer false interrupts on backchannels, but the bot is slower to stop when the user genuinely wants the floor. Lower thresholds mean faster cutoffs, but a greater chance of the bot stopping during a quick “uh-huh.” To visualize this trade-off, we plot a chart showing the relationship between mean interruption time (computed on true-interruption examples) and the false-positive rate (interruptions during backchannels) as the threshold is varied from 0 to 1.

For baseline methods that don’t expose a confidence score (e.g., VAD-based, minimum word count), we plot a single point at their (MIT, FPR) coordinates rather than a curve.

Test datasets

For Turn Prediction, evaluation runs on the Krisp-AI/turn-taking-test-v1 dataset, which we are publishing on HuggingFace alongside this release:

• 4 hours of conversational audio
• 30 speakers
• 976 manually labeled shift cases
• 1,754 manually labeled hold cases

The original recordings consisted of long-form conversations. These recordings were first reviewed and labeled manually by human annotators. After labeling, the conversations were segmented into shorter clips, ensuring that each segment preserved the context needed for accurate interpretation while maintaining alignment with the original annotations. We also retained the last silence segment of each clip and recorded its duration in the metadata, enabling false-positive-rate calculation and analysis of the MST vs FPR trade-off.

Results: Turn Prediction v3

Threshold selection

Turn Prediction v3’s behavior is controlled by a single threshold. We benchmarked across the full threshold range and identified three operating points corresponding to low, medium, and high accuracy levels. The binary prediction evaluations for each are shown below (for the definitions of these metrics, see our first blog post).

We recommend threshold 0.5 as the default operating point — it balances reaction speed and accuracy. All Turn Prediction v3 results below use 0.5; Turn Prediction v2 results use 0.4.

Results: Interruption Prediction v1

Interruption Prediction is a fundamentally different task from end-of-turn detection: instead of judging when a speaker has finished, the model judges whether incoming user speech is a backchannel or a real interruption attempt. The metric framework, however, is structurally parallel.

Mean Interruption Time vs False Positive Rate

The curve shows Interruption Prediction v1 across the full threshold sweep, with two industry baselines plotted as single points (since they don’t expose a confidence score):

• VAD-based: triggers an interrupt after a fixed duration of continuous user speech (Silero VAD, integrated in Pipecat).
• Minimum word count: triggers an interrupt after 3 words are recognized by Deepgram ASR (Pipecat default).

A lower curve means faster interruption at the same false-positive rate, or fewer false interrupts at the same speed. Krisp Interruption Prediction v1’s curve sits decisively below both baseline points across the operating range.

Threshold trade-off

Availability

Both Turn Prediction v3 and Interruption Prediction v1 are available in the Krisp VIVA SDK and integrated into Pipecat:

• Turn Prediction v3 — krisp-viva-tp-v3, ~9M parameters, 30 MB, configurable frame duration, 12+ languages, recommended threshold 0.5
• Interruption Prediction v1 — krisp-viva-ip-v1, ~6M parameters, 24 MB, 40 ms frames, English, recommended threshold 0.4

Both models are designed for low resource consumption on CPU — no GPU required for real-time inference.

FAQ

What is turn-taking in voice AI, and why does it matter?

Turn-taking determines when a voice AI agent should start speaking (after the user finishes) and when it should stop (when the user interrupts). Poor turn-taking causes awkward silences, cut-offs, and agents that halt mid-sentence because the user said “uh-huh.” It’s the single biggest factor in whether a voice agent feels natural or robotic.