How Deepgram benchmarks voice AI
Accuracy results for Deepgram speech-to-text and text-to-speech, with the methodology and testing dates behind every number.
STT, TTS, and the test set
Accuracy across languages and noise, voice quality under pressure, and the audio behind every score.
Batch and streaming
Every vendor runs the same audio both ways, because a model that leads on recordings can fall apart live.
20 languages
Aggregate word error rate across 20 languages, plus a per-language head-to-head on the six with the most production volume.
Accuracy under noise
Scored on customer calls and contact center audio, where background noise, cross-talk and telephony compression are the normal case.
Entities, not sentences
Word error rate on alphanumerics, drug names, currency and dates — the inputs that break voice agents in production.
Time to first audio
Text and audio generate together, so first-audio latency stays under 200ms however long the response runs, instead of growing with it.
Held across a session
Tone, pacing and pronunciation measured over a whole call rather than a single well-chosen line.
Real production audio
About 30 hours per vendor per language of customer calls, contact centre recordings and meetings, across 20 languages.
No public datasets
FLEURS, CommonVoice and LibriSpeech are filtered out, so no vendor is scored on audio it may have trained on.
No demo configs
Every provider is tested on the model it sells for production, named by version, rather than a demo configuration.
Deepgram against the field
Speech-to-text: about 30 hours per vendor, per language of real production audio across 20 languages, public datasets excluded to prevent training-set leakage. Text-to-speech: read-aloud and hard-prompt sets against Inworld 1.5 Max, ElevenLabs v3 and Cartesia Sonic 3.5.
Speech-to-text April 2026 · Text-to-speech August 2026
Word error rate, lower is better| Deepgram | Best competitor | |
|---|---|---|
| Batch accuracy | 13.78% WER | 16.33% WERElevenLabs Scribe |
| Streaming accuracy | 13.44% WER | 18.45% WERSoniox |
| Speech accuracy | 2.2% median WER | 2.7% median WERInworld 1.5 Max |
| Hard prompts | 3.4% WER | 5.0% WERInworld 1.5 Max |
Measured on real, messy production audio
Batch and streaming speech-to-text, text-to-speech accuracy, and the hardest prompts we could find, each against the field.
Most accurate on recorded audio
Across 20 languages and about 30 hours of real production audio per vendor, Nova-3-multi leads the multilingual aggregate. ElevenLabs Scribe is next, 2.55 points behind. On every tier-1 language tested, Deepgram ranks first.
DeepgramNova-3-multi
13.78
ElevenLabsScribe
16.33SonioxSTT RT v4
18.15GoogleChirp 3
18.50AzureAuto
19.26SpeechmaticsEnhanced
19.45AWSTranscribe
20.17AssemblyAIUniversal
22.97OpenAIWhisper
25.30
- 13.78%Multilingual batch WER, best of ten vendors
- 6 of 6Tier-1 languages where Deepgram ranks first
The only sub-10% English model
Same audio, streamed live. Flux-Multi leads the aggregate and Flux-EN is the only streaming model under 10% WER on English. ElevenLabs Scribe falls from second on batch to tenth on streaming.
DeepgramFlux-EN
9.05DeepgramFlux-Multi
13.44DeepgramNova-3-multi
15.92
AssemblyAIUniversal
17.15SonioxSTT RT v4
18.45SpeechmaticsEnhanced
19.23AzureAuto
22.65GoogleChirp 3
22.66ElevenlabsScribe
27.39OpenAIWhisper
33.70
- 9.05%English streaming WER, Flux-EN
- 5.01 ppGap to the nearest non-Deepgram model
Gets the details right
The account number, the drug name, the dollar amount. Flux TTS has the lowest word error rate of any read-aloud text-to-speech model tested.
DeepgramFlux TTS
2.2
Inworld1.5 Max
2.7ElevenlabsV3
4.1CartesiaSonic 3.5
6.1
- 2.2%Median word error rate, about half ElevenLabs
- 80msTime to first audio
Hardest inputs, widest gap
Alphanumerics, drug names, currency and dates break voice agents in production. Flux TTS holds 3.4% where the next-best model reaches 5.0% and the weakest reaches 9.8%.
DeepgramFlux TTS
3.4
Inworld1.5 Max
5.0ElevenlabsV3
6.6CartesiaSonic 3.5
9.8
- 3.4%Hard-prompt word error rate
- 2.9XLower than Cartesia Sonic 3.5 on the same prompts
Run your own benchmarks
Upload your own audio and see how Deepgram performs against the numbers on this page.
FAQs
Because the audio is different. Peer-reviewed clinical research puts the degradation between controlled benchmark conditions and production telephony at 2.8x to 5.7x. Benchmarks use read speech at 16kHz; contact centers run spontaneous conversation at 8kHz with noise, cross-talk, and domain vocabulary.
FLEURS spreads about 12 hours of Wikipedia read-speech evenly across 102 languages. Its own dataset card acknowledges a known mismatch with noisier production settings. Uniform language weighting also tells you little if 80% of your traffic is English.
Word error rate treats every error as equal. Six others predict production success better: keyword recall, target 80 to 90%; entity accuracy, 70 to 90%; real-time factor under 1.0; diarization error rate under 5%; punctuation accuracy; and semantic preservation.
Usually not. ITU-T P.800.2 rules out comparing mean opinion scores from separate experiments unless they were designed for comparison. Even the wording moves the number: asking listeners about quality scored 4.46 where asking about naturalness scored 4.25.
Because it is typically one warm request on the fastest model under ideal network conditions. Tails stretch under concurrency: NVIDIA's tables show P99 first-audio latency of 380.77ms against a 175.62ms average. Derive your budget from the tail at your peak load.
Who ran it and under what conditions. Whether it was measured at production concurrency or as a single request. And whether it names its comparison targets: a named model on a dated API call can be challenged, “a leading competitor” cannot.