Anam solves turn-taking for unscripted avatar conversation with Flux STT

Anam builds photorealistic, real-time interactive avatars for customer-facing agents, training, and simulation. After A/B testing Flux against open-source and home-grown pipelines, Anam deployed Flux self-hosted and cut unwanted avatar barge-ins by about 50% with no p95 latency penalty.

About Anam

Anam builds the face layer for agents. Its photorealistic, emotionally expressive interactive avatars pair sub-second response times with API-first deployment to rank #1 across every independent avatar benchmark, with 70% of users preferring them over voice-only and text-based interfaces. The experience lives or dies on one thing: whether the avatar can hold a natural, real-time back-and-forth that feels as natural and engaging as talking to a human. To deliver that at scale, Anam needed a speech-to-text foundation built specifically for conversational turn-taking, deployable on their own infrastructure.

Key results

• ~50% fewer unwanted avatar barge-ins in Anam's own A/B testing
• 0 latency penalty, with p95 flat across tested alternatives
• Fewer turn-taking complaints on English sessions running Flux
• 2 weeks to self-hosted production, days on Deepgram hosted

The challenge: natural turn-taking in unscripted conversation

Every interactive avatar has to answer a deceptively hard question many times a minute: has the person finished speaking, or are they just pausing to think? Customers described this as a problem with the avatars' "ears": the avatar needed to feel like it was listening properly, not just transcribing audio.

Anam needed a speech-to-text (STT) model purpose-built for conversational turn-taking. And because Anam runs latency-sensitive, high-concurrency workloads, any solution had to be deployable on their own infrastructure rather than adding a network hop to a third-party cloud.

When you are building a real-time avatar, turn-taking quality is not just another backend metric. If the avatar pauses too long, talks over someone, or misses the moment to respond, the illusion breaks immediately.

Ben Carr

CTO

Why Deepgram won

Anam evaluated Flux in A/B testing against the alternative they knew best: a self-hosted, open-source speech-to-text option they could run on their own stack. The evaluation was concrete, scored on the metrics that matter for a voice agent, latency of transcription response and the number of unwanted interruptions, followed by an end-to-end internal test where the whole Anam team held conversations with the agent personas.

Turn-taking accuracy across every speaking style, low end-of-turn latency, resilience to background noise, the ability to self-host on Anam's own Kubernetes cluster, and concurrency that scales with their growth were all mandatory.

Anam ran Flux head-to-head against the open-source option and the result was clear. Flux handled the conversational edge cases the open-source model could not, while still meeting Anam's requirement to own and run the deployment themselves.

We ran thorough real-world A/B testing of Flux against our home-grown cascading pipeline, competitor solutions, and OSS solutions. Across varied talking styles and audio conditions, Flux consistently outperformed on latency and unwanted avatar barge-ins.

Ben Carr

CTO

The solution: Deepgram Flux STT

In conversation with an Anam avatar, audio from the end user streams into Flux, which transcribes speech in real time and, critically, signals when the user has actually finished their turn. The rest of Anam's stack, the language model and the voice output, builds on top of that timing.

Three important differentiators carried into production:

• Room to pause: speakers can stop mid-thought without the avatar jumping in, which is what slower speakers and role-play scenarios need.
• Detection at the end of a turn is fast enough that the avatar starts responding without dead air.
• Sounds that are not the user get ignored, so a barking dog no longer registers as a turn.

Anam deployed Flux self-hosted on their own Kubernetes cluster with H100 GPUs, the same infrastructure they use across their platform. After validation, they rolled Flux turn-taking out to 100% of their turnkey users. And deployment didn’t take long; hosted Deepgram took a matter of days from initial test to production. The self-hosted deployment took around two weeks.

Flux was a step change for us. It kept p95 latency in line with our best home-grown and competitor approaches while roughly halving unwanted barge-ins from the avatar, which is the kind of improvement users actually feel.

Ben Carr

CTO

The results

After deploying Flux, the clearest success signals came from their customers. Complaints about avatar turn-taking accuracy on English sessions running Flux dropped to near zero.

Before Flux, we had to choose between slow latency and unwanted avatar interruptions. Since adopting Flux for supported flows, we no longer see complaints about either issue. The biggest improvement has been the quality of turn-taking: the avatar feels like it is listening, waiting, and responding at the right moment.

Ben Carr

CTO

Some customers who were using their own Speech-to-Text migrated over to Anam's turnkey experience instead. Customers who bring their own Speech-to-Text sit outside the pipeline which limits how much of the conversation quality Anam can control. On the turnkey path, improved conversation quality arrives as part of the product instead of something each customer has to engineer.

This has led to improved outcomes for Anam’s customers as well. Companies use Anam to power customer-facing AI agents and training and simulation experiences, increasing engagement by 44%, accelerating adoption 2x, and lifting conversion by 25%+ when added to the sales cycle.

Looking ahead

Over the next year, Anam expects avatar experiences to become more multilingual, more emotionally expressive, more configurable, and more deeply embedded into real business workflows.

Deepgram fits into the roadmap as a key part of the real-time conversation layer. As Flux expands language coverage, Anam expects to use it more broadly by default. They are also evaluating Deepgram's Flux TTS offering as they look at the next generation of voice quality and real-time avatar presence.

Deepgram and Anam are both pushing toward the same future: real-time AI interactions that feel natural, responsive, and human. That alignment has made the partnership feel very practical and product-driven.

Ben Carr

CTO