A Voice Agent That Sounds Human and Stays in Your VPC

Overview

Voice agents still fail at the things people do without thinking. They talk over the caller, go silent during a lookup, and act on the first version of a request after the caller has already changed it. The models are capable; the problem is the pipeline around them. Every handoff from speech-to-text to the LLM to text-to-speech keeps the words and throws away everything else, and one LLM is asked to do two jobs at once: run the live conversation and do the actual work. In this session, Deepgram and AWS show what changes when those two jobs are split, and how to run the whole package inside your own AWS account on Amazon SageMaker AI, alongside the Amazon Bedrock and AgentCore services you already use.

Deepgram's new Think-Live LLM takes the first job. It manages the conversation, deciding when to listen, when to wait, and when to respond, so the agent backchannels while a caller reads out an account number, waits through a mid-thought pause, and handles an interruption the way a person does. A reasoning LLM keeps the second job. Theresa Foy, product lead for Deepgram's Voice Agent, walks through how Think-Live works alongside the rest of the pipeline, what it exposes as events, and how you tune it. Then she shows the full package, Flux speech-to-text, Think-Live, the reasoning LLM, and Flux text-to-speech, running as Amazon SageMaker AI endpoints inside your VPC, so nothing in the conversation leaves your account.

Sagar Murthy from AWS covers the deployment path: subscribing on AWS Marketplace, standing up SageMaker AI real-time endpoints in your VPC, and how the package fits alongside the rest of your AWS stack, including Amazon Bedrock and Amazon Bedrock AgentCore, with the AWS Guidance for voice agents, a Pipecat pipeline on Amazon ECS, as a reference path. We finish with a live demo on SageMaker and the production guidance you'll need to ship it: concurrency, latency, failover, and cost.

What You'll Walk Away With

• Why one LLM can't run the conversation and do the work at the same time, and how splitting the two jobs changes how a voice agent sounds
• How Think-Live decides when to listen, wait, backchannel, or respond, and which of those decisions you can observe and tune
• How to deploy the full package on Amazon SageMaker AI: Marketplace subscription, real-time endpoints inside your VPC, Pipecat orchestration through the AWS Guidance for voice agents, and where Amazon Bedrock and AgentCore fit alongside it
• A live demo plus production guidance: concurrency, latency, failover, and cost

Who Should Watch

Engineers, architects, and technical leads building voice agents on AWS who need the whole agent to run inside their own VPC. Product and platform leaders weighing speech-to-speech models against cascaded pipelines for customer-facing voice. AWS account teams and solutions architects who want a deployable voice agent to bring to customers before re:Invent.