---
title: "Voice agent function calling: Real-time data lookup guide"
description: "Master real-time data lookups for voice agents: optimize latency, filler functions, failure modes, and server-side execution."
canonical: "https://deepgram.com/blog/voice-agent-function-calling-real-time-data-lookup"
authors:
  - "Deepgram"
date: "2026-09-12"
category: "AI Engineering & Research"
---

# Voice agent function calling: real-time data lookup during a call

Function calling is how a voice agent answers "where's my order?" instead of transferring the caller to someone who can. One order-status lookup on the [inbound telephony agent](https://developers.deepgram.com/docs/inbound-telephony-agent) pattern takes two round trips, three function calls, and a spoken line for every way it can fail.

Time is the binding constraint, since a cascaded streaming pipeline reached first audio at 755ms in [Salesforce AI Research's tutorial](https://arxiv.org/html/2603.05413v1) before any database read. In this guide you'll work through the latency budget, the three failure lines, a complete order-status flow, and the smoke test to run before launch.

## Key takeaways

Five decisions separate a lookup that sounds like a conversation from one that sounds like a dropped call.

- CRM lookup latency lands on top of whatever the pipeline already spends reaching first audio.
- In this pattern, `agent_filler` handles filler as a callable function.
- An order-status question costs two lookup round trips (`find_customer`, then `get_orders`) and three function calls when `agent_filler` is included.
- Every failure path (timeout, empty result, wrong record) ships with a spoken line written before launch.
- In the tutorial's server-side pattern, an `endpoint` in a function definition routes execution through Deepgram, while server credentials stay on your server.

## What changes when the caller is on the phone

Two things change on a call. The caller hears the wait, and the parameters arrive as audio instead of keystrokes. Function calling otherwise works the same whether your agent types or talks.

### Chat tolerates a spinner, a call doesn't

You get no typing indicator on a phone line, so whatever your handler takes, the caller experiences as dead air.

### Parameters arrive as speech

A spoken `ORD0089` reaches your handler only after the model reconstructs it from audio, so the dangerous failure is a confident lookup against the wrong ID. Text fields hand the same ID to a chat agent intact.

The tutorial's `find_customer` description is mirrored in `functions.py`. It tells the model to turn a spoken "42" into `CUST0042`, and to strip spaces, dashes, and parentheses from a phone number before the handler sees it. [Recognition accuracy](https://deepgram.com/learn/what-is-word-error-rate) at the ASR layer still sets the ceiling.

### Client-side and server-side execution

Whether a function runs on your side or through Deepgram depends on one field in its definition. Leave `endpoint` out and your application receives a `FunctionCallRequest`, then returns a `FunctionCallResponse`. Add `endpoint` (`url`, `method`, `headers`) and the [Voice Agent API](https://deepgram.com/product/voice-agent-api) routes the call to your URL instead.

The [function call docs](https://developers.deepgram.com/docs/voice-agents-function-calling) point server-side execution at anything sensitive, including database reads and third-party service calls. Credentials stay on your server. Each item in the request schema includes a `client_side` boolean, which your code can use to identify which functions still need a response.

## The latency budget for voice agent function calling

Your database read gets whatever time is left once the pipeline reaches first audio, and the pipeline usually takes most of it. Plan the fill before you plan the query.

### What the caller hears while the function runs

Dead air is what the caller hears while your handler works, and it costs you fast. Ratings of perceived willingness drop off noticeably at 600ms in the classic 2013 [gap-tolerance study](https://doi.org/10.1121/1.4802900) from Roberts and Francis, and the difference turns statistically significant between 700 and 800ms. `InjectAgentMessage` is what covers that silence in the reference pattern.

Deepgram's [pipeline comparison](https://deepgram.com/learn/voice-agents-vs-automation-platforms) cites the same Salesforce AI Research tutorial for its first-audio figure. Add your backend read time, and a routine turn lands past the threshold.

### Cover the wait with a filler function

The filler is a function the model has to call, not a phrase it can improvise, so every "let me check that" routes through `agent_filler`. The prompt template in Deepgram's reference tutorial forbids the model from saying "One moment please" without a function call.

It requires the model to call `agent_filler` with `message_type="lookup"` first, then the lookup function, then speak only once it has real information. The filler handler returns two parts: a `function_response` with `status: "queued"`, and an `InjectAgentMessage` carrying the spoken phrase. A [2024 filler study](https://doi.org/10.1109/ro-man60168.2024.10731216) measured a 1.1-point gain on a Likert scale in participants' impression of response delay, a statistically significant result.

### When one lookup chains into the next

Chained lookups can't run in parallel, so the caller waits for both hops plus the model's decision between them. The `get_orders` definition requires a `customer_id` that must come from `find_customer` first, so identity resolves before the order does.

## Write conversational failure responses

Write the spoken line for each of the three failure modes before launch, or the model improvises one on the call. A timeout the caller hears as silence costs you the call, even when your logs show the retry succeeded. Deepgram's reference prompt covers all three. Never expose technical detail, and say something like "I'm having trouble accessing that information right now."

### When the lookup times out

Set the clock yourself; the documented workshop pattern provides no numeric limit. [Deepgram's workshop](https://workshops.deepgram.com/voice-agent-py/function-calling) warns that the agent waits indefinitely if your code never sends a `FunctionCallResponse`. It tells you to wrap slow lookups in a timeout.

When the timer fires, return content the agent can speak instead of throwing. Example line: "That's taking longer than usual on my end. Would you like to hold for another moment, or can I take your number and call you back?"

### When the result comes back empty

Give the model speakable data when no account matches. Deepgram's reference prompt tells the agent to ask for a different phone number or email rather than report a failure. Return a boolean like `found: false` so the model can distinguish an empty result from an error.

Ask the caller to repeat or spell rather than guess. Example line: "I couldn't find an account under that number. Could you read it to me once more, or give me the email on the account instead?"

### When the record is the wrong one

When the lookup returns a real order belonging to someone else, the agent reads back a stranger's purchase and your logs look clean. Resolve identity before data. You must bind the `customer_id` returned by `find_customer` to the verified caller identity in your backend.

Requiring that parameter in the `get_orders` schema doesn't enforce identity or authorization. Put authorization in the backend. [OWASP's LLM06 guidance](https://owasp.org/www-project-top-10-for-large-language-model-applications/2_0_vulns/LLM06_ExcessiveAgency.html) says to implement it in downstream systems rather than letting an LLM decide what's allowed. Example line: "Before I read anything out, can you confirm the name on the account and the zip code it ships to?"

## Order status lookup, start to finish

In this flow, getting from the caller's question to a spoken answer takes three calls: filler, customer resolution, and order retrieval. Your Settings message defines them, your handlers run them, and the response payload feeds the read-back.

### The function definition

The linked configuration example places both lookups under `agent.think.functions` in the `Settings` message. It shows each with a `name`, `description`, `parameters` object, and optional `endpoint`, per the [configuration docs](https://developers.deepgram.com/docs/configure-voice-agent). `agent_filler` is a third function definition in the complete flow. The description does real work, since it's where the model learns identifier formats and ordering rules.

```json
{
  "name": "get_orders",
  "description": "Retrieve order history for a customer. Always verify you have the customer's account first using find_customer before checking orders.",
  "parameters": {
    "type": "object",
    "properties": {
      "customer_id": {
        "type": "string",
        "description": "Customer's ID in CUSTXXXX format. Must be obtained from find_customer first."
      }
    },
    "required": ["customer_id"]
  }
}

```

Beside it, `find_customer` accepts `customer_id`, `phone`, or `email`.

### The handler and the response payload

Your handler reads the call's arguments and returns a payload the model can speak. It pulls `functions[].arguments` (a JSON-encoded string) and `functions[].id` from the `FunctionCallRequest`, then echoes that ID back. Deepgram's handler is short:

```python
async def get_orders(params):
    customer_id = params.get("customer_id")
    if not customer_id:
        return {"error": "customer_id is required"}
    result = await get_customer_orders(customer_id)
    return result

```

Wrap `get_customer_orders` in the timeout from the previous section. The reply is a FunctionCallResponse payload with four fields. Its `content` is a string, so serialize before you send:

```json
{
  "type": "FunctionCallResponse",
  "id": "<id from the request>",
  "name": "get_orders",
  "content": "{\"found\": true, \"orders\": [...]}"
}

```

### The spoken read-back

Say the identifier slowly and the status conversationally, because a number read back wrong makes the caller doubt the status too. Deepgram's [Aura-2](https://deepgram.com/product/text-to-speech) [prompting guidance](https://developers.deepgram.com/docs/text-to-speech-prompting) recommends natural pauses between groups of 2–4 letters when reading identifiers.

Its example is "To confirm, is your referral code Queue Why. Eigh Beee?" The reference prompt shapes the order summary as "I can see you have two recent orders. Your most recent order from [date] for $[amount] is currently [status]."

## Where the data actually comes from

Every round trip ends in a system you need to measure directly. The numbers below are alarm thresholds and engineering benchmarks.

### CRM records

Your CRM flags its own slowness long after the caller has noticed. The [Salesforce Trust status guide](https://trust.salesforce.com/en/statususerguide) counts a 500ms average over five consecutive minutes as core-service degradation, with some API services flagged at 300ms. Integration architecture sits in the [voice agents Salesforce guide](https://deepgram.com/learn/integrate-voice-agents-salesforce).

### Order and fulfillment systems

Your order system is the hop with no published number, so measure it yourself. Take the P95 from the agent's region, including any gateway in front of it. Pipeline design for CRM write-back sits in the [CRM integration pipeline](https://deepgram.com/learn/voice-ai-apis-crm-integration-pipeline) guide.

### Internal APIs behind an auth layer

Token validation adds a network hop unless the signing keys are already cached. [Auth0's guidance](https://auth0.com/docs/secure/tokens/json-web-tokens/json-web-key-sets) is to cache JWKS for 5–10 minutes so validation skips the network. Warm that cache when the call connects, not when the lookup fires.

## Ship the lookup before you ship the agent

Run the happy-path call and both controlled failures before you point a phone number at the agent. If all three sound like a conversation, the lookup path is ready.

### A three-call smoke test

Prove the happy path before forcing the controlled failures:

1. Use a record that exists to prove the happy path and the read-back.
1. Test a missing record to prove your empty-result line fires instead of an exception.
1. Stall a handler past your limit to prove the fallback line plays instead of silence.

### What to log on every function call

Log `id`, `name`, `arguments`, and `client_side` on every `FunctionCallRequest`, plus whether the argument the model extracted matched what the caller said.

Four fields per frame give you enough to replay the whole call: `request_id` from the `Welcome` message, a monotonic `seq`, a `ts` timestamp, and `direction`. Tap the WebSocket and persist every non-audio frame this way, as Deepgram's [observability guide](https://developers.deepgram.com/docs/voice-agent-observability) recommends.

### Start with one function

When the toolset grows, selection accuracy falls. The [HumanMCP study](https://doi.org/10.48550/arxiv.2602.23367) measured a drop from 98% at 10 tools to 88% at 100. Add functions one at a time and re-check selection accuracy after each. One lookup, three failure lines, and a filler function make a complete first release.

Try the lookup on a real call. Create a [free Deepgram account](https://console.deepgram.com/signup) and put your $200 free credits against the order that always comes back wrong.

## FAQ

### How long can the agent stay silent before the caller assumes the call dropped?

Your telephony platform decides that for you, and its no-input timer is the hard ceiling. It reprompts or disconnects without consulting your code, so look up the configured value in your own stack. Set filler timing from your measured first-audio and backend latency so the filler plays well before that boundary.

### How does the client know which function calls need a response?

Inspect the `client_side` boolean on each item in the incoming request. Use it to route response ownership and avoid sending a client response for calls assigned to the server-side pattern.

### What happens if the caller interrupts while the function is still running?

The lookup keeps running, so your client halts playback and decides whether that in-flight call finishes or cancels. `UserStartedSpeaking` is the signal to stop audio and discard what's buffered. In the [Flux STT pattern](https://developers.deepgram.com/docs/flux/agent), that logic triggers from `StartOfTurn`, which carries a non-empty transcript. With [Nova-3 streaming](https://developers.deepgram.com/guides/deep-dives/audio-preprocessing-barge-in), it waits for an interim result with at least 2 words.

### How many functions can one agent hold before selection accuracy degrades?

There's no threshold number, since accuracy degrades gradually and overlapping descriptions hurt more than raw count. Two functions with similar descriptions degrade selection faster than ten with distinct ones. Test the phrasings that could plausibly map to two of your functions, and split or merge definitions until each has one obvious trigger.

### Does a long call hit a session limit while lookups are still running?

Yes, at two hours, and an in-flight lookup won't hold the session open. You get a `Warning` with `MAXIMUM_SESSION_LENGTH_APPROACHING` at 1 hour 55 minutes, then an `Error` with `MAXIMUM_SESSION_LENGTH_REACHED` at the [two-hour limit](https://developers.deepgram.com/docs/voice-agent-errors-warnings). `KeepAlive` doesn't extend it, and the docs don't cover a pending `FunctionCallRequest` at close. Finish or abandon it, then reconnect with `agent.context`.

## Related articles

- Anam solves turn-taking for unscripted avatar conversation with Flux STT (AI Agents)
- Audio, decision, composition: Five open-source Jev projects (AI Engineering & Research)
- Introducing Nova-3 Medical multilingual: Medical speech recognition in 10 languages (Announcements)
- AI voice agent to human handoffs: How to warm transfer calls without losing context (AI Engineering & Research)