The challenge
Tier-1 call volume was predictable and repetitive, but every attempt at automation had failed on the same two points: latency that made conversation feel broken, and handoffs that dumped the caller back to the start of a queue with no context.
Voice is unforgiving. A 2-second pause that reads as thoughtful in chat reads as a dropped call on the phone. The latency budget drove nearly every architectural decision.
Approach
01
Budget latency per hop, then defend it
Set a sub-second end-to-end target and allocated milliseconds across transcription, reasoning and synthesis. Any component that exceeded its allocation was replaced rather than tolerated.
02
Stream everything
Deepgram streams partial transcripts so reasoning starts before the caller finishes speaking, and synthesis begins on the first sentence rather than the full response.
03
Escalation carries context
When the agent transfers, the human receives the transcript, the detected intent and what was already attempted. The caller never repeats themselves.
04
Scope defined by confidence, not by keyword list
The agent hands off when its intent confidence drops, which covers cases nobody thought to enumerate during discovery.
Architecture
Transport
LiveKit handles the real-time audio session and call control.
Transcribe
Deepgram streaming speech-to-text with partial results feeding the reasoning layer continuously.
Reason
LangGraph state machine tracks conversation state, tool calls and confidence per turn.
Act
Tool calls hit the ticketing and account systems directly so the agent can actually resolve rather than just answer.
Escalate
Below-threshold confidence triggers a warm transfer with the full context packet attached.
Results
80%+Calls handled autonomously
sub-1sResponse latency
45%Lower handle time
0Repeat-yourself handoffs
24/7Coverage
What I took from it
Streaming partial transcripts into the reasoning layer bought more perceived responsiveness than any model optimisation. Callers judge the pause before the first word, not the total response time.
Stack
Realtime
Speech
Orchestration
Serving