Listen to the whole response path
A caller experiences the time between a request and a useful reply. Network buffering, turn detection, recognition, reasoning, tool execution, synthesis, and playback can all contribute. Improving a single model benchmark does not establish that the interaction feels faster.
Keep slow work from blocking dialogue
An account lookup or business workflow may take longer than a normal conversational pause. A non-blocking design can acknowledge the request, keep listening, and introduce the confirmed result when it is ready. The acknowledgment should describe work in progress without suggesting that an action has already succeeded.
Define what an interruption means
When someone says “wait,” stopping generated speech is only part of the problem. The system also needs a policy for pending actions. Some work can be canceled, some results can be ignored, and completed actions may require a separate corrective step. Make these distinctions explicit in the workflow.
Plan for active sessions
Long-lived audio sessions consume resources differently from short web requests. Routing should account for existing sessions and current load. Worker draining, connection recovery, and context persistence require application design; a load balancer cannot recover conversation state that was never saved.
Measure under realistic conditions
Review response-time distributions and slower turns alongside task completion and tool correctness. Test interruptions, errors, reconnects, and concurrent users. A deployment decision should reflect the conversation people will actually have, including its difficult moments.
Further reading
External publications describe their authors’ work. Reference architectures and examples are conceptual.