Find the pause your caller will actually hear.
Add the five delays between a caller finishing a sentence and the agent beginning its reply. The largest component tells you where optimization work is most likely to matter.
measurement
A budget is useful only after you replace it with traces
Measure each boundary from production calls. Endpointing, first transcript, first model token, first audio byte, and first playable audio should have separate timestamps.
The labels here are planning bands, not a universal quality score. Barge-in, language, caller behavior, network quality, and response length can change what feels natural.
Start with the slowest measured component. Cutting 30 ms from every service is usually less useful than removing a 400 ms endpointing delay.