Why transcription errors happen
How to improve transcription accuracy
Set the correct language and locale
Adjust end-of-speech sensitivity
Add a phrase list (Agentic Autopilot)
Enable Adaptive listening patience (AI Agent Platform)
Guide the agent through the prompt (AI Agent Platform)
Review and troubleshoot conversations
Overview
Autopilot and the AI Agent Platform convert what a contact says on a voice call into text, so the virtual agent can understand and respond. If the transcription is incorrect, the agent may misunderstand the request, ask the contact to repeat information, or take the wrong action.
This article explains why transcription errors happen, what you can configure to reduce them, and how to review conversations when something goes wrong.
Note: This article applies to voice channels only.
Before you begin
Intent-based Autopilots and Agentic Autopilots use different transcription engines. Agentic Autopilot uses more recent technology and tends to be more accurate. All settings below are specific to Agentic Autopilot and AI Agents Platform.
Why transcription errors happen
Transcription accuracy depends on many factors, some of which are outside Talkdesk's control. Common causes include:
- Audio quality: Low volume, a poor phone connection, or poor phone audio quality.
- Background noise: Loud environments or several people talking during the call.
- Accents and pronunciation: Strong accents or regional variations.
- Language and locale mismatch: For example, a call in British English configured as American English.
- Multilingual customers: Customers who use more than one language may confuse the automatic language detection layer.
- Domain-specific terms: Company and product names, technical jargon, and uncommon words.
- Complex values: Phone numbers, email addresses, and addresses.
- Overlapping speech: The contact speaking over the agent or interrupting it (barge-in).
Accuracy by language
Accuracy varies by language, and more widely used languages generally have better accuracy. Because accuracy depends on several different factors, Talkdesk doesn't provide an accuracy rate per language. For the list of supported languages for Agentic Autopilot, see AI Supported Languages.
How to improve transcription accuracy
Set the correct language and locale
- Open the Autopilot Agentic voice settings.
- Select the language and locale that match your contacts.
The locale matters. For example, English (US) and English (UK) can produce slightly different results.
Adjust end-of-speech sensitivity
Set the end-of-speech sensitivity value to 700-900 milliseconds. This way, the Autopilot will give a reasonable amount of time to the user to finish his sentence before starting to speak.
Add a phrase list (Agentic Autopilot)
If contacts use specific vocabulary, or words that are hard to transcribe, add them to the phrase list feature in the Agentic Autopilot voice settings.
Enable Adaptive listening patience (AI Agent Platform)
Adaptive listening patience makes the agent wait longer for the contact to finish speaking. It helps when the agent collects complex information such as phone numbers, emails, account numbers, and addresses.
- In the AI Agent Platform, open an agent and its orchestration settings.
- Enable Adaptive listening patience.
Guide the agent through the prompt (AI Agent Platform)
If important words are still transcribed incorrectly, update the prompt for that step:
- Add examples of the values the agent should expect (for example, the format of an account number).
- Instruct the agent to re-transcribe the value before processing it.
- Instruct the agent to ask the contact to confirm the value.
Alphanumeric and Short words
Capturing alphanumeric values (such as account numbers, codes, and emails) and very short replies (such as “yes”) are known challenges of speech recognition across the industry. They are not a single issue with one fix. Accuracy also depends on audio quality, background noise, accent, and the phone connection.
You can reduce these issues today:
- Alphanumeric values: Add examples of the expected format to the prompt for that step. Instruct the agent to re-transcribe the value and ask the contact to confirm it. Enable Adaptive Listening patience in the AI Agent Platform orchestration settings to give more time to the contact to speak with pauses without being interrupted.
- Short replies: Add the expected answers (for example, “yes” and “no”) to the phrase list in the Agentic Autopilot voice settings.
Review and troubleshoot conversations
Use Autopilot Session Monitor and AI Agent Observability (Preview) to review and listen to conversations.
Important: When a contact interrupts the agent, both tools show what the agent intended to say, not what the contact actually heard. Keep this in mind when you diagnose conversations that include interruptions. For this case, it's always recommended to listen to the conversation to get a better sense of what happened. Talkdesk is continuing to improve interruption handling.
Quick reference guide
| Symptom | What to try |
| Names or terms transcribed incorrectly | Add them to the Phrase list; add examples to the prompt |
| Numbers, emails, or addresses cut off | Enable Adaptive Listening Patience; adjust end-of-speech sensitivity |
| Consistently wrong across many calls | Check the language and locale settings |
| Poor results on noisy or low-quality calls | Limited by audio quality, so use confirmation steps in the flow |