Important Note: This feature is available in Preview for select customers. During the Preview phase, AI Agent Observability is accessed inside the AI Agent Platform. At General Availability (GA), the experience will move to CXA Operations Center (formerly AI Trainer).
- Overview
- How it's Different
- How AI Agent Observability Helps
- Key Capabilities
- Impact
- Faster Troubleshooting
- Better Improvement Decisions
- Operational Transparency
- Important Information
Overview
AI Agent Observability gives administrators and supervisors end-to-end visibility into how AI Agents handle customer interactions. Through this view, teams can review past AI Agent conversations, validate behavior, audit performance, and make informed training decisions based on real interactions.
During the Preview phase, Session History is accessed inside the AI Agent Platform. At General Availability, the experience will move to the CXA Operations Center (formerly AI Trainer), providing a single, consistent place to observe and analyze AI Agent activity across channels.
How it's Different
Until now, teams have had limited visibility into what happened inside an AI Agent session. Reviewing outcomes often required piecing together information across multiple tools, making it difficult to understand why an interaction ended the way it did or how the AI Agent arrived at a given response.
This lack of transparency slowed down troubleshooting, made quality assurance harder, and limited the ability of trainers to continuously improve AI Agent performance.
How AI Agent Observability Helps
AI Agent Observability brings all relevant interaction data into a single, structured view. Admins and supervisors can quickly find the sessions that matter, drill into each conversation turn, and understand the AI Agent's reasoning step by step.
Whether the goal is to investigate a specific error, validate a recent configuration change, or identify training opportunities, Session History provides the context needed to act with confidence.
Key Capabilities
Session History
The Session History view lists recent AI Agent interactions with the key details needed to triage at a glance:
- Contact and Channel: The customer identifier and the channel used (Voice, Webchat, WhatsApp, Digital Connect, API, Autopilot, and more).
- Agent Orchestrator: The orchestrator that handled the interaction (e.g., Account management, Shipping information, FAQs).
- Date/Time & Duration: When the session occurred and how long it lasted.
- End of Automation: The outcome of the session (Completed, Escalated, Timeout, Error, Hangup).
- Error count: A quick indicator of how many errors occurred during the session.
Click Details on any row to open the full session view. Use Export to download a CSV file with all interactions that match the applied filters. Exports are limited to the first 100,000 interactions.
Advanced Filtering
To help navigate high interaction volumes, Session History includes a flexible filter panel. Users can narrow results by:
- When: Last 24 hours, Last 7 days, Last 14 days, Last 30 days, All time, or a custom date range. The default is Last 7 days.
- Channel: Voice, Chat, SMS, and others.
- Agent Orchestrator: Filter by the specific orchestrator(s) that handled the interaction.
- End of Automation: Escalated, Completed, Timeout, Error, Hangup.
- Include simulations: A toggle. When on, sessions created in the AI Agent simulator appear in the list, the insights, and the exports. Off by default.
- Avg Response Time: Shows only sessions whose average response time is above the selected value: "> 3s", "> 5s", or "> 10s".
- Guardrails: Any guardrail hit, or no guardrail hit.
- Guardrail type: One or more guardrail types.
Filters can be combined, cleared individually, or reset all at once.
- Use the search box to find a session by contact or interaction ID.
End of Automation Status
The End of Automation status describes how a session ended.
| Status | Meaning |
| Completed | AI Agent finished handling the session |
| Error | An error ended the session |
| Escalated | Transferred out of automation |
| Timeout | Session ended because the contact stopped responding |
| Hangup | Contact disconnected before the automation ended |
Session Details, Insights, and Errors
Opening a session reveals the full turn-by-turn conversation, along with contextual information such as Interaction ID and Session ID. Two dedicated views help focus the analysis:
- Insights: highlights key moments in the conversation, including the workflows used, inputs and outputs, and notable decisions made by the AI Agent.
- Errors: lists the errors and warnings that occurred during the session to support troubleshooting.
- Latency Summary: shows where response time is spent across the session and in each turn. For details, see “Latency Summary”.
Latency Summary
The Latency Summary tab shows where response time is spent in a session. It appears next to the Transcript tab in Session Details:
- In Session History, select Details on a session.
- Select the Latency Summary tab.
The tab includes the following elements:
- Avg Latency details: The session averages for Response time, STT latency, LLM latency, TTS latency, and Skill execution latency.
- Per-turn breakdown: One horizontal stacked bar per turn, split into STT, LLM, TTS, and Skill execution. A dropdown shows All Turns or Last 5 Turns.
- Session time and turns: Total session time and number of turns.
Note: STT and TTS latency applies to voice turns only.
Session Insights and Guardrails
Session Insights is a panel above the session list. It uses the filters selected in Session History. The Guardrails section in this panel summarizes how often guardrails trigger.
The Guardrails section includes the following elements:
- Guardrail Hits by Type: A ranked bar chart of guardrail types by number of hits.
- Guardrail Hits Over Time: A daily bar chart. Use Group by to show All, By AI Agent, or By version.
When no guardrail triggers, the section shows “No guardrail policies triggered”.
Note: New guardrail types appear in the charts and in the Guardrail type filter as the platform supports them.
Exporting the Guardrails Report
The report contains one row per content-guardrail hit, with the customer’s last message before the hit and how the session ended.
The report has the following columns:
- interaction_id: The interaction identifier.
- ai_agent_orchestration: The AI Agent name and version, for example “Scheduling v2.4”.
- guardrail_type: The type of guardrail that triggered.
- last_customer_message_before_the_hit: The customer message before the guardrail triggered.
- end_of_automation: How the session ended.
- timestamp: The time of the hit.
The report respects the selected date range, AI Agent, and guardrail filters. If more hits match than the report can hold, the dialog shows a message to narrow the date range or filters.
Full Chain of Thought
For a deeper understanding of AI Agent behavior, the Thoughts panel exposes the full Chain of Thought behind each turn. This includes:
- The user message that started the turn.
- The sequence of agent decisions and transitions (e.g., Agent Retail Better Experience → Identify Customer Agent).
- Skill calls executed, with their inputs and outputs.
- Intermediate reasoning produced by the AI Agent.
- The final message returned to the customer.
This transparency helps teams confirm the AI Agent follows expected logic, diagnose unexpected outcomes, and refine configurations precisely.
Impact
Faster Troubleshooting
Errors, latency, and full reasoning are available in one place, reducing the time needed to identify the root cause of an issue and apply a fix. The Latency Summary shows which component slowed a turn.
Better Improvement Decisions
By reviewing real AI Agent interactions end to end, admins and supervisors can identify patterns, validate behavior after configuration changes, and prioritize the improvements that will have the greatest impact.
Operational Transparency
Unified visibility across AI Agent sessions gives admins and supervisors a consistent, auditable view of automated interactions — within the AI Agent Platform during Preview, and in the CXA Operations Center at GA. The guardrails report supports compliance and audit reviews.
Important Information
- AI Agent Observability is available in Preview for select customers.
- During the Preview phase, Session History is accessed inside the AI Agent Platform.
- At General Availability, the experience will move to the CXA Operations Center (formerly AI Trainer).