Chapters

Chapter 10

Operating and improving

Use what happens in production to decide what to fix next.

Conversations

Conversations in the bot menu lists recent conversations with their version, channel, time, and outcome. Filter by outcome, and select a conversation to see the trace of the steps it ran. Sensitive values are removed from the trace.

Recent conversations for a bot.
Figure 28. Recent conversations for a bot.

Analytics

Analytics shows the number of conversations, how many completed, the completion rate, how many failed, and how many unanswered questions are open, with daily volume for the last 30 days.

Conversation analytics.
Figure 29. Conversation analytics.

Untrained queries

A question appears under Untrained queries when the knowledge search finds nothing, when an answer could not be verified against the sources, when an API returned no data, or when a reply was not understood.

Untrained queries, with no open questions.
Figure 30. Untrained queries, with no open questions.

Observability

Administrators open Administration → Observability for the health of the whole workspace. The page refreshes every 30 seconds.

Observability. In this example, event delivery needs attention.
Figure 31. Observability. In this example, event delivery needs attention.
Note

Unanswered questions and outages are kept apart. A question the bot could not answer is a training gap and goes to Untrained queries. A failed API call, model error, or timeout is an outage and goes to Observability.

Audit log

Each bot keeps an Audit log of who published, resolved, tagged, or changed something, and when. It is visible to Administrators and Security Admins, and it records every role change.

The audit log for a bot.
Figure 32. The audit log for a bot.