Analyze message components powered by Navigator can understand customer messages and automate customized content-based decisions on voice and digital channels. These components work by performing detection of the conversation topic by employing Large Language Models (LLMs).
The following best practices are suggestions to improve the results of those topic collections.
- Simplify Topic Names
- Avoid Repetition between Topics
- Avoid Overlapping Sample Contact Messages
- Limit the Number of Sample Contact Messages
- Split Multiple Similar Topics into Different Components
- Test Changes Systematically
Simplify Topic Names
Topics should be descriptive, intuitive, and aligned with customer intents. Avoid using acronyms that will not be known to the system. E.g., “Order Status,” “Technical Support”, “Customer Sales Support”.
Avoid Repetition between Topics
If different topics are similar or have a lot of overlap in their semantic meaning, this will decrease the topic classification accuracy. To improve Navigator's classification, you should avoid these repetitions by splitting similar topics into different components.
Avoid Overlapping Sample Contact Messages
If different topics have the same Sample contact message or similar keywords on their Sample contact message, this will decrease the topic classification accuracy. We recommend that you avoid these repetitions to improve Navigator's classification.
Limit the Number of Sample Contact Messages
Since Navigator is powered by LLM there isn’t a need to provide all the variations of the customer request on the sample contact message (e.g.: “place order”, “place an order”, “place orders”). If there is a need to describe multiple different requests, please use a sentence structure such as: “When a customer requests about X, Y or Z”.
Split Multiple Similar Topics into Different Components
When multiple similar topics are grouped within a single component, there is a higher chance of misclassifying customer requests. To enhance effectiveness, we recommend using multiple consecutive components. The first component should identify the main/generic topic, and the subsequent component will identify the specific topic.
Example: Identify specific topics around orders in a retail scenario:
- The first Analyze message component will have “orders” as a generic topic, and the second Analyze message connected to the “orders” topic/exit will categorize the different subtopics (e.g. “Customer orders”, “Renew order”, “Cancel order” or “Track order”).
- The second component will use the customer's contact message from the first component as an input variable, eliminating the need to request additional information from the customer.
Test Changes Systematically
Improving performance is easier if you can measure it. In some cases, a modification to a field will achieve better performance on a few isolated examples but lead to worse overall performance on a more representative set of examples. We recommend that you test the changes with a sizable subset of examples.