Reports are the only objective source of information on how the chatbot is performing for your shop. Without reviewing them regularly, you won’t know what’s working and what needs improving. In this article, we explain what each metric means and what actions you should take based on it.
Where can you find the reports?
Go to: Reports → AI Agent
Metrics and what to do with them
Number of conversations
How many times customers initiated a chat conversation during a given period.
What this means:
A low number of conversations with high website traffic = the chat is not very visible or the welcome bubbles do not encourage clicks
A rising number of conversations = a good sign; the widget is being noticed
What to do:
If there are few conversions – go back to the widget settings, check auto-open and the content of the welcome bubbles
Compare desktop vs. mobile results to see where there is room for improvement
Number of messages
Average number of messages per conversation.
What this means:
Very short conversations (1–2 messages) may indicate that the chat isn’t providing satisfactory answers and the customer is giving up
Longer conversations (5+) are a sign that the customer is engaged and actively looking for a product
What to do:
Review the transcripts of short conversations in the Inbox – check which questions the chat ‘fails’ on
Add the missing information to the knowledge base
Clicks on product recommendations
How many times customers clicked on a product suggested by the chatbot.
What this means:
This is a key indicator of the accuracy of recommendations
Low click-through rate = the chatbot recommends products that do not meet customers’ needs
What to do:
Enrich product descriptions in the feed – particularly attributes (purpose, size, material)
Add guides to the knowledge base to help match products to needs
Escalations to a human
How many times a customer has requested to speak to a consultant.
What this means:
A high number of escalations regarding after-sales issues (returns, complaints) = it is worth adding this information to the knowledge base
Escalations regarding product-related issues = the chatbot is struggling with this category
What to do:
Check which conversations result in escalations
Add the relevant documents to the knowledge base
Generated orders and revenue
Orders and revenue that can be attributed to chat conversations.
What this means:
A direct ROI metric from the tool
Allows you to compare the effectiveness of chat with other channels
What to do:
Track the trend month-on-month
If conversions are low despite a high number of conversations – check whether the chat is offering products available in stock
How often should you check the reports?
Stage | Frequency |
First 4 weeks after launch | Every 2–3 days |
Stable operation | Once a week |
Campaign periods (Black Friday, Christmas) | Daily |
Signs that something needs improving
Conversion rates are falling despite steady traffic → check the widget’s visibility
Short conversations, few clicks → update the knowledge base
Many escalations → add answers to the most frequently asked questions to the knowledge base
No orders being generated → check the product feed