AI Metrics in Oversai allow you to transform qualitative data into quantifiable insights. With AI, you can automatically evaluate key elements of all your interactions — from soft skills like grammar and greetings to behaviors like reply-to-spam detection or whether agents are following scripts. This helps you measure the performance of your operation, teams, and agents, and take action based on real data to keep improving.
Step-by-Step: How to Create AI Metrics
1. Navigate to the AI Metrics Section
Go to the Logic section in the left-hand menu and select AI Metrics. Then click on “Create New AI Metric” in the top-right corner.
2. Configure Your AI Metric
Give your metric a name, assign relevant filters, choose a status, and select a type.
Filters.
Filters are key to ensuring your AI Metric applies only to relevant interactions. For example, you might want to evaluate only tickets that are marked as "closed" and have a "high priority."
Filters must be pre-built in the Filters section of the platform. [Here’s a guide to help you create filters].
Type
There are two types of AI Metrics in Oversai:
Yes/No – Use this if you want a binary evaluation (e.g., Did the agent greet the customer? Yes or No).
Range – Use this if you want to score something on a scale (e.g., Rate the grammar from 1 to 5).
3. Add a Prompt or Guideline
Your prompt is what guides the AI to evaluate the interaction. Think of the AI as a new team member: it needs clear instructions.
Best practices:
Keep prompts simple and focused.
Only ask one question per prompt to avoid confusion.
Example: ✅ “Did the agent use a greeting in the first message?”
Avoid: ❌ “Did the agent greet the user, use a friendly tone, and ask X, Y, Z?”
👉 [Check out our best practices article for writing great prompts.]
4. Save Your AI Metric
Once you’ve configured the details and added your prompt, click Save. Your AI Metric is now ready to be tested!
Step 5: AI Metrics Playground – Test & Calibrate
This is the most exciting part! Use the AI Metric Playground to test and fine-tune your AI Metric before going live.
When you're in the main AI Metrics menu where all your metrics are listed, click the AI Metrics Playground button in the top right corner.
On the left side, select which AI Metrics you want to test.
Choose the integration and enter the interaction ID you’d like to evaluate.
Click Validate.
Oversai will process the interaction and show:
The interaction itself (for context).
The AI’s answer based on your prompt.
Take this time to review if the results align with your expectations. If needed, you can edit the prompt and re-test.
Final Step: Go Live 🚀
Once you're happy with the test results, reach out to your Account Manager to activate the AI Metric and start running it automatically on daily interactions.
Enjoy accurate, automated QA insights — powered by AI and visible directly in your Oversai reports!
Still need help?
Contact us