QA – Scorecards Scoring Guide

📊 Instructions for Using the Scorecard in Oversai

The Scorecard is an evaluation tool that allows you to assess agent performance based on a defined score goal.

• If the evaluated person meets or exceeds this goal → they are marked as “Passed” ✅

• If they don’t reach it → they are marked as “Failed” ❌

🧩 Scorecard Structure

1. Sections and Questions

• The Scorecard is organized into sections.

• Each section is broken down into evaluation questions.

2. Section Weights

• Each section has a percentage weight.

• The total of all section weights must add up to 100%.

• The weight of each section is defined manually.

3. Questions and Scoring

• Each question has a maximum possible score (the value of the highest-scoring option).

• The maximum score of a section is the sum of the maximum scores of its questions.

4. Question Option Types

• default → Adds only the indicated points.

• fail_section → If selected, the entire section is scored with 0 points, regardless of the other answers.

• fail_all → If selected, the overall Scorecard score is automatically set to 0.

• N/A (Not Applicable) → Excludes the question from the calculation (removes its points from the total available, adjusting the final score proportionally).

🧮 Final Score Calculation

The final score is obtained by summing the results of each section, weighted by its percentage.

🔹Example 1: Step-by-step

1. A section has a maximum of 10 points.

2. The agent earns 5 points.

   Section calculation:

   (5/10) *100 = 50%

3. If that section is weighted at 40%, then:

   50% * 40 = 20%

4. Repeat the calculation for each section.

5. The final Scorecard score is the sum of all section results.
   

🔹Example 2: Step-by-step (With some N/A within a section)

1. A section has a maximum of 20 points, distributed across 2 questions (10 points each).

2. One question is marked as N/A, and the other is answered correctly.

Adjusted calculation:

• Question 1: N/A → excluded from the calculation

• Question 2: 10 / 10 points

Adjusted total possible points: 10

3. Section calculation:
   (10/10) * 100 = 100%
   

4. If that section is weighted at 25%, then:
   100% * 25 = 25%

5. This means the section contributes 25 points to the final Scorecard score.

Important:
When a question is marked as N/A, its points are removed from the total available points, and the section score is recalculated proportionally using only the applicable questions.

🔹Example 3: Step-by-step (With all questions marked as N/A within a section)

1. A section has a maximum of 20 points, distributed across 2 questions (10 points each).

2. All questions in this section are marked as N/A.
   

Adjusted calculation:

Question 1: N/A → excluded from the calculation
Question 2: N/A → excluded from the calculation

Adjusted total possible points: 0

3. Important - Section calculation:

Since all questions were marked as N/A, the section score is considered 100%.

If that section is weighted at 25%, then:

100% × 25 = 25%

🏆 Example Outcome

• Agent’s final score: 90%

• Defined score goal: 80%

• Since the agent exceeds the goal → Result: “Passed” ✅

✨ Benefits of Using Scorecards in Oversai

• Objective and consistent evaluations.

• Flexibility to adapt the structure and weights according to your policies.

• Ability to include critical criteria (fail_section, fail_all).

• Clear visibility of agent performance with simple, transparent metrics.