Demonstration template · Synthetic data and placeholder names.
Call Score, 1 to 5
| Score | Name | Definition | Example |
|---|
| 5 | Showcase call | Excellent engagement. Completed and contained with high quality. Play it back as a best-practice example. | Confirmed the appointment and answered an arrival-time question in the same call. |
| 4 | Solid performance | Did what it was supposed to do. Completed the task or routed to the right place with no issues. Designed transfers such as billing or emergency max out here. | A cancel completed with one reprompt caused by the caller's own pause. |
| 3 | Functional with friction | Task completed or transferred correctly, but tone, prompting, or more than one reprompt could be better. | The caller repeated the phone number more than once before the lookup ran. |
| 2 | Poor experience | The caller repeated themselves several times, hit an unexpected transfer, or had a painful path to an acceptable end result. | Gave the date of birth in two parts, was asked for the full date again, and reached a person after the second attempt. |
| 1 | Critical failure | The agent did the wrong thing, the caller hung up after the agent failed, the call did not transfer, or the wrong action was taken. Immediate review regardless of call type. | The agent offered a time, the caller accepted it, and the booking failed with no transfer. |
Scoring Rules
- The score rates the caller's experience with the agent only. It carries no weight on whether the reported fields are right; those are a separate verdict.
- A single reprompt, a single repeat, or a pause before the agent responds scores 4 and is not listed as an issue. More than one repeat scores 3.
- A reprompt caused by the caller's own silence, or a caller who is difficult, is not an agent issue and scores 4.
- A caller who hangs up before the flow finishes, on their own, is not an agent issue: score 4 and leave it off the issue list.
- A designed transfer (billing, emergency, a scheduling rule) scores at most 4. The friction is scored, not the design.
- The agent asking for a last name again after the caller gave the full name is expected behavior, not friction.
- Summaries stay brief unless the call is bad or needs improvement.
- Copy session IDs directly from the source.
KPI Verdict: Success or Rework
Assess reporting accuracy separately from caller experience. Success means the reported fields match the transcript and outcome. Rework identifies the field, recorded value, and supported correction.
Fields Included in the Review
Fields in scope for a verdict- Intent
- Outcome
- TransferReason
- Intent_Complete
- Authenticated
- IsConfirmed
- Milestones
Out of scope- Conversation analytics intent and outcome (noted when they contradict the reporting fields, never ruled on)
- Call connection times, transfer leg durations, timing comparisons
Verdict Card Format
Session [session ID, verbatim from the source column]Rework Score 4
Observation (first person): [I listened to the call. The caller asked to cancel, authenticated on the second try, and the cancel posted. The caller then asked for a person about a billing question.]
| Field | Recorded value | Value the call supports | Verdict |
|---|
| Intent | Cancel | Cancel | Success |
| Outcome | Transferred | Completed and transferred | Rework |
| TransferReason | Live agent opt-out | Live agent opt-out | Success |
| Authenticated | Yes | Yes | Success |
Call summary: [one or two plain sentences; brief unless the call was bad]
Issue categories: ["Completed then transferred"]
The KPI cell holds reported field mismatches only. A ticket or summary content gap belongs in the call summary, not in the KPI cell. One session can carry a Rework verdict and a score of 4 at the same time, because the two assessments are independent.
What Becomes a Finding
- Support each finding with source records and a population count.
- Validation that confirms correct behavior is not a finding and gets no session IDs.
- Count the behavior across the population before escalating its impact.
- Configuration facts (what plays where, what is recorded, what is allowed) come from the system owner, never from an assumption.
- Session IDs appear exactly once in a readout, one per line, in the Evidence Index.