AI grading AI is not enough.Grade what actually happened.
A transcript can sound perfect while the refund never went through. Scorecards read the conversation and your systems. They check every promise against the result, for AI and human agents alike.
Text-only grading
"The reply said the refund was done. Polite. Clear. 9/10."Pronomy Scorecards
"The reply said refunded. Billing shows no refund. Right actions: failed."Four questions. Every conversation.Always in this order.
Each check uses the right tool. Facts get deterministic checks against your systems. Content gets AI evaluation against your rules. Disputed cases get a human.
Tone of voice
- Sounds like your brand guide
- Acknowledges the problem
- No blame, no template coldness
Right actions
- Promised action → confirmed done
- Required steps in the right order
- Required tags and fields set
Right response
- Correct for this customer's plan and case
- Matches the policy in force
- Nothing invented, nothing missing
Right timing
- First reply inside your SLA
- Follow-ups when they were promised
- Handoffs before it's too late
Before we show you ours,grade this reply yourself.
Read the conversation. Rate the reply from 1 (bad) to 5 (great) on each of the four dimensions. Then see how the scorecard graded it, and what it could see that you couldn't.
Reads fine, right? Rate it before you peek.
Illustrative data. Not a real customer or company. Real scorecards score 0–100 against your own rules; we use 1–5 here to keep it quick.
Break the conversation.Watch the scorecard catch it.
Here's a clean double-charge ticket. Switch on a fault and see what the scorecard sees: the score, the verdict, and the evidence behind it.
The conversation
System state (what the scorecard reads)
Evidence
Graded against: Refund policy v7 · Tone guide v3 · SLA 2 h (rules in force on Oct 21)
Illustrative data. Scores, threshold and weights are examples; you agree yours with us. Not a live system or a real customer.
No generic rubric.Scorecards built from how you work.
From your flows and policies
Each flow brings its own checks. If your policy says it, the scorecard checks it.
- Refunds only inside the 14-day window
- Retention offer shown before a cancel
- Identity check before an email change
- Damaged item → photo requested before replacement
Example checks. Yours come from your own flows.
Plus universal checks
Some things are always wrong, for every company.
- Promised action → action done
- Escalation promised → escalation done
- Required tags set
- No "done" without a confirmed result
Graded against the rules in force that day
Policies change. A conversation from last Tuesday is graded against last Tuesday's rules, not today's. Nobody gets punished for a policy that didn't exist yet.
30-day refund window. A refund on day 20, on Oct 8, passes.
14-day window. The same refund on day 20, on Oct 15, fails Right response.
Next: one shared, versioned rule set that drives both the Agent and the scorecards, so changing a rule updates both at once. Building
Score your people, fairly.Coaching, not catching.
Manual QA reads a small sample and guesses the rest. Scorecards score every conversation and turn the misses into coaching flags with the exact ticket attached.
Start here, before you automate anything
- Connect your helpdesk. Zendesk natively. Others via API or webhook.
- Score your team's conversations. Same four dimensions, same rules.
- See where requests break. Which steps get skipped, which promises go missing.
- Automate what's ready. Or don't. Scorecards are useful on their own.
- ✕Callback promised for Tuesday, never scheduled
Right actions · #PRN-52877open - ✕First reply 3 h 10 m · SLA 2 h
Right timing · #PRN-52901open - ✓Tone flagged as cold, now fixed
Tone of voice · 4 tickets since: cleanclosed
Each flag links to the conversation and the evidence. Team leads see patterns. Agents see their own tickets first, so nobody hears about a miss for the first time in a team meeting.
Illustrative example.
Every score shows its work.And every score can be appealed.
A score with no evidence is just an opinion, and your team already gets plenty of those. Every check links to the message, the timestamp or the system response it used. If someone thinks it's wrong, they appeal. A person reviews it, and the change is logged.
- Evidence on every check, not just a number
- Appeals from agents or team leads
- Human review for disputed cases
- Full history: who changed what, and why
- ✕Follow-up late: 26 h after last message
Evidence: timestamps 10:02 → 12:07 next dayauto - →Appealed by team lead
"Customer asked us to write back tomorrow at noon."+2 h - ✓Human review: appeal upheld
Customer message at 10:01 confirms it+1 d - ✓Score updated · 40 → 100
Change logged with reasondone
Illustrative example.
Not an add-on. Not per seat.Grading shouldn't cost more than your team.
extra on AI conversations in Scale
Scorecards on AI conversations are included in the Scale plan. A failed scorecard also means that resolution isn't billed.
convo
for human-agent QA
Priced per scored conversation, not per seat. We set the price with you on a call. Talk to us.
For comparison, Fin lists "Custom AI Scorecards" in a Pro add-on from $100 per month.1
1. Fin pricing page, checked October 2026. Features and limits differ by plan; check their page for your case. Details: vs-fin.
Scorecard questions.No marketing answers.
Isn't this just an LLM grading another LLM?
Do we need the AI Agent to use Scorecards?
What if we disagree with a score?
Which helpdesks does it work with?
How do we get set up?
Your team keeps a lot of small promises every week, and almost nobody sees them kept. Bring a few of your own tickets to a call and we’ll grade them together. If we can’t find a single broken promise, I owe you a coffee.