Skills Quality assurance
Voice QA: grading calls you cannot read
Voice QA looks like text QA with an extra step. It is not. You are grading a transcript, not a call, and the transcription pipeline introduces errors that become grading errors — errors that are not randomly distributed.
npx rulebase-skills install cx-voice-qaWhen to use it
Reach for this when someone says any of these — they are the phrases the skill itself triggers on:
- “QA our calls”
- “score phone conversations”
How it works
The method, in the order the skill runs it. The full procedure — tables, worked examples and the edge cases — is in the skill itself.
The problem that makes voice QA different
Automatic speech recognition and speaker diarisation fail systematically, not randomly. Word error rate is consistently worse for: Accented and non-native speech, Regional dialects and code-switching, Poor audio, Overlapping speech and interruptions, Domain vocabulary, product names, alphanumerics.
Step 1: validate the pipeline before writing any criteria
Build a gold set of 30–50 calls with human transcription, deliberately spanning your real conditions: accents, languages, mobile and landline, quiet and noisy, short and long, and at least a few with heavy crosstalk.
Step 2: choose criteria the transcript can actually support
Anything that can end in an auto-fail must be confirmed against the audio by a human. A "missing" disclosure is at least as likely to be an ASR miss as a real one, and the consequence of getting it wrong is disciplinary or regulatory.
Step 3: adapt the scorecard rather than reusing it
An email or chat scorecard does not transfer. Follow the general design procedure in cx-qa-scorecard-design and then change these.
Step 4: expect AI grading to be worse on voice, and quantify it
If you grade transcripts with a model, the errors compound: transcription error feeds grading error. Follow the AI-grading practice from cx-qa-scorecard-design — evidence quotes, blinding to outcomes, per-criterion agreement — with three voice-specific additions: Report agreement per audio condition, not just per criterion. Model, Never auto-fail on transcript evidence alone.
Step 5: handle the data properly, because voice is worse
Call recordings carry obligations that text does not.
Step 6: report the measurement quality alongside the scores
Every voice QA report should carry: WER overall and by segment, so the reader knows the precision of the, The share of calls excluded for poor transcript quality, Which criteria are transcript-graded vs human-confirmed, Agreement by audio condition, if AI-graded.
Troubleshooting
Some agents consistently score lower — check WER by agent before concluding anything about performance. Accent-correlated WER produces exactly this pattern.
Related skills
Free and open source, and vendor-neutral — it reads the conversations from whichever helpdesk you already run. Browse all 149 skills · connect your helpdesk over MCP · source on GitHub
