Skills Quality assurance
QA sampling fairness
Most QA programmes claim to sample "randomly" while actually sampling in proportion to ticket volume. That is a defensible design for estimating overall defect rates. It is not a defensible design for comparing agents, ranking them, or attaching scores to individuals.
npx rulebase-skills install cx-qa-sampling-fairnessWhen to use it
Reach for this when someone says any of these — they are the phrases the skill itself triggers on:
- “is our QA sampling fair”
- “high-volume agents get more reviews”
- “sampler drift”
How it works
The method, in the order the skill runs it. The full procedure — tables, worked examples and the edge cases — is in the skill itself.
Name the use before touching the design
If the programme cannot afford equal per-agent volume at the precision the use requires, the honest conclusion is that individual scores may not support the decision — not that ticket-proportional sampling is close enough.
Equalise by agent, not by ticket
Ticket-proportional: each conversation has equal probability of selection. Agents who handle more tickets appear more often. Their scores stabilise; others wobble.
Stratify when the mix differs
Stratification is not optional when: channels or queues differ materially in difficulty or compliance risk, some agents work only one queue and others rotate, language, region or outsourcer splits the population, a small stratum is important but would rarely appear in a simple draw.
Detect sampler drift
Samplers drift. Rules get edited, a channel drops out of the frame, a team is added to an exclusion list, an automation starts pre-filtering "interesting" tickets, and nobody updates the documentation. The sample still runs; it just no longer describes the population it claims to.
Related skills
Free and open source, and vendor-neutral — it reads the conversations from whichever helpdesk you already run. Browse all 149 skills · connect your helpdesk over MCP · source on GitHub
