Skills Quality assurance
Prompt change management
Prompts are treated like copy someone edits in a dashboard: no diff, no rollback, no record of who changed what. Then a "small tweak" breaks deferral behaviour in Dutch and the first signal is a spike in complaints.
npx rulebase-skills install cx-prompt-change-managementWhen to use it
Reach for this when someone says any of these — they are the phrases the skill itself triggers on:
- “how do we deploy prompt changes”
- “prompt rollback”
- “version our system instructions”
- “CI for prompts”
- “staging before promoting the bot prompt”
How it works
The method, in the order the skill runs it. The full procedure — tables, worked examples and the edge cases — is in the skill itself.
Prompt as config-as-code
Store system instructions, tool descriptions, and grader prompts in version control, not only in a vendor UI: One canonical source per environment (or export that syncs from git), Semantic or dated versions, Pull requests for changes.
What counts as a prompt change
Treat all of these as deploys requiring the same gate: System / developer instructions, Few-shot examples embedded in the prompt, Tool-use descriptions and parameter schemas exposed to the model, Grader or classifier prompts downstream of the agent, Retrieval query templates and reranker instructions.
Staging environments
Minimum viable pipeline.
Eval gate before promote
No promote without: Fixed eval set frozen and versioned (see agent evaluation practice), Per-dimension grades, Case diff against previous prompt version on the same model and KB pin, Grader validation if an automated grader gates the run.
Rollback path
Before every promote: Know time-to-rollback on your platform, Define who can rollback without a full change window.
Changelog
Every promoted version gets a changelog entry.
Non-determinism and canary
Same input can yield different outputs. For borderline promotes: Run critical cases multiple times in staging; document variance, Canary in production.
Related skills
Free and open source, and vendor-neutral — it reads the conversations from whichever helpdesk you already run. Browse all 149 skills · connect your helpdesk over MCP · source on GitHub
