Model cost management

When to use it

Reach for this when someone says any of these — they are the phrases the skill itself triggers on:

• “LLM cost per ticket”
• “why is our bot so expensive”
• “token usage breakdown”
• “reduce AI support costs”
• “cost vs quality tradeoff”

How it works

The method, in the order the skill runs it. The full procedure — tables, worked examples and the edge cases — is in the skill itself.

1. Define cost per resolution honestly

   Pick a resolution definition and stick to it: Bot-contained conversation, Any bot participation, Issue resolved.

2. Where tokens actually go

   Decompose a sample of conversations (or all if logging allows).

3. Cost vs quality frontier

   Optimisation is not "cheapest model." Map points on your own frontier: Fix quality bar (eval set or production sample grade), Vary one lever, Measure quality delta and cost delta on the same cases.

4. Practical reductions (after measurement)

   Only after attribution: Trim system prompt, Cap history, Right-size retrieval, Stop double model calls, Cache stable context, Fix retry storms.

5. Monitoring

   Track weekly: Total spend, conversations, resolutions, cost per resolution, Tool iterations p50 / p95, Spend by model version (detect accidental prod on flagship tier).

Related skills

• AI agent evaluationTest an AI agent before shipping a prompt or model change
• Bot safety auditFind where a bot did harm, and where it was pushed to
• Trusting an AI graderCheck an AI grader agrees with your humans before trusting it
• Copilot evaluationMeasure whether the copilot actually helps agents
• Limits on AI actionsLimit what an AI agent may do to a customer’s record
• What the bot can findFix what the bot retrieves before blaming what it says