Back to blog

AI agents

Containment counts a customer who gave up as a win. Score AI like your people

Your people are scored on quality and your AI on containment, so the riskiest queue looks safest to automate. One rubric for both, handover included.

Two reports land on the same desk on the same Monday. The first is the contact centre’s quality scorecard: the human agents are averaging 84 out of 100 on the rubric, with the usual notes about disclosures missed and empathy under pressure. The second is the AI agent’s performance summary: 78% containment, 61% of inbound volume handled, average handle time a third of the people’s. The operations director has a decision to make off the back of them. The payment-difficulty queue still routes to people, the AI is cheaper and contains well, so the obvious move is to send more of that queue to the bot.

The two numbers in front of her don’t share a unit. One says how well her people handled customers. The other says how often the AI kept customers away from her people. She is about to route some of her most delicate conversations, customers who can’t make a payment, on the strength of a comparison that isn’t one.

One quality bar means scoring every conversation on the same rubric, whoever handled it, human or AI. Most operations judge their people on a quality scorecard and their AI on containment, and two yardsticks can’t be compared, even on one dashboard. The bar is a single rubric applied to the conversation itself: the same criteria, the same evidence behind each score, the same right to dispute and the same causal re-measurement against a baseline, across the whole workforce.

A shared dashboard can still hold two standards

Workforce platforms now sell one console for the hybrid workforce, people and AI side by side. The consolidation is worth having: quality data, performance data and AI operations have lived in tools that don’t talk to each other.

But putting the human scorecard and the AI performance summary on one screen doesn’t make them the same measurement. You can now see two incompatible numbers without switching tabs. The standard sits underneath the screen: what you measure, against which criteria, with what evidence. That is where many hybrid operations still run two separate systems, long after the dashboard was unified.

The two yardsticks measure different things

Look at what each half of the workforce is scored on. Your people are scored on a quality rubric. Did they verify identity, give the disclosure, acknowledge the customer’s problem before solving it, hold steady when the call got difficult. Your AI agent is scored on a different vocabulary altogether: containment, deflection, automation rate, intent-recognition accuracy, fallback rate. These are operational metrics, and most of them come down to one question. Did the AI finish the conversation without passing it to a person?

That question measures avoidance: how often the customer didn’t reach a person. Containment can’t separate “the AI solved the problem” from “the customer gave up trying to reach a human.” A contained conversation counts as a win whether the customer rang off satisfied or furious. The satisfaction score won’t catch the difference either: an aggregate customer-satisfaction number can sit still while the experience on contained calls gets worse, because the few contained customers who left unhappy are diluted by the many the AI never needed to help.

So “78% containment” tells the director almost nothing about whether the conversations went well. She has a quality number for her people and a volume number for her AI, and she treats them as comparable because they share a screen.

Two yardsticks push the automation decision onto cost

This bites hardest when it’s most expensive to get wrong: deciding which conversations to automate and which to keep human. That decision is the hybrid strategy, and a wrong call puts your hardest conversations in front of the agent least equipped to handle them.

You can only make that call well if both sides are measured on the same thing: how well the customer was served. With two yardsticks you fall back on the one number that is comparable, which is cost. The bot is cheaper per contact and it contains well, so the queue moves.

Watch that play out on the payment-difficulty queue. It automates cleanly. Containment climbs, cost per contact drops and the dashboard turns green. What the containment number can’t show is the customer in financial distress who asked a question, got a half-answer, didn’t understand it, and rang off: contained, logged as resolved, and failed. On a quality rubric that call is an obvious miss. On containment it’s a success.

Illustration: the AI takes 1,000 payment-difficulty calls in a month and contains 780 of them. Score those 780 on the rubric your people are held to, and say 120 end with the customer’s question unanswered.

One month, payment-difficulty queueContainmentYour rubric
Calls the AI handled1,0001,000
Counted as a success780 (78%)660 (66%)
Customers failed but logged as resolved0120

The queue that most needed care is the one containment made look safest to automate.

One rubric, run on every conversation

One quality bar scores the conversation, whoever handled it. The rubric is the one you would hold a person to, observable behaviours written so the score is consistent and traceable, and it runs on every conversation on both sides.

Many operations sample their humans and don’t quality-score their AI at all. The half of the workforce growing fastest is measured least, and on the one metric that hides its failures. Put both on the same rubric and you see that half for the first time.

The mechanism is closed-loop quality assurance: score on one rubric, fix the gap, then re-measure causally against a baseline of comparable agents who didn’t get the fix. Only who it runs on has changed, now that the word “agent” includes the software ones. For the operations director, that means one comparable measure of how well customers were served, so the Monday decision turns on quality.

Nobody scores the handover

Two separate systems never measure one part of the hybrid workforce: the handover. Many conversations start with the AI and finish with a person. The customer experiences one conversation. The two systems see two fragments: the AI’s leg, scored on whether it contained (so the handover itself is logged as a failure), and the human’s leg, scored on the rubric from the moment they picked it up. Nobody scores the seam between them.

That blind spot has a cost. A customer who has explained a bereavement or a missed payment to the AI, then has to explain it again from the top to the person it transfers them to, has had one badly handled conversation, even if each system scores its own half well. For an insurer or a bank, the handover is the worst place to fumble a call.

Score the whole conversation, end to end, and the handover becomes something you can see and improve. Our recommendation for a first pass: flag every conversation, human or AI, where

  1. the required disclosure is missing;
  2. the customer had to ask the same question twice, or rang off with it unanswered;
  3. the customer had to repeat to a person what they had already told the AI;
  4. the customer asked for a person and the AI kept going, because escalating counts against its containment;
  5. the customer mentioned a missed payment, illness or a death, and the conversation carried on as normal.

“But you sell the agents too”

There’s an obvious objection to a company that sells AI agents and also sells the bar those agents are measured against. Future Ready is that company, so the objection lands on us. If we grade the agents we sold you, “they score well” is worth roughly nothing on its own.

We aren’t a neutral referee, and because we sell agents we don’t pitch this as vendor-agnostic oversight. The bar only means something if it belongs to you: scored on your rubric, every score tied to the transcript moment that earned it, every score open to challenge, and any improvement measured causally against a baseline. Hold any vendor’s agents to that, ours included. The four questions to put to anyone selling you an AI agent apply to us as hard as to anyone. If a vendor can’t show you a single automated conversation scored against your own criteria, all you have is their word.

One workforce, or two that share a queue

An operations leader who runs people on a quality rubric and AI on a containment rate has two operations that happen to share a queue, judged by numbers never built to be compared. Another dashboard moves the two numbers closer together on the screen. One rubric underneath both makes every conversation answer the same question: was this customer well served, and how do we know?

Future Ready was built around this loop, and our customers see an average 3.7x return on it. It runs on the whole workforce: the same rubric that holds your best human agents to a standard holds the AI ones there too.

Which of your automated queues would pass on containment and fail on your own rubric?

Want to see your whole workforce on one bar? We’ll score a week of your real conversations, human and AI, on your own rubric and show you both halves on the same measure, queue by queue: where each is strong and where each needs work.