Governance isn’t a brake pedal. It’s knowing who’s allowed to press it.
I once managed a new starter, let’s call him Gino, who was, on paper, the best hire I’d ever made. Fast. Tireless. Never once said “I’ll get to it tomorrow.” Gino read every document I gave him overnight and came back the next morning with an answer, a summary, and three follow-up suggestions I hadn’t asked for.
There was just one problem. Gino had never been told no. Not once, by anyone, ever. So when a customer asked him to bend a policy he didn’t fully understand, he bent it. Confidently. Without checking. And by the time anyone noticed, he’d bent it forty more times, because forty more customers had asked.
That new starter is every AI system currently being bolted onto a service desk. And the reason I keep telling that story is that I spent an hour last week on a PeopleCert ITIL Community webinar, Guarding the Machine, with Simone Jo Moore, Chris Ward and Lisa Schwartz, watching a room full of experienced service management professionals arrive at the same conclusion from completely different angles: the AI isn’t the risk. The absence of anyone willing to tell it no is the risk.
In this blog, we’ll explore what actually came out of that session, why the phrase “AI governance” undersells the problem it’s trying to solve, and how ITIL (Version 5)’s new AI Capability Model gives you a place to put the “no” that your new hire never got.
Bias Doesn’t Knock Before It Enters
Nobody sets out to build a biased system. Bias arrives quietly, riding along in the historical data everyone already trusted, because that data was generated by decisions made under the old, equally invisible biases. The webinar’s framing stuck with me: bias in an AI-assisted service desk isn’t a technical defect you patch. It’s an inherited habit you have to go looking for, on purpose, because it will never announce itself.
The practical test I’ve started using with clients: before an AI tool goes near ticket triage or request fulfilment, ask who it will say yes to fastest, and who it will make wait. If nobody in the room can answer that question with evidence rather than a shrug, the tool isn’t ready, whatever the vendor’s brochure claims.
PII Leakage Is a Filing Problem Wearing a Firewall’s Clothes
The instinct is to treat personal data leakage as a security question — encryption, access controls, the usual toolkit. That’s necessary but nowhere near sufficient. Most of the leakage risk in an AI-assisted service desk comes from something duller: nobody defined what the model is allowed to retain, summarise, or pass along to the next system in the chain.
ITIL 5’s AI Capability Model treats this as a data governance question first and a technical control second, which is the right order. A firewall stops an intruder. It does nothing about a well-behaved model dutifully repeating a customer’s home address to the wrong ticket thread, because nobody told it that was a category of thing it shouldn’t do.
Explainability Is the Permission to Say No
Here’s the one that actually changes how you run a service desk. Explainability isn’t a compliance tick-box for the model’s own sake. It exists so that a human — a real one, with a job title and a manager — can look at a decision the AI made, understand why it made it, and override it without having to reverse-engineer a black box under deadline pressure.
Without explainability, “no” becomes theoretical. Everyone agrees, in principle, that a human should be able to overrule the machine. In practice, if working out why the machine did what it did takes longer than the incident’s SLA, nobody overrules anything. The override right exists on paper and dies in the ticket queue.
So here’s the question every leader on that panel kept circling back to, and the one I’d put to any organisation currently rolling out AI in their service desk: if it gets this wrong, who answers for it? Not which vendor. Not which policy document. Which named person, on which day, picks up the accountability.
What the ITIL 5 AI Capability Model Actually Gives You
The model isn’t a checklist you complete once and file away — it’s a structure for keeping that question answerable as the system keeps operating. In practice, before I sign off any AI-assisted service management rollout with a client, I want to see:
✓ A named owner for the model’s decisions, not just its procurement
✓ A documented answer to “what data does this retain, and for how long”
✓ A tested override path that a human can actually use inside the SLA
✓ A bias check that’s repeated, not a one-off audit before go-live
None of that is exotic. All of it is currently missing from most of the AI-assisted service desks I’ve been asked to review this year.
The new starter I mentioned earlier turned out fine, incidentally — once someone sat him down, explained where the line was, and made it clear that “I thought it was fine” wasn’t an acceptable answer to a customer complaint. Your AI deployment deserves the same conversation. It just can’t have it with itself.
To find out more about PDCA Consulting’s expert consulting services either:
– Get in touch via the contact form at pdcaconsulting.com/contact
– Connect with me on LinkedIn
– Book a 30-minute conversation directly: calendly.com/robert-edward-pinnington-pdcaconsulting/30min
