How to spot the mistakes AI makes
AI writes fluently and is confidently wrong. The specific failure modes that matter in care records, and how to read a draft so you catch them.
The problem with AI mistakes is not that they are frequent. It is that they look exactly like the correct answer.
CQC’s own list of AI risks puts it plainly: AI “can hallucinate and present untrue information, authoritatively as fact”. It adds that AI mistakes “can be difficult to spot, sometimes only after people experience harm when using a service”.
That second sentence is the one to sit with. The errors that matter are the ones nobody catches.
Why fluency makes it worse
When a colleague writes something in a rush, the writing itself signals the rush. Short sentences, gaps, a note that says “check this”. You read it differently because it looks provisional.
AI output never looks provisional. A completely invented medication dose arrives in the same calm, well-organised prose as an accurate one. There is no tonal difference between a sentence the model is sure about and one it produced to fill a gap, because the model does not have a sense of being sure.
So the usual reading instinct — this looks fine — is exactly the instinct that fails.
The five failure modes that matter in care
1. The tidy summary that drops the odd detail
This is the most common, the least dramatic, and the most dangerous in care work.
A care worker mentions six routine things and one unusual one: she seemed a bit confused about what day it was. The AI produces a clean summary organised under personal care, nutrition and mood. The confusion does not appear, because it did not fit a heading and did not look important next to everything else.
Nothing was invented. Nothing is wrong. The record is worse than useless, because it reads like a complete account.
How to catch it. Read the draft against what you actually said or heard, not on its own. Ask: what was unusual today, and is it in here? Summaries lose outliers, and in care the outlier is usually the point.
2. Confident invention of specifics
Names, dates, dosages, times. Where the source was unclear, a model will often produce a plausible specific rather than leave a gap.
How to catch it. Treat every number and every proper noun as unverified. Medicines, doses, times, NHS numbers and names get checked against the source every time. If you cannot check it, it does not go in the record.
3. Mishearing that produces a real word
Transcription does not fail by producing gibberish; it fails by producing something plausible. Medication names are the worst case, because many sound alike and all of them are real words to the model.
Local place names, street names and unusual surnames fail the same way.
How to catch it. Read medication names aloud against the MAR chart. Do not skim them — they are the exact thing that looks right and is not.
4. Smoothing that changes clinical meaning
“She said her hip hurt again” becomes “reported ongoing hip discomfort”. “He refused his medication and shouted” becomes “he declined his medication”.
Each rewrite is defensible. Together they move a record away from what happened and towards what sounds professional. Over weeks, a pattern of escalation can be smoothed into a pattern of mild events — and the record is what a safeguarding review will read.
How to catch it. Where the original language was strong, keep the original language. Plainer is not always more accurate, and in incident and safeguarding records accuracy beats tone every time.
5. Filling your template rather than reporting reality
Give a model headings and it will fill them. Give it a heading for something that did not happen, and it will often produce something anyway.
How to catch it. Empty headings should stay empty. If a section says nothing happened, that is a finding, not a gap to be filled.
How to actually read a draft
Not “read it carefully”. That is advice nobody can follow at the end of a shift. Three specific passes, each fast:
- Facts pass. Numbers, names, medicines, times. Check against source. Nothing else.
- Missing pass. What do I remember that is not here? Especially anything unusual.
- Meaning pass. Does this say what happened, or a politer version of it?
Most drafts take under a minute. The ones that take longer are the ones that were going to cause a problem.
Budget the checking time honestly
If a care worker saves six minutes and spends four checking, the saving is two minutes, not six. That is still a saving. But a service that assumed six will conclude AI failed, when what actually failed was the estimate.
Measure it rather than guessing. See measuring a baseline and our admin time tracker.
When to stop using it for a task
Some honest thresholds:
- If more than about one draft in five needs a substantive correction, the tool is making work rather than saving it for that task.
- If staff have started approving drafts without reading them, stop. That is worse than not having the tool, because the record now carries an implied check that did not happen.
- If you cannot tell whether it is accurate — because nobody is sampling — you do not have a tool, you have an unmeasured risk. CQC asks for “monitoring outputs” under Regulation 17 for exactly this reason.
What this does not cover
Everything above assumes a human reads the draft before it is used. That is rule three of the golden rules, and nothing on this page is a substitute for it. These are techniques for reading better, not for reading less.