Why Does AI Sometimes Get Things Wrong?
AI can be remarkably useful.
It can explain complicated subjects, answer questions, help write letters and find connections between information in seconds.
But there's something important to understand:
AI can be wrong.
More importantly, it can be wrong while sounding completely confident.
It doesn't always know when it doesn't know
If you ask another person a difficult question, they might say:
"I don't know."
AI doesn't always do that.
Sometimes, when it doesn't have a reliable answer, it produces an answer that looks right instead.
It might give you the wrong date, confuse two people, invent a book that doesn't exist or provide a convincing explanation for something that never happened.
The answer may be beautifully written.
It can still be completely wrong.
Why does this happen?
An AI chatbot doesn't work like an encyclopaedia containing millions of stored answers. If you're new to how AI systems actually work, it helps to understand the fundamentals—how AI learns patterns from data is the foundation of everything it does.
It has learnt patterns and relationships from enormous amounts of information and uses those patterns to generate a response to your question.
At a very simple level, it is working out what should come next.
Usually, those patterns allow it to produce useful and accurate answers.
But sometimes they lead it in the wrong direction.
AI can produce something that sounds like the right answer without actually being the right answer.
Why AI Sometimes Makes Things Up: Hallucinations
You'll sometimes hear people say that an AI has "hallucinated."
It doesn't mean the AI is actually seeing or experiencing something. The word "hallucination" is just the term we use in AI to describe when a system confidently invents false information.
Think of it like this: A person who hallucinates is experiencing something that isn't real. When AI "hallucinates," it's generating false information that sounds convincing. It's not the same thing at all, but the term has stuck.
When AI generates false or unsupported information as though it were real, it might invent:
- a quotation somebody never said
- a book that was never published
- a court case that doesn't exist
- a website address that leads nowhere
- a feature that a product doesn't have
- statistics that sound plausible but aren't real
That's why an impressive-looking answer isn't necessarily a correct one. The solution isn't to distrust everything AI says—it's to check important answers before acting on them.
Confidence doesn't mean accuracy
This is one of the biggest differences between talking to AI and talking to another person.
People often give clues when they're uncertain:
"I think…"
"I'm not completely sure, but…"
"You'd better check this…"
AI can do this too, but it doesn't always accurately recognise when it should.
A completely wrong answer might therefore be delivered in exactly the same confident style as a correct one.
Don't use confidence as your test of whether an AI answer is true.
Does AI always get things wrong?
No.
AI can provide accurate and extremely useful information. While you can't break AI by asking the wrong thing—in fact, AI is designed to handle messy, unclear questions—that doesn't mean you should assume all answers are correct.
The sensible approach isn't to distrust everything it says. It's to consider what would happen if the answer were wrong.
If you ask:
"Give me some ideas for using leftover potatoes."
you probably don't need to conduct extensive research into every suggestion.
If you ask:
"Can I safely take these two medicines together?"
that's very different.
The consequences of an incorrect answer could be serious.
The more important the answer, the more important the check
A useful rule is:
The greater the consequences of getting something wrong, the more carefully you should check the answer.
Be particularly cautious with subjects such as:
- health and medicines
- money and investments
- legal matters
- safety
- important work decisions
- information about other people
For important matters, check reliable sources or speak to an appropriate professional rather than relying solely on an AI response.
Ask where the information came from
You can ask an AI:
"What sources support this?"
Or:
"Can you find a reliable source for that information?"
But there's another important catch.
AI can sometimes get its sources wrong too.
So don't simply assume that a reference is genuine because the AI supplied it. Open the source yourself and check that it exists and actually says what the AI claims it says.
Give it permission to say "I don't know"
You can also change the way you ask questions.
Try adding:
"If you're not sure, say so rather than guessing."
Or:
"Tell me which parts of your answer you're uncertain about."
This doesn't guarantee an accurate answer, but it encourages the AI to distinguish between what it can support and what may be uncertain.
AI is a tool, not an authority
Perhaps the easiest mistake to make with AI is assuming that because it can answer almost any question, it must know the answer to almost any question.
Those aren't the same thing.
AI can be extraordinarily useful for explaining, exploring, drafting, comparing and helping you think.
But don't hand over your judgement to it.
Use AI for answers. Use your judgement to decide which answers need checking.
