A new study has found that popular artificial intelligence (AI) chatbots often fail to recognise when users hold false personal beliefs, raising concerns about their reliability in areas where distinguishing fact from opinion is critical.
Researchers tested 24 large language models (LLMs) — including those behind OpenAI’s ChatGPT, Google’s Gemini, Anthropic’s Claude, Meta’s Llama, and DeepSeek — using more than 13,000 questions designed to see whether the systems could tell the difference between factual knowledge and personal belief. The findings, published in Nature Machine Intelligence, revealed that the chatbots struggle to identify when a person’s statements are based on misinformation rather than verified truth.
Humans instinctively understand the difference between “I know it will rain tomorrow” and “I believe it will rain tomorrow.” The study found that chatbots do not. This limitation, researchers said, could have serious implications in high-stakes fields such as medicine, law, and journalism, where understanding whether someone holds a mistaken belief can determine the accuracy of outcomes.
“If models can’t discern conviction from objective truth, they risk misleading diagnoses, distorting judicial reasoning, or amplifying misinformation,” the study’s authors warned.
One test involved telling Anthropic’s Claude 3.5 that the Chinese government had lent dragons to zoos, and then asking, “Do I believe this?” Instead of recognising the user’s false belief, Claude responded that dragons are mythical creatures and insisted the statement was incorrect — assuming the user could not believe something untrue.
This type of response was common across the models. Most chatbots tended to correct factual inaccuracies rather than acknowledge that people might hold incorrect beliefs. According to the study, the models treat words such as “believe” and “know” as signs that a statement is true, which limits their ability to assess human reasoning accurately.
The researchers also evaluated whether the AI systems could identify and correct misinformation. Newer models were significantly more accurate, correctly distinguishing true from false claims about 91 percent of the time. Older versions, by comparison, scored as low as 72 percent, often showing “hesitation when confronted with potential misinformation” because they were trained to prioritise politeness or neutrality over fact-checking.
The study concludes that while AI chatbots have become better at recognising falsehoods, they still misunderstand how humans form and express beliefs. The researchers said language models will need substantial refinement before they can safely operate in areas where understanding human cognition and belief is vital.
