A new investigation by the BBC has revealed that leading artificial intelligence (AI) chatbots frequently misrepresent and distort news stories, raising concerns over the accuracy and reliability of AI-generated content.
The research evaluated four major AI models—OpenAI’s ChatGPT, Microsoft’s Copilot, Google’s Gemini, and Perplexity AI—by feeding them content from the BBC website and asking them questions about the news. The findings showed that more than half of the AI-generated responses contained significant inaccuracies.
AI ‘Playing with Fire’ in News Summarization
Deborah Turness, CEO of BBC News and Current Affairs, warned in a blog post that while AI presents “endless opportunities,” it also carries serious risks.
“How long will it be before an AI-distorted headline causes significant real-world harm?” she asked, calling on AI companies to “pull back” their news summaries.
Key Findings from the BBC Study
The BBC’s investigation involved 100 news stories, with journalists—who were experts in the respective fields—fact-checking the AI-generated summaries. The results were concerning:
- 51% of AI responses contained significant issues
- 19% of responses that cited BBC content contained factual errors, including incorrect dates, numbers, and statements
- AI models failed to differentiate between fact and opinion, often editorializing content instead of summarizing it accurately
Specific inaccuracies included:
- Google’s Gemini falsely stating that the NHS does not recommend vaping as a smoking cessation aid
- ChatGPT and Copilot incorrectly claiming that Rishi Sunak and Nicola Sturgeon were still in office after they had left
- Perplexity misquoting BBC News in a story about the Middle East, incorrectly stating that Iran initially showed “restraint” while describing Israel’s actions as “aggressive”
Among the four AI models, the study found that Microsoft’s Copilot and Google’s Gemini produced the most significant inaccuracies, while OpenAI’s ChatGPT and Perplexity performed slightly better.
BBC Calls for AI Transparency and Control
In response to the findings, Pete Archer, the BBC’s Programme Director for Generative AI, emphasized that publishers should have control over how their content is used.
He urged AI companies to provide transparency on how their models process and summarize news while also revealing the scale and frequency of errors in their outputs.
Although the BBC typically blocks AI bots from accessing its content, it temporarily lifted these restrictions for the study in December 2024.
Turness stressed the need for a collaborative effort between news organizations and AI developers to find solutions, citing Apple’s decision to pull back its AI-generated news summaries after the BBC raised similar concerns.
As AI-powered news summaries become increasingly prevalent, the report highlights the potential dangers of misinformation and distortion, reinforcing the urgent need for greater oversight and responsible AI development in journalism.
