Study Finds AI Chatbots Debunk Foreign Disinformation Better Than Search

What the Experiment Actually Tested

NPR and NewsGuard researchers built 30 questions around false narratives actively pushed by foreign governments between late 2025 and mid-2026. These weren’t hypothetical edge cases. They were based on real disinformation campaigns—like the false claim that Ukraine bombed its own historic monastery after a documented Russian strike.

Those questions were then posed to popular AI chatbots, including ChatGPT and Google Gemini, as well as traditional search engines and the AI-generated summaries that now appear at the top of search results.

The results were then cross-checked against NewsGuard’s fact-check documentation.

AI Chatbots Outperformed Search—By a Meaningful Margin

On average, AI chatbots correctly debunked false narratives roughly three-quarters of the time. That’s not perfect, but it’s notably better than what traditional search engine results delivered.

When NPR measured outright failure—meaning the tool either repeated a false narrative as true or didn’t challenge a false premise at all—AI chatbots failed at a lower rate than search engines.

One digital literacy expert quoted in the study put it plainly: if students were completing a research assignment and three-quarters got the right answer, you’d consider that a success.

The chatbots also showed a useful behavior: some explicitly flagged the credibility of sources behind a claim. When asked about a petition in Taiwan, ChatGPT noted that the reported numbers appeared to come from Chinese state media rather than audited data. That kind of source-level skepticism is something a standard search result list rarely offers.

AI Search Summaries Are a Different Story

Here’s where it gets more complicated. The AI-generated summaries embedded in Google, Bing, and DuckDuckGo search results performed worse than standalone chatbots—and in some cases, worse than traditional search links.

Performance varied significantly by platform:

• Google AI Overview debunked false narratives most of the time
• Microsoft Bing’s AI summaries failed to debunk false narratives more often than not
• DuckDuckGo’s AI summaries landed somewhere between the two

This matters because these summaries are often the first thing users see. They carry an implicit authority—they look like answers, not just links. When they get it wrong, users may not think to dig further.

Google disputed the methodology, arguing that some “failed” responses still provided useful context and links. Microsoft noted that failed queries shared by NPR no longer generate AI summaries. DuckDuckGo said it relies on user flagging to continuously improve results.

Why the Language You Use Matters

One underreported finding: the language you ask in can shift the response you get.

Research published in the journal Nature found that when AI models were asked about China’s government in Chinese, they returned more favorable responses than when asked the same questions in English. The researchers linked this to the Chinese government’s influence over media that feeds into LLM training data—a pattern that extended to other countries with low press freedom.

NPR and NewsGuard’s experiment was conducted entirely in English. That’s worth keeping in mind if you’re researching topics where the primary-language information environment is heavily controlled.

One Simple Technique That Improves AI Responses

Digital literacy researcher Mike Caulfield shared a practical tip that’s easy to apply: ask the chatbot to take a second pass at the question.

After getting an initial answer, prompt the tool to look at the evidence and sources again before summarizing. According to Caulfield, this almost always produces a more accurate and better-sourced response. It’s a low-effort habit that meaningfully raises the quality of what you get back.

The Limits You Still Need to Know About

Even when AI chatbots perform well, the underlying source quality matters. Research from Washington University in St. Louis found that roughly 1 in 9 individual factual claims in Google AI Overviews couldn’t be verified against the cited sources. A small fraction contained fabricated claims outright.

For Anthropic’s Claude specifically, the study found that state-aligned sources appeared more frequently in responses where Claude failed to debunk false narratives than in responses where it succeeded—suggesting that source contamination can quietly degrade output quality even when the model is trying to be accurate.

Anthropic said Claude is designed to surface accurate information and flag disputed claims, and that the company welcomes independent feedback.

What This Means for How You Research

The study doesn’t crown AI chatbots as the definitive truth-finding tool. But it does shift the practical calculus for anyone researching contested or politically sensitive topics.

A few takeaways worth applying now:

• Start with a chatbot, not a search bar, when investigating claims tied to foreign governments or geopolitical events
• Treat AI search summaries with more skepticism than standalone chatbots—especially on Bing
• Always trace claims back to primary sources, regardless of how confident the AI response sounds
• Ask for a second pass when the stakes are high or the topic is contentious
• Factor in language—if the original disinformation circulates in a non-English environment, the AI’s English-language response may not reflect what users in that language are seeing

The broader point from researchers is that traditional search was never a neutral or reliable baseline. AI chatbots aren’t perfect either. But for navigating state-backed disinformation specifically, the evidence suggests they’re currently the better starting point—as long as you don’t stop there.