Benchmarking Online AI-and-Cancer Content: Only 33% Meets High-Quality Standards

The Study at a Glance

The research team, led by Internal Medicine Resident Pearl Subramanian, MD, began with the first 320 results from Google and YouTube searches using common cancer- and AI-related keywords. The content pool was deliberately broad: news articles, hospital websites, government agency resources, medical society publications, and influencer videos were all included.

After filtering out academic papers and content not intended for lay audiences, 52 webpages and 29 videos remained for analysis. Validated consumer health information tools were applied to assess readability and overall quality. Researchers also tracked whether content addressed critical AI safety concepts — specifically, the risks patients face as direct users of AI tools.

The final numbers are stark. Only 17 webpages (33%) and 7 videos (23%) qualified as high quality. The median readability of webpages sat at college level, well above the 6th–8th grade standard recommended by both the American Medical Association and the National Institutes of Health for consumer health information.

Readability as a Barrier

Reading level is not a cosmetic concern. When a patient navigating a cancer diagnosis encounters content written at college level, comprehension drops — and so does the likelihood that safety information will be absorbed and acted upon. The AMA and NIH benchmarks exist precisely because health decisions made under stress require plain language. The current content landscape fails that standard by a wide margin.

AI Hallucinations: Almost Invisible in Patient Resources

Only 15% of the analyzed webpages mentioned the risk of AI hallucinations. This is a critical omission. A patient who asks a chatbot whether a treatment side effect is normal may receive an accurate, well-sourced answer — or they may receive fabricated information that leads them to dismiss a potentially serious symptom without consulting their care team.

The distinction matters enormously in oncology, where delayed responses to side effects can have severe consequences. Yet the majority of resources that did address AI safety framed it from the perspective of clinical deployment — how hospitals and physicians are integrating AI — rather than the risks patients face when they use these tools independently.

Why This Matters Now

AI adoption in oncology is accelerating. Patients are already using large language models and AI-powered search tools to interpret diagnoses, evaluate treatment options, and assess prognosis — often before or between clinical appointments. As Hematology-Oncology Fellow Henry Litt, MD, senior author of the study, noted:

“In the clinic, we hear from patients all the time, asking us about something an AI tool told them.”

Clinicians are trained to communicate treatment risks. They are not systematically trained to help patients evaluate the epistemic risks of AI-generated health information. That gap in clinical communication mirrors the gap in publicly available content — and both need to close simultaneously.

The study also surfaces a structural issue in how AI safety is currently framed in public-facing oncology content. Safety messaging oriented toward institutional AI deployment does not translate into actionable guidance for a patient independently querying a chatbot at midnight. These are categorically different use cases, and the content ecosystem has not caught up with that distinction.

The Call to Action for Health Systems and Oncology Organizations

The research team frames the findings as a clear mandate. Health systems, cancer centers, and oncology organizations need to develop patient-facing resources that meet established readability standards, explicitly address the risks of direct AI use, and are discoverable through the search pathways patients actually use.

Co-author Ronac Mamtani, MD, put it directly:

“As AI is further integrated into oncology, patient education should be prioritized as a key part of AI implementation strategies.”

That framing is significant — it positions patient education not as a supplementary communication effort, but as a core component of responsible AI deployment in clinical settings.

Practically, this means setting institutional standards for reading level, requiring safety disclosures in AI-related patient content, and ensuring that resources address the patient-as-user scenario, not just the clinician-as-deployer scenario.

What This Signals for the AI Tools Ecosystem

For developers and evaluators of AI tools operating in health-adjacent spaces, this study functions as a benchmark and a warning. The quality deficit documented here is not hypothetical — it is measurable, reproducible, and already affecting how patients interact with AI in one of the highest-stakes domains imaginable.

Tools that surface health content, power patient-facing chatbots, or assist in medical communication carry an implicit responsibility to meet the standards that the current web largely ignores. Readability, accuracy, hallucination disclosure, and clinical oversight guidance are not optional features in oncology contexts — they are baseline requirements.

The 33% figure is not just a research finding. It is a quality floor that the entire ecosystem — platforms, health systems, AI developers, and medical societies — needs to collectively raise.

Rigorous benchmarking of this kind is exactly what the AI tools space needs more of: not promotional claims about capability, but structured, evidence-based assessment of what patients actually encounter and whether it serves them safely. The ASCO 2026 study sets a methodological precedent worth replicating across other high-stakes AI application domains.