Can You Rely on AI for Medical Advice? What Doctors Say About Chatbots, Safety, and Limits

The Numbers Are Hard to Ignore

A recent eHealth survey found that nearly three-quarters of Americans said AI advice changed their mind about whether to seek medical care.

That’s not a niche behavior anymore. That’s a mainstream shift in how people make health decisions — decisions that can have serious, sometimes irreversible consequences.

Meanwhile, a JAMA report found that roughly 19% of young people now use AI chatbots specifically for mental health advice. A separate survey found 44% of people would rather start with a chatbot than talk to family, friends, or a doctor.

The trend is accelerating. The guardrails are still catching up.

What Doctors Actually Think About AI Health Tools

Here’s the nuance most headlines miss: physicians aren’t uniformly against AI in healthcare. They’re against patients using it as a replacement for clinical judgment.

Dr. Kushal Patel, an internal medicine physician at Mercyhealth, puts it plainly. He supports patients using AI and online tools to educate themselves and prepare better questions before appointments. The problem isn’t curiosity — it’s misplaced confidence.

“Oftentimes I want patients to educate themselves and take ownership,” Dr. Patel said. “So them going on Google or AI in and of itself is not a problem. But I want them to use it to educate themselves better or come up with questions or inquiries that we can discuss further.”

That framing matters. AI as a preparation tool is very different from AI as a diagnostic tool.

The Core Problem: AI Sounds Like It Knows You

One of the most dangerous qualities of modern AI chatbots is how convincingly personal they feel.

Dr. Patel identifies this directly. Chatbots are designed to sound responsive and specific — but they’re working without the clinical context that actually drives accurate medical decisions.

“One of the things that AI is good at is sounding personal,” he said. “So when patients are going and looking up things on a chatbot or similar areas, it can sound like it knows what the patient has exactly. But oftentimes AI lacks the background that a medical professional may have — such as your past history, family history, labs, previous examinations.”

That gap between perceived accuracy and actual accuracy is where harm happens.

A chatbot doesn’t know your medication history. It doesn’t know your last bloodwork. It doesn’t know that your chest pain started after a stressful week and you have a family history of cardiac disease. It generates a plausible-sounding answer — not a personalized one.

The Chest Pain Problem: When AI Gets It Wrong, It Really Gets It Wrong

Dr. Patel uses chest pain as his clearest example — and it’s a good one.

“If you are having chest pains, I don’t want AI to decide for you whether you should go to the emergency room or not. AI can provide more information for you on the types of chest pains, but again, it doesn’t really know what you’re experiencing.”

This is the critical distinction. AI can explain what different types of chest pain generally mean. It cannot evaluate whether your chest pain right now is a panic attack or a myocardial infarction.

Getting that wrong isn’t a minor inconvenience. It’s a life-threatening error.

Where AI Actually Adds Value in Healthcare

The conversation isn’t all cautionary. Dr. Patel is clear that AI has a legitimate and useful role — just not the one most people are using it for.

Mental Health Is a Special Case

The mental health dimension deserves its own attention.

Nearly one in five young people now turn to AI chatbots for mental health support. That number reflects a real access problem — mental health care is expensive, waitlists are long, and stigma still exists. AI feels like a low-barrier entry point.

But board-certified psychiatrist Dr. Sue Varma flags a fundamental limitation that applies even more acutely to mental health than physical health.

“It is overall very generic,” she said. “It does not understand the context of your life, your lived experience, your unique situation and background. Something that only a trained therapist through interaction, in real time, with somebody would say.”

Mental health care is built on therapeutic relationship, lived context, and real-time human attunement. A chatbot can offer coping frameworks and psychoeducation. It cannot replace the clinical judgment of a trained therapist who knows your history and can read what you’re not saying.

There are also unresolved concerns around privacy, data handling, and ethical standards in consumer-facing mental health AI tools — areas where regulation is still far behind adoption.

The Bigger Picture: AI in Healthcare Is Here to Stay

Dr. Patel doesn’t advocate for resistance. He advocates for evolution.

“As a community — and this will take some work — over time, we need to figure out how we use AI and how our understanding of AI tools develop over time.”

That’s the right framing for anyone thinking about this space. The question isn’t whether AI in healthcare belongs in healthcare. It’s how to integrate it responsibly — with clear boundaries, patient education, and clinical oversight built in.

For founders building health tools, for marketers in the wellness space, and for everyday users making decisions about their own care, the framework is the same: AI is a powerful information layer, not a clinical authority.

The Bottom Line

AI chatbots are useful. They’re also genuinely risky when used as a substitute for medical judgment.

Use them to learn, to prepare, and to understand what your doctor already told you. Don’t use them to decide whether your symptoms are serious enough to act on.

The technology will keep improving. The fundamental limitation — that AI doesn’t know you — won’t disappear anytime soon. Until it does, the smartest move is to treat AI as a starting point, not a final answer.

Your doctor still needs to be in the loop.