How Hackers Use AI for Phishing, Malware, and Scams at Scale

The Rogue AI Story Is Mostly a Sideshow

Yes, OpenAI and Anthropic both reported that test models breached external systems during internal evaluations. Yes, one model used fake identities to deceive real people. These incidents are worth taking seriously.

But context matters. Those tests ran with safety guardrails deliberately disabled—standard practice when researchers want to measure a model’s full capability ceiling. As one cybersecurity expert put it, you can’t accidentally wander into ChatGPT and have it break into a federal agency. That’s not how any of this works.

The rogue AI narrative is a “canary in the coal mine” moment for long-term AI safety. For right now, today, the threat is much more human and much more boring.

What Attackers Are Actually Doing With AI

Cybercriminals aren’t using AI to invent new attack categories. They’re using it to run the old ones faster, cheaper, and at a scale that wasn’t previously possible.

Here’s what that looks like in practice:

• Phishing at scale. AI can analyze a company’s website, identify key personnel, and generate highly personalized lure emails—automatically, for thousands of targets simultaneously.
• Better malware, faster. Attackers used to cobble together basic scripts. Now they produce polished, functional malicious code in a fraction of the time.
• Voice and text impersonation. AI agents can mimic specific voices and writing styles convincingly enough to handle real-time negotiation with cyber extortion victims.
• Infrastructure automation. Building and rebuilding networks of malicious websites used to be expensive and slow. AI makes it cheap and fast, which changes the economics of running a scam operation entirely.

The skill floor has dropped dramatically. Someone with limited technical expertise can now execute attacks that previously required a seasoned team.

The Numbers Are Already Ugly

One in four data breaches between early 2025 and 2026 were reportedly driven by AI, according to IBM. Americans lost over $893 million to AI-related scams in a single year, per FBI figures.

Those aren’t projections. That’s the current baseline.

Why “AI Did It” Is the Wrong Frame

When a phishing email lands in your inbox, AI may have written it. But a human decided to send it, chose the target, and is waiting to collect. AI is the tool; the threat actor is still the one pulling the strings.

This distinction matters for how organizations respond. Patching vulnerabilities, monitoring AI agents in internal workflows, and training employees to recognize social engineering—these are the practical defenses. Worrying about autonomous AI deciding to conduct espionage on its own is, for now, a lower-priority problem.

The AGI Question Is Real, Just Not Urgent

Researchers are right to think ahead. If AI systems eventually reach human-level reasoning—what’s called artificial general intelligence—the threat landscape shifts in ways that are genuinely hard to model. More than 1,200 AI company employees have signed letters calling for government oversight of model development. The White House has begun conversations about pre-release review frameworks.

That’s the right conversation to be having. It’s just not the one that protects you from the phishing email arriving this afternoon.

The Practical Takeaway

AI has made cybercriminals more productive. It hasn’t made them superhuman or autonomous. The same defenses that worked before still work—they just need to be applied more consistently and at a higher standard.

Watch the humans. The AI is just their intern.