Grok, X, and the FTC: How Musk’s AI Data Grab Tests US Privacy Enforcement

The Order X Wants Gone

The backstory is mundane by tech scandal standards. Twitter once had a coding error that let user phone numbers—submitted for two-factor authentication—get quietly redirected to ad targeting. The FTC penalized them. The resulting consent order requires independent audits and gives the agency authority to demand compliance documents without filing fresh lawsuits.

Musk bought Twitter, rebranded it X, folded it into his corporate empire, and is now arguing the order no longer applies. The company’s position: we’re a different business, the order is burdensome, and the EU’s GDPR already covers similar ground anyway.

The FTC opened a public comment period with a July 2 deadline. Fifteen advocacy groups—including the EFF, EPIC, and the National Consumers League—responded with a letter that can be summarized as: absolutely not.

What Actually Changed Under Musk

The advocates’ letter doesn’t just defend the old order. It argues that Musk’s changes to X have added reasons for scrutiny, not removed them.

The list is uncomfortable reading:

• 2.8 billion records leaked from the platform in a single year
• Grok, X’s chatbot, faced a lawsuit alleging it generated child sexual abuse material and non-consensual intimate images
• The FTC had already found that Musk directed employees toward actions that would have violated the existing order
• X scraped hundreds of millions of posts for AI training without explicit user consent—updating its terms of service and apparently hoping nobody noticed

That last point is where the story gets structurally interesting for anyone watching the AI tools ecosystem.

Surveillance Capitalism, Industrialized

Cambridge Analytica—yes, that Cambridge Analytica—published a pointed analysis arguing that X’s Grok model isn’t a new business model. It’s the old one, scaled up and moved in-house.

Their framing: when Facebook’s API enabled behavioral profiling at population scale, it was eventually shut down after public outrage. Musk didn’t invent something new. He just rebuilt the same extraction pipeline natively inside X’s infrastructure, replacing third-party data sales with direct AI deployment.

The numbers are striking. Research cited in the advocacy letter found that 73% of X users were unaware their tweets were being used to train Grok. Opt-out mechanisms exist but are, as Cambridge Analytica put it,

“practically invisible.”

There’s also a detail that tends to unsettle people once they hear it: deleting your posts from X doesn’t delete the behavioral signal already absorbed by the model. The algorithm may keep targeting you based on content you chose to remove.

The GDPR Substitution Argument Doesn’t Hold

X’s claim that GDPR oversight makes FTC monitoring redundant has a fairly obvious problem: X is currently under investigation in Europe for collecting user data to train Grok without valid GDPR consent.

Using an ongoing GDPR investigation as proof that GDPR is sufficient oversight is a creative legal argument. Advocates called it “misleading,” which is the polite version.

The two legal precedents X cited to support its “transformed company” theory also don’t hold up on closer reading. One involved a 20-year sunset clause. The other saw an order modified after 16 years of demonstrated compliance. X’s order is four years old, and compliance has been, at best, contested.

What’s Actually at Stake for AI Regulation

This case is worth watching beyond the Musk-versus-regulators drama, because it’s testing something genuinely unresolved: whether consent frameworks built for individual users can meaningfully constrain industrial-scale behavioral data extraction for AI training.

The advocates’ letter argues they can’t—at least not without active enforcement. Cambridge Analytica’s analysis goes further, suggesting that X’s unusual transparency about its data extraction scope might paradoxically make it harder to regulate, because the surveillance mechanism is obvious rather than hidden in ambiguous terms of service.

That’s a pattern worth tracking across the AI tools ecosystem. Platforms that quietly harvest behavioral data for model training while maintaining plausible deniability in their ToS may face less scrutiny than one that announces it openly. That’s a perverse incentive structure.

The Practical Takeaway

If you’re building on top of AI tools that train on user-generated content—or advising clients who use platforms like X—the consent question is no longer theoretical. Regulators are paying attention, advocacy groups are organized, and the legal frameworks are being actively contested.

The FTC keeping its consent order intact won’t resolve the broader question of how AI training and user data rights coexist. But losing it would signal that rebranding is a viable exit strategy from regulatory accountability.

For anyone choosing AI tools right now: the data provenance question—where did this model’s training data come from, and did users actually consent—is becoming a due diligence item, not just an ethics footnote.