AI Cybersecurity Guardrails: The Tradeoff Between Safer Models and Effective Vulnerability Research

The Core Problem: Guardrails That Cannot Distinguish Intent

Offensive security work is not inherently malicious. Penetration testers, vulnerability researchers, and zero-day developers probe systems for weaknesses before criminals do. Their workflows often require asking an AI model to do exactly what a guardrail is designed to prevent: analyze exploitable code, reason through attack paths, or confirm whether a bug is genuinely dangerous.

Chris Anley, chief scientist at NCC Group, frames the problem precisely. Asking a model to exploit a bug is a standard step in validating whether a vulnerability is real and worth fixing. When a guardrail blocks that query, it does not just stop an attacker—it stops a defender mid-workflow.

“Fix this code” is simultaneously a defensive prompt and a roadmap for finding critical vulnerabilities. The two uses cannot be cleanly separated, which means a guardrail calibrated for one will inevitably interfere with the other.

Vetted Access Programs: A Partial Solution With Real Friction

Both OpenAI and Anthropic have introduced structured programs—OpenAI’s Trusted Access for Cyber and Anthropic’s Cyber Verification Program—that offer approved researchers access to models with reduced cybersecurity restrictions. The intent is reasonable: differentiate legitimate researchers from bad actors.

In practice, the friction is significant. Researchers report that even within these programs, guardrail behavior is inconsistent. The same prompt may be answered one day and refused the next. Chris Thompson, CEO of Remote Threat and founder of Offensive AI Con, describes the practical result clearly: researchers spend time negotiating with the model rather than analyzing vulnerabilities.

That negotiation cost is not trivial. It shifts cognitive load away from the actual security problem and toward managing model behavior—an overhead that compounds across a research workflow.

Organizations that are not enrolled in these programs face a harder ceiling. One anonymous researcher at a smartphone-component manufacturer described frontier models as nearly unusable for security work: any hint of security-related context triggers a refusal.

The Open-Source Shift: Capability Without Restriction, Privacy Without Risk

When frontier models become unreliable or inaccessible, researchers move to open-source alternatives. This pattern appears consistently across the researchers interviewed for this analysis.

The appeal is twofold:

• No guardrails. Models like GLM, downloaded and locally run, impose no usage restrictions and require no vetting.
• No data exposure. Cloud-based models carry the risk that sensitive vulnerability data—unpublished zero-days, proprietary exploit code—could be logged, reviewed, or absorbed into future training runs. Local models eliminate that risk entirely.

Paolo Stagno, CTO at Crowdfense, illustrates the resulting workflow split: frontier cloud models for reverse engineering and general code analysis, local open-source models for anything touching vulnerability discovery or exploit development. The division is not ideological—it is operational.

What Researchers Actually Use AI For

It is worth being precise about where AI currently fits in offensive security workflows, because the picture is more nuanced than “AI writes exploits.”

Several researchers describe a consistent pattern: AI accelerates the early, analytical stages of security work—understanding unfamiliar code, identifying code structure, building supporting tooling—while the core vulnerability discovery and weaponization remain human-driven.

Giuseppe Cali, a zero-day researcher, states this directly. He uses AI for reverse engineering and tooling support, not for finding bugs or building exploits. His reasoning is partly practical and partly professional: he wants to own the discovery process. Even if all guardrails were removed tomorrow, his workflow would not change significantly.

This suggests that guardrails are most disruptive not at the exploit-writing stage—where many researchers prefer to work manually—but at the earlier analytical stages where AI is already widely integrated and where the line between offensive and defensive intent is genuinely blurry.

The Geopolitical Dimension

Thompson raises a concern that extends beyond individual researcher productivity. Responsible security professionals being pushed away from U.S.-governed AI systems toward unrestricted foreign-owned models is not a neutral outcome. It shifts the tools ecosystem in a direction that U.S. AI labs presumably do not intend.

The argument is not that Chinese open-source models are inherently dangerous to use. It is that the incentive structure created by overly restrictive guardrails produces an unintended migration—one that reduces the leverage U.S. labs have over how their tools are used in high-stakes security contexts.

The Underlying Tradeoff

The guardrail debate in cybersecurity is not simply about whether AI should help with hacking. It is about who gets to define “safe” in a domain where safety is context-dependent, and whether the current gatekeeping mechanisms are calibrated well enough to make that distinction reliably.

Mark Dowd’s criticism—that large companies are making arbitrary decisions about what is safe in security—points to a legitimacy problem as much as a technical one. Security researchers operate under professional norms, legal frameworks, and institutional accountability structures. Treating them as undifferentiated risk vectors produces friction without proportionate safety benefit.

What to Watch

The practical trajectory here is fairly clear. As long as frontier model guardrails remain inconsistent and access programs remain narrow, the open-source local model ecosystem will continue to absorb the overflow demand from professional security researchers.

For teams evaluating AI tools for security workflows, the relevant questions are:

• Does your use case require cloud-based frontier models, or can a capable local model handle the analytical workload?
• What is your organization’s risk tolerance for feeding vulnerability-adjacent data into cloud APIs?
• Are you eligible for vetted access programs, and is the reduced friction worth the enrollment overhead?

The labs that figure out how to extend reliable, accountable access to professional security researchers—without the inconsistency that currently characterizes even their vetted programs—will be better positioned to remain relevant in offensive security workflows. The alternative is continued migration to tools they do not govern.