The Central Governance Model Has Limits
Most states have centralized the creation of AI rules. CIO offices and emerging technology teams develop policies, review use cases, maintain inventories, and set risk standards before deployment. That structure has matured quickly: according to data shared from NASCIO’s forthcoming 2026 State CIO Survey, 98 percent of states now have enterprise policies governing generative AI—up from 76 percent in 2025.
But centralized rule-setting does not equal centralized accountability. Once a system is approved and deployed, ownership typically shifts to the individual agency running it.
Maryland illustrates this clearly. The state’s Department of Information Technology establishes guardrails covering security, privacy, and data standards. Agencies must complete an intake process before deployment. After approval, however, day-to-day accountability returns to the agency that owns the use case—including monitoring outputs, auditing for bias, maintaining error logs, and following incident response procedures.
California operates on a similar logic. The California Department of Technology sets statewide policy and governance requirements. Agencies remain responsible for operation, monitoring, risk management, and compliance on the systems they use.
Pennsylvania adds a layer before the handoff. Generative AI deployments undergo a multidisciplinary review involving the Emerging Technology Office, a Generative AI Governing Board, privacy and security officers, and legal counsel. But the agency implementing the use case still shares responsibility after launch.
The Operational Gap
Building an approval process is structurally straightforward. Defining what happens after a system produces inaccurate recommendations, introduces bias, or generates unexpected outcomes is considerably harder.
Andrew Merluzzi, AI Innovation and Incubation Fellow at the Beeck Center, has observed that while states have made significant progress on central governance structures, operational accountability has not evolved at the same pace. Executive orders and statewide AI inventories establish frameworks—but they do not automatically answer the question of who handles errors when they occur.
The risk Merluzzi identifies is a diffusion of responsibility: multiple offices are involved, but no single person is named as accountable for a system’s performance, monitoring, or improvement. His recommendation is direct—every deployed AI system should have a specific, named individual responsible for it.
The best-functioning models, in his assessment, combine central standards with operational decision-making pushed down to agencies or individual teams. Chief AI officers are not meant to become the sole authority over every deployed system. Their role is to translate broad principles into repeatable practices: risk tiers, evaluation standards, escalation procedures. This aligns with broader debates about governance and decision quality.
Embedded AI: The Governance Blind Spot
One of the more underappreciated challenges is that AI does not always arrive with a label.
Software updates, enterprise platform upgrades, and customer service application changes can introduce AI capabilities into government environments without any intentional AI procurement decision. A productivity tool update becomes an AI deployment by default.
Meredith Ward, deputy executive director of NASCIO, identified embedded AI as one of the most significant governance challenges states currently face. Each software update that introduces new AI features expands the potential attack surface while reducing visibility into where AI is actually entering government operations.
The concern is not theoretical. The 2026 NASCIO-Deloitte Cybersecurity Study found that 94 percent of state CISOs are actively participating in developing generative AI security policies, and 84 percent are involved in strategy development and use case reviews. One state CISO quoted in the report described vendors as “effectively inflicting AI on operational environments before comprehensive risk assessments or policy frameworks can be applied.”
States have responded by making procurement a checkpoint.
- Pennsylvania requires enterprise security and emerging technology reviews whenever AI is purchased—whether as a standalone product or embedded in another system.
- California expects agencies to evaluate embedded AI features during procurement and implementation.
- Maryland routes AI capabilities, including features built into existing enterprise software, through its intake process before deployment.
The principle is consistent: embedded AI should receive the same scrutiny as any explicitly purchased AI tool, especially where security and privacy concerns are involved.
Shadow AI and the Human Variable
Even well-designed governance frameworks encounter a persistent human problem. When approved tools are difficult to access or approval processes move slowly, employees find alternatives on their own.
Ward noted that acceptable use policies have become a common tool for managing what many call shadow AI—the use of unauthorized AI tools by government employees. But policies alone do not resolve the underlying adoption dynamic.
Merluzzi frames it plainly: banning AI does not eliminate its use. It drives use underground. Employees turn to unauthorized tools when approved options are unavailable, when approval timelines are too long, or when they simply do not know which tools they are permitted to use.
The implication for governance designers is that restriction without accessible alternatives creates a compliance gap that policies cannot close.
The Questions That Will Define the Next Phase
AI governance in U.S. state government has moved from early-stage debate to operational reality faster than most observers expected. The structural foundations—policies, advisory committees, AI officer roles—are now broadly in place.
What remains unresolved is the harder layer: performance accountability over time.
Merluzzi identifies the questions that will define the next phase of state AI governance:
- Who owns each deployed system?
- Who monitors its performance on an ongoing basis?
- What happens when results decline or outputs drift?
- When should an AI tool be expanded, modified, or retired?
These are not policy questions. They are operational and organizational ones—and they require named individuals, defined escalation paths, and monitoring practices that outlast the initial deployment decision.
The 2026 picture suggests that states have built the road. The more demanding work is deciding who drives, and who is responsible when something goes wrong.
Comments (0) No comments yet
Want to join this discussion? Login or Register.
No comments yet. Be the first to share your thoughts!