The Wrong Metric for Institutional AI
Productivity metrics have a gravitational pull. They’re easy to track, easy to report, and easy to justify to stakeholders. When an institution invests in generative AI, the natural instinct is to measure return through efficiency gains — tasks completed faster, meetings eliminated, response times shortened.
This framing creates a trap. If you measure AI’s impact solely through time savings, you’ll either be disappointed or you’ll miss the transformation entirely. Work in complex organizations doesn’t simply compress. It changes composition.
At the Community College of Philadelphia, staffing levels and work hours remained stable across the four-year observation window. Nobody worked fewer hours. But the nature of what they produced — the texture of decisions, the clarity of communication, the speed at which deliberation reached closure — shifted in measurable ways.
The implication matters beyond higher education. Any institution with layered decision-making, distributed authority, and professional knowledge work faces the same measurement challenge. Government agencies, healthcare systems, legal practices, and large nonprofits all operate in environments where coordination costs often exceed execution costs. In these settings, time saved is the least interesting outcome.
How Three Professional Groups Changed
The research examined three distinct roles within one administrative unit: executive leaders, operational leaders, and student-facing professionals. Each group used generative AI differently, and each experienced different shifts in work composition.
Executive Leaders: From Deliberation to Direction
Executive leaders traditionally spend significant time in sensemaking — gathering input, weighing alternatives, and navigating ambiguity before issuing direction. Generative AI didn’t eliminate this work. It compressed the sensemaking phase.
Leaders used the tools to generate structured options, surface tradeoffs, and draft decision frameworks before entering deliberation with their teams. The result wasn’t faster decisions in the stopwatch sense. It was decisions that reached closure with fewer iterative rounds. Meetings that once required three sessions to align stakeholders now often concluded in one.
The coordination cost shifted from synchronous discussion to asynchronous preparation. Leaders wrote more and met less, but the total cognitive load remained comparable. What changed was the quality of the output: clearer rationales, more explicit assumptions, and decisions that operational teams could execute with fewer clarifying questions.
Operational Leaders: Translation Without Friction
Operational leaders sit at the critical junction between executive intent and frontline execution. Their work involves translating strategic decisions into processes, workflows, and guidance that student-facing staff can act on.
Before generative AI, this translation work was labor-intensive and error-prone. Operational leaders spent hours interpreting executive communications, drafting procedural documents, and fielding clarification requests from frontline staff. The back-and-forth was a feature of the system, not a bug — it’s how alignment got built.
After 2026, the pattern changed. Operational leaders used generative AI to produce clearer first drafts of procedural guidance. They could generate multiple versions tailored to different audiences, anticipate edge cases before they reached frontline staff, and preemptively address the questions that typically triggered clarification loops.
The volume of work didn’t decrease. But the nature of the work shifted from reactive clarification to proactive design. Operational leaders spent more time thinking about process architecture and less time answering the same questions repeatedly.
Student-Facing Professionals: Decision Quality Over Speed
Student-facing professionals — those handling admissions, advising, and program coordination — experienced perhaps the most counterintuitive shift. Their work didn’t accelerate. In some cases, individual interactions took longer.
What changed was the depth and consistency of their decision-making. Generative AI helped these professionals access institutional knowledge, policy details, and precedent cases during student interactions. They could make informed decisions in the moment rather than deferring to supervisors or scheduling follow-up conversations.
The metric that improved wasn’t throughput. It was decision quality and student experience. Fewer cases required escalation. Fewer decisions got reversed. Fewer students fell through procedural cracks because a frontline professional lacked the full context needed to resolve their situation.
Governance Implications Beyond the Obvious
When work composition changes, governance structures need to follow. The Community College of Philadelphia’s experience surfaces three governance considerations that most institutional AI discussions miss.
First, coordination patterns are a governance concern, not just an operational one. If generative AI shifts coordination from meetings to writing, from synchronous to asynchronous, from iterative clarification to clearer first passes — then governance bodies need to adapt how they oversee decisions. Reviewing meeting minutes won’t capture what’s happening. The artifacts of governance shift toward written rationales, decision frameworks, and the prompts that generated them.
Second, professionalism itself gets redefined. The research team noted that generative AI changed what it meant to be “professional” in each role. For executives, professionalism increasingly meant producing clear, structured decision frameworks. For operational leaders, it meant designing processes that anticipated edge cases. For student-facing staff, it meant exercising judgment informed by institutional knowledge that AI made accessible in real time.
These shifts don’t happen automatically. They require explicit conversation about what good work looks like when the tools change. Organizations that skip this conversation risk evaluating people against outdated standards while the actual work has already moved elsewhere.
Third, measurement systems need to catch up. Most institutional dashboards track throughput, turnaround time, and volume. None of these capture decision quality, coordination efficiency, or the reduction in clarification loops. Building measurement systems that reflect what actually changed requires understanding work composition before and after AI adoption — not just counting outputs.
What This Means for AI Adoption Strategy
The Community College of Philadelphia’s experience points toward a different adoption playbook for institutions.
Start by mapping coordination patterns before introducing tools. Where does work get stuck? Where do clarification loops multiply? Where do decisions cycle through multiple rounds of deliberation? These friction points are where generative AI is most likely to shift work composition — not by eliminating the work, but by changing its form.
Then define success in terms of work quality, not time savings. Decision closure speed, reduction in escalation rates, clarity of first-pass communication, and consistency of frontline decisions are all measurable. They’re just not the metrics most organizations track by default.
Finally, invest in the governance conversation early. When generative AI changes how professionals write, decide, and coordinate, it changes what accountability looks like. Supervisors need new frameworks for evaluating work. Teams need shared norms about when and how to use AI in decision processes. Institutions need updated policies that reflect the actual workflows, not the pre-AI assumptions about how work gets done, especially in higher education.
The Takeaway
Generative AI in institutions isn’t an automation story. It’s a work composition story. The tools don’t replace coordination — they shift where and how it happens. They don’t eliminate deliberation — they compress the path to closure. They don’t make professionals faster — they change what professionalism requires.
Organizations that measure only time savings will miss these shifts and conclude that AI underdelivers. Organizations that look at what kind of work gets produced, how decisions reach closure, and where coordination costs move will see something different: a quiet restructuring of institutional work that’s already underway.
The question isn’t whether generative AI is working. It’s whether your metrics can see what’s actually changing.
Comments (0) No comments yet
Want to join this discussion? Login or Register.
No comments yet. Be the first to share your thoughts!