AI governance is decentralizing from committees to departments, where real oversight happens.
When Epic implementations matured, many health systems discovered that central IT was not enough. Sustainable success required physician builders embedded within departments — clinicians with defined access, protected time, and shared accountability for how the system functioned locally.
AI governance is approaching a similar moment. Over the past two years, organizations have stood up centralized AI governance committees to manage intake, risk classification, and approval. That structure was necessary in the early phase. But as AI functionality spreads across service lines — and increasingly appears inside broader vendor platforms — the limits of a purely centralized model are becoming apparent.
The next phase is distributed governance.
Health systems are no longer reviewing a handful of discrete algorithms. They are managing ambient documentation tools, predictive risk models, imaging and pathology systems, EHR-embedded decision support, and vendor solutions with integrated AI features.
Central governance teams can define risk tiers and set monitoring expectations. What they cannot realistically sustain is ongoing oversight across every department.
Post-deployment monitoring requires understanding how a tool performs within specific clinical contexts, how clinicians actually use it, and whether unintended consequences or concerns emerge over time. These assessments are deeply contextual. They depend on proximity to the work. No central committee can maintain that level of granular visibility across an entire health system.
The Accountability Gap
In many organizations, AI tools are assigned to clinical or operational program owners. These individuals are expected to monitor performance and report against predefined metrics.
In reality, most program owners are managing full clinical practices or operational portfolios. Few have formal training in model evaluation, performance drift, or bias analysis. Even fewer have protected time to conduct structured oversight. This dynamic echoes the early EHR era. Without embedded physician builders, HealthIT teams became bottlenecks, and departments lacked ownership of local optimization. The solution was not to eliminate central IT, but to complement it with distributed expertise.
AI governance is encountering a similar constraint.
The physician builder model offers a practical template for what AI governance may require next. In the EHR era, physician builders committed defined portions of their time to system design and optimization, operated within clear governance boundaries, and partnered closely with IT. Service lines funded the role, reinforcing accountability and alignment with local priorities.
AI oversight is likely to demand a comparable structure: a designated Departmental AI Lead within each service line, with defined responsibilities for monitoring, documentation, and escalation; structured partnership with central governance and data teams; and protected time proportional to the department’s AI footprint. Central governance would continue to set policy, classify risk, and conduct audits, while departments assume responsibility for local stewardship.
Importantly, this shift will require deliberate investment. Historically, radiology and pathology developed dense clinical informatics expertise, which makes the emergence of AI leads in those departments more straightforward. Most other service lines did not build that depth of informatics capacity.
For CIOs, CMIOs, CAIOs, the design questions are immediate: whether to formalize departmental AI oversight before it emerges informally; how to allocate protected time and funding; and what monitoring standards to require. Organizations that invested early in physician builder programs strengthened both system performance and clinician engagement. As AI becomes routine infrastructure, governance will need to move closer to clinical and operational workflows. The Departmental AI Lead represents a structural evolution in stewardship — bringing oversight to where the work actually happens.
Explore more of our research and insights on health AI safety.
Originally published on the CHARGE blog. Republished here as part of the archive.
CHARGE publishes independently. Grid Health, its incubator, builds an AI operations center health systems use to monitor models, detect drift, and govern AI in production.
Go deeper with Grid