In an ideal world, regulation of digital healthcare vendors would inspire confidence. Providers should see seals of regulatory compliance, and trust that “compliant” vendors can service them with products they sorely need. Conversely, “compliance” that only exists on paper, accomplishes the opposite effect: it forces distrusting providers to approach new digital health solutions with hesitation and unease.
Unfortunately, a research consortium centered at Georgetown University found that ASTP/ONC regulation of EHR AI-developers fails to meet standards for transparency or consistency. Healthcare leaders, unfortunately, navigate an environment of perfunctory regulation.
Incomplete reports
As EHR systems support documentation, PHI exchange, and clinical workflows, EHR vendors often develop AI-enabled predictive tools. Under the HT-1 final rule, ASTP/ONC requires that vendors publish Risk Management Summaries (RMS) for predictive tools, which navigate complex administrative geographies and sociotechnical environments. Through these summaries, vendors are meant to describe risk analysis processes, governance frameworks, and mitigation strategies for each of their marketed AI-algorithms. Additionally, RMSs must address eight core attributes: validity, reliability, robustness, fairness, intelligibility, safety, security, and privacy of their models.
The research, published in npj Digital Medicine, found that, while 71% of 35 probed developers described their risk analysis processes, only 57% identified the specific risks they monitored. Similarly, while 77% addressed all eight safety attributes, many offered single-sentence affirmation statements for considered attributes.
A governance crisis
Health providers require consistent, robust, and transparent reporting from developers to assess whether predictive AI tools are appropriate and safe for different administrative geographies, patient populations, and governance contexts. A seal of HT-1 compliance is meant to deliver assurance to providers that their EHR systems are, in fact, safe, reliable, and secure.
Instead, ASTP/ONC provides blanket compliance for all 35 developers, including those who provided only brief or high-level statements to support their self-reported “risk management” protocols. That type of inconsistent reporting can complicate governance processes, procurement decisions, and AI-evaluation for health providers, thereby challenging already strained hospital governance structures. As many EHR systems are auto-embedded within hospital workflows, the risk of faulty compliance practices is nascent.
More critically, however, that gap between seals of compliance and the robustness of actual mitigation strategies problematizes ASTP/ONC’s entire regulation regime. A patchworked governance environment makes consistent evaluation of AI safety and efficacy difficult. A regulatory level mired by its own inconsistency and inadequacy makes the task of AI governance even more arduous.
Instead, healthcare providers should demand regulatory consistency and transparency. That is a luxury that has not yet been afforded: the average AI/ML-enabled devices cleared by the FDA between 1970 and December 2024 scored a transparency rating of 3.3 out of 17, a shockingly low figure.
