A groundbreaking Nature study published in late June is questioning the efficacy of the privacy, vulnerability, and cyber attack tests regularly conducted by pre-deployment risk evaluators. Cyber vulnerability research typically quantifies cyber attacks across all records in a dataset. The Technical University of Munich (TUM) based research group, however, stress tested open source models – among them TorchXray Vision, PadChest, CheXpert, and MIMIC-CXR – with MIAs (membership inference attacks). MIAs seek to identify whether individual patients belong to a given model’s training data set. Because models are trained on deidentified patient information, MIAs obtain identified health records by proxy, exposing patients’ sensitive medical information.
The research team demonstrated that, even where aggregate data-set cyberattacks failed, MIAs achieved high success rates for individual patient identification. At dataset fringes, that success rate neared perfection. MIA capacity, for example, scaled by orders of magnitude with the size of attacked models (that is, the risk was higher for larger, more capable models). Similarly, underrepresented groups, like racial minorities or medicare recipients, were more exposed to substantial risk.
The results are jarring. Auditors have, thus far, failed to adequately capture MIAs within their evaluation protocols. Similarly, as hospitals seek to integrate AI with expanded scope and capacity, they expose their network to greater risk.
That risk is structural and suggestive: AI models integrated into a hospital’s EHR infrastructure are trained on insecure data, and auditing procedures clearly fail to map an enterprise’s entire risk surface.
