On July 22, Scott Winters, a Florida pastor, sued OpenAI for “unauthorized practice of medicine,” claiming that ChatGPT-4o’s “extremely dangerous” medical advice led him to avoid seeking professional care for what was later diagnosed as a pulmonary embolism. Winters and Tech Justice Law, his counsel, sought damages and a preliminary injunction to halt OpenAI’s planned ChatGPT Health expansion until evaluators could independently confirm its safety. A day later, OpenAI launched ChatGPT Health for all U.S. users.
The risk of shadow AI
Hospitals should know that untracked “shadow AI” proliferates in the wake of that tech expansion, a long-term fear of health professionals. This year, Wolters Kluwer reported that roughly 58% of healthcare professionals have confronted use of unapproved consumer-grade AI tools at least once at work.
“Shadow AI,” however, is also patient-facing. According to OpenAI’s own reporting, more than 40 million people consult ChatGPT for health advice daily. That prompted ECRI to rank chatbot misuse – like incorrect diagnosis, faulty testing recommendations, or recommendation of faux medical supplies – as the top health technology hazard of 2026.
The new risk
But Winters’ case is different. With GPT-4o, the malpractice was suggested inaction rather than diagnostic inaccuracy. According to the lawsuit, after Winters reportedly consulted ChatGPT about his dizziness, the chatbot downplayed his systems, recommending he stay “recliner-bound” rather than visit a hospital. On the day of his eventual cardiac arrest, ChatGPT dismissed the advice of Winters’ congregants to visit the hospital, assuring Winters that “God did not design your body to endlessly fail.”
Winters’ litigation follows a string of other lawsuits against OpenAI: a lawsuit filed in May sued OpenAI claiming that ChatGPT had engaged in the unauthorized practice of medicine when it recommended a 19 year-old take a drug cocktail that would eventually kill him; seven suits filed in 2025 sued OpenAI, claiming it prematurely released ChatGPT-4o as a “dangerously sycophantic and psychologically manipulative model.” Winters’ case, however, appears to be the first of its kind where the dangerous, unauthorized medical advice is negligent inaction rather than dangerous prescription diagnosis.
That new risk anecdotally corroborates emergent research on AI accuracy. Though AI models typically score well on medical benchmarks – written exams which grade AI diagnostic accuracy – highly scripted tests rarely mirror real-world conditions. A Stanford Research team that scored 20 models on 1,100 real world prompts found that AI advice risked severe harm in 24.6% of cases. Over 80% of those severe recommendations were omissions or suggested inaction.
OpenAI's impunity
Last month, CHARGE spoke with Jason G. Cooper, formerly of Blue Cross Blue Shield, about consumer health consultation with what he called “Dr. GPT.” In his assessment, when patients arrive with health information gathered from ChatGPT it makes consultations “more efficient and therefore leads to deeper human connection.” Though gathered information may be inaccurate, it can be checked and pushed back upon by providers.
Winters’ case spotlights a different question: what happens when patients don’t show up at all? Similarly, what happens when shadow AI creeps into hospital workflows not by introducing error artifacts but by failing to introduce critical information?
Unfortunately, OpenAI won’t provide us the answers. Though enterprise integrations can help cut down on shadow AI, consider the hubris for a moment: OpenAI is so confident they can eat any lawsuit and any scandal that they rolled out ChatGPT Health, which allows subscribers to “securely” upload their medical records and receive advice for “routine” health management, a day after a Times article claimed that it almost killed a man. Though not earned on trust, the self-assurance reflects the market: tech expansion is so outpacing regulation that developers hardly pay lip service to safety optics.
The practical implication is that shadow AI error modes are discoverable only through pulmonary embolism. Providers should demand clarity.
References
On the lawsuit:
[2] https://www.nytimes.com/2026/07/22/well/openai-chatgpt-health-lawsuit.html
[3] https://www.bbc.com/news/articles/cwylp3nxp5yo
On shadow AI risk:
[4] Wolters Kluwer: https://www.wolterskluwer.com/en/news/wolters-kluwer-survey-finds-broad-presence-of-unsanctioned-ai-tools-in-hospitals-and-health-systems
On past lawsuits:
The Stanford study:
