AI Regulation: States Increasingly Frown on “Doctor AI”

I confess to being one of those hypochondriacs who has often turned to the Internet to diagnose my latest supposed ailment, even though every time I plugged in my symptoms “Ricin poisoning” popped up as one of the possibilities. To my knowledge, I’ve never gotten on the wrong side of Bulgaria’s intelligence agency, so eventually this undermined the Internet’s diagnostic credibility with me.  However, my past experience hasn’t deterred me from using AI tools in the same way.  According to this McDermott blog, states are increasingly concerned with protecting idiots like me.

The blog notes that Pennsylvania has sued Character.AI over a chatbot that allegedly allows people to interact with “characters” portraying themselves as licensed medical professionals, and says that Pennsylvania is far from the only state that finds “Dr. AI” troubling:

Pennsylvania’s lawsuit stands out as the first enforcement action alleging that an AI chatbot engaged in the unauthorized practice of medicine. However, this action falls under a broader national trend of increasing state scrutiny of AI in healthcare, especially AI chatbots.

For example, on June 1, 2026, the Florida Attorney General filed a lawsuit against OpenAI and its CEO, Sam Altman. The complaint claims that the company violated consumer protection laws by promoting ChatGPT in a way that made people think it could diagnose medical conditions. According to the lawsuit, some users even relied on ChatGPT to make healthcare decisions. Texas has taken a similar approach, although it has not filed a lawsuit yet. In August 2025, Texas Attorney General Ken Paxton opened an investigation into Meta AI Studio and Character.AI. The Texas Attorney General’s office is looking into whether these companies misled consumers by presenting chatbots as mental health tools even though the bots may not have proper medical training or oversight.

Several states also have enacted (and many more have considered) legislation specifically addressing AI chatbots, the use of AI in connection with mental healthcare, and prohibitions on representations or other statements that an AI system is licensed to practice healthcare.

The blog cautions that the Pennsylvania lawsuit indicates that disclaimers may be insufficient to protect developers against liability, and that “the AI platforms must also have sufficient guardrails in place to prevent misuse, and companies must monitor and correct the performance of their platforms over time.”