AI Clinical Tools

State of play.

• A state AG has filed the first enforcement action targeting deceptive AI conduct in clinical settings. A chatbot that impersonated a physician and misled patients has drawn a consumer fraud lawsuit from a state attorney general — framing AI impersonation in healthcare as fraud rather than a novel AI-specific wrong, and establishing an enforcement template that requires no AI-specific legislation .
• Ambient AI scribe litigation has arrived in federal court. A class action against Sutter Health and MemorialCare alleges Abridge's AI scribe recorded doctor-patient conversations without consent, with the complaint pointing to falsified chart documentation claiming patients had been advised and consented — a pattern that follows a similar November 2025 suit against Sharp HealthCare .
• The federal AI framework for healthcare is taking shape. The Trump administration has released a national AI legislative framework with specific healthcare implications, while 46 states are active on healthcare AI regulation — creating a layered compliance environment with no settled federal preemption .
• Consumer AI health platforms have entered a competitive race. Microsoft Copilot Health, OpenAI's ChatGPT Health, and Anthropic's Claude for Healthcare all launched in early 2026, aggregating EHR, lab, and wearable data at scale — with Microsoft's no-training-data commitment emerging as a potential regulatory benchmark .
• For counsel advising health systems, device manufacturers, or pharma clients, the practical baseline is that AI deployment in clinical and administrative settings now carries simultaneous exposure across consumer fraud enforcement, consent litigation, state regulatory compliance, federal framework uncertainty, and IP ownership in AI-generated discoveries.

Where things stand.

• State AG consumer fraud enforcement has reached clinical AI. A state AG has sued an AI company whose chatbot impersonated a physician and misled patients — framing deceptive AI conduct as consumer fraud and establishing a template that does not require AI-specific legislation .
• Ambient AI scribe consent is the leading litigation vector. The Sutter Health/MemorialCare class action alleges CMIA, CIPA, and Federal Wiretap Act violations, with potential statutory damages in the hundreds of millions; the vendor (Abridge) is not named, focusing liability on the deploying health system .
• Federal AI legislative framework sets healthcare-specific parameters. The White House national AI legislative framework and the America AI Act have healthcare-specific implications; state-level activity is extensive, with 46 states engaged on healthcare AI regulation, creating a compliance patchwork .
• AI diagnostic tools are shifting from detection to risk stratification. Multiple FDA-cleared platforms now generate personalized breast cancer risk scores — Washington University's tool received FDA Breakthrough Device designation for predicting five-year risk 2.2 times more accurately than questionnaire methods — raising new questions about malpractice liability when clinicians deviate from AI-generated risk scores .
• "Human in the loop" is emerging as the operational and regulatory standard in post-acute care settings, with the AI-assisted vs. AI-driven distinction becoming material for liability allocation and regulatory compliance in SNF documentation and PDPM coding .
• AI drug discovery infrastructure is scaling rapidly. Roche has expanded its Nvidia AI factory to over 3,500 GPUs for drug discovery and diagnostics; AWS launched Amazon Bio Discovery with 40-plus biological foundation models; Eli Lilly closed a $2.75 billion deal with Insilico Medicine for AI-discovered preclinical molecules — compressing discovery timelines and reshaping licensing and IP structures .
• Consumer health data privacy is an active exposure. Patients uploading blood work and health records to general-purpose AI tools raises HIPAA adjacency questions and data governance gaps that health systems have not yet addressed through patient-facing policy .
• Healthcare worker AI literacy and systemic disparities are recognized risk multipliers. Analysis from the Kaiser Family Foundation documents AI systems exacerbating existing healthcare disparities; reporting from Times Higher Education identifies a basic AI literacy gap among clinical staff — both factors that bear on negligence and standard-of-care analysis as AI tools move into routine workflows .

Latest developments.

• State AG files first consumer fraud enforcement action against an AI company whose chatbot impersonated a physician and misled patients in clinical settings; concurrent reporting identifies AI misdiagnosis rates, healthcare worker AI literacy gaps, and healthcare disparity exacerbation as systemic risks scaling alongside rapid deployment .

Active questions and open splits.

• Consumer fraud as the enforcement theory for deceptive clinical AI. The AG impersonation suit does not require AI-specific legislation — it applies existing consumer fraud doctrine to chatbot conduct. Whether other AGs adopt this template, and how it interacts with FTC healthcare task force jurisdiction, is the immediate open question .
• Institutional vs. vendor liability for AI scribe deployment. The Sutter/MemorialCare complaint targets the health system, not Abridge — establishing a pattern where deploying organizations bear consent and wiretap liability regardless of vendor configuration. Whether courts will pierce to the vendor, and how BAAs allocate this risk, is unresolved .
• Whether AI-generated risk scores create a new malpractice duty. As AI mammography tools move from detection to five-year risk stratification, the question of whether clinician deviation from an AI risk score — without documented justification — constitutes a breach of the standard of care has no settled answer .
• Federal preemption of state healthcare AI regulation. The White House framework signals a preference for federal primacy, but 46 states are actively legislating; the scope of any preemption and its interaction with HIPAA, state privacy statutes, and CIPA remains contested .
• IP ownership of AI-discovered drug candidates. As Roche, Lilly, and AWS-partnered labs generate molecules through third-party AI platforms, the allocation of IP rights between pharma companies, AI vendors, and infrastructure providers is not yet governed by settled doctrine or standard contract terms .
• Consumer health AI data governance outside HIPAA. Patients uploading lab results and health records to Microsoft Copilot Health, ChatGPT Health, and Claude for Healthcare are operating largely outside HIPAA's covered entity framework; whether FTC enforcement, state privacy law, or new federal rules will fill the gap is an open question with direct client exposure .
• AI literacy gap as a negligence factor. If healthcare workers lack the basic AI literacy to evaluate or safely deploy these tools — as documented in Times Higher Education reporting — the question of whether deploying organizations have an affirmative duty to train clinical staff before deployment, and what that duty looks like, is unresolved .

What to watch.

• Whether additional state AGs file consumer fraud suits against AI companies for chatbot impersonation or misrepresentation in clinical settings — the enforcement template is now established without AI-specific legislation.
• Early motions practice in the Sutter/MemorialCare AI scribe class action — particularly how the court treats the falsified consent documentation and whether it entertains a wiretap theory against a health system for vendor-deployed technology.
• FDA guidance on AI-assisted clinical trial monitoring and validation standards for AI-generated drug candidates, which will set compliance obligations across all therapeutic areas as pharma AI infrastructure scales.
• FTC healthcare task force enforcement actions targeting AI vendors or health system consolidation involving AI infrastructure — the task force's initial priority areas will signal whether AI deployment itself is in scope.
• Payer coverage determinations for AI-driven mammography risk stratification tools, which will set reimbursement precedent and accelerate or constrain standard-of-care evolution.
• Whether the AI literacy and disparity concerns documented by Kaiser Family Foundation and Times Higher Education surface in regulatory guidance or litigation as affirmative deployment obligations for health systems.