Healthcare AI data breaches: Why companies need to protect more than patient records

AI & Algorithms, Artificial Intelligence , , , , , , , , , , ,

Healthcare and life science companies are not only protecting traditional patient records – things like names, diagnoses, treatment notes, lab results, and insurance information. They may also be protecting clinical trial data, research data, vendor systems, proprietary models, and other information used to support patients and drug development. When these companies and systems are involved in data breaches, the legal risks can become broader than a standard data breach. 

A useful example is the recent Novo Nordisk incident. Novo Nordisk is the pharmaceutical company that makes drugs such as Ozempic and Wegovy and recently disclosed a cyber incident involving patient data from clinical trials. It has also been reported that a hacking group claimed to have stolen over a terabyte of data and attempted to extort the company, though Novo Nordisk has not confirmed the entire scope of those claims. 

The Novo Nordisk incident shows that healthcare breaches can involve more than names and medical records. Companies also need to consider whether a cyber incident involves clinical trial information, healthcare provider data, research files, AI model information, or other confidential business materials. When this type of data is involved, companies may need to take precautions to protect the data, document their response, and comply with applicable privacy and cybersecurity obligations.

Recent health-related breaches also highlight how these incidents create major legal and business consequences. The Change Healthcare cyberattack that occurred in 2024 shows the scale of healthcare cybersecurity risks with reports that 197.2 million individuals were impacted by this breach. For scale, 197.2 million people would be more than half of Americans. The 23andMe breach also highlights the litigation risks tied to sensitive genetic information, with a bankruptcy judge approving a $46.75 million settlement for victims in the breach. While the facts of these examples differ, they all highlight why companies handling health-related data need to treat cybersecurity, AI governance and breach response as connected compliance and a priority in business practices. 

Why are AI-related healthcare breaches unique?

A standard healthcare breach analysis often focuses on whether names, medical records, insurance information, or other identifiable patient information was exposed. These concerns are highly important and sensitive. But as artificial intelligence becomes more integrated into healthcare, it creates additional risks because AI tools rely on large volumes of sensitive data that may also be tied to valuable research and business information.

For example, an AI model used in drug development may involve clinical trial data, molecular research, imaging data, prompts, outputs and other model training information. If these systems are accessed by an unauthorized actor, the incident may raise privacy concerns and may also create IP and competitive risks. 

This means healthcare AI security should not only focus on protecting patient records. Companies should also consider whether their AI models, training datasets, research systems, and vendor environments are adequately secured.

Why does de-identification matter?

De-identification can be key when healthcare data is used with AI. In simple terms, de-identification means removing information that could identify a person up to a certain legal standard. This matters because companies may want to use patient data or clinical data to train, test or improve their models. 

If health data is properly de-identified before being used with AI, the company may reduce privacy risks. However, if the data is not properly de-identified, several questions remain. The company may need to ask whether its privacy notices clearly explained that health data could be used for AI training or analysis. State privacy laws may also require clear disclosures about how personal information is collected and used. For example, California’s CCPA requires covered businesses to provide notice about the categories of personal information collected and the purposes for which that information is collected or used. This is especially important when health-related data is later used in a way that patients or consumers may not have expected. 

The company may also need to consider whether protected health information was shared with an outside AI vendor, whether a business associate agreement was required, and whether patient authorization was needed.

What happens if PHI may have been compromised? 

Breach response is also an important issue. If a healthcare company experiences a ransomware attack or an unauthorized access incident, they may need to assess whether protected health information (PHI) was compromised, This may include reviewing what information was involved, whether it could identify an individual, who accessed it, whether it was viewed or acquired and if the risk was mitigated or not.

The Novo Nordisk incident highlights why this analysis matters. Novo Nordisk stated that the clinical trial information involved was not directly linked to patient names or other direct identifiers. However, the incident still raised questions about patient-related data, de-identification, and whether any protected health information may have been compromised. This is why companies should be able to document how health-related data was stored, protected, and separated from information that could identify individuals.

What should companies take away? 

For healthcare and life science-related companies, the key takeaway is that AI governance should include privacy and cybersecurity from the start. Before using health-related data with AI, companies should ask:

  • What data goes into the AI system?
  • Is the data identifiable or properly de-identified?
  • Who can access the data, prompts, outputs, or model?
  • Are outside vendors involved?
  • Can the vendor use the data to train or improve its own models?
  • What security controls apply if the system is breached?

Companies should also treat AI models, training datasets, prompts, outputs, and research tools as sensitive assets as well since they can carry PHI and may lead to inadvertent disclosures. Vendor contracts should address confidentiality, data retention, security controls and model training. 

AI may help healthcare and life science companies innovate and operate faster, but innovation cannot replace privacy and regulatory accountability. Companies using AI with health-related data need to confirm de-identification practices, strengthen vendor contracts, and properly prepare for responses before incidents occur.

Discover the privacy, confidentiality, and compliance challenges of AI in clinical trials, from HIPAA and FDA rules to AI vendor risks.

AI Use in Clinical Trials: Privacy, Confidentiality, and Compliance Risks

AI & Algorithms , , ,

Artificial Intelligence (AI) is becoming increasingly common in health care clinical trial operations. Sponsors, contract research organizations, research sites and vendors may use AI for many tasks such as patient recruitment, eligibility screening, and informed consent support among other things. While this can improve efficiency, these tools may also create privacy, confidentiality and compliance risks in an already highly regulated environment. 

Clinical trials often involve sensitive patient information, confidential study data, and strict documentation obligations. This means that AI use in clinical research cannot be treated like traditional workplace software. Before deploying an AI tool, organizations should understand what data the tool will access, where that data will go, whether the vendor can use it to train models, and whether the output can be explained, reviewed, and preserved.

How is AI being used in clinical trials? 

AI tools may be used throughout the clinical trial process. For example, AI can be used to help identify potential participants, screen patient eligibility, summarize medical records, draft study documents, monitor data and flag adverse events. Generative AI may also be used for other tasks in the clinical trial process such as summarizing meeting notes, clinical records or other trial related documents.

These uses can be helpful, but they also raise basic compliance questions. Before using an AI tool, organizations need to ask whether the tool fits within the trial’s legal and contractual framework. Clinical trial agreements and site agreements often control who may access trial data and how that data can be used, but those contracts may have been drafted without considering the use of AI in the trial. If an AI vendor is introduced without updating those agreements, contracts that are not AI-specific may create gaps in data access, confidentiality and regulatory accountability.

What legal obligations matter?

AI use in clinical trials must also fit within existing privacy, research and documentation obligations. HIPAA and its related rules may apply when AI vendors receive, store and analyze PHI. This means, organizations may need privacy and security safeguards before sharing their trial data with the AI vendor.

FDA electronic records requirements should also be considered as clinical trial records have to be reliable, auditable and traceable. If an AI tool creates a summary or other output without prompt logs or documented human review, it may create problems for recordkeeping and data integrity. 

Consent for the use of AI in the clinical trial process is also a key factor, especially when AI is used in recruitment processes or eligibility screenings. In these situations, participants should receive clear information about how AI is involved and how their information may be used.

Finally, confidentiality should remain at the forefront. Clinical trials do not only involve patient data, but may also involve sensitive commercial information, such as investigational product data, interim results, safety signals, and proprietary research methods. If an AI tool takes in this information for transcription, summarization, or analysis without clear confidentiality restrictions, it may create risks for sponsor confidentiality, trade secret protection, or attorney-client privilege.

Why are AI vendors a risk?

AI vendors can be one source of risk in clinical trials because they may receive and process very sensitive trial data. Standard AI vendor terms may not be enough when clinical trials involve protected health information or FDA-regulated records. Before using a specific AI vendor, organizations should review whether the vendor can retain prompts or outputs, use trial data to train its models, allow employee access to uploaded data, or store information outside approved systems.

Vendor contracts should clearly address data retention, model training, confidentiality, audit rights, security controls and human oversight. Additionally, companies should not rely only on what the vendor promises in the contract. They should verify how the vendor actually handles trial data before using the AI tool.

What should companies take away? 

For sponsors, meaning the organizations responsible for clinical trials, the main takeaway is that AI should be reviewed before it is used with clinical trial data. The organization should review what the AI tool will be used for, whether it will access PHI or confidential study information, whether the vendor can use trial data to train its models and if AI-generated outputs can become part of the trial record.

There are AI vendors that offer privacy-protective features like PHI redaction and de-identification. However, companies should not assume these features automatically make the tools compliant. Companies still need to verify how the vendor stores, processes, deletes and protects clinical trial data before deployment. 

AI may help clinical trials become more efficient, but efficiency does not replace privacy, confidentiality, consent, or regulatory accountability. Companies should update agreements, limit data exposure, verify vendor practices, document human review, and clearly disclose AI involvement where appropriate.

AI Companion Chatbot Regulations: New State Laws Target Child Safety, Disclosures & Crisis Response

Eyes on AI Chatbots: New State Laws Target Child Safety, Disclosures and Crisis Response 

AI & Algorithms , , ,

AI companion chatbots have recently become a focus of state AI regulation. Unlike task-oriented chatbots, companion chatbots may be designed to simulate conversation, friendship, emotional support or other ongoing personal relationships. These features may create heightened legal and safety concerns, especially when users are minors, emotionally vulnerable or may mistake automated responses for human support. 

State legislatures are beginning to address some of these risks.  While some enacted laws focus more on transparency and AI disclosures, proposed bills go further by addressing youth safety, emotional dependence, crisis response, age verification, data protection and human oversight. The Future of Privacy Forum is currently tracking 98 chatbot-specific bills across 34 states and three federal proposals, showing how quickly and unevenly chatbot regulation is developing.

Why are companion chatbots receiving regulatory attention?

Companion chatbots raise unique risks compared to traditional automated tools as they are often designed to boost user interaction, and keep users engaged over longer periods of time. In some cases, they may remember previous conversations, provide emotionally validating responses and create the impression of a meaningful relationship. These features may make the product engaging but also raise concerns about manipulation, emotional dependency and the collection of sensitive and personal data.

These risks are especially significant when the user is a minor. Because minors may have a harder time recognizing the limits of AI systems or identifying persuasive design techniques, they may be more vulnerable to mistaking automated responses for genuine human support. Regulators are also paying closer attention to situations in which a user discloses mental health concerns or expresses self-harm thoughts during a chatbot interaction. 

Across these proposals, there are several recurring regulatory themes: transparency, age verification, content safety, harm prevention, data protection, liability and enforcement practices. These themes suggest that lawmakers are not only concerned with whether users know they are interacting with AI but also with how chatbot systems are designed, how they collect data, and how they respond if a user may be at risk. 

What laws have already been passed?

Connecticut recently enacted one of the broader state laws addressing youth online safety and AI-related protections. On June 2, 2026, Governor Lamont signed Public Act 26-15, describing it as a bipartisan law intended to protect children and adults from digital-age harms, including youth social media addiction and concerns over the growing use of AI. The law also includes chatbot-related protections, such as requiring chatbot operators to make reasonable efforts to detect suicidal ideation or indicators of self-harm expressed by users and to maintain a protocol for responding with appropriate resources. Connecticut’s law goes into effect October 1, 2026.

California has also already taken steps to regulate companion chatbots through SB 243, which established baseline disclosure and safety requirements for companion chatbot operators. This law is in effect, with additional requirements for operators beginning July 1, 2027. 

Together, these enacted laws show that states are beginning to regulate AI systems directly, even without a comprehensive federal AI law. For businesses, this means AI compliance may increasingly depend on tracking different state requirements rather than relying on one national standard.

What pending proposals should companies watch?

Pending California bills highlight how companion chatbot regulations may become more specific. For example, SB 1119 focuses on chatbot interactions with child users, defined as consumers under 18 years of age. If enacted, it would require operators to take a more proactive approach to safety by conducting annual child safety risk assessments, creating public child safety policies, setting privacy and safety defaults for minors, providing parental controls, and conducting audits. It would also restrict certain chatbot behaviors like responses that encourage self-harm, substance use, disordered eating, or harm to others. This bill is currently active in the Assembly committee process.

California’s AB 1988, also known as the PAUSE Act, focuses more directly on crisis response. Unlike SB 1119, this bill is not limited to minors. If enacted, AB 1988 would require companion chatbot operators to identify and respond to credible crisis expressions, provide 988 Suicide and Crisis Lifeline information, pause chatbot responses after repeated crisis expressions, and require human moderator review before ending that pause. This bill is currently active in the Senate committee process.

These proposals show that chatbot regulation is moving beyond basic transparency requirements. Lawmakers are increasingly focused on how AI systems are designed, how they interact with vulnerable users, and whether companies have real safety procedures in place when a chatbot conversation becomes harmful or high risk.

What should companies take away?

For companies that are developing or deploying companion chatbots, one key takeaway is that basic AI disclosure may not be sufficient. Emerging state laws are narrowing in on how chatbots interact with minors, respond to self-harm or crisis-related statements, collect sensitive data, and whether meaningful human oversight is available. Businesses operating across multiple states should track both enacted laws and pending proposals as chatbot obligations may differ by state and are continuing to rapidly change. 

0

Deepfakes: A New Form of Workplace Sexual Harassment

Cybersecurity , , ,

In recent years, there has been an uptick in the number of cases where images generated or edited by artificial intelligence have given rise to workplace harassment claims. Regardless of whether the conduct at issue occurred in person or off duty, courts have shown a willingness to hold employers liable, leaving employers vulnerable to significant costs from employee misconduct. 

Current Cases

Employer liability arising from AI-generated content may stem from actionable workplace harassment claims. This could include media such as falsified videos, audio and images containing sexually explicit material which features a real person without their consent. 

Current and pending litigation involving these types of claims includes:  

  • Carranza v. City of Los Angeles (Cal. Ct. App. 2025). A decision from the California Court of Appeals confirmed a $4 million dollar award issued to a female police captain where a deepfake photo of her topless circulated in the workplace. There, the dissemination in the workplace was considered actionable workplace harassment. 
  • Pearson v. State of Washington (Wash. Super. Ct. 2025). Washington State Patrol trooper Collin Pearson alleges coworkers circulated an AI-generated kissing video that created a hostile work environment based on sexual orientation.
  • Friedrichs v. Scripps Media, Inc. (M.D. Tenn. 2025). Former Nashville meteorologist Bree Smith Friedrichs alleges her employer failed to address sexually explicit deepfake images and retaliation tied to workplace sexism claims. 

What about other federal statutes? 

Workplace harassment claims often interact with Title VII of the Civil Rights Act of 1964, which prohibits discrimination on the basis of sex. Additionally, Section 230 limits liability for platforms where harmful content is posted, meaning that if, for example, an employee distributes an AI-generated non-consensual image on a workplace messaging system (e.g. Slack, Microsoft Teams Chat, etc.) the employer, as opposed to the platform, may still be held liable. Additional claims at play may include: 

  • Title VII of the Civil Rights Act of 1964. The primary federal employment law used in deepfake cases. It supports hostile work environment, sexual harassment, sex discrimination, and retaliation claims. Employers face liability if they knew of the conduct and failed to investigate or stop it.
  • TAKE IT DOWN Act. The first major federal deepfake-specific law. It criminalizes knowingly publishing nonconsensual intimate imagery, including AI-generated “digital forgeries.” Requires covered platforms to remove reported content rapidly.

Are state laws involved? 

State laws cover three categories of harm—nonconsensual intimate deepfakes, election deepfakes, and identity impersonation. Additionally, nonconsensual intimate imagery and revenge porn statutes now often explicitly include AI-generated content, prohibiting the distribution of intimate images without consent and adding an additional legal framework supportive of employee claims against employers. 

In California, there are a handful of specific laws addressing this type of AI use, which may include: 

  • AB 602 creates a civil cause of action against anyone who either creates and intentionally shares digitized sexually explicit material without the depicted person’s consent, providing broad protection against deepfake pornography. Claims arising under this statute are supplemented by strong privacy torts, publicity rights, and CA FEHA for workplace claims. 
  • SB 926 explicitly adds AI-generated depictions to CA’s existing revenge porn law. 
  • SB 1381 and AB 1831 extend CA’s protections to include AI-generated content depicting minors. 

Additional laws have been enacted in Connecticut, Michigan, New Jersey, and New York, among other states. Additionally, state and common law claims for defamation may be relevant when deepfakes create false representations that create reputational harm. Deepfake audio and video may be considered evidence of injury. 

What are my potential responsibilities as an employer? 

While the issue is specific, the issue may require comprehensive action in order to preempt potential liability. Employers may consider the following actions: 

    • Updating Policies: Ensure that workplace policies clearly prohibit dissemination of sexually explicit material, real or doctored. Draft or update a standalone AI Acceptable Use Policy that names prohibited conduct (creating, possessing, distributing deepfakes targeting coworkers) and specifies that violations are grounds for discipline up to and including termination. 
    • Incorporating Training: Equip HR, legal, and IT teams to recognize and respond to deepfake incidents effectively.
    • Refreshing Investigation and Response Protocols: Encourage prompt investigations, which may include forensic analysis, verification of metadata, and ensuring fairness in credibility assessments for both alleged victims and accused parties. 
    • Reviewing Insurance: Review employment practices liability insurance coverage to confirm whether deepfake-related harassment claims and related cyber incidents are covered. Many existing EPL policies predate generative AI and may contain gaps.

What’s next? 

This is a rapidly evolving area of employment litigation—the applications of state deepfake and AI-related statutes in workplace harassment claims are likely to turn on pending federal agency actions and court decisions, ultimately determining the limits of employer liability for their employee’s potentially harassing conduct. Concerned employers may consider monitoring this landscape closely and adjusting compliance programs as litigation continues to contour this area of law. 

0

AI Chats and Law Enforcement: What Are You Sharing? 

AI & Algorithms , ,
AI chat platforms are increasingly becoming repositories of sensitive personal, professional, and legal information, and the legal frameworks governing what can be done with that information remain unsettled. This can have serious repercussions for individuals, businesses, and their advisors who happen to find themselves in the complex intersection of law enforcement and information privacy.  

What are users actually sharing?

The volume and sensitivity of information flowing into AI chat platforms go beyond what many users fully appreciate. Chatbots prompt users to provide background, context, and points of view, all of which may reveal intentions. This interface allows AI models to respond conversationally and prompt further explanation, inviting more disclosure than traditional searches. Below, we have highlighted two key reasons this leads to additional information being disclosed in this context:

The Illusion of the Advisor

Users increasingly interact with AI platforms as they would with a trusted professional, an attorney, therapist, or financial planner. However, AI chat platforms are not bound by traditional confidentiality obligations that govern licensed professionals. There is no attorney-client privilege, no therapist-patient privilege, and no fiduciary duty attached to a chatbot conversation. The sensitivity of the content does not create the protection the user may assume exists.

Agentic AI’s increased access

As the industry moves from chat interfaces to AI agents, this risk may continue to grow. Agentic AI is a tool that streamlines workflows; however, it requires broad, constant access to a user’s data across devices and applications. Major technology companies have already released early versions. As these agents become standard, the question of what an AI platform “knows” will no longer be limited to what was typed into a chat window, but may instead extend to digital communications such as email and text, documents, financial records, and location history.

What Can the Government Access?

Prosecutors and investigators have already begun seeking access to chatbot conversation histories in criminal investigations, and the legal framework governing those requests is still taking shape. However, there are a few current frameworks governing the chatbot’s permissible uses and disclosures of user intentions. 

Subpoenas and Third-Party Doctrine

Under the traditional application of the third-party doctrine, information voluntarily shared with a third-party platform has lesser protection than the Fourth Amendment typically affords. A government agency seeking chat transcripts may obtain them via subpoena without meeting the higher probable cause standard required for a warrant. The Supreme Court introduced some limits in Carpenter v. United States (2018), but its application to AI conversation logs is entirely untested.

National Security Demands

AI platforms may be subject to National Security Letters and Foreign Intelligence Surveillance Act (FISA) orders requiring disclosure of user data, with limited judicial oversight and strict non-disclosure obligations. A platform that receives such a demand often cannot notify the affected user, who has no opportunity to contest the disclosure. For businesses using AI tools for sensitive professional work, this exposure can be far-reaching and hard to foresee until it materializes. 

The Regulatory Gap

Currently, frameworks are designed for passive content-hosting platforms. However, these privacy frameworks are a poor fit for conversational AI.  

Ambiguity in Section 230 Protections

Section 230 of the Communications Decency Act shields platforms from liability for user-generated content. Whether that shield extends to AI chatbot outputs generated by the platform, not merely hosted by it, remains unresolved. A chatbot that produces a harmful response is authoring a reply, not hosting a post. Courts have not yet answered whether Section 230 immunity applies, and platforms that assume it does may find that assumption is not correct.

Consent Frameworks and Cross-border Complexity

Most AI platforms rely on broad, scroll-past consent mechanisms that regulators increasingly consider inadequate to secure meaningful consent. In the absence of comprehensive federal privacy legislation, compliance obligations vary by state and sector, and for multinational organizations, cross-border data flows through AI platforms may simultaneously implicate GDPR transfer requirements and foreign mandatory access regimes.

Key Takeaways

As AI use becomes more and more prevalent for use of everyday tasks and sensitive information alike, individuals and businesses may want to consider the following key takeaways: 
  • Establish policies governing employee use of AI chat platforms for work matters, with explicit restrictions on sharing confidential, privileged, or regulated information.
  • Review data retention and third-party sharing policies for any AI platforms in use, and update litigation hold procedures to treat AI chat logs as a discoverable data category.
  • Assess AI agent tools – those requiring broad device and application access – before deployment, with legal review of data exposure and applicable frameworks.
  • Brief leadership on the government access risk: AI chat transcripts may be subpoenaed or compelled under national security processes, often without user notification.
  • For multinational organizations, conduct a cross-border data flow analysis covering AI platform use and compliance with GDPR and analogous transfer frameworks.
When using these AI tools, it’s important to remember that the legal protections available for information shared with AI are not proportional to the information’s sensitivity or the user’s reasonable expectations. Closing that gap is, at this moment, primarily the responsibility of the user and the organizations that employ them. While legal frameworks are developing to align these interests, it is best to implement best practices early. 
1 2