0

AI Chats and Law Enforcement: What Are You Sharing? 

AI & Algorithms , ,
AI chat platforms are increasingly becoming repositories of sensitive personal, professional, and legal information, and the legal frameworks governing what can be done with that information remain unsettled. This can have serious repercussions for individuals, businesses, and their advisors who happen to find themselves in the complex intersection of law enforcement and information privacy.  

What are users actually sharing?

The volume and sensitivity of information flowing into AI chat platforms go beyond what many users fully appreciate. Chatbots prompt users to provide background, context, and points of view, all of which may reveal intentions. This interface allows AI models to respond conversationally and prompt further explanation, inviting more disclosure than traditional searches. Below, we have highlighted two key reasons this leads to additional information being disclosed in this context:

The Illusion of the Advisor

Users increasingly interact with AI platforms as they would with a trusted professional, an attorney, therapist, or financial planner. However, AI chat platforms are not bound by traditional confidentiality obligations that govern licensed professionals. There is no attorney-client privilege, no therapist-patient privilege, and no fiduciary duty attached to a chatbot conversation. The sensitivity of the content does not create the protection the user may assume exists.

Agentic AI’s increased access

As the industry moves from chat interfaces to AI agents, this risk may continue to grow. Agentic AI is a tool that streamlines workflows; however, it requires broad, constant access to a user’s data across devices and applications. Major technology companies have already released early versions. As these agents become standard, the question of what an AI platform “knows” will no longer be limited to what was typed into a chat window, but may instead extend to digital communications such as email and text, documents, financial records, and location history.

What Can the Government Access?

Prosecutors and investigators have already begun seeking access to chatbot conversation histories in criminal investigations, and the legal framework governing those requests is still taking shape. However, there are a few current frameworks governing the chatbot’s permissible uses and disclosures of user intentions. 

Subpoenas and Third-Party Doctrine

Under the traditional application of the third-party doctrine, information voluntarily shared with a third-party platform has lesser protection than the Fourth Amendment typically affords. A government agency seeking chat transcripts may obtain them via subpoena without meeting the higher probable cause standard required for a warrant. The Supreme Court introduced some limits in Carpenter v. United States (2018), but its application to AI conversation logs is entirely untested.

National Security Demands

AI platforms may be subject to National Security Letters and Foreign Intelligence Surveillance Act (FISA) orders requiring disclosure of user data, with limited judicial oversight and strict non-disclosure obligations. A platform that receives such a demand often cannot notify the affected user, who has no opportunity to contest the disclosure. For businesses using AI tools for sensitive professional work, this exposure can be far-reaching and hard to foresee until it materializes. 

The Regulatory Gap

Currently, frameworks are designed for passive content-hosting platforms. However, these privacy frameworks are a poor fit for conversational AI.  

Ambiguity in Section 230 Protections

Section 230 of the Communications Decency Act shields platforms from liability for user-generated content. Whether that shield extends to AI chatbot outputs generated by the platform, not merely hosted by it, remains unresolved. A chatbot that produces a harmful response is authoring a reply, not hosting a post. Courts have not yet answered whether Section 230 immunity applies, and platforms that assume it does may find that assumption is not correct.

Consent Frameworks and Cross-border Complexity

Most AI platforms rely on broad, scroll-past consent mechanisms that regulators increasingly consider inadequate to secure meaningful consent. In the absence of comprehensive federal privacy legislation, compliance obligations vary by state and sector, and for multinational organizations, cross-border data flows through AI platforms may simultaneously implicate GDPR transfer requirements and foreign mandatory access regimes.

Key Takeaways

As AI use becomes more and more prevalent for use of everyday tasks and sensitive information alike, individuals and businesses may want to consider the following key takeaways: 
  • Establish policies governing employee use of AI chat platforms for work matters, with explicit restrictions on sharing confidential, privileged, or regulated information.
  • Review data retention and third-party sharing policies for any AI platforms in use, and update litigation hold procedures to treat AI chat logs as a discoverable data category.
  • Assess AI agent tools – those requiring broad device and application access – before deployment, with legal review of data exposure and applicable frameworks.
  • Brief leadership on the government access risk: AI chat transcripts may be subpoenaed or compelled under national security processes, often without user notification.
  • For multinational organizations, conduct a cross-border data flow analysis covering AI platform use and compliance with GDPR and analogous transfer frameworks.
When using these AI tools, it’s important to remember that the legal protections available for information shared with AI are not proportional to the information’s sensitivity or the user’s reasonable expectations. Closing that gap is, at this moment, primarily the responsibility of the user and the organizations that employ them. While legal frameworks are developing to align these interests, it is best to implement best practices early. 
0

Risks of Shared AI Workspaces and Confidentiality, Security, and Privacy Concerns

AI & Algorithms , ,
Traditionally, the relationship between a company and its outside advisors, law firms, consultants, and financial advisors has been governed by confidentiality agreements, attorney-client privilege, and codes of professional ethics. These agreements assure that these outside advisors have access only to the information necessary for the scope of the project. However, artificial intelligence is becoming a mainstay in these working relationships, dismantling that clear separation.  AI-powered productivity tools are increasingly deployed not just within a single organization, but across shared digital workspaces, the collaborative platforms where companies and their external advisors jointly draft documents, manage new projects, exchange data, and make decisions. This shift represents a fundamentally new risk landscape, one that most organizations and their advisors have not yet adequately mapped.  This post identifies the three primary risk categories that arise when AI enters these shared spaces and the key considerations to mitigate them.  

Risk 1: Confidentiality

When AI tools operate within a shared workspace, there are two primary threats to client confidentiality:  1) Cross-client training and model contamination, and  2) over-input of information.  

Cross-Client Training Model Contamination

Many AI tools learn continuously from user interactions. For example, if a law firm’s AI assistant is trained, even implicitly, on documents, queries, and outputs across multiple client engagements sharing a platform environment. In this case, client information can become embedded in the model’s behavior. The AI may begin surfacing language, structures, or strategic approaches drawn from one client’s confidential materials when assisting another.  This is an example of cross-client training contamination. 

Over-Input of Information

When processing the information above, AI tools may ask follow-up questions, or the user may want to include additional context and guidance for the tool. These prompts and the need for greater contextual clarity may drive users to input additional information, information that may not normally be shared or be strictly necessary for the task at hand. This could lead to AI tools being trained on, and potentially re-sharing, information that is not strictly necessary. 

Risk 2: Overexposure

AI processes operating across shared workspaces introduce a new failure mode: overexposure through automated workflow. When an AI agent is tasked with summarizing documents, preparing briefings, or surfacing relevant materials, it may draw on content from across the workspace without respecting the role-based and project-based permissions designed to contain that information.

Misconfiguration and Permission Gaps

AI tools in shared workspaces are typically configured by IT or platform administrators, not by the lawyers or compliance officers who understand the sensitivity of the underlying information. Permissioning structures that may be technically correct for human access often fail to account for how AI agents traverse and aggregate information. A consultant with project-scoped access to a workspace may, through the AI layer, receive synthesized summaries that draw on materials outside their authorized scope.

Role and Project Segmentation Failures

Even well-intentioned configurations can break down when AI tools are updated. For example, this could occur when team membership changes or when workspace structures evolve mid-engagement. Unlike a human employee who is subject to ongoing supervision, an AI system with broad access will continue operating at that level until it is explicitly restricted. The moment of overexposure may be difficult to trace, making the discovery of these failures especially challenging. 

Risk 3: Accountability

Who is Responsible when AI makes the decision? Professional service relationships often assign responsibilities clearly; for example, the lawyer is responsible for legal advice, the auditor for the audit opinion, and the consultant for the recommendation. These lines of responsibility are the foundation of malpractice liability, professional licensing, and regulatory compliance. However, AI tools make this division more complicated. 

The Absence of Auditable Decision Trails

Many AI tools used in professional services do not generate decision logs or explainable outputs in a meaningful sense. When a deal recommendation, a compliance conclusion, or a litigation strategy is influenced by an AI-generated analysis, there may be no record of what data the AI considered, what it weighted, or what it excluded. If the decision is later challenged in litigation, a regulatory proceeding, or a malpractice claim, the AI’s contribution cannot be reconstructed or audited.

Diffused Liability Across a Shared Platform

In a shared workspace involving the company, its law firm, its auditors, and potentially a technology platform provider, an AI-assisted error may have no clear owner. Did the AI fail because of a platform defect? Because the law firm configured it incorrectly? Because the company provided bad inputs? Because no human professional adequately reviewed the output? Engagement letters, platform terms of service, and professional liability policies may not be drafted to answer these questions.

Key Considerations in Light of these Risks

The risks described may be present in any organization that has extended its advisory relationships (law firms, consultants, and financial advisors, to name a few) into AI-enabled collaborative platforms. To minimize these risks, organizations may want to consider the following tips:  Consider…
  • Auditing shared platforms and tools currently used with outside advisors to identify any AI features, and map what data those features can access. 
  • Reviewing engagement agreements, NDAs, and platform terms of service for AI-specific confidentiality provisions. 
  • Assessing whether AI access controls in shared workspaces respect role-based and project-based information silos and construct limitations where they do not. 
  • Establishing AI decision-logging protocols with outside advisors, including requirements for human review and sign-off before AI-influenced advice is acted upon. 
  • Negotiating clear contractual allocation of liability for AI-related errors across the full advisory chain, company, advisors, and platform providers. 
  • Briefing executive leadership and the board on AI-specific risks in advisory relationships, particularly in regulated industries where privilege and data protection obligations are most acute. 
Establishing governance frameworks for AI early in advisory relationships may enable companies to reduce their own exposure and hold advisors accountable if one of the risks of use materializes. 
0

Overview of New York’s Child Data Protection Act

Children, Cybersecurity, New York, State Privacy Laws, United States , , , , ,
In June 2024, New York Governor Kathy Hochul signed the New York Child Data Protection Act (Act) into law, which will go into effect on June 20, 2025. Per the Act’s justification, “[c]hildren now live much of their lives online,” including learning, socializing, shopping. They also “make mistakes online, and they discover who they are online,” and, accordingly, they should be able to do so without the “concern of omnipresent monitoring and recording.” The Act enables this through two major provisions:
  1. if a digital service knows a user is a minor (or if the service is primarily directed to minors), it will “default to only being able to use that child’s data in a way that is strictly necessary to provide the service;” and
  2. digital services using third-party service providers must “contractually restrict those third parties from using the personal data of minors except for specified purposes” and include additional safeguards to help ensure compliance.
The Office of the New York State Attorney General has also released Implementation Guidance to clarify key questions raised in the rulemaking process.

Scope & Applicability

This Act applies only to conduct occurring in the state of New York. This means that commercial conduct that takes place outside of New York is not covered by the Act if: 1)  the user was outside of the state or 2) no data collected while the user was in the state was used.
  • Covered Users. The Act imposes restrictions on processing information of “covered users.” This includes users of websites, online services, or connected devices (the “Websites”) who are: 1) actually known by the operator to be a minor (under 18), or 2) who are using Websites primarily directed to minors.
  • Operator. An operator is defined as any person who offers Websites, who alone – or jointly with others – controls the purposes and means of processing personal data. Notably, one who acts as both a controller and processor shall comply with obligations for both roles, depending on the purposes and means of processing personal data.
  • Personal data. This definition includes any data that identifies or could be reasonably linked, directly or indirectly, with a specific natural person or device.

Substantive Provisions

Processing Restrictions. The Act provides that, among other things, an operator shall not process the personal data of a covered user collected through the Sites, unless one of the following applies:
  1. the user is 12 or younger, and processing is permitted under COPPA;
  2. the user is 13 or older and the processing is “strictly necessary”; or
  3. the user is 13 or older and the processor has received informed consent.
Strictly Necessary Processing. The term “strictly necessary” includes, among other things, processing that is required to:
  • Provide or maintain a specific product or service requested by the covered user;
  • Conduct the operator’s internal business operations (excluding those that relate to marketing, advertising, research and development, providing products or services to third parties, pr prompting covers users to use the Site when it is not in use); and
  • Identify and repair technical errors that impair functionality.
According to the Implementation Guidance, processing that is “strictly necessary” to provide a process or service required by a covered user depends on the “expectations of a reasonable covered user,” similar to the guidance provided under the CCPA regulations. The Guidance also clarifies that business operations “shall not include any activities relating to marketing, advertising, research and development, [or] providing products or services to third parties.” Informed Consent. If the information being processed is not “strictly necessary,” the operator will need informed consent, through either: 1) a device communication or signal, or 2) an informed consent request. A request for informed consent should, among other things:
  1. be made separately from any part of the transaction.
  2. clearly and conspicuously state that the processing is not strictly necessary, and consent is not mandatory to continue using the Websites.
  3. clearly present an option to refuse to provide consent as the most prominent option.
Additionally, the user should be able to revoke consent at any time as easily as they provided it.

Enforcement

The New York Attorney General may bring an action or special proceeding to enjoin any violation of this Act, and to obtain civil penalties of up to $5,000 per violation. Further, the Act gives the New York Attorney General authority to issue rules and regulations ad necessary, and according to the Implementation Guidance, the Office of the Attorney General intends to issue these rules. The Implementation Guidance also states that, until such rules are finalized, the Office of the Attorney General will exercise discretion in pursuing enforcement actions, taking good-faith compliance efforts of covered businesses into account.

Effective Date

The Act goes into effect on June 20, 2025.
0
Chicago Grand Central Looking Up

DOJ Issues Final Rule on US Bulk Sensitive Data

Data Brokers, Data Transfers, United States , , , ,
The International Emergency Economic Powers Act (IEEPA) vests the President with authority to deal with extraordinary threats to national security and foreign policy that have their source in part or in whole outside of the United States. Acting pursuant to the IEEPA, President Biden issued Executive Order 14117, “Preventing Access to Americans’ Bulk Sensitive Personal Data and United States Government-Related Data By Countries of Concern” (the EO). The EO directed the Department of Justice (DOJ or Department) to establish and implement regulations addressing threats from certain countries of concern attempting to access and exploit bulk amounts of US sensitive data, including personal and government data. On December 27, 2024, the DOJ issued the Final Rule, which went into effect on April 8, 2025. Additional compliance provisions for certain transactions take effect on October 6, 2025. The Final Rule prohibits or restricts a range of transactions involving categories of bulk sensitive personal data or government-related data between the US and countries of concern or covered persons. In assisting businesses to adapt to this comprehensive update, the DOJ provided a Fact Sheet, a Compliance Guide, and over 100 FAQs on the Final Rule, along with an Implementation and Enforcement Policy. Below are five main takeaways that US entities may want to consider in light of these regulations.
  1. Enforcement May Be More Lenient Until July 8, 2025 
The DOJ’s Implementation and Enforcement Policy, states that the Department will “target its enforcement efforts during the first 90 days to allow US persons (e.g., individuals and companies) additional time to continue implementing the necessary changes to comply with the [Final Rule].” The Department’s civil enforcement actions for violations of the Final Rule will not be a priority “so long as the person is engaging in good faith efforts to comply with or come into compliance with the [Final Rule] during that time.” However, the Department makes clear that it will “pursue penalties and other enforcement actions as appropriate for egregious, willful violations” during the delayed enforcement period.
  1. DOJ Will Consider Good Faith Efforts to Comply
While the Implementation and Enforcement Policy reflects that civil actions for violations of the Final Rule will not be a priority, this depends on the entity’s good faith effort to comply. According to this Policy, examples of evidence of good faith efforts may include, but are not limited to:
  • Conducting internal reviews of access to sensitive data.
  • Conducting internal reviews to determine whether transactions involving access to such data flows constitute data brokerage.
  • Reviewing internal datasets and datatypes to determine if they are subject to the Final Rule.
  • Conducting due diligence on potential new vendors.
  • Renegotiating vendor agreements or negotiating contracts with or transferring products or services to new vendors.
  • Adjusting employee work locations, roles or responsibilities.
  • Evaluating investments from countries of concern or covered persons.
  • Implementing the CISA Security Requirements.
  1. “Good Faith” May Include Satisfying CISA Security Requirements 
A good-faith effort to comply may be demonstrated, in part, by implementing the CISA Security Requirements, which were developed concurrently with the Final Rule pursuant to the EO. The security requirements are intended to address threats that arise when conducting restricted transactions, as detailed below. These security requirements are divided into two sections: i) organizational- and covered system-level requirements; and ii) data-level requirements.
  1. Before October 6, 2025, Determine if Your Company is Conducting Restricted Transactions
US entities engaged in restricted transactions under the Final Rule have affirmative data compliance program and audit obligations, among other obligations. In addition, the Final Rule provides that data brokerage transactions are prohibited with any foreign entity unless the US person contractually binds the foreign entity from subsequent transactions of that data with a country of concern or covered person. They must also report any known or suspected violation of this requirement.
  1. An Iterative Review Plan May be Needed for Covered Transactions 
With the Final Rule coming into effect and enforcement nearing, US companies that engage in certain data transactions or share information with third parties that may be covered persons or countries of concern should evaluate their transactions and data practices. After a thorough review of the types of information collected, who that information is shared with, and who is involved in the processing of that data, it may be helpful to adopt a compliance policy to ensure transactions are being handled appropriately in light of the Final Rule.
0

Metaverse Law in Orange County Lawyer Magazine

AI & Algorithms, CCPA, Cybersecurity, GDPR, Health Data, State Privacy Laws , , , , , , ,
The January 2025 edition of Orange County Lawyer magazine features an article written by Metaverse Law’s Lily Li. Read “AI and Machine Learning in Drug Development and Clinical Trials” below or in Orange County Lawyer magazine.
[Originally published as a Feature Article: AI and Machine Learning in Drug Development and Clinical Trials, by Lily Li, in Orange County Lawyer Magazine, January 2025, Vol. 67 No.1, page 28.]   AI and Machine Learning in Drug Development and Clinical Trials by Lily Li   In 2013, sleep medication zolpidem (Ambien, Ambien CR, and Edluar) swept headlines. Marie Claire reported on an alarming and suspicious rise in users experiencing irrational eating, gambling, and even “sleep-driving” while in a hypnotic trance—waking with no memories of their actions.[1] In several cases, women arrested and convicted for driving under the influence contested their convictions, arguing that they were not liable for these undisclosed drug-related side effects. At the same time, several clinical studies suggested that women metabolized zolpidem differently from men. By reviewing existing literature, Japanese researchers out of Shimane University identified 40% higher concentrations of zolpidem in women than men following use, and higher rates of visual hallucinations and sensory distortions.[2] The FDA released a safety advisory, warning users of the risks of “next-morning impairment” for the use of Ambien and related drugs.[3] In addition, the FDA took the unusual step of recommending a 50% cut in the dosage for women. When asked about the change, an FDA director told ABCNews.com: “The changes are different in women and men . . .We don’t understand why yet, but women are more susceptible to next-morning impairment.”[4] Yet, a decade later, the evidence supporting different zolpidem dosages for women and men is unclear.[5] In part, this is due to the lack of research surrounding sex differences in drug impact and drug treatment, as well as substantial gaps in the inclusion of women in clinical studies. From 1977 to 1993, FDA policy recommended excluding women of childbearing potential from Phase 1 and early Phase II drug trials.[6] Even after this policy was removed in 1993, industry fears remained with respect to drug interactions with pregnancy. This episode with zolpidem raised several concerns in the drug development and clinical trial process:
  • How do we recruit representative candidates for drug trials?
  • How do we ensure the quality and availability of datasets for clinical research?
  • How do we measure potential impacts of drug dosing on different populations?
  • What are the legal implications for failing to address appropriate drug doses?
  AI and ML to the Rescue? Now that artificial intelligence is being used in research and development, one wonders: Can artificial intelligence (AI) and machine learning (ML) reduce bias and risks during drug development? Or will it create new legal risks due to bias, privacy intrusions, and lack of transparency? The FDA released a discussion paper on AI, Using Artificial Intelligence and Machine Learning in the Development of Drug and Biological Products, to discuss potential regulatory frameworks to address the use of AI and ML.[7] In this discussion paper, the FDA released a set of fascinating case studies into existing research and uses of AI in the clinical trial process. Several of these case studies are discussed below, as well as an analysis of their potential impact on the zolpidem example.
  1. Recruitment. According to the FDA, “AI/ML is being used to mine vast amounts of data, such as data from clinical trial databases, trial announcements, social media, medical literature, registries, and structured and unstructured data in EHRs [electronic health records], which can be used to match individuals to trials (Harrer, 219 Shah, Antony, & Hu, 2019).” In this manner, researchers can combine huge quantities of publicly available data and individual health data from prior research to identify participants with certain medical conditions (or lack of adverse conditions) for investigational treatments. For zolpidem, the use of AI/ML may have been able to identify a much broader list of participants for initial clinical testing, making it easier to assess and identify adverse reactions.
  2. Selection and Stratification of Trial Participants. In addition to initial recruitment, AI/ ML has the capability improve intake, selection, and classification of clinical trial participants. Based on baseline characteristics selected by the researchers, such as prior clinical data, and vitals/labs taken during intake, predictive algorithms can help identify high-risk participants.[8] These groups can then be randomized and then subject to more strict monitoring protocols. In the case of zolpidem, alcohol use is associated with sometimes severe adverse effects from the drug, and so it would be beneficial to screen out candidates with a history of alcoholism or, on the flip side, assess drug interactions for this high-risk group with additional support, monitoring, or counseling.
  3. Dose/Dosing Regimen Optimization. AI/ML can be used to predict drug exposure for different populations based on factors such as weight, height, sex, and other characteristics that might impact drug metabolism. Based on prior drug exposure and response profiles for similar drugs and similar populations, AI/ML can help to narrow the dose/dosing regimen selected for a study. As noted by the FDA’s discussion paper, this can help optimize drug dosing “in special populations where there may be limited data (e.g., rare disease studies, pediatric and pregnant populations).” Based on this research, we can imagine future scenarios where AI/ML could have avoided zolpidem dosing concerns, where graduated and limited dosing was tested and applied to different sex, age, and metabolism categories to determine ideal dosing.
  4. Data Analysis. On a more intriguing level, the FDA AI discussion paper discussed the concept of creating “digital twins” of patients for clinical trials. Essentially, an AI version of the clinical participant is created, using the existing candidate’s electronic health records, vital signs, labs and other records. Researchers can assess how the digital twin would react under normal conditions using AI/ML modeling based on data gathered from similar individuals. This digital twin would then act as a substitute for a placebo candidate in a clinical trial, and act as a benchmark against the actual patient undergoing investigational treatment. For zolpidem, this could be used to assess candidates that already have underlying medical conditions such as anxiety, depression, or other confounding factors, to see whether an adverse effect from a trial is due to the investigational treatment or something that is likely to occur to the same individual from anxiety alone.
  5. Postmarketing Safety Surveillance. Finally, AI/ML can help detect and assess adverse events once the drug enters the market. This is not just limited to individual case safety reports (ICSR), required by regulators, but can include adverse events reported publicly on social media and the wider internet. This type of postmarketing safety surveillance could assist researchers and drug companies in identifying potential drug risks, prior to landing on primetime news.
  Quality and Reliability Risks While AI/ML can help to address the costs and efficiency of clinical trials, this relies substantially on the underlying data used to train AI. The quality and reliability of any AI/ML model requires similar quality controls for underlying training data. Given the safety risks of inappropriate drug dosing, or recruiting candidates with severe medical conditions, AI developers cannot rely solely on self-reported healthcare data with no external medical testing or validation. Developers should be equally wary of training on third-party data sets that do not provide documentation on the collection of data and data validation. Within an existing healthcare organization, if the organization is big enough, aggregate and de-identified data may be obtained from existing electronic health care records and prior clinical trials. Yet, even within these large datasets, errors may surface during training. Medical providers may code the same procedure, and similar symptoms, a dozen different ways. Even drug names can be misspelled and coded incorrectly within existing records. While many of these errors may end up being statistically insignificant with enough data, there is the risk of missing one or two major adverse events, or “black swan” events, that would otherwise change the entire risk profile of a drug. In addition to quality and reliability, the underlying dataset needs to be representative of the population that will be studied for the clinical trial. If the underlying dataset is only trained on a handful of individuals with a certain medical predisposition, age, sex, weight, etc., it will be difficult for the AI model to make predictions for that group. As an example, if the training data only contains the medical information for two individuals over the age of sixty, and shows no adverse effects from a particular drug dose, this information is not enough to generalize that the drug at that dosage is appropriate for all individuals over the age of sixty. For all we know, these two candidates could be a former Olympic diver and a nutrition coach, two outliers that completely skew the data. Consequently, the underlying training data for any AI model should also be assessed for bias and representativeness as it applies to the proposed clinical trial.   Data Privacy, Cybersecurity, and AI Risks The data privacy and cybersecurity risks associated with the foregoing uses of AI/ML cannot be underestimated. The quality and representativeness of any AI system in this field will rely heavily on large swathes of healthcare data, fine-tuned and, at times, personalized in the case of digital twins. This is sensitive or special category data at its finest, triggering heightened scrutiny under the EU’s data privacy law, the GDPR, and U.S. data privacy and data breach laws. To date, most healthcare organizations have sidestepped data privacy concerns by relying on HIPAA’s de-identification standard to remove personal information and other identifiers from healthcare data, making it difficult to associate with an individual. While the FDA requires Institutional Review Board (IRB) review of most biomedical research involving human subjects, this generally does not apply to de-identified personal information that cannot be linked to an individual. Simply de-identifying data and then running with it is not enough, however. Under the California Consumer Privacy Act and similar state laws, for example, recipients of de-identified data need to affirm that they will not attempt to reidentify the data (except to test their de-identification methods). The GDPR has a much higher “anonymization” standard, which looks at the re-identifiability of personal information, given all the different datasets that an organization may have access to. AI/ML itself is making the de-identification process harder. As it is capable of slicing and dicing data by age, race, sex, and medical condition, and combining multiple large datasets, it is easy to run the risk of re-identifying data. While several thousand people might have the same configuration of eye color, age, gender, and weight, only one or two may have participated in a clinical trial at a particular location, or have specific allergies or side effects to certain types of medication. As a result, in circumstances where healthcare data is not de-identified, or the risk of reidentification is heightened, then it behooves clinical organizations and their AI developers to implement written information security programs and associated privacy and security controls.   Legal Liability and Drug Dosing In several notable cases, defendants on zolpidem were able to contest or overturn DWI or even vehicular manslaughter cases. Essentially, these defendants argued that they were not aware of the potential dangers of zolpidem, and so could not be liable for their actions while “sleep driving.” This raises the question: If AI gets good enough, and can tell you exactly the right dose to take of a drug, will you (or your doctor) be liable if you deviate from the AI’s recommendations? Will the AI’s recommendations be discoverable in court (and surfaced via AI-enhanced search)? Only time will tell what this brave new world will bring.   ENDNOTES [1] Kai Falkenberg, While You Were Sleeping (September 27, 2012), Marie Claire, https://www.marieclaire.com/culture/news/a7302/while-you-were-sleeping/.   [2] Takuji Inagaki, Tsuyoshi Miyaoka, Seiichi Tsuji, Yasushi Inami, Akira Nishida, and Jun Horiguchi, Adverse Reactions to Zolpidem: Case Reports and a Review of the Literature, 12 Prim Care Companion J Clin Psychiatry 6 (2010), https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3067983/.   [3] U.S. FDA, Drug Safety Communication: FDA approves new label changes and dosing for zolpidem products and a recommendation to avoid driving the day after using Ambien CR (May 14, 2013), https://www.fda.gov/drugs/drug-safety-and-availability/fda-drug-safety-communication-fda-approves-new-label-changes-and-dosing-zolpidem-products-and.   [4] FDA: Cut Ambien Dosage for Women, ABC News (January 10, 2013, 6:03AM), https://abcnews.go.com/Health/fda-recommends-slashing-sleeping-pill-dosage-half-women/story?id=18182165.   [5] David J Greenblatt, Jerold S Harmatz, & Thomas Roth, Zolpidem and Gender: Are Women Really At Risk?, 39(3) J. Clinical Psychopharmacol. 189 (May/Jun 2019), https://pubmed.ncbi.nlm.nih.gov/30939589/.   [6] NIH Inclusion Outreach Toolkit: How to Engage, Recruit, and Retain Women in Clinical Research, last accessed September 16, 2024: https://orwh.od.nih.gov/toolkit/recruitment/history.   [7] FDA, Using Artificial Intelligence and Machine Learning in the Development of Drug and Biological Products (May 10, 2023), https://www.fda.gov/media/167973/download; see also Using Artificial Intelligence and Machine Learning in the Development of Drug and Biological Products; Availability, 88 FR 30313 (May 11, 2023), https://www.federalregister.gov/documents/2023/05/11/2023-09985/using-artificial-intelligence-and-machine-learning-in-the-development-of-drug-and-biological.   [8] Thi Tuyet Van Tran, Hilal Tayara, and Kil To Chong, Artificial Intelligence in Drug Metabolism and Excretion Prediction: Recent Advances, Challenges, and Future Perspectives, 15 Pharmaceutics. 1260 (Apr 17, 2023), https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10143484/.   Lily Li is an AI, data privacy, and cybersecurity lawyer and founder of Metaverse Law. She is a certified information privacy professional for the United States and Europe and is a GIAC Certified Forensic Analyst for advanced incident response and computer forensics. She can be reached at info@metaverselaw.com.
1 2 3 4