0
Chicago Grand Central Looking Up

DOJ Issues Final Rule on US Bulk Sensitive Data

Data Brokers, Data Transfers, United States , , , ,
The International Emergency Economic Powers Act (IEEPA) vests the President with authority to deal with extraordinary threats to national security and foreign policy that have their source in part or in whole outside of the United States. Acting pursuant to the IEEPA, President Biden issued Executive Order 14117, “Preventing Access to Americans’ Bulk Sensitive Personal Data and United States Government-Related Data By Countries of Concern” (the EO). The EO directed the Department of Justice (DOJ or Department) to establish and implement regulations addressing threats from certain countries of concern attempting to access and exploit bulk amounts of US sensitive data, including personal and government data. On December 27, 2024, the DOJ issued the Final Rule, which went into effect on April 8, 2025. Additional compliance provisions for certain transactions take effect on October 6, 2025. The Final Rule prohibits or restricts a range of transactions involving categories of bulk sensitive personal data or government-related data between the US and countries of concern or covered persons. In assisting businesses to adapt to this comprehensive update, the DOJ provided a Fact Sheet, a Compliance Guide, and over 100 FAQs on the Final Rule, along with an Implementation and Enforcement Policy. Below are five main takeaways that US entities may want to consider in light of these regulations.
  1. Enforcement May Be More Lenient Until July 8, 2025 
The DOJ’s Implementation and Enforcement Policy, states that the Department will “target its enforcement efforts during the first 90 days to allow US persons (e.g., individuals and companies) additional time to continue implementing the necessary changes to comply with the [Final Rule].” The Department’s civil enforcement actions for violations of the Final Rule will not be a priority “so long as the person is engaging in good faith efforts to comply with or come into compliance with the [Final Rule] during that time.” However, the Department makes clear that it will “pursue penalties and other enforcement actions as appropriate for egregious, willful violations” during the delayed enforcement period.
  1. DOJ Will Consider Good Faith Efforts to Comply
While the Implementation and Enforcement Policy reflects that civil actions for violations of the Final Rule will not be a priority, this depends on the entity’s good faith effort to comply. According to this Policy, examples of evidence of good faith efforts may include, but are not limited to:
  • Conducting internal reviews of access to sensitive data.
  • Conducting internal reviews to determine whether transactions involving access to such data flows constitute data brokerage.
  • Reviewing internal datasets and datatypes to determine if they are subject to the Final Rule.
  • Conducting due diligence on potential new vendors.
  • Renegotiating vendor agreements or negotiating contracts with or transferring products or services to new vendors.
  • Adjusting employee work locations, roles or responsibilities.
  • Evaluating investments from countries of concern or covered persons.
  • Implementing the CISA Security Requirements.
  1. “Good Faith” May Include Satisfying CISA Security Requirements 
A good-faith effort to comply may be demonstrated, in part, by implementing the CISA Security Requirements, which were developed concurrently with the Final Rule pursuant to the EO. The security requirements are intended to address threats that arise when conducting restricted transactions, as detailed below. These security requirements are divided into two sections: i) organizational- and covered system-level requirements; and ii) data-level requirements.
  1. Before October 6, 2025, Determine if Your Company is Conducting Restricted Transactions
US entities engaged in restricted transactions under the Final Rule have affirmative data compliance program and audit obligations, among other obligations. In addition, the Final Rule provides that data brokerage transactions are prohibited with any foreign entity unless the US person contractually binds the foreign entity from subsequent transactions of that data with a country of concern or covered person. They must also report any known or suspected violation of this requirement.
  1. An Iterative Review Plan May be Needed for Covered Transactions 
With the Final Rule coming into effect and enforcement nearing, US companies that engage in certain data transactions or share information with third parties that may be covered persons or countries of concern should evaluate their transactions and data practices. After a thorough review of the types of information collected, who that information is shared with, and who is involved in the processing of that data, it may be helpful to adopt a compliance policy to ensure transactions are being handled appropriately in light of the Final Rule.
0

The Do’s and Don’ts of DSARs: A Practical Guide for Responding to Data Subject Access Requests

DSAR, EU, GDPR, State Privacy Laws, United States , , , , , ,
Handling data subject access requests (DSARs) isn’t as easy as ticking a compliance checkbox. It can be a test of an entity’s data organization, internal communication, and understanding of legal requirements. Between navigating jurisdictional nuances and meeting strict deadlines, the DSAR response process can quickly unravel without a clear plan. In this guide, we suggest best practices for handling and responding to DSARs, along with tips and common pitfalls to avoid when planning effective responses.

1.    Understand the Individual’s Ask

Under international data privacy laws, including those in the US and EU, individuals may have rights over the personal data collected about them by covered entities. The way individuals generally actualize those rights are through DSARs submitted to the relevant entities. These rights can include, but are not limited to:
  • Accessing Data: Individuals may request access to all or specific categories of their personal data.
  • Ceasing Data Processing: Individuals may request the entity stop processing their personal data.
  • Data Correction or Deletion: Individuals may request rectification of inaccurate or outdated personal data or even request the deletion of their personal data.
  • Processing Information: Individuals may request what their personal data is used for and why.
  • Portability: Individuals may request to receive a copy of their personal data in a portable format.
When an individual makes a request to exercise one of these rights, the entity must then respond to the request within a set time frame determined by the applicable law. These time frames differ between applicable laws, so the first step is ensuring you know the appropriate time frame to apply. Who can submit a DSAR? DSARs may be submitted by individuals whose data is processed by entities under the scope of laws like the GDPR and US state privacy laws. Depending on the jurisdiction, DSARs may also be submitted by employees of the covered entity or by agents appointed by the individual and authorized to submit DSARs on the individual’s behalf. Why are DSARs important? DSARs allow individuals to determine what information a covered entity holds about them, how it’s being used, and why it is being processed. In short, they empower individuals to understand and exert some control over their personal data. Additionally, DSARs serve as a tool to confirm that covered entities are upholding their promises: by using these requests, individuals can check whether entities are adhering to both privacy laws and customer privacy notices. This allows individuals to better hold entities accountable for lawful data processing.

2.    Build A Response Team

Given the complexity of modern data systems, internal collaboration is essential when handling DSARs. Clear communication helps ensure DSARs are handled effectively—especially for more comprehensive requests, like deleting or accessing an individual’s data. To build your response team, start by identifying key players. Privacy officers can help oversee legal and regulatory compliance, data experts can help retrieve and process data securely, and communication teams can help draft clear responses to requests and questions. While the specific structure of each team will vary based on the covered entity’s size and complexity, every member of the team should understand the DSAR requirements and specific responsibilities, and get proper training based on their role. Do: Train Your Team       Training is critical to help every member of the team understand the importance of DSARs and their role in maintaining compliance. This isn’t about knowing the legal jargon—each team member should be able to recognize these requests (even if worded in a vague or informal way) and how to execute the steps required to meet deadlines. Since each DSAR is unique, teams should also have a clear point of contact for guidance and next steps if there is any confusion. Don’t: Delay Decisions Effective responses generally take effective planning. Because of the tight DSAR response deadlines imposed by applicable laws, covered entities should plan for these requests before they arrive. By defining clear rules, covered entities can avoid last-minute confusion and chaos when responding to DSARs.

3.    Prepare A Playbook

The regulatory landscape governing DSARs is far from uniform. Because each law may have its own requirements and response timeline, it is essential to understand jurisdiction-specific obligations. A playbook is a simple way to address these obligations in one place and guide the response team through a step-by-step process. To create a playbook, consider:
  • Legal scope: Identify applicable laws based on where the entity operates and whose personal data they process.
  • Verification requirements: Confirm the verification requirements, if any, under each law to determine what steps are needed to confirm the identity of the individual submitting the DSAR.
  • Data retrieval methods: Determine what tools and workflows are needed to locate and compile data efficiently, and how this information may be transmitted to the individual, if necessary.
  • Template responses: Draft standardized responses for anticipated outcomes, like fulfillment or denial of requests, or requests for additional information.
  • Escalation plans: Provide guidance for handling complex requests.
Playbooks should be regularly reviewed to reflect changes in regulations or operational processes. Do: Note the Nuances of Each Law Laws that provide individuals with rights over their personal data commonly include exemptions, such as data that is covered by other laws. Double-check and note these requirements for each jurisdiction and ensure that the playbook is marked in a way that users can easily understand it. Don’t: Forget to Customize Using the same strategy for every DSAR risks a misstep in responses. Privacy laws are often unique, and failing to adapt to these nuances can lead to delays, incomplete responses, or even regulatory penalties. By making your playbook specific to both your entity’s needs and the requirements of each jurisdiction, you are better preparing your team to handle DSARs.

4.    Respond Effectively

Most data privacy laws require a response within a certain time frame from when the request was received. In other words, once a DSAR is received, a clock usually starts ticking. We suggest the following steps as a starting place for a well-executed response, but your steps should be tailored to the applicable legal requirements:
  1. Acknowledge the Request: Confirm the request and provide a clear timeline for how the request will be handled.
  2. Verify the Identify (as needed): Ensure the individual’s identity is confirmed, if required by the relevant laws.
  3. Locate and Collect Data: Collaborate across departments as needed to gather the relevant information.
  4. Review Data for Exceptions: Identify data that may be exempt from disclosures or require redaction, like data that pertains to another individual.
  5. Respond Clearly: Deliver the response in a clear, accessible format with an explanation of how that response was arrived at.
  6. Record and Learn: Maintain detailed records for accountability and review the process regularly.
 Do: Build a Feedback Loop    The best way to learn is by doing. After developing your playbook, perform a trial exercise to ensure your communication is streamlined and a test request is handled as expected. Then, talk to your team to review what went well and what improvements are needed. By viewing this process as iterative, with modifications and refinements made along the way, the DSAR response team can effectively grow and shift with the volume of requests or any regulatory changes. Don’t: Overlook Redaction and Exemptions Redaction and exemptions can easily be overlooked, but neglecting these steps can lead to non-compliance, or even a breach. Always double-check any information before it is disclosed and verify that all information is accounted for and handled appropriately.   While typically seen as a compliance obligation, DSARs can also present an opportunity for entities to demonstrate data privacy and transparency. Each DSAR is a chance to refine operations, and with a capable response team and a detailed playbook, entities can approach the process with a better understanding of compliance.
0

Hoyoverse, developer of Genshin Impact, to pay $20 million to settle FTC complaint

Children, Gaming , , ,
On January 17, the Federal Trade Commission (FTC) announced a proposed settlement with Cognosphere Pte. Ltd and its subsidiary Cognosphere, LLC, doing business as Hoyoverse, developer of gacha video games such as Genshin Impact and Zenless Zone Zero, over allegations that Hoyoverse’s loot boxes and children’s data collection practices violated various federal laws. What is a gacha video game? Generally, a “gacha” video game is one that can be downloaded and played for free but is monetized by selling in-game currency that can be spent on chance-based rewards, which the FTC refers to as “loot boxes.” The loot box rewards range from playable characters to cosmetics to equipment for specific characters, but the reward a player receives is based on chance (e.g., one percent chance to receive X reward) and which reward a player received is revealed only after the player has paid to open the loot box. In games such as Genshin Impact, certain rewards are often featured and available for limited periods of time. For example, if a new character is introduced into the game, the character is typically only available as a rare loot box reward for, say, three weeks. The character is not available for direct purchase, and if the player misses the character as a reward, a rerun of the character as a loot box reward may not happen for months or even years. According to the FTC, this causes players to “purchase dozens of loot boxes, at the cost of hundreds of dollars,” to obtain the featured characters within the limited availability time frame. What did Hoyoverse allegedly do? According to the FTC’s complaint, Hoyoverse violated the FTC Act by misrepresenting the odds and cost of their loot boxes and violated the Children’s Online Privacy Protection Act (COPPA) by failing to provide notice to and collect sufficient consent for children younger than 13 years old.
  • The FTC Act
The FTC claims that Hoyoverse violated the FTC act by making false or misleading representations in advertisements, marketing, and promotions about the odds of obtaining a particular reward in a loot box. For example, Hoyoverse’s social media ads claimed that certain rewards would have a “huge drop-rate boost,” when in reality, the purported “boost” in odds was referring to a featured prize being available to obtain “at all” during the limited availability period, “while the underlying odds of obtaining the featured prize remain[ed] the same.” So, the odds for the reward essentially went from zero percent to the standard percent for the given reward tier (e.g., 5-star rewards may be one percent). The FTC further claims that Hoyoverse’s loot box system constitutes an unfair act or practice, because purchasing a loot box requires the player to navigate a “complex and confusing multi-tier virtual currency exchange system” to purchase a loot box. This system typically requires the player – including children and teenagers, according to the FTC – to purchase in-game currency with actual money and transform that in-game currency into other in-game currency, sometimes multiple times, before being able to purchase a loot box. This multi-tier virtual currency system poses an increased risk for children and teenagers, “whose executive function skills are not yet fully developed” and therefore are “particularly susceptible” to the system’s monetization and pressure to spend money on virtual currency. As such, because children and teenagers can purchase virtual currency in the multi-tier system without first obtaining parental consent to such purchases, the FTC alleged that the system, in the context of children and teenagers, violated the FTC Act as an unfair act or practice.
  • COPPA
According to the FTC, Hoyoverse’s Genshin Impact is covered by COPPA because it is an online service directed to children, and Hoyoverse had actual knowledge that it collected personal information from children under the age of 13. The FTC alleged that Genshin Impact is directed to children under 13 because, in part, the game features matter, visual content, animated characters, and activities that are directed to children. For example, the gameplay and subject matter revolve around exploring, role-playing and collecting a team of heroes, and “engaging in fantasy combat with no blood or gore,” which the FTC claimed are all mechanics like those in other games “popular with children.” And Genshin Impact’s use of anime-style cartoon graphics and colorful animation, according to the FTC, further emphasizes the game’s appeal to children. In particular, Hoyoverse’s use of child-like characters such as Paimon and Klee in promotional materials (e.g., the game’s icon in app stores) serves as evidence of the game’s appeal to children. Despite the applicability of COPPA to Genshin Impact, Hoyoverse failed to satisfy COPPA’s requirements. Specifically, the FTC alleged that Hoyoverse violated COPPA by:
  1. Failing to provide notice on their website or in Genshin Impact of the information collected from children, how they used that information, and to whom they disclosed the information;
  2. Failing to provide the above information directly to parents; and,
  3. Failing to obtain consent from parents before collecting personal information from children.
In addition, because violations of COPPA can constitute an unfair or deceptive act or practice under the FTC Act, the FTC also included such a violation amongst their COPPA-related allegations. What does the $20 million settlement obligate Hoyoverse to do? To settle the FTC’s claims against Hoyoverse, Hoyoverse entered into a proposed settlement order, which will require Hoyoverse to pay a $20 million fine and make changes to address the allegations in the complaint. Hoyoverse will be:
  • Prohibited from allowing children under 16 to purchase loot boxes in Genshin Impact or other Hoyoverse video games without a parent’s affirmative express consent;
  • Prohibited from selling loot boxes using virtual currency without providing an option for consumers to purchase loot boxes directly with real money;
  • Prohibited from misrepresenting loot box odds, prices, and features;
  • Required to disclose loot box odds and exchange rates for multi-tiered virtual currency;
  • Required to delete any personal data previously collected from children under 13 unless they obtain parental consent to retain such data; and,
  • Required to comply with COPPA, including its notice and consent requirements.
Key takeaways? While much of the FTC’s complaint and proposed settlement order references loot boxes in the alleged violations, the FTC’s allegations are more focused on how Hoyoverse promoted and operated its loot box system and to whom they were selling loot boxes. First, if a video game company seeks to monetize their game using loot boxes, the company should consider whether their advertising and promotional material obscures or otherwise inaccurately details the odds of winning a particular reward. For example, there is greater risk in saying a particular reward is “boosted” or its odds are “increased,” when the odds are going from zero percent to the usual percentage rate for a given reward rarity. Second, if a video game uses anime-style graphics, child-like characters, and no blood or gore, the video game should consider satisfying COPPA’s obligations, which may include informing children and parents about the game’s information practices and collecting consent from parents to collect such information. Lastly, if the game sells loot boxes to children under 13 and teenagers under 16, the game may need to satisfy a parental consent requirement before allowing either the children or teenagers to purchase virtual currency or loot boxes.
0
Image depicting the flag of Texas, which is blue, white, and red, with a lone white star.

Texas sues Allstate, continuing Lone Star’s focus on vehicle data regulation

Data / Web Scraping, Data Brokers, United States, Vehicle Data , ,
Update: On Jan. 29, 2025, it was reported that on Jan. 12, 2025, Texas sent Kia America, Inc., a notice of their alleged violations of the Texas Data Privacy and Security Act. Kia has 30 days to cure the alleged violations. Everything is bigger in Texas, including data privacy enforcement. On January 13, 2025, Texas continued its recent regulatory focus on vehicular and geolocation data by initiating a lawsuit against Allstate and its subsidiary, Arity, alleging the companies violated numerous consumer protection and data privacy laws by unlawfully collecting, using, and selling personal, vehicular, and location data without consumers’ knowledge or consent. What led to this lawsuit? For more than half a year, Texas has been leading the regulatory enforcement of vehicular and geolocation data practices. On June 6, 2024, Texas Attorney General Ken Paxton announced that his office had opened an investigation into various car manufacturers “after widespread reporting” that those manufacturers had been secretly collecting mass amounts of data about drivers and selling that data to third parties. The “widespread reporting” cited by Paxton as the seed for the investigation was most likely a nod toward the Mozilla Foundation’s “Privacy Not Included” report published in September of 2023. The report expressly declared that modern vehicles are a “privacy nightmare” and that all 25 car brands researched for the report were labeled as having the worst privacy ever reviewed by the Foundation. The investigation initiated by Paxton in June of 2024 eventually led to a lawsuit against General Motors, filed on August 13, 2024, alleging the company engaged in false, deceptive, and misleading business practices related to its unlawful collection and sale of driving data to insurance companies without the consumers’ knowledge or consent. Following this suit, Paxton’s office sent a notice in November of 2024 to Arity, LLC, a data analytics company founded in 2016 by Allstate, alleging that Arity was in violation of Texas’s recently enacted state privacy law, the Texas Data Privacy and Security Act (the “TDPSA”). The notice identified specific provisions of the TDPSA that Arity was allegedly violating and requested that Arity cure the violations within 30 days, in accordance with the TDPSA’s cure period. But according to Texas’s petition against Allstate and Arity filed on January 13, 2025, Arity failed to cure the alleged TDPSA violations within the 30-day cure period, thereby allowing Texas to include these alleged violations in the lawsuit. What did Allstate and Arity allegedly do? According to Texas’s petition, defendants Allstate and Arity developed and integrated software into third-party apps so that when consumers downloaded the third-party app, they also “unwittingly” downloaded the defendants’ software. The defendants presented the software as “providing a necessary function,” but Texas claims the software does little more than scrape data from the third-party app. Once downloaded, the defendants’ software, through the third-party apps, monitored the consumer’s location and movement “in real-time” and collected trillions of miles of consumer driving data, including geolocation data, accelerometer data, gyroscopic data, and more. The defendants then sold that data to third parties or used it for Allstate’s insurance underwriting. To encourage third parties to integrate the defendants’ software, the defendants paid app developers and offered an incentive program that provided “generous bonus incentives” if developers increased the size of their dataset. All the while, according to Texas, consumers did not consent to, nor were they made aware of, the full extent of defendants’ collection and sale of data. Instead, defendants entered into agreements with the third-party app developers to mandate, to some degree, that certain privacy disclosures and consent language were presented by the third-party apps to consumers, but those third-party disclosures and consent, according to Texas, never mentioned the existence of the defendants, “let alone any of Defendants’ data collection or sales.” Nor did the defendants provide consumers with any of their own notices regarding their data collection practices, and even if consumers did happen to take the extra step to investigate defendants’ policies, those policies contained “untrue and contradictory statements that do not reflect Defendants’ practices.” For example, the policies expressly stated that the defendants do not sell personal data for monetary value, which Texas alleges is untrue, and the policies do not provide consumers with the ability to request that defendants stop selling their data. Taken together, Texas claims these alleged facts establish the basis for numerous legal violations, including violations of the TDPSA, the Texas Data Broker Law, and the Texas Insurance Code. Key takeaways?
  1. Texas is – and will likely remain – focused on regulating vehicular data practices. As the saying goes, once is a coincidence, twice is a pattern, and thrice is a regulatory enforcement focus. Within the short span of half a year, Texas opened an investigation, submitted a notice to cure under the TDPSA, and initiated two lawsuits, all targeting vehicular data practices. Given the rapidity in which Texas is bringing these actions, Texas will likely continue making this an enforcement priority for the near future.
  2. Relying on third-parties to provide notices and collect consent on your behalf may not be enough. The facts allege that the defendants had entered into agreements that, to some degree, obligated the third-party apps to provide notices and collect consumer consent for the collection and sharing of data with the defendants. Yet, according to Texas’s petition, these third-party disclosures and consent collection mechanisms failed to sufficiently inform consumers about the defendants’ data practices.
  3. SDKs remain an area of risk. In recent years, there has been a string of federal and state enforcement action over the use of software development kits (SDKs) to collect and share data. The Federal Trade Commission entered a settlement agreement with InMarket and others; California entered a stipulated judgment of $500,000 with Tilting Point Media; and now a core fact of Texas’s petition is that the defendants developed and integrated SDKs into third-party apps to scrape data.
  4. Carefully consider whether data is being “sold.” Under the TDPSA, a “sale” occurs when personal data is shared, disclosed, or transferred for monetary or other valuable consideration, and Texas alleges that Allstate and Arity “sold” personal data when they sold “data-based products and services for monetary value that linked a specific [consumer] to their alleged driving behavior.” Often, the language of “selling” something conjures to mind ideas of direct financial transactions – exchanging personal data expressly for money or other benefits – but regulators, including those in Texas and California, interpret “selling” personal data more broadly. Thus, companies should carefully review whether their data disclosure and access practices may constitute “selling” personal data, and if so, whether they satisfy the relevant obligations when data is being “sold.”
0

Metaverse Law in Orange County Lawyer Magazine

AI & Algorithms, CCPA, Cybersecurity, GDPR, Health Data, State Privacy Laws , , , , , , ,
The January 2025 edition of Orange County Lawyer magazine features an article written by Metaverse Law’s Lily Li. Read “AI and Machine Learning in Drug Development and Clinical Trials” below or in Orange County Lawyer magazine.
[Originally published as a Feature Article: AI and Machine Learning in Drug Development and Clinical Trials, by Lily Li, in Orange County Lawyer Magazine, January 2025, Vol. 67 No.1, page 28.]   AI and Machine Learning in Drug Development and Clinical Trials by Lily Li   In 2013, sleep medication zolpidem (Ambien, Ambien CR, and Edluar) swept headlines. Marie Claire reported on an alarming and suspicious rise in users experiencing irrational eating, gambling, and even “sleep-driving” while in a hypnotic trance—waking with no memories of their actions.[1] In several cases, women arrested and convicted for driving under the influence contested their convictions, arguing that they were not liable for these undisclosed drug-related side effects. At the same time, several clinical studies suggested that women metabolized zolpidem differently from men. By reviewing existing literature, Japanese researchers out of Shimane University identified 40% higher concentrations of zolpidem in women than men following use, and higher rates of visual hallucinations and sensory distortions.[2] The FDA released a safety advisory, warning users of the risks of “next-morning impairment” for the use of Ambien and related drugs.[3] In addition, the FDA took the unusual step of recommending a 50% cut in the dosage for women. When asked about the change, an FDA director told ABCNews.com: “The changes are different in women and men . . .We don’t understand why yet, but women are more susceptible to next-morning impairment.”[4] Yet, a decade later, the evidence supporting different zolpidem dosages for women and men is unclear.[5] In part, this is due to the lack of research surrounding sex differences in drug impact and drug treatment, as well as substantial gaps in the inclusion of women in clinical studies. From 1977 to 1993, FDA policy recommended excluding women of childbearing potential from Phase 1 and early Phase II drug trials.[6] Even after this policy was removed in 1993, industry fears remained with respect to drug interactions with pregnancy. This episode with zolpidem raised several concerns in the drug development and clinical trial process:
  • How do we recruit representative candidates for drug trials?
  • How do we ensure the quality and availability of datasets for clinical research?
  • How do we measure potential impacts of drug dosing on different populations?
  • What are the legal implications for failing to address appropriate drug doses?
  AI and ML to the Rescue? Now that artificial intelligence is being used in research and development, one wonders: Can artificial intelligence (AI) and machine learning (ML) reduce bias and risks during drug development? Or will it create new legal risks due to bias, privacy intrusions, and lack of transparency? The FDA released a discussion paper on AI, Using Artificial Intelligence and Machine Learning in the Development of Drug and Biological Products, to discuss potential regulatory frameworks to address the use of AI and ML.[7] In this discussion paper, the FDA released a set of fascinating case studies into existing research and uses of AI in the clinical trial process. Several of these case studies are discussed below, as well as an analysis of their potential impact on the zolpidem example.
  1. Recruitment. According to the FDA, “AI/ML is being used to mine vast amounts of data, such as data from clinical trial databases, trial announcements, social media, medical literature, registries, and structured and unstructured data in EHRs [electronic health records], which can be used to match individuals to trials (Harrer, 219 Shah, Antony, & Hu, 2019).” In this manner, researchers can combine huge quantities of publicly available data and individual health data from prior research to identify participants with certain medical conditions (or lack of adverse conditions) for investigational treatments. For zolpidem, the use of AI/ML may have been able to identify a much broader list of participants for initial clinical testing, making it easier to assess and identify adverse reactions.
  2. Selection and Stratification of Trial Participants. In addition to initial recruitment, AI/ ML has the capability improve intake, selection, and classification of clinical trial participants. Based on baseline characteristics selected by the researchers, such as prior clinical data, and vitals/labs taken during intake, predictive algorithms can help identify high-risk participants.[8] These groups can then be randomized and then subject to more strict monitoring protocols. In the case of zolpidem, alcohol use is associated with sometimes severe adverse effects from the drug, and so it would be beneficial to screen out candidates with a history of alcoholism or, on the flip side, assess drug interactions for this high-risk group with additional support, monitoring, or counseling.
  3. Dose/Dosing Regimen Optimization. AI/ML can be used to predict drug exposure for different populations based on factors such as weight, height, sex, and other characteristics that might impact drug metabolism. Based on prior drug exposure and response profiles for similar drugs and similar populations, AI/ML can help to narrow the dose/dosing regimen selected for a study. As noted by the FDA’s discussion paper, this can help optimize drug dosing “in special populations where there may be limited data (e.g., rare disease studies, pediatric and pregnant populations).” Based on this research, we can imagine future scenarios where AI/ML could have avoided zolpidem dosing concerns, where graduated and limited dosing was tested and applied to different sex, age, and metabolism categories to determine ideal dosing.
  4. Data Analysis. On a more intriguing level, the FDA AI discussion paper discussed the concept of creating “digital twins” of patients for clinical trials. Essentially, an AI version of the clinical participant is created, using the existing candidate’s electronic health records, vital signs, labs and other records. Researchers can assess how the digital twin would react under normal conditions using AI/ML modeling based on data gathered from similar individuals. This digital twin would then act as a substitute for a placebo candidate in a clinical trial, and act as a benchmark against the actual patient undergoing investigational treatment. For zolpidem, this could be used to assess candidates that already have underlying medical conditions such as anxiety, depression, or other confounding factors, to see whether an adverse effect from a trial is due to the investigational treatment or something that is likely to occur to the same individual from anxiety alone.
  5. Postmarketing Safety Surveillance. Finally, AI/ML can help detect and assess adverse events once the drug enters the market. This is not just limited to individual case safety reports (ICSR), required by regulators, but can include adverse events reported publicly on social media and the wider internet. This type of postmarketing safety surveillance could assist researchers and drug companies in identifying potential drug risks, prior to landing on primetime news.
  Quality and Reliability Risks While AI/ML can help to address the costs and efficiency of clinical trials, this relies substantially on the underlying data used to train AI. The quality and reliability of any AI/ML model requires similar quality controls for underlying training data. Given the safety risks of inappropriate drug dosing, or recruiting candidates with severe medical conditions, AI developers cannot rely solely on self-reported healthcare data with no external medical testing or validation. Developers should be equally wary of training on third-party data sets that do not provide documentation on the collection of data and data validation. Within an existing healthcare organization, if the organization is big enough, aggregate and de-identified data may be obtained from existing electronic health care records and prior clinical trials. Yet, even within these large datasets, errors may surface during training. Medical providers may code the same procedure, and similar symptoms, a dozen different ways. Even drug names can be misspelled and coded incorrectly within existing records. While many of these errors may end up being statistically insignificant with enough data, there is the risk of missing one or two major adverse events, or “black swan” events, that would otherwise change the entire risk profile of a drug. In addition to quality and reliability, the underlying dataset needs to be representative of the population that will be studied for the clinical trial. If the underlying dataset is only trained on a handful of individuals with a certain medical predisposition, age, sex, weight, etc., it will be difficult for the AI model to make predictions for that group. As an example, if the training data only contains the medical information for two individuals over the age of sixty, and shows no adverse effects from a particular drug dose, this information is not enough to generalize that the drug at that dosage is appropriate for all individuals over the age of sixty. For all we know, these two candidates could be a former Olympic diver and a nutrition coach, two outliers that completely skew the data. Consequently, the underlying training data for any AI model should also be assessed for bias and representativeness as it applies to the proposed clinical trial.   Data Privacy, Cybersecurity, and AI Risks The data privacy and cybersecurity risks associated with the foregoing uses of AI/ML cannot be underestimated. The quality and representativeness of any AI system in this field will rely heavily on large swathes of healthcare data, fine-tuned and, at times, personalized in the case of digital twins. This is sensitive or special category data at its finest, triggering heightened scrutiny under the EU’s data privacy law, the GDPR, and U.S. data privacy and data breach laws. To date, most healthcare organizations have sidestepped data privacy concerns by relying on HIPAA’s de-identification standard to remove personal information and other identifiers from healthcare data, making it difficult to associate with an individual. While the FDA requires Institutional Review Board (IRB) review of most biomedical research involving human subjects, this generally does not apply to de-identified personal information that cannot be linked to an individual. Simply de-identifying data and then running with it is not enough, however. Under the California Consumer Privacy Act and similar state laws, for example, recipients of de-identified data need to affirm that they will not attempt to reidentify the data (except to test their de-identification methods). The GDPR has a much higher “anonymization” standard, which looks at the re-identifiability of personal information, given all the different datasets that an organization may have access to. AI/ML itself is making the de-identification process harder. As it is capable of slicing and dicing data by age, race, sex, and medical condition, and combining multiple large datasets, it is easy to run the risk of re-identifying data. While several thousand people might have the same configuration of eye color, age, gender, and weight, only one or two may have participated in a clinical trial at a particular location, or have specific allergies or side effects to certain types of medication. As a result, in circumstances where healthcare data is not de-identified, or the risk of reidentification is heightened, then it behooves clinical organizations and their AI developers to implement written information security programs and associated privacy and security controls.   Legal Liability and Drug Dosing In several notable cases, defendants on zolpidem were able to contest or overturn DWI or even vehicular manslaughter cases. Essentially, these defendants argued that they were not aware of the potential dangers of zolpidem, and so could not be liable for their actions while “sleep driving.” This raises the question: If AI gets good enough, and can tell you exactly the right dose to take of a drug, will you (or your doctor) be liable if you deviate from the AI’s recommendations? Will the AI’s recommendations be discoverable in court (and surfaced via AI-enhanced search)? Only time will tell what this brave new world will bring.   ENDNOTES [1] Kai Falkenberg, While You Were Sleeping (September 27, 2012), Marie Claire, https://www.marieclaire.com/culture/news/a7302/while-you-were-sleeping/.   [2] Takuji Inagaki, Tsuyoshi Miyaoka, Seiichi Tsuji, Yasushi Inami, Akira Nishida, and Jun Horiguchi, Adverse Reactions to Zolpidem: Case Reports and a Review of the Literature, 12 Prim Care Companion J Clin Psychiatry 6 (2010), https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3067983/.   [3] U.S. FDA, Drug Safety Communication: FDA approves new label changes and dosing for zolpidem products and a recommendation to avoid driving the day after using Ambien CR (May 14, 2013), https://www.fda.gov/drugs/drug-safety-and-availability/fda-drug-safety-communication-fda-approves-new-label-changes-and-dosing-zolpidem-products-and.   [4] FDA: Cut Ambien Dosage for Women, ABC News (January 10, 2013, 6:03AM), https://abcnews.go.com/Health/fda-recommends-slashing-sleeping-pill-dosage-half-women/story?id=18182165.   [5] David J Greenblatt, Jerold S Harmatz, & Thomas Roth, Zolpidem and Gender: Are Women Really At Risk?, 39(3) J. Clinical Psychopharmacol. 189 (May/Jun 2019), https://pubmed.ncbi.nlm.nih.gov/30939589/.   [6] NIH Inclusion Outreach Toolkit: How to Engage, Recruit, and Retain Women in Clinical Research, last accessed September 16, 2024: https://orwh.od.nih.gov/toolkit/recruitment/history.   [7] FDA, Using Artificial Intelligence and Machine Learning in the Development of Drug and Biological Products (May 10, 2023), https://www.fda.gov/media/167973/download; see also Using Artificial Intelligence and Machine Learning in the Development of Drug and Biological Products; Availability, 88 FR 30313 (May 11, 2023), https://www.federalregister.gov/documents/2023/05/11/2023-09985/using-artificial-intelligence-and-machine-learning-in-the-development-of-drug-and-biological.   [8] Thi Tuyet Van Tran, Hilal Tayara, and Kil To Chong, Artificial Intelligence in Drug Metabolism and Excretion Prediction: Recent Advances, Challenges, and Future Perspectives, 15 Pharmaceutics. 1260 (Apr 17, 2023), https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10143484/.   Lily Li is an AI, data privacy, and cybersecurity lawyer and founder of Metaverse Law. She is a certified information privacy professional for the United States and Europe and is a GIAC Certified Forensic Analyst for advanced incident response and computer forensics. She can be reached at info@metaverselaw.com.
1 2 3 … 5