0
Flag of California, depicting a large brown bear beside a red star, above the words "California Republic."

CCPA Draft Regulations Sent for Final Approval

California, CCPA, Cybersecurity , , ,
On July 24, 2025 the California Privacy Protection Agency (CCPA) board voted 5-0 to finalize Draft Regulations to the California Consumer Privacy Act (CCPA). The CPPA sent the rulemaking package to the Office of Administrative Law, which has 30 days to approve the regulations. The rulemaking process for these Draft Regulations began in 2022, and while the regulations have been narrowed since the prior proposal, the Draft Regulations will significantly impact how companies manage automated decisionmaking technology (ADMT), conduct risk assessments, and implement cybersecurity audits. Additionally, California’s regulatory process requires the CCPA to respond to public comments with their rationale for accepting or rejecting the suggestion. This requirement provides additional context and guidance for interpreting the intent of the Draft Regulations as they go into effect. What’s New? A Summary of Key Changes The Draft Regulations contain significant changes from the prior proposal – along with a 9-page explanation of changes. Most notably, the Draft Regulations roll back several of the most highly debated elements, while streamlining and clarifying other requirements:
  • References to “Artificial Intelligence” have been removed, significantly tightening the scope of ADMT systems.
  • First-party advertising removed from ADMT definition, narrowing the requirements needed for this type of processing.
  • Risk assessments are streamlined, and the scope of the types of data processing activities that trigger risk assessments has been narrowed.
  • Cybersecurity audits are clarified, and the CPPA included a “cybersecurity audit report” which should be produced during the audit process.

ADMT: Narrower Definition, Clearer Application

The Draft Regulations significantly narrow the scope of ADMT systems. Previously, ADMT systems included any technology that “substantially facilitated” human decisionmaking. Now, the Draft Regulations limits ADMT to systems which “substantially replace” human decisions. In practical terms, this may mean that only technologies which operate without human review or override fall under the ADMT rules. Importantly, the CPPA also removed first-party behavioral advertising from the definition of ADMT. Previously, businesses raised strong concerns that including this category within the ADMT definition would impose unnecessary burdens on common advertising practices. Businesses also voiced that including first-party behavioral advertising in the definition of ADMT went beyond Proposition 24, which provides the basis for amending the CCPA.

Risk Assessments: Who, What, and When?

While risk assessments remain a key part of the Draft Regulations, the CPPA has refined when they apply and what they must include. Who Needs to Conduct a Risk Assessment? Under the Draft Regulations, covered businesses that fall under the California Consumer Privacy Act (CCPA) “whose processing…presents significant risk to consumers’ privacy” must conduct a risk assessment. However, the newest version of the Regulations narrows what processing activities present “significant risk.” These activities include but are not limited to:
  • Selling or sharing personal information, which may require specific contractual obligations per the CCPA and current CCPA Regulations.
  • Processing sensitive personal information, as defined in the CCPA, including financial information, precise geolocation, health information and children’s personal information.
  • Using automated decisionmaking technology for a “significant decision” concerning a consumer, including those that impact availability of financial or lending services, housing, education enrollment or opportunities, employment or independent contracting opportunities or compensation, or healthcare services.
  • Using automated processing to profile a consumer through systematic observation when the individual is acting as an educational program application, job applicant, student, employee, or independent contractor for the covered business.
  • Using automated processing to profile a consumer based on their presence in a sensitive location, including healthcare facilities, domestic violence shelters, food pantries, housing/emergency shelters, educational institutions, political party offices, legal services offices, union offices, and places of worship.
  • Using personal information to train AI that could be used to make significant decisions concerning consumers, train facial- or emotional-recognition or other technology to verify a consumer’s identify or conducts physical or biological identification or profiling of a consumer.
While these risk assessments no longer apply to the previous expanded version of ADMT, they will apply to processing if the technology substantially replaces human decisionmaking for “significant decisions.” For example, if a covered business videotapes job interviews and uses AI to determine who to hire without human involvement, the covered business must conduct a risk assessment because of its use of ADMT for a significant decision concerning the consumer. What is Required for a Risk Assessment? As part of an effort to streamline and clarify the risk assessments required under the Draft Regulations, the CPPA defined a “risk assessment report” as the document that every covered business is required to create upon conducting the assessment. The CPPA provides a newly articulated goal for risk assessments: “[R]estricting or prohibiting the processing of personal information if the risks to privacy of the consumer outweigh the benefits resulting from processing to the consumer, the business, other stakeholders, and the public.” Additionally, the addition of the risk assessment report and changes in requirements may ease compliance efforts. To complete a risk assessment, a covered business should document, among other things:
  • The purpose of processing, the types of data involved, and any sensitive categories of personal information.
  • How the business plans to use the data, or otherwise collect, disclose or process the information, along with the retention period for the information.
  • How the business interacts with consumers, and whose data they process, along with the number of consumers whose information will be processed.
  • The disclosures made to consumers, and any other disclosures that the covered business plans to make, along with the names of service provides, contractors, or third parties to whom the information will be disclosed and the purpose for that disclosure.
  • The benefits, negative impacts, and safeguards of the planned processing.
  • Whether or not the business will initiate the processing subject to the risk assessment.
  • The individuals who provided information, as well as who the document was reviewed and approved by.
  If a covered business is using ADMT, the business must also identify:
  • The logic of the ADMT, including any assumptions or limitations of the logic; and
  • The output of the ADMT and how the covered business will use that output to make a significant decision.
The CPPA also clarifies that the risk assessment process may include involvement by external parties. Finally, a covered business must submit the following risk assessment information, among other things, to the Agency:
  • The business’s contact information, the information of the person submitting the assessment, and the date of certification.
  • The time period covered by the submission, and the number of risk assessments conducted or updated during that time.
  • Whether the risk assessments involved the processing of each of the categories of personal information identified in the CCPA.
  • A specific attestation, which certifies the business conducted a risk assessment for the processing activities involving significant decisions, subject to the penalty or perjury.
The individual submitting the information to the Agency must be a member of the covered business’s executive management team who is: 1) directly responsible for the business’s risk assessment compliance; 2) has sufficient knowledge to provide accurate information regarding the assessment; and 3) has the authority to submit the assessment information to the Agency. In addition, the Agency or Attorney General may require a covered business to submit its risk assessment reports at any time, within 30 days of the request. When Should Risk Assessments Be Conducted? According to the Proposed Rules, a covered business must conduct and document a risk assessment before beginning any processing activities that present a significant risk to consumers’ privacy. At least once every three years, the covered business must review and update their assessment. The covered business must also update a risk assessment whenever there is a material change relating to the processing activity, no later than 45 days from the material change. The covered business must retain its risk assessments – including original and updated versions – for as long as the processing continues or for five years after the completion of the risk assessment, whichever is later. What if I Have Already Conducted A Risk Assessment? There have been significant changes to the Draft Regulations regarding how covered businesses can use comparable assessments to satisfy the risk assessment criteria. New additions provide that a covered business may use a risk assessment that it has prepared for another purpose, provided that the assessment contains or is paired with all the required information to meet the Proposed Regulation’s requirements.

Cybersecurity Audits: Who, What, and When?

Among the added definitions is the “cybersecurity audit report” – the document that covered businesses must create as part of the cybersecurity audit. Similar to changes regarding risk assessments, this inclusion was part of the streamlining and clarification efforts of the CPPA. The scope and requirements of the cybersecurity audit – and the resulting audit report – have also been modified. Who Needs to Complete a Cybersecurity Audit? According to the Draft Regulations, every covered business whose processing of information presents a “significant risk” to consumers’ security must complete a security audit. While this language is similar to the requirements of the risk assessment, “significant risk” is defined slightly differently in the context of a cybersecurity audit. According to the Draft Regulations, a “significant risk” that warrants a cybersecurity audit includes but is not limited to covered businesses which:
  1. Derive 50% of more of its annual revenue from selling or sharing consumer’s personal information; or
  2. Had a gross annual revenue of $25M in the preceding calendar year (adjusted for inflation), and
    1. Processed the information of 250,000 or more consumers or households in the last year; or
    2. Processed the sensitive information of 50,000 or more consumers in the last year.
Covered businesses that are required to complete a cybersecurity audit must do so using a “qualified, objective, independent processional (‘auditor’) using procedures and standards accepted in the profession of auditing.” This audit may be internal or external to the covered business, but a qualified auditor must have knowledge of cybersecurity and know how to audit a business’s cybersecurity program, according to the changes in the Draft Regulations. What Should the Cybersecurity Audit Assess? Initially, the cybersecurity audit must assess how the covered business’s cybersecurity program protects personal information against unauthorized access, destruction, use, modification and disclosure, as well as how the program protects against unauthorized activity resulting in the loss of availability to that information. The cybersecurity audit must also assess the strength of a covered business’s cybersecurity program across such as, but not limited to:
  • Authentication and encryption;
  • Access control and account management;
  • Software and hardware inventories;
  • Patch and configuration management;
  • Network security, antivirus, and antimalware;
  • Incident response and business continuity;
  • Vendor oversight;
  • Data retention and disposal; and
  • Employee and contractor training.
The covered business’s auditor must also create a detailed cybersecurity audit report, documenting:
  • What was assessed and why. The report should describe the processes, activities, and components of the business’s cybersecurity program, the criteria used for the audit, along with the specific evidence examined to make decisions and assessments.
  • Evidence reviewed. The report must also include why these elements were appropriate for the audit, and how the evidence examined supports the findings.
  • Gaps or weaknesses found. The report should describe, in detail, the status of any gaps or weaknesses and any additional components that the auditor deemed to increase the risk of unauthorized activity. The report should also document the business’s plan to address these gaps and/or weaknesses.
  • Auditor information and certification. The report should also include the auditor’s information, as well as a statement by the highest-ranking auditor that certifies that they completed an independent review of the business’s cybersecurity program and information system, exercised objective and impartial judgement on all issues within the scope of the audit and did not rely primarily on assertations or attestations by business management to create the audit.
When Should Cybersecurity Audits Be Conducted? The final determination of when a covered business must conduct their first cybersecurity audit is based on the business’s annual gross revenue. If a business meets the audit thresholds, it may be time to start thinking about a compliance plan. First audit reports will be due:
  • April 1, 2028, for covered businesses with over $100 million in gross annual revenue;
  • April 1, 2029, for covered businesses with $50 million to $100 million in gross annual revenue; and
  • April 1, 2030, for covered businesses with under $50 million in revenue.
Each audit must cover the previous calendar year from January to January, with reports completed within the following three months. What if I Have Already Conducted A Cybersecurity Audit? As with the risk assessment, a covered business may use a cybersecurity audit, assessment, or evaluation that it has prepared for another purpose – provided that the audit meets all the requirements of the Draft Regulations, on its own or through supplemental information. The Draft Regulations provide, as an example, that a covered business may use the NIST Cybersecurity Framework 2.0 “and meets all the requirements of this Article.”

What Comes Next?

On July 24, 2025, the CPPA sent the rulemaking package to the Office of Administrative Law, which has 30 days to approve the regulations. The CPPA’s Draft Regulations signal a more measured approach to emerging technologies, such as AI. Still, these Draft Regulations carry out the CPPA’s mandate to issue regulations, reinforcing the agency’s commitment to privacy and security. For executives, the potential adoption of the Draft Regulations could be a strategic inflection point: Whether they are responsible for legal, compliance, data governance or information security, these Draft Regulations should prompt a reassessment of data practices, internal documentation and audit readiness. The publication of these Draft Regulations is also an opportunity to engage more deeply with operational teams. These rules will require clear cross-functional coordination, and organizations that begin building these bridges sooner will be better positioned to meet regulatory expectations and reinforce consumer trust in coming years. Compliance Deadlines: Compliance with these Draft Regulations will be required once they are approved by the Office of Administrative Law. The deadlines include:
  • ADMT Regulations: January 1, 2027
  • Privacy Risk Assessments: December 31, 2027
  • Cybersecurity Audits:
    • For businesses with $100+ million in annual gross revenue: April 1, 2028.
    • For businesses between $50 million and $100 million in annual gross revenue: April 1, 2029.
    • For businesses with less than $50 million in annual gross revenue: April 1, 2030.
 
0

Overview of New York’s Child Data Protection Act

Children, Cybersecurity, New York, State Privacy Laws, United States , , , , ,
In June 2024, New York Governor Kathy Hochul signed the New York Child Data Protection Act (Act) into law, which will go into effect on June 20, 2025. Per the Act’s justification, “[c]hildren now live much of their lives online,” including learning, socializing, shopping. They also “make mistakes online, and they discover who they are online,” and, accordingly, they should be able to do so without the “concern of omnipresent monitoring and recording.” The Act enables this through two major provisions:
  1. if a digital service knows a user is a minor (or if the service is primarily directed to minors), it will “default to only being able to use that child’s data in a way that is strictly necessary to provide the service;” and
  2. digital services using third-party service providers must “contractually restrict those third parties from using the personal data of minors except for specified purposes” and include additional safeguards to help ensure compliance.
The Office of the New York State Attorney General has also released Implementation Guidance to clarify key questions raised in the rulemaking process.

Scope & Applicability

This Act applies only to conduct occurring in the state of New York. This means that commercial conduct that takes place outside of New York is not covered by the Act if: 1)  the user was outside of the state or 2) no data collected while the user was in the state was used.
  • Covered Users. The Act imposes restrictions on processing information of “covered users.” This includes users of websites, online services, or connected devices (the “Websites”) who are: 1) actually known by the operator to be a minor (under 18), or 2) who are using Websites primarily directed to minors.
  • Operator. An operator is defined as any person who offers Websites, who alone – or jointly with others – controls the purposes and means of processing personal data. Notably, one who acts as both a controller and processor shall comply with obligations for both roles, depending on the purposes and means of processing personal data.
  • Personal data. This definition includes any data that identifies or could be reasonably linked, directly or indirectly, with a specific natural person or device.

Substantive Provisions

Processing Restrictions. The Act provides that, among other things, an operator shall not process the personal data of a covered user collected through the Sites, unless one of the following applies:
  1. the user is 12 or younger, and processing is permitted under COPPA;
  2. the user is 13 or older and the processing is “strictly necessary”; or
  3. the user is 13 or older and the processor has received informed consent.
Strictly Necessary Processing. The term “strictly necessary” includes, among other things, processing that is required to:
  • Provide or maintain a specific product or service requested by the covered user;
  • Conduct the operator’s internal business operations (excluding those that relate to marketing, advertising, research and development, providing products or services to third parties, pr prompting covers users to use the Site when it is not in use); and
  • Identify and repair technical errors that impair functionality.
According to the Implementation Guidance, processing that is “strictly necessary” to provide a process or service required by a covered user depends on the “expectations of a reasonable covered user,” similar to the guidance provided under the CCPA regulations. The Guidance also clarifies that business operations “shall not include any activities relating to marketing, advertising, research and development, [or] providing products or services to third parties.” Informed Consent. If the information being processed is not “strictly necessary,” the operator will need informed consent, through either: 1) a device communication or signal, or 2) an informed consent request. A request for informed consent should, among other things:
  1. be made separately from any part of the transaction.
  2. clearly and conspicuously state that the processing is not strictly necessary, and consent is not mandatory to continue using the Websites.
  3. clearly present an option to refuse to provide consent as the most prominent option.
Additionally, the user should be able to revoke consent at any time as easily as they provided it.

Enforcement

The New York Attorney General may bring an action or special proceeding to enjoin any violation of this Act, and to obtain civil penalties of up to $5,000 per violation. Further, the Act gives the New York Attorney General authority to issue rules and regulations ad necessary, and according to the Implementation Guidance, the Office of the Attorney General intends to issue these rules. The Implementation Guidance also states that, until such rules are finalized, the Office of the Attorney General will exercise discretion in pursuing enforcement actions, taking good-faith compliance efforts of covered businesses into account.

Effective Date

The Act goes into effect on June 20, 2025.
0

Metaverse Law in Orange County Lawyer Magazine

AI & Algorithms, CCPA, Cybersecurity, GDPR, Health Data, State Privacy Laws , , , , , , ,
The January 2025 edition of Orange County Lawyer magazine features an article written by Metaverse Law’s Lily Li. Read “AI and Machine Learning in Drug Development and Clinical Trials” below or in Orange County Lawyer magazine.
[Originally published as a Feature Article: AI and Machine Learning in Drug Development and Clinical Trials, by Lily Li, in Orange County Lawyer Magazine, January 2025, Vol. 67 No.1, page 28.]   AI and Machine Learning in Drug Development and Clinical Trials by Lily Li   In 2013, sleep medication zolpidem (Ambien, Ambien CR, and Edluar) swept headlines. Marie Claire reported on an alarming and suspicious rise in users experiencing irrational eating, gambling, and even “sleep-driving” while in a hypnotic trance—waking with no memories of their actions.[1] In several cases, women arrested and convicted for driving under the influence contested their convictions, arguing that they were not liable for these undisclosed drug-related side effects. At the same time, several clinical studies suggested that women metabolized zolpidem differently from men. By reviewing existing literature, Japanese researchers out of Shimane University identified 40% higher concentrations of zolpidem in women than men following use, and higher rates of visual hallucinations and sensory distortions.[2] The FDA released a safety advisory, warning users of the risks of “next-morning impairment” for the use of Ambien and related drugs.[3] In addition, the FDA took the unusual step of recommending a 50% cut in the dosage for women. When asked about the change, an FDA director told ABCNews.com: “The changes are different in women and men . . .We don’t understand why yet, but women are more susceptible to next-morning impairment.”[4] Yet, a decade later, the evidence supporting different zolpidem dosages for women and men is unclear.[5] In part, this is due to the lack of research surrounding sex differences in drug impact and drug treatment, as well as substantial gaps in the inclusion of women in clinical studies. From 1977 to 1993, FDA policy recommended excluding women of childbearing potential from Phase 1 and early Phase II drug trials.[6] Even after this policy was removed in 1993, industry fears remained with respect to drug interactions with pregnancy. This episode with zolpidem raised several concerns in the drug development and clinical trial process:
  • How do we recruit representative candidates for drug trials?
  • How do we ensure the quality and availability of datasets for clinical research?
  • How do we measure potential impacts of drug dosing on different populations?
  • What are the legal implications for failing to address appropriate drug doses?
  AI and ML to the Rescue? Now that artificial intelligence is being used in research and development, one wonders: Can artificial intelligence (AI) and machine learning (ML) reduce bias and risks during drug development? Or will it create new legal risks due to bias, privacy intrusions, and lack of transparency? The FDA released a discussion paper on AI, Using Artificial Intelligence and Machine Learning in the Development of Drug and Biological Products, to discuss potential regulatory frameworks to address the use of AI and ML.[7] In this discussion paper, the FDA released a set of fascinating case studies into existing research and uses of AI in the clinical trial process. Several of these case studies are discussed below, as well as an analysis of their potential impact on the zolpidem example.
  1. Recruitment. According to the FDA, “AI/ML is being used to mine vast amounts of data, such as data from clinical trial databases, trial announcements, social media, medical literature, registries, and structured and unstructured data in EHRs [electronic health records], which can be used to match individuals to trials (Harrer, 219 Shah, Antony, & Hu, 2019).” In this manner, researchers can combine huge quantities of publicly available data and individual health data from prior research to identify participants with certain medical conditions (or lack of adverse conditions) for investigational treatments. For zolpidem, the use of AI/ML may have been able to identify a much broader list of participants for initial clinical testing, making it easier to assess and identify adverse reactions.
  2. Selection and Stratification of Trial Participants. In addition to initial recruitment, AI/ ML has the capability improve intake, selection, and classification of clinical trial participants. Based on baseline characteristics selected by the researchers, such as prior clinical data, and vitals/labs taken during intake, predictive algorithms can help identify high-risk participants.[8] These groups can then be randomized and then subject to more strict monitoring protocols. In the case of zolpidem, alcohol use is associated with sometimes severe adverse effects from the drug, and so it would be beneficial to screen out candidates with a history of alcoholism or, on the flip side, assess drug interactions for this high-risk group with additional support, monitoring, or counseling.
  3. Dose/Dosing Regimen Optimization. AI/ML can be used to predict drug exposure for different populations based on factors such as weight, height, sex, and other characteristics that might impact drug metabolism. Based on prior drug exposure and response profiles for similar drugs and similar populations, AI/ML can help to narrow the dose/dosing regimen selected for a study. As noted by the FDA’s discussion paper, this can help optimize drug dosing “in special populations where there may be limited data (e.g., rare disease studies, pediatric and pregnant populations).” Based on this research, we can imagine future scenarios where AI/ML could have avoided zolpidem dosing concerns, where graduated and limited dosing was tested and applied to different sex, age, and metabolism categories to determine ideal dosing.
  4. Data Analysis. On a more intriguing level, the FDA AI discussion paper discussed the concept of creating “digital twins” of patients for clinical trials. Essentially, an AI version of the clinical participant is created, using the existing candidate’s electronic health records, vital signs, labs and other records. Researchers can assess how the digital twin would react under normal conditions using AI/ML modeling based on data gathered from similar individuals. This digital twin would then act as a substitute for a placebo candidate in a clinical trial, and act as a benchmark against the actual patient undergoing investigational treatment. For zolpidem, this could be used to assess candidates that already have underlying medical conditions such as anxiety, depression, or other confounding factors, to see whether an adverse effect from a trial is due to the investigational treatment or something that is likely to occur to the same individual from anxiety alone.
  5. Postmarketing Safety Surveillance. Finally, AI/ML can help detect and assess adverse events once the drug enters the market. This is not just limited to individual case safety reports (ICSR), required by regulators, but can include adverse events reported publicly on social media and the wider internet. This type of postmarketing safety surveillance could assist researchers and drug companies in identifying potential drug risks, prior to landing on primetime news.
  Quality and Reliability Risks While AI/ML can help to address the costs and efficiency of clinical trials, this relies substantially on the underlying data used to train AI. The quality and reliability of any AI/ML model requires similar quality controls for underlying training data. Given the safety risks of inappropriate drug dosing, or recruiting candidates with severe medical conditions, AI developers cannot rely solely on self-reported healthcare data with no external medical testing or validation. Developers should be equally wary of training on third-party data sets that do not provide documentation on the collection of data and data validation. Within an existing healthcare organization, if the organization is big enough, aggregate and de-identified data may be obtained from existing electronic health care records and prior clinical trials. Yet, even within these large datasets, errors may surface during training. Medical providers may code the same procedure, and similar symptoms, a dozen different ways. Even drug names can be misspelled and coded incorrectly within existing records. While many of these errors may end up being statistically insignificant with enough data, there is the risk of missing one or two major adverse events, or “black swan” events, that would otherwise change the entire risk profile of a drug. In addition to quality and reliability, the underlying dataset needs to be representative of the population that will be studied for the clinical trial. If the underlying dataset is only trained on a handful of individuals with a certain medical predisposition, age, sex, weight, etc., it will be difficult for the AI model to make predictions for that group. As an example, if the training data only contains the medical information for two individuals over the age of sixty, and shows no adverse effects from a particular drug dose, this information is not enough to generalize that the drug at that dosage is appropriate for all individuals over the age of sixty. For all we know, these two candidates could be a former Olympic diver and a nutrition coach, two outliers that completely skew the data. Consequently, the underlying training data for any AI model should also be assessed for bias and representativeness as it applies to the proposed clinical trial.   Data Privacy, Cybersecurity, and AI Risks The data privacy and cybersecurity risks associated with the foregoing uses of AI/ML cannot be underestimated. The quality and representativeness of any AI system in this field will rely heavily on large swathes of healthcare data, fine-tuned and, at times, personalized in the case of digital twins. This is sensitive or special category data at its finest, triggering heightened scrutiny under the EU’s data privacy law, the GDPR, and U.S. data privacy and data breach laws. To date, most healthcare organizations have sidestepped data privacy concerns by relying on HIPAA’s de-identification standard to remove personal information and other identifiers from healthcare data, making it difficult to associate with an individual. While the FDA requires Institutional Review Board (IRB) review of most biomedical research involving human subjects, this generally does not apply to de-identified personal information that cannot be linked to an individual. Simply de-identifying data and then running with it is not enough, however. Under the California Consumer Privacy Act and similar state laws, for example, recipients of de-identified data need to affirm that they will not attempt to reidentify the data (except to test their de-identification methods). The GDPR has a much higher “anonymization” standard, which looks at the re-identifiability of personal information, given all the different datasets that an organization may have access to. AI/ML itself is making the de-identification process harder. As it is capable of slicing and dicing data by age, race, sex, and medical condition, and combining multiple large datasets, it is easy to run the risk of re-identifying data. While several thousand people might have the same configuration of eye color, age, gender, and weight, only one or two may have participated in a clinical trial at a particular location, or have specific allergies or side effects to certain types of medication. As a result, in circumstances where healthcare data is not de-identified, or the risk of reidentification is heightened, then it behooves clinical organizations and their AI developers to implement written information security programs and associated privacy and security controls.   Legal Liability and Drug Dosing In several notable cases, defendants on zolpidem were able to contest or overturn DWI or even vehicular manslaughter cases. Essentially, these defendants argued that they were not aware of the potential dangers of zolpidem, and so could not be liable for their actions while “sleep driving.” This raises the question: If AI gets good enough, and can tell you exactly the right dose to take of a drug, will you (or your doctor) be liable if you deviate from the AI’s recommendations? Will the AI’s recommendations be discoverable in court (and surfaced via AI-enhanced search)? Only time will tell what this brave new world will bring.   ENDNOTES [1] Kai Falkenberg, While You Were Sleeping (September 27, 2012), Marie Claire, https://www.marieclaire.com/culture/news/a7302/while-you-were-sleeping/.   [2] Takuji Inagaki, Tsuyoshi Miyaoka, Seiichi Tsuji, Yasushi Inami, Akira Nishida, and Jun Horiguchi, Adverse Reactions to Zolpidem: Case Reports and a Review of the Literature, 12 Prim Care Companion J Clin Psychiatry 6 (2010), https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3067983/.   [3] U.S. FDA, Drug Safety Communication: FDA approves new label changes and dosing for zolpidem products and a recommendation to avoid driving the day after using Ambien CR (May 14, 2013), https://www.fda.gov/drugs/drug-safety-and-availability/fda-drug-safety-communication-fda-approves-new-label-changes-and-dosing-zolpidem-products-and.   [4] FDA: Cut Ambien Dosage for Women, ABC News (January 10, 2013, 6:03AM), https://abcnews.go.com/Health/fda-recommends-slashing-sleeping-pill-dosage-half-women/story?id=18182165.   [5] David J Greenblatt, Jerold S Harmatz, & Thomas Roth, Zolpidem and Gender: Are Women Really At Risk?, 39(3) J. Clinical Psychopharmacol. 189 (May/Jun 2019), https://pubmed.ncbi.nlm.nih.gov/30939589/.   [6] NIH Inclusion Outreach Toolkit: How to Engage, Recruit, and Retain Women in Clinical Research, last accessed September 16, 2024: https://orwh.od.nih.gov/toolkit/recruitment/history.   [7] FDA, Using Artificial Intelligence and Machine Learning in the Development of Drug and Biological Products (May 10, 2023), https://www.fda.gov/media/167973/download; see also Using Artificial Intelligence and Machine Learning in the Development of Drug and Biological Products; Availability, 88 FR 30313 (May 11, 2023), https://www.federalregister.gov/documents/2023/05/11/2023-09985/using-artificial-intelligence-and-machine-learning-in-the-development-of-drug-and-biological.   [8] Thi Tuyet Van Tran, Hilal Tayara, and Kil To Chong, Artificial Intelligence in Drug Metabolism and Excretion Prediction: Recent Advances, Challenges, and Future Perspectives, 15 Pharmaceutics. 1260 (Apr 17, 2023), https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10143484/.   Lily Li is an AI, data privacy, and cybersecurity lawyer and founder of Metaverse Law. She is a certified information privacy professional for the United States and Europe and is a GIAC Certified Forensic Analyst for advanced incident response and computer forensics. She can be reached at info@metaverselaw.com.
0
Photo of a judges gavel and block next to each other.

CCPA Board Meeting: Key Takeaways from November 8, 2024

Adtech, AI & Algorithms, California, CCPA, Cybersecurity, State Privacy Laws, United States , , , , , , , , , , , , , , , , ,

In a vote of 4-1, the California Privacy Protection Agency (CPPA) has decided to move forward with rulemaking of its draft regulations concerning AI, cyber audits, profiling and risk assessments, despite complaints of regulatory overreach.

 

On Friday, November 8, the CCPA held a public meeting to discuss proposed updates to the California Consumer Privacy Act (CCPA) regulations. The hybrid meeting included public comments from a broad range of stakeholders – nearly 45 public comments were heard from business representatives, privacy advocates, and industry experts. While the passing vote would have typically triggered a 45-day public comment period on the draft regulations, Chairperson Urban requested flexibility, considering the upcoming holidays.

 

Legal Challenges

During the meeting, the CPPA stated that it was sued for failing to promulgate regulations, specifically on opt-out rights of information processed by automated decisionmaking tools (ADMTs). At the same time, commentators argued that the breadth of the proposed rules overstepped the intent of the CCPA.

 

Board Member Alastair Mactaggart–who helped draft the CCPA–voiced concerns about the regulations, arguing that the current proposed regulation is excessively broad to the point of being unworkable. He pointed out that these regulations, as written, apply to nearly all businesses that use any kind of software to generate any type of output–whether it’s AI-powered or not. For example, a simple tool like a spreadsheet or a school admission application could fall under these rules, forcing a large swath of low-risk businesses to conduct risk assessments. Mactaggart referred to this as statutory overreach and claimed that regulations should be focused on issues that genuinely impact privacy or security.

 

Economic Forecasts

The CPPA also issued a Standardized Regulatory Impact Assessment (SRIA) which was discussed during the meeting. In this assessment, the CPPA estimates the total cost of this regulatory initiative to be around $3.5 billion for the first year of implementation, with an average of $1 billion each subsequent year for the first ten years. The CPPA justifies this cost, asserting that the direct benefits to California businesses will be $1.5 billion in 2027, and $66.3 billion in 2036.

 

However, the California Chamber of Commerce states that “[b]usinesses, consumers and governments in California will suffer net losses from the proposed rules pending before the [CPPA] this week.” This statement stems from a report prepared for the Chamber of Commerce by Capitol Matrix Consulting, which concludes that the regulations are likely to “result in a substantial net losses to businesses, consumers, and governments in this state, both in the near and long term.”

 

Industry groups including TechNet, the Civil Justice Association of California, and the Interactive Advertising Bureau voiced concern about the heavy compliance burden that regulations place on businesses–especially small businesses that may not have the recourses to implement the required risk assessments or redesign their services to accommodate opt-out provisions.

 

Behavioral Advertising & Opt-Out Provisions

Another key point of contention during the meeting was the opt-out provision for consumers related to decisions made by AI systems.

 

The draft regulations govern a large range of AI. Under the draft, AI is defined as a “machine-based system that infers, from the input it receives, how to generate outputs that can influence physical or virtual environments.” Additionally, the draft defines ADMTs as “any technology that processes personal information and uses computation to execute a decision, replace human decisionmaking, or substantially facilitate human decisionmaking.”

 

Together, these definitions are more expansive than the definition of the high-risk automated processing addressed in Article 22 of the EU’s GDPR, the source of the original opt-out language. Under Article 22, a consumer has the right to opt out of decisions made by solely automated systems. The intent of this provision is to give consumers the ability to opt out of decisions that may be made on solely automated processes, such as targeted advertising.

 

However, critics argue that including the opt-out language in the draft in combination with an expansive definition of AI and ADMTs could have unintended consequences, especially for small businesses. Mactaggart, for instance, is concerned that applying this opt-out rule too broadly could lead to a breakdown of essential services. For example, online booking services for airlines and automated reservation software for hotels may rely on software that would be categorized as “AI” under this definition. Allowing users to opt out of using AI when asking for these services may be untenable, which could cause friction in these industries and ultimately could cause harm to consumers by limiting access to these services or increasing costs.

 

Risk Assessments

A central component of the draft regulation is for businesses who use AI, as defined above, to conduct risk assessments. While the goal of this requirement is to ensure that businesses are aware of and mitigate any potential privacy risks that arise from these technologies, critics believe the regulations go too far by applying the requirement to low risk, everyday activities.

 

For example, a representative from the California Grocery Association expressed concerns about how the opt-out provision would impact a chain of small rural grocery stores with whom she conducts business. While these AI tools could be used to help consumers save money, the cost of compliance to integrate these tools might not be within reach, especially given the thin profit margins within the grocery industry.

 

Again, Mactaggart questioned the scope of the draft. He and other advocates called for a narrower focus for risk assessments that centers on significant decisions–such as those that deny individuals access to essential goods and services. This could include the denial of a loan application, exclusion from an online platform, or an adverse employment decision. One commenter stated that there have been no public comments against regulating high-risk systems, and by focusing on these issues, the CPPA could better mitigate potential harms. At the same time, this would free low-risk systems from potential overregulation.

 

Additionally, a commentor suggested that risk assessments should be streamlined and aligned with other state standards to reduce compliance costs.  Mactaggart notes that accepting risk standards from other US jurisdictions could help businesses avoid duplicative efforts, cut compliance costs, and reduce the overall regulatory burden.

 

AI Training

The ability to opt out of training for AI datasets was of lesser concern but was still addressed by a number of commentors. For example, a representative from the Software and Data Industry Association argued that requiring an opt-out from consumers from AI dataset training could create a substantial burden on small businesses who already have trouble accumulating representative training data. Other commentors shares concerns that these opt-outs could compromise the quality and effectiveness for AI systems.

 

Ultimately, California faces a delicate balance in regulating AI and ADMT. On one hand, the state must work toward protecting consumers from privacy risks, potential discrimination, and other adverse impacts of AI. At the same time, the CPPA must ensure that rulemaking does not stifle innovation, create excessive compliance costs, or diminish competition between businesses that rely on AI.

 

As formal rulemaking moves forward, it will be crucial for the CPPA to consider feedback from the public comment period and to refine the regulations to ensure that they strike a balance between privacy concerns and costs to consumers and businesses alike.

0
American dollars and European Euros stacked on top of each other.

Cybersecurity Laws for the Fintech Industry

Cybersecurity, EU, Financial Services , , , , , , , , ,
In our modern digital landscape, the intersection of cybersecurity, finance and tech has become a focal point for regulators. With the rise of fintech, insurtech, personal financial management, alternative investments, and complex financial APIs, legal frameworks are evolving to keep pace.   Below are five notable cybersecurity legal updates within the financial sector, impacting financial institutions, fintech companies, and their service providers both domestically and abroad:  
  1. EU’s Digital Operational Resilience Act (DORA);
  2. SEC Amendments to Regulation S-P;
  3. FTC Standards for Safeguarding Consumer Information;
  4. Nacha’s Updates to Operating Rules; and
  5. CFPB’s Rulemaking on Personal Financial Data Rights.
 
  1. EU’s Digital Operational Resilience Act (DORA)
The Digital Operational Resilience Act (DORA) is an EU regulation that applies to financial entities and third parties that support them. DORA requires that applicable organizations must “follow rules for the protection, detection, containment, recovery and repair capabilities against [information and communication technology]-related incidents,” per the DORA website.   When Does it Take Effect? DORA entered into force on January 16, 2023, and will apply to each member state of the EU beginning January 17, 2025.   Who Does This Apply to? Financial Entities: Under DORA, financial entities are defined broadly to include banks, insurance providers, investment firms, payment institutions, credit institutions and credit rating agencies, and more.   ICT Third-Party Service Providers: DORA’s scope also includes Information Communication Technology (ICT) third-party service providers.  ICT third-party service providers are companies that provide digital and data services to financial entities. These providers include hardware providers as well as cloud computing services, software, data analytics services and providers of data center services. After identification, these providers are then be deemed critical or non-critical, with critical ICT service providers subject to additional requirements.   Key Takeaways DORA establishes uniform requirements regarding network security and information systems that support financial entities.   To establish this uniform framework, the Act requires:  
  • Managing risk of ICT resources. Financial entities are required to create and maintain an internal governance and control framework for the effective management of ICT risk.
 
  • Reporting on ICT-related incidents and major operational or security payment-related incidents. Financial entities are required to report major ICT-related incidents, and to voluntarily report cyber threats to competent authorities.
 
  • Digital operational resilience testing. Financial entities are required to establish, maintain and review a sound and comprehensive digital operational resilience testing program, including a range of assessments, tests, methodologies, practices and tools.
 
  • Contracting with ICT third-party service providers. Financial entities and ICT third-party service providers are required to clearly set out relevant rights and obligations in writing, including specific elements defined in the Act. Additionally, critical ICT-providers are subject to additional requirements.
 
  • Implementing measures for management of ICT third-party risk. Financial entities are required to adopt, and regularly review, a strategy on ICT third-party risk including a register of information related to the required contractual agreements between financial entities and ICT third-party service providers.
  Because the definition of “ICT third-party service providers” includes a range of entities that provide digital and data services, it is important that both financial entities and providers of ICT services are familiar with the requirements imposed by DORA.  
  1. SEC Amendments to Regulation S-P
Regulation S-P is a set of rules created by the Security and Exchange Commission (SEC). It requires certain parties to adopt written policies and procedures for the protection of customer records and information. The amendments to the Regulation are designed to address the expanded use of technology and associated risks that have emerged since the Regulation’s original adoption in 2000.   When Does it Take Effect? The SEC adopted the amendments to Regulation S-P on May 16, 2024, with an effective date of August 2, 2024. Larger entities will need to comply by December 3, 2025 while smaller entities will need to comply by June 1, 2026.   Who Does This Apply To? Regulation S-P applies to “covered institutions”, including broker-dealers, registered investment companies, as well as registered investment advisors (RIAs), funding portals, and transfer agents registered with the SEC or another appropriate regulatory agency.   Key Takeaways: The amendments to Regulation S-P modernize the rules regarding the treatment of consumers’ nonpublic personal information by imposing privacy-related protections.   Among other things, the amended Regulation requires:  
  • Adopting an incident response program. Covered institutions must adopt written policies and procedures for incident response programs to handle unauthorized access of information. This policy should be reasonably designed to detect, respond to, and recover from unauthorized access or use of customer information.
 
  • Updating consumer notification protocols. As part of the required incident response programs, covered institutions are required to notify consumers whose sensitive information was or is reasonably likely to have been accessed or used without authorization. This notice must be as soon as reasonably practicable, but no later than 30 days after the Covered Institution has become aware of the unauthorized access.
 
  • Providing oversight of service providers. Covered institutions are required to establish, maintain and enforce written policies that are reasonably designed to require oversight – including through monitoring of service providers to ensure that any individuals impacted by breach of sensitive information receive any required notices.
 
  • Expanding the scope of the Regulation. The amended Regulation aligns more closely to the FTC’s Safeguards Rule. Both rules apply to “customer information,” defined as “any record containing nonpublic personal information” about a customer of a financial institution. Additionally, the amendments broaden the group of customers whose information is protected under this Regulation.
 
  • Updating recordkeeping and annual privacy notices. The amended Regulation will add requirements to certain covered institutions to maintain written documentation of compliance. Additionally, certain covered institutions must provide a clear and conspicuous privacy notice at least annually during the customer relationship.
   
  1. FTC Standards for Safeguarding Consumer Information
The Federal Trade Commission’s (FTC’s) Standards for Safeguarding Consumer Information (the Safeguards Rule) is a set of regulations that requires certain financial institutions to protect consumer information.   When Does it Take Effect? In October 2023, the FTC announced the revised provisions of the Safeguards Rule, and the Rule took effect on May 13, 2024.   Who Does This Apply To? The Safeguards Rule applies to “financial institutions” that are covered by the FTC’s jurisdiction. This includes mortgage and payday lenders, finance companies, mortgage brokers, account services, check cashers, and investment advisors that are not required to register with the FTC, among others. This rule does not apply to those financial institutions subject to the authority of another regulator under §505 of the Gramm-Leach-Bliley Act.   Additionally, there are exemptions to this rule, including financial institutions that maintain consumer information concerning fewer than 5,000 consumers.   Key Takeaways The Safeguards Rule requires financial institutions to develop and maintain an information security program to protect consumer information. The amendments to the Safeguards Rule require entities to report data and security breaches affecting 500 people or more.   Among other things, the Safeguards Rule requires:  
  • Implementation of a security program. Financial institutions are required to develop, implement, and maintain a comprehensive security program. This program should be appropriate to the size, complexity, nature and scope of activities, and sensitivity of consumer information. The FTC Safeguards Rule also imposes minimum security controls on financial institutions, including but not limited to secure development, encryption and MFA.
 
  • Notifying the FTC. The amendment requires financial institutions to notify the FTC as soon as possible, and no later than 30 days after discovery, of a security breach involving at least 500 consumers.
 
  1. Nacha’s Updates to Operating Rules
The National Automated Clearing House Association (Nacha) Operating Rules govern how the Automated Clearing House (ACH) Network functions. The Nacha Rules cover all ACH payments, providing guidelines for securely storing, accessing, and transmitting sensitive customer information.   When Does it Take Effect? The changes to the Nacha Operating Rules became effective on October 1, 2024.   Who Does This Apply To? The Nacha Operating Rules apply to entities that collect and store non-public sensitive information in ACH transactions, including bank account and routing numbers, social security numbers, and driver’s license numbers, among other information.   Key Takeaways In 2024, the Nacha Operating Rules underwent amendments as part of a larger risk management package. These amendments are intended to reduce fraud and improve the recovery funds after fraud has occurred.   Among other things, the amendments to the Rules include:  
  • Allowing financial institutions to return entries via R17. A receiving depository financial institution (RDFI) may, but is not required, to use return code R17 to return an entry it believes is fraudulent. This amendment defines the return code for this use and is designed improve the recovery of funds that originated from fraud.
 
  • Expanding the uses of Request for Return. An originating depository financial entity (ODFI) may request a return from the RDFI for any reason. Under this amendment, the ODFI would still indemnify the RDFI for compliance with the request, and compliance by the RDFI remains optional.
 
  • Creating additional funds availability exceptions. This amendment provides RDFIs with an additional exception from the existing funds availability requirements, including credit entries that the RDFI suspects are fraudulent. This rule is intended to improve the recovery of funds obtained by fraud.
 
  • Modifying the timing of Written Statement of Unauthorized Debit (WSUD). While the rule previously allowed that a WSUD could be date on or after the Settlement Date of Entry, this amendment will allow a WSUD to be signed and dated by the receiver on or after the date on which the entry is presented to the receiver – even if the debit has not yet been posted to the account.
 
  • Requiring RDFI to return unauthorized debit. When returning a consumer debit as unauthorized, the RDFI must make the return by the sixth banking day following the completion of its review of the consumer’s signed WSUD. This prompt return will is intended to alert the ODFI of potential issues, and is intended to improve the recovery of funds and occurrence of future fraud.
   
  1. CFPB’s Rulemaking on Personal Financial Data Rights
The Consumer Financial Protection Bureau (CFPB) issued a final Rule to carry out the personal financial rights established by the Consumer Financial Protection Act of 2010 (CFPA).  This Rule allows consumers to access account data controlled by certain providers of consumer financial products in a safe, secure manner.   When Does it Take Effect? The data providers covered under this Rule must comply with the requirements in phases: the largest institutions will have to comply by April 1, 2026, while the smallest institutions must comply by April 1, 2030.   Who Does This Rule Apply To? Under this Rule, a “data provider” is required to make the covered data available, in electronic form, to consumers and certain authorized third parties.   A “data provider” includes depository institutions, such as credit unions, and non-depository institutions that issue credit cards, hold transaction accounts, issue devices to access an account, or provide other types of payment facilitation products or services. However, the rule does not apply to certain small depository institutions.   Key Takeaways This Rule enables consumers and authorized third parties to access consumer account information. This enables account holders to make more informed and freely made decisions regarding their providers.   Among other things, the Rule requires:  
  • Disclosing certain information. Data providers must provide certain data – including information about transactions, costs, charges, and usage – available to consumers and authorized third parties upon request.
 
  • Adhering to disclosure requirements. Disclosures must be made in a standardized and machine-readable format and in a commercially reasonable manner, among other disclosure requirements.
 
  • Banning “screen scraping” by third parties. A data provider cannot comply with the requirement to make certain data available to third parties by allowing the third party to use “screen scraping” – an access method using consumer credentials to log in to the consumer account to retrieve data.
1 2 3 4 5