0
Flag of California, depicting a large brown bear beside a red star, above the words "California Republic."

CCPA Draft Regulations Sent for Final Approval

California, CCPA, Cybersecurity , , ,
On July 24, 2025 the California Privacy Protection Agency (CCPA) board voted 5-0 to finalize Draft Regulations to the California Consumer Privacy Act (CCPA). The CPPA sent the rulemaking package to the Office of Administrative Law, which has 30 days to approve the regulations. The rulemaking process for these Draft Regulations began in 2022, and while the regulations have been narrowed since the prior proposal, the Draft Regulations will significantly impact how companies manage automated decisionmaking technology (ADMT), conduct risk assessments, and implement cybersecurity audits. Additionally, California’s regulatory process requires the CCPA to respond to public comments with their rationale for accepting or rejecting the suggestion. This requirement provides additional context and guidance for interpreting the intent of the Draft Regulations as they go into effect. What’s New? A Summary of Key Changes The Draft Regulations contain significant changes from the prior proposal – along with a 9-page explanation of changes. Most notably, the Draft Regulations roll back several of the most highly debated elements, while streamlining and clarifying other requirements:
  • References to “Artificial Intelligence” have been removed, significantly tightening the scope of ADMT systems.
  • First-party advertising removed from ADMT definition, narrowing the requirements needed for this type of processing.
  • Risk assessments are streamlined, and the scope of the types of data processing activities that trigger risk assessments has been narrowed.
  • Cybersecurity audits are clarified, and the CPPA included a “cybersecurity audit report” which should be produced during the audit process.

ADMT: Narrower Definition, Clearer Application

The Draft Regulations significantly narrow the scope of ADMT systems. Previously, ADMT systems included any technology that “substantially facilitated” human decisionmaking. Now, the Draft Regulations limits ADMT to systems which “substantially replace” human decisions. In practical terms, this may mean that only technologies which operate without human review or override fall under the ADMT rules. Importantly, the CPPA also removed first-party behavioral advertising from the definition of ADMT. Previously, businesses raised strong concerns that including this category within the ADMT definition would impose unnecessary burdens on common advertising practices. Businesses also voiced that including first-party behavioral advertising in the definition of ADMT went beyond Proposition 24, which provides the basis for amending the CCPA.

Risk Assessments: Who, What, and When?

While risk assessments remain a key part of the Draft Regulations, the CPPA has refined when they apply and what they must include. Who Needs to Conduct a Risk Assessment? Under the Draft Regulations, covered businesses that fall under the California Consumer Privacy Act (CCPA) “whose processing…presents significant risk to consumers’ privacy” must conduct a risk assessment. However, the newest version of the Regulations narrows what processing activities present “significant risk.” These activities include but are not limited to:
  • Selling or sharing personal information, which may require specific contractual obligations per the CCPA and current CCPA Regulations.
  • Processing sensitive personal information, as defined in the CCPA, including financial information, precise geolocation, health information and children’s personal information.
  • Using automated decisionmaking technology for a “significant decision” concerning a consumer, including those that impact availability of financial or lending services, housing, education enrollment or opportunities, employment or independent contracting opportunities or compensation, or healthcare services.
  • Using automated processing to profile a consumer through systematic observation when the individual is acting as an educational program application, job applicant, student, employee, or independent contractor for the covered business.
  • Using automated processing to profile a consumer based on their presence in a sensitive location, including healthcare facilities, domestic violence shelters, food pantries, housing/emergency shelters, educational institutions, political party offices, legal services offices, union offices, and places of worship.
  • Using personal information to train AI that could be used to make significant decisions concerning consumers, train facial- or emotional-recognition or other technology to verify a consumer’s identify or conducts physical or biological identification or profiling of a consumer.
While these risk assessments no longer apply to the previous expanded version of ADMT, they will apply to processing if the technology substantially replaces human decisionmaking for “significant decisions.” For example, if a covered business videotapes job interviews and uses AI to determine who to hire without human involvement, the covered business must conduct a risk assessment because of its use of ADMT for a significant decision concerning the consumer. What is Required for a Risk Assessment? As part of an effort to streamline and clarify the risk assessments required under the Draft Regulations, the CPPA defined a “risk assessment report” as the document that every covered business is required to create upon conducting the assessment. The CPPA provides a newly articulated goal for risk assessments: “[R]estricting or prohibiting the processing of personal information if the risks to privacy of the consumer outweigh the benefits resulting from processing to the consumer, the business, other stakeholders, and the public.” Additionally, the addition of the risk assessment report and changes in requirements may ease compliance efforts. To complete a risk assessment, a covered business should document, among other things:
  • The purpose of processing, the types of data involved, and any sensitive categories of personal information.
  • How the business plans to use the data, or otherwise collect, disclose or process the information, along with the retention period for the information.
  • How the business interacts with consumers, and whose data they process, along with the number of consumers whose information will be processed.
  • The disclosures made to consumers, and any other disclosures that the covered business plans to make, along with the names of service provides, contractors, or third parties to whom the information will be disclosed and the purpose for that disclosure.
  • The benefits, negative impacts, and safeguards of the planned processing.
  • Whether or not the business will initiate the processing subject to the risk assessment.
  • The individuals who provided information, as well as who the document was reviewed and approved by.
  If a covered business is using ADMT, the business must also identify:
  • The logic of the ADMT, including any assumptions or limitations of the logic; and
  • The output of the ADMT and how the covered business will use that output to make a significant decision.
The CPPA also clarifies that the risk assessment process may include involvement by external parties. Finally, a covered business must submit the following risk assessment information, among other things, to the Agency:
  • The business’s contact information, the information of the person submitting the assessment, and the date of certification.
  • The time period covered by the submission, and the number of risk assessments conducted or updated during that time.
  • Whether the risk assessments involved the processing of each of the categories of personal information identified in the CCPA.
  • A specific attestation, which certifies the business conducted a risk assessment for the processing activities involving significant decisions, subject to the penalty or perjury.
The individual submitting the information to the Agency must be a member of the covered business’s executive management team who is: 1) directly responsible for the business’s risk assessment compliance; 2) has sufficient knowledge to provide accurate information regarding the assessment; and 3) has the authority to submit the assessment information to the Agency. In addition, the Agency or Attorney General may require a covered business to submit its risk assessment reports at any time, within 30 days of the request. When Should Risk Assessments Be Conducted? According to the Proposed Rules, a covered business must conduct and document a risk assessment before beginning any processing activities that present a significant risk to consumers’ privacy. At least once every three years, the covered business must review and update their assessment. The covered business must also update a risk assessment whenever there is a material change relating to the processing activity, no later than 45 days from the material change. The covered business must retain its risk assessments – including original and updated versions – for as long as the processing continues or for five years after the completion of the risk assessment, whichever is later. What if I Have Already Conducted A Risk Assessment? There have been significant changes to the Draft Regulations regarding how covered businesses can use comparable assessments to satisfy the risk assessment criteria. New additions provide that a covered business may use a risk assessment that it has prepared for another purpose, provided that the assessment contains or is paired with all the required information to meet the Proposed Regulation’s requirements.

Cybersecurity Audits: Who, What, and When?

Among the added definitions is the “cybersecurity audit report” – the document that covered businesses must create as part of the cybersecurity audit. Similar to changes regarding risk assessments, this inclusion was part of the streamlining and clarification efforts of the CPPA. The scope and requirements of the cybersecurity audit – and the resulting audit report – have also been modified. Who Needs to Complete a Cybersecurity Audit? According to the Draft Regulations, every covered business whose processing of information presents a “significant risk” to consumers’ security must complete a security audit. While this language is similar to the requirements of the risk assessment, “significant risk” is defined slightly differently in the context of a cybersecurity audit. According to the Draft Regulations, a “significant risk” that warrants a cybersecurity audit includes but is not limited to covered businesses which:
  1. Derive 50% of more of its annual revenue from selling or sharing consumer’s personal information; or
  2. Had a gross annual revenue of $25M in the preceding calendar year (adjusted for inflation), and
    1. Processed the information of 250,000 or more consumers or households in the last year; or
    2. Processed the sensitive information of 50,000 or more consumers in the last year.
Covered businesses that are required to complete a cybersecurity audit must do so using a “qualified, objective, independent processional (‘auditor’) using procedures and standards accepted in the profession of auditing.” This audit may be internal or external to the covered business, but a qualified auditor must have knowledge of cybersecurity and know how to audit a business’s cybersecurity program, according to the changes in the Draft Regulations. What Should the Cybersecurity Audit Assess? Initially, the cybersecurity audit must assess how the covered business’s cybersecurity program protects personal information against unauthorized access, destruction, use, modification and disclosure, as well as how the program protects against unauthorized activity resulting in the loss of availability to that information. The cybersecurity audit must also assess the strength of a covered business’s cybersecurity program across such as, but not limited to:
  • Authentication and encryption;
  • Access control and account management;
  • Software and hardware inventories;
  • Patch and configuration management;
  • Network security, antivirus, and antimalware;
  • Incident response and business continuity;
  • Vendor oversight;
  • Data retention and disposal; and
  • Employee and contractor training.
The covered business’s auditor must also create a detailed cybersecurity audit report, documenting:
  • What was assessed and why. The report should describe the processes, activities, and components of the business’s cybersecurity program, the criteria used for the audit, along with the specific evidence examined to make decisions and assessments.
  • Evidence reviewed. The report must also include why these elements were appropriate for the audit, and how the evidence examined supports the findings.
  • Gaps or weaknesses found. The report should describe, in detail, the status of any gaps or weaknesses and any additional components that the auditor deemed to increase the risk of unauthorized activity. The report should also document the business’s plan to address these gaps and/or weaknesses.
  • Auditor information and certification. The report should also include the auditor’s information, as well as a statement by the highest-ranking auditor that certifies that they completed an independent review of the business’s cybersecurity program and information system, exercised objective and impartial judgement on all issues within the scope of the audit and did not rely primarily on assertations or attestations by business management to create the audit.
When Should Cybersecurity Audits Be Conducted? The final determination of when a covered business must conduct their first cybersecurity audit is based on the business’s annual gross revenue. If a business meets the audit thresholds, it may be time to start thinking about a compliance plan. First audit reports will be due:
  • April 1, 2028, for covered businesses with over $100 million in gross annual revenue;
  • April 1, 2029, for covered businesses with $50 million to $100 million in gross annual revenue; and
  • April 1, 2030, for covered businesses with under $50 million in revenue.
Each audit must cover the previous calendar year from January to January, with reports completed within the following three months. What if I Have Already Conducted A Cybersecurity Audit? As with the risk assessment, a covered business may use a cybersecurity audit, assessment, or evaluation that it has prepared for another purpose – provided that the audit meets all the requirements of the Draft Regulations, on its own or through supplemental information. The Draft Regulations provide, as an example, that a covered business may use the NIST Cybersecurity Framework 2.0 “and meets all the requirements of this Article.”

What Comes Next?

On July 24, 2025, the CPPA sent the rulemaking package to the Office of Administrative Law, which has 30 days to approve the regulations. The CPPA’s Draft Regulations signal a more measured approach to emerging technologies, such as AI. Still, these Draft Regulations carry out the CPPA’s mandate to issue regulations, reinforcing the agency’s commitment to privacy and security. For executives, the potential adoption of the Draft Regulations could be a strategic inflection point: Whether they are responsible for legal, compliance, data governance or information security, these Draft Regulations should prompt a reassessment of data practices, internal documentation and audit readiness. The publication of these Draft Regulations is also an opportunity to engage more deeply with operational teams. These rules will require clear cross-functional coordination, and organizations that begin building these bridges sooner will be better positioned to meet regulatory expectations and reinforce consumer trust in coming years. Compliance Deadlines: Compliance with these Draft Regulations will be required once they are approved by the Office of Administrative Law. The deadlines include:
  • ADMT Regulations: January 1, 2027
  • Privacy Risk Assessments: December 31, 2027
  • Cybersecurity Audits:
    • For businesses with $100+ million in annual gross revenue: April 1, 2028.
    • For businesses between $50 million and $100 million in annual gross revenue: April 1, 2029.
    • For businesses with less than $50 million in annual gross revenue: April 1, 2030.
 
0
Image of a computer circuit board with "AI" written on one of the chips.

AI Updates: An Overview of the Legal Landscape

AI & Algorithms, California, CCPA, EU , , , , , ,
As AI continues to advance, so do regulatory efforts. During the 2024 legislative session, 45 states along with Puerto Rico, the Virgin Islands, and Washington D.C. all introduced AI bills. With the legislative session for 2025 wrapping up, we are seeing similar tends this year. As new legal requirements emerge, organizations across the U.S. and EU may face overlapping – yet not identical – regulations that touch on issues of bias, safety, privacy, and transparency. Additionally, these laws may categorize the same AI system differently in different jurisdictions, requiring a nuanced approach to navigating these laws. Keeping this in mind, this article provides a brief overview of a handful of these laws. The practical takeaway? Businesses operating in the U.S. or EU should be aware of their legal requirements. Additionally, these organizations may want to consider a programmatic, auditable, and documented approach to AI governance, which may allow the business to map their AI controls to multiple legal frameworks.

Converging Themes

While details of AI laws differ across jurisdictions, trends seem to be converging on risk-based classification, transparency requirements, and enforcement efforts. Regulators are moving toward risk-based classification. This means AI uses are categorized according to their use case (and the risk associated with that use case). As seen in the EU AI Act, the Colorado AI Act, and TRAIGA, systems may be prohibited or classified by risk. High-risk systems tend to have stricter governance, testing and documentation requirements. Another shared theme is transparency. Laws including the EU AI Act, Colorado AI Act, Utah AI Policy Act, may require covered entities to tell people when AI is in use, while other laws may require the developer or deployer to explain the logic behind certain outputs, and provide consumers with a methods of contesting certain decisions, or opt out of certain types of decisionmaking entirely. The California AI Transparency Act and the EU AI Act may also require labeling of certain AI-generated content. Finally, enforcement is sharpening. The EU AI Act comes with regulatory teeth, with fines of the higher of €35,000,000 or 7% global annual turnover for violation of prohibited practices. In the U.S., state attorneys general and regulators have been active in monitoring AI missteps, including consumer protection and privacy violations. For example, attorneys general in Massachusetts and Oregon have issued advisories on how consumer protection laws apply to AI, while Texas Attorney General Ken Paxton reached the first-of-its-kind settlement in a healthcare generative AI investigation.

The European Union Artificial Intelligence Act (EU AI Act)  

Overview: The EU AI Act is the world’s first comprehensive AI regulation and sets a high-water mark for governance expectations. The Act is technology neutral and uses risk-based classification to sort AI systems into risk-tiers, each with escalating obligations. Key Provisions:
  • Prohibited systems include cognitive behavioral manipulation, most real-time biometric identification, and systems used for social scoring. These systems are considered to pose an unacceptable risk to safety or fundamental rights.
  • High-risk systems include hiring tools, biometric identification, and critical safety technology. They must undergo conformity assessments, maintain technical documentation, and ensure human oversight.
  • Limited-risk systemsinclude chatbots, deepfake generators, and public facing generative AI. These systems have transparency obligations to ensure users understand they are interacting with AI.
  • Minimal-risk systems include AI-enabled spam filers, grammar checkers, and basic AI in video games. These systems have no specific obligations under the Act, but best practices are encouraged.
Key Dates & Enforcement:
  • February 2, 2025: Prohibitions on certain AI systems and requirements on AI literacy start to apply.
  • August 2, 2025: Rules on general practice AI models, governance, confidentiality, and penalties start to apply.
  • August 2, 2026: The remainder of the AI Act (except for Article 6(1)) applies.
The Act will be enforced by European AI Office and national market surveillance authorities. Non-compliance with the prohibition of AI practices is subject to an administrative fine of up to €35,000,000 or up to 7% worldwide annual turnover, whichever is higher. Non-compliance with other provisions shall be subject to administrative fines of up to €15,000,000 of up to 3% of its total worldwide annual turnover, whichever is higher.

Colorado: Consumer Protections for Artificial Intelligence Act (CO AI Act)

Overview: Enacted in May 2024, the CO AI Act was the first far-reaching AI law in the United States. This Act primarily focuses on high-risk AI systems, including but not limited to those which influence “consequential decisions” – those impacting areas such as employment, education, housing, healthcare, finance, insurance, legal services, and essential government services. Key Provisions: Developer and deployers must both exercise “reasonable care” to protect consumers from known or reasonably foreseeable risks of algorithmic discrimination. For both, this may include providing notice to the Colorado Attorney General within 90 days of becoming aware of new discrimination risks.
  • Developers. There is a rebuttable presumption that the developer used reasonable care if they disclose, among other things:
    • reasonably foreseeable uses and known inappropriate or harmful uses of the AI system (including of algorithmic discrimination) and the measures taken to mitigate them;
    • the intended purpose, benefits, uses and outputs of the AI system; and
    • high-level summaries of the data types used to train the AI system, including data governance measures.
  • Deployers must also exercise reasonable care to protect consumers from any known or reasonably foreseeable risks of algorithmic discrimination. Similarly, there is a rebuttable presumption that the deployer used reasonable care if they complete the following, among other things:
    • a risk-management program that considers the NIST AI Risk Management Framework (AI RMF) or another similarly recognized risk management framework with substantially similar requirements (for more information about conducting an AI Risk Assessment, you can check out our post here);
    • an impact assessment, that includes the purpose, use cases, deployment context, and an analysis of whether it poses any foreseeable risks of discrimination, along with steps taken to mitigate those risks;
    • notice to consumers when certain systems are being used that include the system purpose, contact information, and options to opt-out of AI processing for that purpose, correct personal information used in the decisionmaking process, and appeal the decisionmaking process.
  • Disclosure should be clear. Regardless of risk level, any AI system that is directly interacting with Colorado consumers must disclose that it is an AI system, unless that would be obvious to a reasonable person.
Key Dates & Enforcement: While this law was originally set to take effect in 2026, Colorado Governor Polis called a special legislative session to address budget issues, taking place on August 21. The impact of SB24-05 (Consumer Protections for AI) is on the agenda, which may result in a delayed enforcement deadline and substantive changes to the law’s provisions. Violations are treated as deceptive trade practices under Colorado’s Consumer Protection Act, subject to enforcement by the Colorado Attorney General and penalties of up to $20,000 per violation.

Texas Responsible AI Governance Act (TRAIGA)

Overview: While TRAIGA originally provided a comprehensive AI framework, the final version has been significantly pared down. With narrow substantive provisions, TRAIGA focuses on harms caused by AI, and the Act regulates – or completely bans – certain uses of these systems. TRAIGA applies broadly to private sector companies if they provide AI-generated content or services to Texas residents, even if they are located outside the state of Texas. Additionally, government agencies interacting with the public fall squarely within the scope of the Act. You can read more about TRAIGA at our blog post covering the Act here. Key Provisions:
  • Prohibited AI For Public and Private Sectors include but are not limited to intentionally inciting self-harm, violence or crime; infringing on an individual’s rights; or unlawfully discriminating (with purposeful intent). The Act also prohibits deploying AI systems that intentionally generate illegal content, as well as child sexual abuse material or sexually explicit chat systems that impersonate children.
  • Prohibited AI uses for the Public Sector include but are not limited to social scoring and uniquely identifying individuals with biometric data (with limited exceptions).
  • Transparency Requirements for Public Sector may require governmental agencies to, among other things, provide conspicuous notice to consumers that they are acting with an AI system.
Key Dates & Enforcement:   TRAIGA was signed into law in June 2025 and takes effect on January 1, 2026. With no private right of action, the Act can only be enforced by the Texas Attorney General. The Act requires the Attorney General to create an “online mechanism” on their website where consumers can submit complaints of potential violations. If the Attorney General determines a violation has occurred, there is a 60-day cure period. If the violation continues after this period, the Attorney General may bring a claim for, among other things:
  • an injunction;
  • a civil penalty for curable breaches between $10,000 and $12,000;
  • a civil penalty for uncurable breaches between $80,000 and $200,000; and
  • a civil penalty for each day of continued violation between $2,000 and $40,000.
 

California CCPA Draft Regulations

Overview: On July 24, 2025, the California Privacy Protection Agency (CCPA) board voted 5-0 to finalize Draft Regulations to the California Consumer Privacy Act (CCPA). The CPPA sent the rulemaking package to the Office of Administrative Law, which has 30 days to approve the regulations. For a deeper dive on the CCPA Draft Regulations, please see our post here. Key Provisions:
  • Automated-decisionmaking (ADMT): Businesses must inform consumers with a pre-use notice and provide opt-out rights when AI or automated tools influence “significant decisions,” including those about employment, education, housing, healthcare, financial or lending services, and similar areas.
  • Risk Assessments: Organizations engaging in high-risk data processing (such as the decisions covered in ADMT, above) must conduct risk assessments before beginning processing, and must update them regularly, including within 45 days of any material change of the system. For more information about conducting an AI Risk Assessment, you can check out our post here.
  • Cybersecurity Audits: Businesses meeting certain thresholds must undergo annual, evidence-based audits carried out by a “qualified, objective, independent professional.” The audits must rely on specific evidence (as opposed to assertions by the business management), and all information related to the audit should be kept for a minimum of five years after completion.
Key Dates & Enforcement: Compliance with these Draft Regulations will be required once they are approved by the Office of Administrative Law. The deadlines include:
  • ADMT Regulations: January 1, 2027
  • Privacy Risk Assessments: December 31, 2027
  • Cybersecurity Audits:
    • For businesses with $100+ million in annual gross revenue: April 1, 2028.
    • For businesses between $50 million and $100 million in annual gross revenue: April 1, 2029.
    • For businesses with less than $50 million in annual gross revenue: April 1, 2030.

Other Laws to Consider

Along with the more far-reaching laws provided above, there are additional laws that businesses may want to consider when building, implementing, or otherwise engaging with AI tools or systems.
  • Utah’s Artificial Intelligence Policy Act
    • Effective as of May 2024, this Act mandates certain disclosures when businesses use generative AI to interact with consumers. This applies specifically to “regulated professions,” where the provider shall make the disclosure prominently, regardless of whether it is obvious the person is interacting with an AI system or not.
  • New York City’s Local Law 144 (and other AI employment regulations)
    • Signed in 2021, this law applies to employers and employment agencies in New York City that use “automated employment decision tools” to screen candidates or employees. It requires that an independent bias audit be conducted within one year of using the AI tools. For more information on AI in employment, see our article on AI In the Workplace: Legal Considerations for Leadership Teams.
  • California’s AI Transparency Law (SB 942)
    • Effective January 1, 2026, this law applies to “covered providers” – those offering generative AI systems with over 1 million monthly users in California. These providers must provide: 1) a free, public AI detection tool; and 2) certain disclosures as a label or embedded within their content.
  • California’s Data Transparency Law (AB 2013)
    • Effective January 1, 2026, developers of generative AI systems must post a disclosure on their website including documentation used to train the AI system. This documentation includes high-level summary of datasets used in the development of the AI system – the sources or owners of the datasets, how they further the purpose of the AI system, the number of datapoints in the datasets, and more.

Key Takeaway

As lawmakers race to keep up with the breakneck speed of AI implementation, guidance is quickly becoming enforcement. While specific requirements between these laws vary, the common thread is clear: covered entities are expected to understand, document, and justify their AI systems’ design, data, and impact. Additionally, organizations utilizing AI should consider building responsible AI governance into their operations. By incorporating these governance processes into everyday systems and – similar to those for privacy and cybersecurity – organizations may proactively protect against legal, ethical and operational risk when implementing AI.
0

Data Collection Practices and CCPA Compliance: Key Takeaways from Honda’s CPPA Settlement

California, CCPA, Vehicle Data
On March 12, 2025, the California Privacy Protection Agency (CPPA), one of the enforcement agencies for the California Consumer Privacy Act (CCPA), announced a settlement of over $630,000 with American Honda Motor Co. (Honda) for alleged privacy violations. This is the first time the CPPA has fined an automaker since the CPPA announced in July, 2023 that it was reviewing privacy practices related to connected vehicles. The CPPA’s Order defines four key areas of Honda’s alleged non-compliance:
  1. Verifying information for requests to opt out/limit sensitive information.
  2. Verifying information for requests to opt out/limit sensitive information through agents.
  3. Providing lack of symmetry through the website’s cookie management tool.
  4. Engaging in insufficient contracts with advertising technology vendors.
This post will walk through each of these issues in turn, providing key takeaways to consider based on the CPPA’s Order.

1.    Issue: Verifying Information for Requests to Opt Out/Limit Sensitive Information

The CPPA alleges that Honda’s webform, as depicted in the Order, requires individuals to include information for verification purposes when submitting requests to opt out of sale/sharing or limit the use of sharing sensitive information. Overview: Per §7060(b) of the California Consumer Privacy Act Regulations (Regulations), there is no verification requirement to process requests to opt-out of the sale/sharing of personal information or for requests to limit the use of sensitive personal information. The CPPA alleges that Honda’s “Submit A Privacy Request” webform required eight separate data points for a range of data subject access requests (DSARs), including the right to opt out of sale/sharing of personal information and limit use of sensitive information. Covered entities should not require verification before processing the requests. According to the CPPA’s Order, from July 1, 2023 to September 23, 2023, Honda improperly required at least 119 individuals to provide excessive information and denied at least 20 individuals requests based on unlawful verification standards. Takeaway: Under the CCPA, opt out and limit requests are non-verifiable and covered entities should only collect the minimal data points necessary to fulfill the request. You can learn more about responding to DSARs on our blog.

2.    Issue: Verifying Information for Requests to Opt Out/Limit Sensitive Information through Agents

The CPPA alleges that Honda unlawfully required individuals to confirm with Honda directly that they had authorized an agent to submit requests on their behalf to opt out of sales/sharing or to limit use of sensitive information. Overview: While covered entities may request proof of the individuals’ signed permission for an agent to act on their behalf, this is only permitted by verifiable requests – requests to know, delete or correct information, per §7063(a) of the Regulations. The CPPA alleges that Honda’s direct confirmation requirement for request to opt out and limit goes beyond what is permitted in the CCPA and Regulations. The Agency alleges that these unlawful practices impacted at least 14 consumers during the reviewed period from July to September 2023. Takeaway: The CCPA prohibits covered entities from requiring direct confirmation from consumers for non-verifiable requests – even when using an agent to effectuate this request. Again, as opposed to requiring the same verification standards for all DSARs, covered entities should distinguish which types of requests are verifiable. This may vary between jurisdictions, so be sure to check all applicable laws when building your DSAR playbook. You can refer to our U.S. state privacy law post for relevant jurisdictional thresholds within the US, and covered entities should also consider international laws, like the GDPR, which may impose other DSAR or verification requirements.

3.    Issue: Lack of Symmetry on the Website’s Cookie Management Tool

The CPPA alleges that Honda’s cookie management tool (the cookie banner at the bottom of their webpage) required more steps to opt out of sharing than to opt in, violating the symmetrical choice requirements of the CCPA. Overview: According to the Order, individuals using Honda’s cookie banner needed to complete two steps to disable advertising – a “change” step and a “save” step. However, opting in required a single “change & save” step. Per §7004(a)(2) of the Regulations, “[t]he path for a consumer to exercise a more privacy-protective option shall not be longer or more difficult or more time-consuming than the path to exercise a less privacy-protective option,” because an imbalance in options “would impair or interfere with the consumer’s ability to make a choice.” According to the examples in the Regulations, “[a]n equal or symmetrical choice [in a website banner] could be between ‘Accept All’ and ‘Decline All.’” Takeaway: Entities covered by the CCPA should ensure that the process to submit opt out requests – including those through cookie management tools – is no more difficult than the process to opt in. According to the Regulations, this standard also applies when the individual uses the “Do Not Sell or Share My Personal Information” or “Your Privacy Choices” link. The number of steps for submitting a request to opt out is measured from when the consumer first clicks the link to the completion of the request. Similarly, the number of steps to opt in is measured from the first indication the consumer makes of their interest to opt in to the completion of the request.

4.    Issue: Insufficient Contracts with Advertising Technology Vendors

The CPPA alleges that Honda failed to produce contracts (such as data protection agreements, or DPAs) that required technology vendors to sufficiently protect consumer information. Overview: Under the CCPA §1798.100(d), when a covered entity collects  a consumer’s personal information and discloses it to a service provider or contractor, the covered entity should enter into an agreement with that party, requiring them to protect the consumer’s personal information. According to the Order, Honda lacked proper contractual agreements, despite collecting and disclosing individuals’ information with third-party vendors. These vendors included businesses that conducted targeted advertising, which may constitute “selling” or “sharing” personal information under the CCPA. Without agreements with these third-party vendors in place, the CPPA alleges that individuals’ information may be improperly used or shared without sufficient privacy protections. Takeaway: The CCPA requires covered entities to maintain agreements, such as a DPA, that specify data use limitations, require CCPA compliance, and ensure a certain standard of privacy protection. If a covered entity is disclosing personal information to third-party vendors, it should ensure that these contracts are in place and meet the law’s requirements.

Conclusion

The Order against Honda serves as a cautionary example for covered entities managing individuals’ information under the CCPA. In addition to the fine, the Order requires Honda to “certify its compliance, train its employees, and consult a user experience (UX) designer to evaluate its methods for submitting privacy requests. Honda must also change its contracting process to ensure appropriate mechanisms are in place to protect personal information.” Additionally, the CPPA’s head of the Enforcement Division stated that “[the Agency] won’t hesitate to use our cease-and-desist authority to change business practices,” indicating that the Agency is serious about its enforcement authority. By taking proactive steps, covered entities can better protect against regulatory enforcement actions while working to safeguard individuals’ privacy.
0

Metaverse Law in Orange County Lawyer Magazine

AI & Algorithms, CCPA, Cybersecurity, GDPR, Health Data, State Privacy Laws , , , , , , ,
The January 2025 edition of Orange County Lawyer magazine features an article written by Metaverse Law’s Lily Li. Read “AI and Machine Learning in Drug Development and Clinical Trials” below or in Orange County Lawyer magazine.
[Originally published as a Feature Article: AI and Machine Learning in Drug Development and Clinical Trials, by Lily Li, in Orange County Lawyer Magazine, January 2025, Vol. 67 No.1, page 28.]   AI and Machine Learning in Drug Development and Clinical Trials by Lily Li   In 2013, sleep medication zolpidem (Ambien, Ambien CR, and Edluar) swept headlines. Marie Claire reported on an alarming and suspicious rise in users experiencing irrational eating, gambling, and even “sleep-driving” while in a hypnotic trance—waking with no memories of their actions.[1] In several cases, women arrested and convicted for driving under the influence contested their convictions, arguing that they were not liable for these undisclosed drug-related side effects. At the same time, several clinical studies suggested that women metabolized zolpidem differently from men. By reviewing existing literature, Japanese researchers out of Shimane University identified 40% higher concentrations of zolpidem in women than men following use, and higher rates of visual hallucinations and sensory distortions.[2] The FDA released a safety advisory, warning users of the risks of “next-morning impairment” for the use of Ambien and related drugs.[3] In addition, the FDA took the unusual step of recommending a 50% cut in the dosage for women. When asked about the change, an FDA director told ABCNews.com: “The changes are different in women and men . . .We don’t understand why yet, but women are more susceptible to next-morning impairment.”[4] Yet, a decade later, the evidence supporting different zolpidem dosages for women and men is unclear.[5] In part, this is due to the lack of research surrounding sex differences in drug impact and drug treatment, as well as substantial gaps in the inclusion of women in clinical studies. From 1977 to 1993, FDA policy recommended excluding women of childbearing potential from Phase 1 and early Phase II drug trials.[6] Even after this policy was removed in 1993, industry fears remained with respect to drug interactions with pregnancy. This episode with zolpidem raised several concerns in the drug development and clinical trial process:
  • How do we recruit representative candidates for drug trials?
  • How do we ensure the quality and availability of datasets for clinical research?
  • How do we measure potential impacts of drug dosing on different populations?
  • What are the legal implications for failing to address appropriate drug doses?
  AI and ML to the Rescue? Now that artificial intelligence is being used in research and development, one wonders: Can artificial intelligence (AI) and machine learning (ML) reduce bias and risks during drug development? Or will it create new legal risks due to bias, privacy intrusions, and lack of transparency? The FDA released a discussion paper on AI, Using Artificial Intelligence and Machine Learning in the Development of Drug and Biological Products, to discuss potential regulatory frameworks to address the use of AI and ML.[7] In this discussion paper, the FDA released a set of fascinating case studies into existing research and uses of AI in the clinical trial process. Several of these case studies are discussed below, as well as an analysis of their potential impact on the zolpidem example.
  1. Recruitment. According to the FDA, “AI/ML is being used to mine vast amounts of data, such as data from clinical trial databases, trial announcements, social media, medical literature, registries, and structured and unstructured data in EHRs [electronic health records], which can be used to match individuals to trials (Harrer, 219 Shah, Antony, & Hu, 2019).” In this manner, researchers can combine huge quantities of publicly available data and individual health data from prior research to identify participants with certain medical conditions (or lack of adverse conditions) for investigational treatments. For zolpidem, the use of AI/ML may have been able to identify a much broader list of participants for initial clinical testing, making it easier to assess and identify adverse reactions.
  2. Selection and Stratification of Trial Participants. In addition to initial recruitment, AI/ ML has the capability improve intake, selection, and classification of clinical trial participants. Based on baseline characteristics selected by the researchers, such as prior clinical data, and vitals/labs taken during intake, predictive algorithms can help identify high-risk participants.[8] These groups can then be randomized and then subject to more strict monitoring protocols. In the case of zolpidem, alcohol use is associated with sometimes severe adverse effects from the drug, and so it would be beneficial to screen out candidates with a history of alcoholism or, on the flip side, assess drug interactions for this high-risk group with additional support, monitoring, or counseling.
  3. Dose/Dosing Regimen Optimization. AI/ML can be used to predict drug exposure for different populations based on factors such as weight, height, sex, and other characteristics that might impact drug metabolism. Based on prior drug exposure and response profiles for similar drugs and similar populations, AI/ML can help to narrow the dose/dosing regimen selected for a study. As noted by the FDA’s discussion paper, this can help optimize drug dosing “in special populations where there may be limited data (e.g., rare disease studies, pediatric and pregnant populations).” Based on this research, we can imagine future scenarios where AI/ML could have avoided zolpidem dosing concerns, where graduated and limited dosing was tested and applied to different sex, age, and metabolism categories to determine ideal dosing.
  4. Data Analysis. On a more intriguing level, the FDA AI discussion paper discussed the concept of creating “digital twins” of patients for clinical trials. Essentially, an AI version of the clinical participant is created, using the existing candidate’s electronic health records, vital signs, labs and other records. Researchers can assess how the digital twin would react under normal conditions using AI/ML modeling based on data gathered from similar individuals. This digital twin would then act as a substitute for a placebo candidate in a clinical trial, and act as a benchmark against the actual patient undergoing investigational treatment. For zolpidem, this could be used to assess candidates that already have underlying medical conditions such as anxiety, depression, or other confounding factors, to see whether an adverse effect from a trial is due to the investigational treatment or something that is likely to occur to the same individual from anxiety alone.
  5. Postmarketing Safety Surveillance. Finally, AI/ML can help detect and assess adverse events once the drug enters the market. This is not just limited to individual case safety reports (ICSR), required by regulators, but can include adverse events reported publicly on social media and the wider internet. This type of postmarketing safety surveillance could assist researchers and drug companies in identifying potential drug risks, prior to landing on primetime news.
  Quality and Reliability Risks While AI/ML can help to address the costs and efficiency of clinical trials, this relies substantially on the underlying data used to train AI. The quality and reliability of any AI/ML model requires similar quality controls for underlying training data. Given the safety risks of inappropriate drug dosing, or recruiting candidates with severe medical conditions, AI developers cannot rely solely on self-reported healthcare data with no external medical testing or validation. Developers should be equally wary of training on third-party data sets that do not provide documentation on the collection of data and data validation. Within an existing healthcare organization, if the organization is big enough, aggregate and de-identified data may be obtained from existing electronic health care records and prior clinical trials. Yet, even within these large datasets, errors may surface during training. Medical providers may code the same procedure, and similar symptoms, a dozen different ways. Even drug names can be misspelled and coded incorrectly within existing records. While many of these errors may end up being statistically insignificant with enough data, there is the risk of missing one or two major adverse events, or “black swan” events, that would otherwise change the entire risk profile of a drug. In addition to quality and reliability, the underlying dataset needs to be representative of the population that will be studied for the clinical trial. If the underlying dataset is only trained on a handful of individuals with a certain medical predisposition, age, sex, weight, etc., it will be difficult for the AI model to make predictions for that group. As an example, if the training data only contains the medical information for two individuals over the age of sixty, and shows no adverse effects from a particular drug dose, this information is not enough to generalize that the drug at that dosage is appropriate for all individuals over the age of sixty. For all we know, these two candidates could be a former Olympic diver and a nutrition coach, two outliers that completely skew the data. Consequently, the underlying training data for any AI model should also be assessed for bias and representativeness as it applies to the proposed clinical trial.   Data Privacy, Cybersecurity, and AI Risks The data privacy and cybersecurity risks associated with the foregoing uses of AI/ML cannot be underestimated. The quality and representativeness of any AI system in this field will rely heavily on large swathes of healthcare data, fine-tuned and, at times, personalized in the case of digital twins. This is sensitive or special category data at its finest, triggering heightened scrutiny under the EU’s data privacy law, the GDPR, and U.S. data privacy and data breach laws. To date, most healthcare organizations have sidestepped data privacy concerns by relying on HIPAA’s de-identification standard to remove personal information and other identifiers from healthcare data, making it difficult to associate with an individual. While the FDA requires Institutional Review Board (IRB) review of most biomedical research involving human subjects, this generally does not apply to de-identified personal information that cannot be linked to an individual. Simply de-identifying data and then running with it is not enough, however. Under the California Consumer Privacy Act and similar state laws, for example, recipients of de-identified data need to affirm that they will not attempt to reidentify the data (except to test their de-identification methods). The GDPR has a much higher “anonymization” standard, which looks at the re-identifiability of personal information, given all the different datasets that an organization may have access to. AI/ML itself is making the de-identification process harder. As it is capable of slicing and dicing data by age, race, sex, and medical condition, and combining multiple large datasets, it is easy to run the risk of re-identifying data. While several thousand people might have the same configuration of eye color, age, gender, and weight, only one or two may have participated in a clinical trial at a particular location, or have specific allergies or side effects to certain types of medication. As a result, in circumstances where healthcare data is not de-identified, or the risk of reidentification is heightened, then it behooves clinical organizations and their AI developers to implement written information security programs and associated privacy and security controls.   Legal Liability and Drug Dosing In several notable cases, defendants on zolpidem were able to contest or overturn DWI or even vehicular manslaughter cases. Essentially, these defendants argued that they were not aware of the potential dangers of zolpidem, and so could not be liable for their actions while “sleep driving.” This raises the question: If AI gets good enough, and can tell you exactly the right dose to take of a drug, will you (or your doctor) be liable if you deviate from the AI’s recommendations? Will the AI’s recommendations be discoverable in court (and surfaced via AI-enhanced search)? Only time will tell what this brave new world will bring.   ENDNOTES [1] Kai Falkenberg, While You Were Sleeping (September 27, 2012), Marie Claire, https://www.marieclaire.com/culture/news/a7302/while-you-were-sleeping/.   [2] Takuji Inagaki, Tsuyoshi Miyaoka, Seiichi Tsuji, Yasushi Inami, Akira Nishida, and Jun Horiguchi, Adverse Reactions to Zolpidem: Case Reports and a Review of the Literature, 12 Prim Care Companion J Clin Psychiatry 6 (2010), https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3067983/.   [3] U.S. FDA, Drug Safety Communication: FDA approves new label changes and dosing for zolpidem products and a recommendation to avoid driving the day after using Ambien CR (May 14, 2013), https://www.fda.gov/drugs/drug-safety-and-availability/fda-drug-safety-communication-fda-approves-new-label-changes-and-dosing-zolpidem-products-and.   [4] FDA: Cut Ambien Dosage for Women, ABC News (January 10, 2013, 6:03AM), https://abcnews.go.com/Health/fda-recommends-slashing-sleeping-pill-dosage-half-women/story?id=18182165.   [5] David J Greenblatt, Jerold S Harmatz, & Thomas Roth, Zolpidem and Gender: Are Women Really At Risk?, 39(3) J. Clinical Psychopharmacol. 189 (May/Jun 2019), https://pubmed.ncbi.nlm.nih.gov/30939589/.   [6] NIH Inclusion Outreach Toolkit: How to Engage, Recruit, and Retain Women in Clinical Research, last accessed September 16, 2024: https://orwh.od.nih.gov/toolkit/recruitment/history.   [7] FDA, Using Artificial Intelligence and Machine Learning in the Development of Drug and Biological Products (May 10, 2023), https://www.fda.gov/media/167973/download; see also Using Artificial Intelligence and Machine Learning in the Development of Drug and Biological Products; Availability, 88 FR 30313 (May 11, 2023), https://www.federalregister.gov/documents/2023/05/11/2023-09985/using-artificial-intelligence-and-machine-learning-in-the-development-of-drug-and-biological.   [8] Thi Tuyet Van Tran, Hilal Tayara, and Kil To Chong, Artificial Intelligence in Drug Metabolism and Excretion Prediction: Recent Advances, Challenges, and Future Perspectives, 15 Pharmaceutics. 1260 (Apr 17, 2023), https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10143484/.   Lily Li is an AI, data privacy, and cybersecurity lawyer and founder of Metaverse Law. She is a certified information privacy professional for the United States and Europe and is a GIAC Certified Forensic Analyst for advanced incident response and computer forensics. She can be reached at info@metaverselaw.com.
0
Photo of a judges gavel and block next to each other.

CCPA Board Meeting: Key Takeaways from November 8, 2024

Adtech, AI & Algorithms, California, CCPA, Cybersecurity, State Privacy Laws, United States , , , , , , , , , , , , , , , , ,

In a vote of 4-1, the California Privacy Protection Agency (CPPA) has decided to move forward with rulemaking of its draft regulations concerning AI, cyber audits, profiling and risk assessments, despite complaints of regulatory overreach.

 

On Friday, November 8, the CCPA held a public meeting to discuss proposed updates to the California Consumer Privacy Act (CCPA) regulations. The hybrid meeting included public comments from a broad range of stakeholders – nearly 45 public comments were heard from business representatives, privacy advocates, and industry experts. While the passing vote would have typically triggered a 45-day public comment period on the draft regulations, Chairperson Urban requested flexibility, considering the upcoming holidays.

 

Legal Challenges

During the meeting, the CPPA stated that it was sued for failing to promulgate regulations, specifically on opt-out rights of information processed by automated decisionmaking tools (ADMTs). At the same time, commentators argued that the breadth of the proposed rules overstepped the intent of the CCPA.

 

Board Member Alastair Mactaggart–who helped draft the CCPA–voiced concerns about the regulations, arguing that the current proposed regulation is excessively broad to the point of being unworkable. He pointed out that these regulations, as written, apply to nearly all businesses that use any kind of software to generate any type of output–whether it’s AI-powered or not. For example, a simple tool like a spreadsheet or a school admission application could fall under these rules, forcing a large swath of low-risk businesses to conduct risk assessments. Mactaggart referred to this as statutory overreach and claimed that regulations should be focused on issues that genuinely impact privacy or security.

 

Economic Forecasts

The CPPA also issued a Standardized Regulatory Impact Assessment (SRIA) which was discussed during the meeting. In this assessment, the CPPA estimates the total cost of this regulatory initiative to be around $3.5 billion for the first year of implementation, with an average of $1 billion each subsequent year for the first ten years. The CPPA justifies this cost, asserting that the direct benefits to California businesses will be $1.5 billion in 2027, and $66.3 billion in 2036.

 

However, the California Chamber of Commerce states that “[b]usinesses, consumers and governments in California will suffer net losses from the proposed rules pending before the [CPPA] this week.” This statement stems from a report prepared for the Chamber of Commerce by Capitol Matrix Consulting, which concludes that the regulations are likely to “result in a substantial net losses to businesses, consumers, and governments in this state, both in the near and long term.”

 

Industry groups including TechNet, the Civil Justice Association of California, and the Interactive Advertising Bureau voiced concern about the heavy compliance burden that regulations place on businesses–especially small businesses that may not have the recourses to implement the required risk assessments or redesign their services to accommodate opt-out provisions.

 

Behavioral Advertising & Opt-Out Provisions

Another key point of contention during the meeting was the opt-out provision for consumers related to decisions made by AI systems.

 

The draft regulations govern a large range of AI. Under the draft, AI is defined as a “machine-based system that infers, from the input it receives, how to generate outputs that can influence physical or virtual environments.” Additionally, the draft defines ADMTs as “any technology that processes personal information and uses computation to execute a decision, replace human decisionmaking, or substantially facilitate human decisionmaking.”

 

Together, these definitions are more expansive than the definition of the high-risk automated processing addressed in Article 22 of the EU’s GDPR, the source of the original opt-out language. Under Article 22, a consumer has the right to opt out of decisions made by solely automated systems. The intent of this provision is to give consumers the ability to opt out of decisions that may be made on solely automated processes, such as targeted advertising.

 

However, critics argue that including the opt-out language in the draft in combination with an expansive definition of AI and ADMTs could have unintended consequences, especially for small businesses. Mactaggart, for instance, is concerned that applying this opt-out rule too broadly could lead to a breakdown of essential services. For example, online booking services for airlines and automated reservation software for hotels may rely on software that would be categorized as “AI” under this definition. Allowing users to opt out of using AI when asking for these services may be untenable, which could cause friction in these industries and ultimately could cause harm to consumers by limiting access to these services or increasing costs.

 

Risk Assessments

A central component of the draft regulation is for businesses who use AI, as defined above, to conduct risk assessments. While the goal of this requirement is to ensure that businesses are aware of and mitigate any potential privacy risks that arise from these technologies, critics believe the regulations go too far by applying the requirement to low risk, everyday activities.

 

For example, a representative from the California Grocery Association expressed concerns about how the opt-out provision would impact a chain of small rural grocery stores with whom she conducts business. While these AI tools could be used to help consumers save money, the cost of compliance to integrate these tools might not be within reach, especially given the thin profit margins within the grocery industry.

 

Again, Mactaggart questioned the scope of the draft. He and other advocates called for a narrower focus for risk assessments that centers on significant decisions–such as those that deny individuals access to essential goods and services. This could include the denial of a loan application, exclusion from an online platform, or an adverse employment decision. One commenter stated that there have been no public comments against regulating high-risk systems, and by focusing on these issues, the CPPA could better mitigate potential harms. At the same time, this would free low-risk systems from potential overregulation.

 

Additionally, a commentor suggested that risk assessments should be streamlined and aligned with other state standards to reduce compliance costs.  Mactaggart notes that accepting risk standards from other US jurisdictions could help businesses avoid duplicative efforts, cut compliance costs, and reduce the overall regulatory burden.

 

AI Training

The ability to opt out of training for AI datasets was of lesser concern but was still addressed by a number of commentors. For example, a representative from the Software and Data Industry Association argued that requiring an opt-out from consumers from AI dataset training could create a substantial burden on small businesses who already have trouble accumulating representative training data. Other commentors shares concerns that these opt-outs could compromise the quality and effectiveness for AI systems.

 

Ultimately, California faces a delicate balance in regulating AI and ADMT. On one hand, the state must work toward protecting consumers from privacy risks, potential discrimination, and other adverse impacts of AI. At the same time, the CPPA must ensure that rulemaking does not stifle innovation, create excessive compliance costs, or diminish competition between businesses that rely on AI.

 

As formal rulemaking moves forward, it will be crucial for the CPPA to consider feedback from the public comment period and to refine the regulations to ensure that they strike a balance between privacy concerns and costs to consumers and businesses alike.

1 2 3 4