0
Flag of California, depicting a large brown bear beside a red star, above the words "California Republic."

California: New AI laws in California – roundup of the 2025 legislative session

AI & Algorithms, California, Children, Employment Law, Healthcare, Social Media , , , , , , , ,
This article was originally published by OneTrust DataGuidance on November 24, 2025 and can be found on the DataGuidance website here.

California introduces comprehensive AI laws focusing on transparency, children’s safety, healthcare, antitrust, and law enforcement.

California has taken an aggressive stance towards artificial intelligence (AI) legislation and will likely set the standard for other US states. Back in 2024, Governor Newsom vetoed comprehensive AI safety legislation under bill SB 1047 and advised caution on regulations for this nascent and important technology. This year, Governor Newsom pressed ahead with a full slate of new AI laws. The reasons for this change in approach are many, including but not limited to the lack of federal AI legislation, the growing concern over children’s interactions with AI, especially sexualized content, and harmonization with more stringent requirements in the EU and elsewhere.

This year’s legislative session set records for the number and scope of new AI laws. For the roundup this year, Lily Li, of Metaverse Law Corporation, breaks down the new AI laws by scope and sector, noting where this may add on to existing California legislation and rulemaking from 2024-2025.

General AI safety, transparency, and risk assessments

  • SB 53: Transparency in Frontier Artificial Intelligence Act (Wiener) – Starting in January 2026, California will require large frontier AI developers to publish a framework detailing how they incorporate safety, security, and testing standards into their AI models. SB 53 also creates a mechanism for AI developers and the public to report critical safety incidents, and protects internal whistleblowers who report risks posed by frontier AI models. The law establishes significant penalties for companies that fail to comply, with fines of up to $1 million per violation.
  • AB 316: Artificial Intelligence defenses (Krell) – This amends California’s Civil Code. If a party to a lawsuit develops, modifies, or uses AI, this law prohibits them from asserting as a defense that the AI autonomously caused the harm.
  • AB 853: California AI Transparency Act (Wicks) – This bill expands the existing AI Transparency Act and modifies the effective date from January 1, 2026, to August 2, 2026. The California AI Transparency Act requires covered generative AI developers to provide an AI-detection tool to assess whether image, video, or audio content is created or altered by generative AI. This bill adds to the existing law by requiring large online platforms to embed provenance data into generated content. Starting January 1, 2028, users will also have the option to include latent disclosures on ‘capture devices’ such as cameras, video recorders, and other recorders.

This new California approach to AI transparency and safety legislation needs to be read in conjunction with the following existing laws.

  • California Privacy Protection Agency’s (CPPA’s) recently approved Cyber, Risk, ADMT, and Insurance Regulations – The CPPA’s most recently updated 127-page regulation package contains requirements governing cybersecurity audits, risk assessments, and automated decision-making technology. AI developers and systems that process personal information and meet certain California privacy thresholds will now face new cybersecurity audit and risk assessment requirements. In addition, automated and significant decisions concerning the provision or denial of financial or lending services, housing, education enrollment or opportunities, employment or independent contracting opportunities or compensation, or healthcare services will trigger significant notice, opt-out, and risk assessment requirements.
  • AB 2013: AI Training Data Transparency Act (Irwin-2024) – Passed last year, this law will require covered generative AI developers to publish online a high-level summary of the datasets used in the development of the generative AI system or service, including but not limited to whether personal information or copyrighted information is included in the training data. The law is scheduled to go into effect on January 1, 2026.

Children’s safety, age verifications, and companion chatbots

  • SB243: Companion Chatbots (Padilla) – This law applies to chatbots that provide human-like interactions and are capable of sustaining relationships across multiple interactions. Beginning July 1, 2027, developers of these ‘companion chatbots’ will need to develop and report protocols addressing suicidal ideation and self-harm to regulators and the public. The law requires AI disclosures, referrals to suicide hotlines or crisis text lines, and break reminders. SB 243 further requires developers to institute reasonable measures to prevent the chatbot from producing visual material of sexually explicit conduct or directly stating that the minor should engage in sexually explicit conduct. The legislation includes a private right of action to individuals who suffer ‘an injury in fact’ with statutory damages of $1,000 per violation, or actual damages if greater.
  • AB 1043 – Digital Age Assurance Act (Wicks) – Starting January 1, 2027, operating systems and covered application stores will be required to obtain age data from users and pass on age bracket data to developers when users download and launch an application.
  • AB 56: Social Media Warning Law (Bauer-Kahan) – Starting January 1, 2027, covered social media platforms will need to display a warning label to minors the first time a user accesses the platform each day, after three hours of active use, as well as once per hour of cumulative active use after that. The warning label must say ‘The Surgeon General has warned that while social media may have benefits for some young users, social media is associated with significant mental health harms and has not been proven safe for young users.’
  • AB 621: Deepfake pornography (Bauer-Kahan) – This amends California’s Civil Code and expands protections against deepfake pornography. The law explicitly provides a cause of action against individuals who create or disclose deepfake pornography if they know, or reasonably should know, that the depicted individual was a minor and also provides a cause of action against individuals who knowingly facilitate or recklessly aid or abet the creation or disclosure of such nonconsensual deepfake pornography. The bill confirms that a minor cannot consent to the creation or distribution of deepfake pornography.

California’s approach to AI and children has a long and complicated history, and these new laws should be read in conjunction with the following laws on the books.

  • California Age Appropriate Design Code (Wicks) – This law was signed on September 15, 2022, and was scheduled to go into effect on July 1, 2024. Modeled after the UK Age Appropriate Design Code, this law requires businesses to conduct impact assessments, provide Privacy by Default, estimate the age of all users, and restrict dark patterns. The law was enjoined in March 2025, but is being appealed by the California Attorney General.
  • Protecting Our Kids from Social Media Addiction Act (Skinner-2024) – This law is scheduled to go into effect on January 1, 2027, and prohibits covered social media platforms from providing addictive feeds to minors without verifiable parental consent. The law has so far escaped a constitutional challenge, but may face other court challenges prior to the effective date.

Healthcare AI and chatbots

  • AB 489: Health care professions: deceptive terms or letters: artificial intelligence (Bonta) – This law prohibits AI systems from falsely indicating or implying possession of a medical license or certificate through advertising, marketing, or other functionality. AB 489 also makes AI developers directly subject to the healthcare professional licensing board or enforcement agency if they develop such a system. Each use of a prohibited term, letter, or phrase shall constitute a separate violation.

California’s approach to AI in healthcare also needs to be read in conjunction with the following laws and guidance.

  • Legal Advisory on the Application of Existing California Law to Artificial Intelligence in Healthcare – In January 2025, California Attorney General Rob Bonta issued this advisory, setting forth California’s existing consumer protection, civil rights, competition, and data privacy laws governing healthcare AI.
  • SB 1120: Physicians Make Decisions Act (Becker-2024) – This law prohibits covered healthcare service plans from denying, delaying, or changing healthcare services based, in whole or in part, on medical necessity using AI, algorithms, or other software tools. Such determinations shall require a physician or licensed healthcare professional and review of individual circumstances. This law also requires written policies and procedures governing such determinations.
  • AB 3030: Artificial Intelligence in Health Care Services (Calderon – 2024) – This law applies to health facilities, clinics, physicians’ offices, or other health group practices that use generative AI for communications about patient clinical information. Under this bill, generative AI, which pertains to clinical information, must include:
    • a disclaimer that indicates the communication was generated by AI at the beginning of the interaction; and
    • clear instructions on how the patient can contact the appropriate person.

Antitrust and pricing discrimination

  • AB 325: Cartwright Act violations (Aguiar-Curry) – This amends California’s existing antitrust law, the Cartwright Act, to explicitly cover ‘common pricing algorithms.’ The law prohibits:
    • the use or distribution of a ‘common pricing algorithm’ as part of a contract, combination in the form of a trust, or conspiracy to restrain trade or commerce; or
    • coercion to set or adopt a recommended price or term, recommended by the common pricing algorithm for the same or similar products or services.

Complaints shall not be required to allege facts tending to exclude the possibility of independent action.

Law enforcement use of AI

  • SB 524 Law Enforcement Agencies (Arreguín) – SB 524 requires law enforcement to disclose if an official report was written either fully or in part using AI, as well as retain the first draft created by AI and an associated audit trail that, at minimum, identifies both the officer who used AI to create a report and the video and audio footage used to create a report, if any. SB 524 also prohibits AI vendors from sharing, selling, or otherwise using information, except as provided in the bill (e.g., troubleshooting, bias mitigation, quality control, legal purposes, etc.).

Employment and bias

While Governor Newsom vetoed SB 7, the No Robo Bosses Act, the Governor’s veto letter pointed to the CPPA’s ADMT regulations as addressing some of the bill’s requirements. Per Governor Newsom, SB 7 is ‘partially covered’ by these regulations, as they ‘allow employees and independent contractors to better understand how their personal data is used by automated decision technology.’ In addition, the California Civil Rights Council’s recently promulgated regulations state that California’s antidiscrimination laws apply to AI workplace tools. These regulations address another concern raised in SB 7, which sought to prohibit ADS systems from inferring a worker’s protected status.

0
Image of a cellular phone with the ChatGPT app open.

Overview: The EU General-Purpose AI Code of Practice

AI & Algorithms, Cybersecurity, EU , , , ,

Why Do We Need a Code of Practice?

On August 2, 2025, the general-purpose AI (GPAI) provisions of the EU AI Act went into effect. GPAI models (including models that support most generative AI, like ChatGPT), now face certain obligations in the EU, including requirements around transparency, copyright and systemic risk. However, the EU AI Act is a framework: it defines obligations but leaves technical details to harmonized standards and codes of practice. While this approach sets certain expectations and allows the EU AI Act to remain technology-neutral, it also leaves questions about how businesses substantially comply with the EU AI Act. To bridge this gap, a multi-stakeholder group drafted the General-Purpose AI Code of Practice (GPAI Code). On August 1, 2025, the European Commission issued a formal opinion confirming the GPAI Code is an “adequate tool” to help demonstrate compliance with the EU AI Act. Why is the Code significant? This opinion signals that organizations who adopt the GPAI Code may be able to demonstrate good-faith efforts to comply with the relevant provisions of the EU AI Act –  according to the Commission’s website: “The Code of Practice helps industry comply with the AI Act legal obligations…of general-purpose AI models.” In its opinion, the Commission notes that the Code provides actionable commitments and reporting mechanisms, especially for high-risk models. Additionally, the Commission emphasized that the Code provides a practical framework to demonstrate regulatory compliance. Following this endorsement, providers of GPAI models can voluntarily sign the Code, which “will reduce their administrative burden and give them more legal certainty than if they proved compliance through other methods.” Still, signatories should be aware that the Code explicitly states that adherence to the Code does not necessarily constitute evidence of compliance with the EU AI Act.

What is a General-Purpose AI Model?

A GPAI model is a component of an AI system with a wide range of possible uses, whether intentional or unintentional. It is important to note that these models are not systems in themselves but are part of AI systems. Additional elements, like user interfaces, are necessary to make these models fully operational systems. Under Article 3(63) of the EU AI Act, a GPAI model includes those trained on a “large amount of data using self-supervision at scale.”  They can be applied across sectors or tasks, usually without substantial modification, meaning GPAI models “can be integrated into a variety of downstream systems or applications.” Recital 98 of the EU AI Act states that the generality of the model can also be determined by the number of parameters, and “models with at least a billion parameters…should be considered to display significant generality and to competently perform a wide range of distinctive tasks.” GPAI models are sometimes called “foundation” or “frontier” models, and while they may include large language models (LLMs), they can also process audio, physical, textual or visual data, powering systems like DALL-E, GPT-4, Gemini, LaMDA, SEER, ALIGN, and more.

How are general-purpose AI models regulated?

Under the EU AI Act, the chapter on GPAI both addresses generative AI and outlines some of the most stringent requirements under the Act. However, all requirements for GPAI under the EU AI Act are directed to providers as opposed to deployers. Providers of GPAI models have a range of obligations under the EU AI act, both directly to supervising authorities and onward to AI providers who integrate the GPAI models into their systems. Obligations of Providers of GPAI Models If a provider places a GPAI model on the EU market, or integrates such a model into its own AI system on the EU market, it must:
  • Prepare and maintain technical documentation for regulators. This should include at least a general description of the GPAI model, including the tasks it’s designed to perform and the types of systems in which it can be integrated; acceptable use policies; and information on training process.
  • Prepare and maintain documentation for downstream providers. This should include information that allows the downstream AI system providers to comply with their own obligations under Article 53(1)(b). Similar to the technical documentation, this includes but is not limited to a general description of the model, and a description of its elements and development process.
  • Prepare an EU copyright policy. This policy should establish a means to comply with EU regulations on copyright and related rights.
  • Prepare and publish a summary of training content. Using the template provided by the AI Office, providers of GPAI must share a comprehensive summary of AI training information. This should allow stakeholders to exercise their rights by informing them of the information used to train the GPAI model.
  • Cooperate with relevant authorities and appoint an authorized representative. Providers must also cooperate with relevant authorities, and if they are established outside the EU, appoint an authorized representative located in the EU.
It is notable that under Recital 85, the EU AI Act states that GPAI systems “may be used as high-risk systems by themselves or be components of other high-risk systems.” Therefore, the providers of GPAI systems must work closely with providers of high-risk AI systems to ensure compliance with any requirements of high-risk systems under the Act. Obligations of Providers of GPAI Models with Systemic Risk What does “systemic risk” mean? GPAI models with systemic risk include models that reasonably pose foreseeable negative effects relating to major accidents, disruption of critical sectors, serious consequences to public health and safety, public and economic security, democratic processes, and the dissemination of false or discriminatory content, or other similar effect. Under Article 51(1) of the EU AI Act, a GPAI model will be classified as having systemic risk if:
  • It has high impact capabilities, or
  • It is designated by the Commission to have high impact capabilities based on the criteria in Annex XIII (i.e., the number of parameters in the model, the size of the data set, the amount of computation used to train the model, etc.).
What are the additional obligations for these models? In addition to the requirements for all GPAI models, those with systemic risk have additional obligations related to:
  • Model evaluation, assessment, and mitigation of systemic risks;
  • Incident management and reporting; and
  • Cybersecurity protections and technical documentation.
Because there are differences in the obligations between GPAI systems generally and GPAI systems with systemic risk, this classification procedure should be noted by providers of GPAI systems; it is essential to understand where each GPAI model falls, and what requirements the model has under the EU AI Act. According to Article 52(6), a list of GPAI models with systemic risk will be published and updated by the European Commission, but it has not been published at the time of writing.

What is the General-Purpose AI Code of Practice?

While not legally binding, providers of GPAI models can use the Code of Practice to demonstrate compliance with their obligations under the EU AI Act. The Code consists of three chapters on 1) transparency, 2) copyright, and 3) safety and security. The first two chapters apply to all providers of general-purpose AI models, providing a way to demonstrate compliance with obligations under Article 53 of the AI Act. The final chapter applies only to general-purpose AI models with systemic risk under Article 55 of the AI Act. Chapter 1: Transparency Among other things, this chapter requires signatories to create and maintain documentation for all GPAI models distributed within the EU for up to ten years. There are exceptions for models that are free, open-source, and do not pose systemic risk. When completing this documentation, signatories must use a standard Model Documentation Form, which includes information on licensing, technical specifications, training data, and other parameters of the GPAI model. The Code encourages publication of this information to promote transparency. Chapter 2: Copyright This chapter requires signatories to create and maintain a copyright policy that complies with the EU’s legal standards. This includes, but is not limited to, ensuring that data collected by web crawling is lawfully accessible, and certain websites flagged for copyright infringement are avoided. Importantly, signatories must designate a contact for copyright holders to submit complaints, along with a process for handling those complaints. Chapter 3: Safety & Security (GPAI with systemic risk only) One of the main elements of this chapter is the requirement for signatories to develop a state-of-the-art Safety and Security Framework before releasing any GPAI model categorized as posing a systemic risk. Additionally, systemic risks should be identified and inventoried, and before progressing with development or deployment, the signatories should weigh the relative risks and determine if they are acceptable, among other requirements.

What’s next?

The Code will be monitored and reviewed at regular intervals by the AI Office, and may be updated in response to emerging risks, technological developments, or incidents involving general-purpose AI models.
0
Image of a computer circuit board with "AI" written on one of the chips.

AI Updates: An Overview of the Legal Landscape

AI & Algorithms, California, CCPA, EU , , , , , ,
As AI continues to advance, so do regulatory efforts. During the 2024 legislative session, 45 states along with Puerto Rico, the Virgin Islands, and Washington D.C. all introduced AI bills. With the legislative session for 2025 wrapping up, we are seeing similar tends this year. As new legal requirements emerge, organizations across the U.S. and EU may face overlapping – yet not identical – regulations that touch on issues of bias, safety, privacy, and transparency. Additionally, these laws may categorize the same AI system differently in different jurisdictions, requiring a nuanced approach to navigating these laws. Keeping this in mind, this article provides a brief overview of a handful of these laws. The practical takeaway? Businesses operating in the U.S. or EU should be aware of their legal requirements. Additionally, these organizations may want to consider a programmatic, auditable, and documented approach to AI governance, which may allow the business to map their AI controls to multiple legal frameworks.

Converging Themes

While details of AI laws differ across jurisdictions, trends seem to be converging on risk-based classification, transparency requirements, and enforcement efforts. Regulators are moving toward risk-based classification. This means AI uses are categorized according to their use case (and the risk associated with that use case). As seen in the EU AI Act, the Colorado AI Act, and TRAIGA, systems may be prohibited or classified by risk. High-risk systems tend to have stricter governance, testing and documentation requirements. Another shared theme is transparency. Laws including the EU AI Act, Colorado AI Act, Utah AI Policy Act, may require covered entities to tell people when AI is in use, while other laws may require the developer or deployer to explain the logic behind certain outputs, and provide consumers with a methods of contesting certain decisions, or opt out of certain types of decisionmaking entirely. The California AI Transparency Act and the EU AI Act may also require labeling of certain AI-generated content. Finally, enforcement is sharpening. The EU AI Act comes with regulatory teeth, with fines of the higher of €35,000,000 or 7% global annual turnover for violation of prohibited practices. In the U.S., state attorneys general and regulators have been active in monitoring AI missteps, including consumer protection and privacy violations. For example, attorneys general in Massachusetts and Oregon have issued advisories on how consumer protection laws apply to AI, while Texas Attorney General Ken Paxton reached the first-of-its-kind settlement in a healthcare generative AI investigation.

The European Union Artificial Intelligence Act (EU AI Act)  

Overview: The EU AI Act is the world’s first comprehensive AI regulation and sets a high-water mark for governance expectations. The Act is technology neutral and uses risk-based classification to sort AI systems into risk-tiers, each with escalating obligations. Key Provisions:
  • Prohibited systems include cognitive behavioral manipulation, most real-time biometric identification, and systems used for social scoring. These systems are considered to pose an unacceptable risk to safety or fundamental rights.
  • High-risk systems include hiring tools, biometric identification, and critical safety technology. They must undergo conformity assessments, maintain technical documentation, and ensure human oversight.
  • Limited-risk systemsinclude chatbots, deepfake generators, and public facing generative AI. These systems have transparency obligations to ensure users understand they are interacting with AI.
  • Minimal-risk systems include AI-enabled spam filers, grammar checkers, and basic AI in video games. These systems have no specific obligations under the Act, but best practices are encouraged.
Key Dates & Enforcement:
  • February 2, 2025: Prohibitions on certain AI systems and requirements on AI literacy start to apply.
  • August 2, 2025: Rules on general practice AI models, governance, confidentiality, and penalties start to apply.
  • August 2, 2026: The remainder of the AI Act (except for Article 6(1)) applies.
The Act will be enforced by European AI Office and national market surveillance authorities. Non-compliance with the prohibition of AI practices is subject to an administrative fine of up to €35,000,000 or up to 7% worldwide annual turnover, whichever is higher. Non-compliance with other provisions shall be subject to administrative fines of up to €15,000,000 of up to 3% of its total worldwide annual turnover, whichever is higher.

Colorado: Consumer Protections for Artificial Intelligence Act (CO AI Act)

Overview: Enacted in May 2024, the CO AI Act was the first far-reaching AI law in the United States. This Act primarily focuses on high-risk AI systems, including but not limited to those which influence “consequential decisions” – those impacting areas such as employment, education, housing, healthcare, finance, insurance, legal services, and essential government services. Key Provisions: Developer and deployers must both exercise “reasonable care” to protect consumers from known or reasonably foreseeable risks of algorithmic discrimination. For both, this may include providing notice to the Colorado Attorney General within 90 days of becoming aware of new discrimination risks.
  • Developers. There is a rebuttable presumption that the developer used reasonable care if they disclose, among other things:
    • reasonably foreseeable uses and known inappropriate or harmful uses of the AI system (including of algorithmic discrimination) and the measures taken to mitigate them;
    • the intended purpose, benefits, uses and outputs of the AI system; and
    • high-level summaries of the data types used to train the AI system, including data governance measures.
  • Deployers must also exercise reasonable care to protect consumers from any known or reasonably foreseeable risks of algorithmic discrimination. Similarly, there is a rebuttable presumption that the deployer used reasonable care if they complete the following, among other things:
    • a risk-management program that considers the NIST AI Risk Management Framework (AI RMF) or another similarly recognized risk management framework with substantially similar requirements (for more information about conducting an AI Risk Assessment, you can check out our post here);
    • an impact assessment, that includes the purpose, use cases, deployment context, and an analysis of whether it poses any foreseeable risks of discrimination, along with steps taken to mitigate those risks;
    • notice to consumers when certain systems are being used that include the system purpose, contact information, and options to opt-out of AI processing for that purpose, correct personal information used in the decisionmaking process, and appeal the decisionmaking process.
  • Disclosure should be clear. Regardless of risk level, any AI system that is directly interacting with Colorado consumers must disclose that it is an AI system, unless that would be obvious to a reasonable person.
Key Dates & Enforcement: While this law was originally set to take effect in 2026, Colorado Governor Polis called a special legislative session to address budget issues, taking place on August 21. The impact of SB24-05 (Consumer Protections for AI) is on the agenda, which may result in a delayed enforcement deadline and substantive changes to the law’s provisions. Violations are treated as deceptive trade practices under Colorado’s Consumer Protection Act, subject to enforcement by the Colorado Attorney General and penalties of up to $20,000 per violation.

Texas Responsible AI Governance Act (TRAIGA)

Overview: While TRAIGA originally provided a comprehensive AI framework, the final version has been significantly pared down. With narrow substantive provisions, TRAIGA focuses on harms caused by AI, and the Act regulates – or completely bans – certain uses of these systems. TRAIGA applies broadly to private sector companies if they provide AI-generated content or services to Texas residents, even if they are located outside the state of Texas. Additionally, government agencies interacting with the public fall squarely within the scope of the Act. You can read more about TRAIGA at our blog post covering the Act here. Key Provisions:
  • Prohibited AI For Public and Private Sectors include but are not limited to intentionally inciting self-harm, violence or crime; infringing on an individual’s rights; or unlawfully discriminating (with purposeful intent). The Act also prohibits deploying AI systems that intentionally generate illegal content, as well as child sexual abuse material or sexually explicit chat systems that impersonate children.
  • Prohibited AI uses for the Public Sector include but are not limited to social scoring and uniquely identifying individuals with biometric data (with limited exceptions).
  • Transparency Requirements for Public Sector may require governmental agencies to, among other things, provide conspicuous notice to consumers that they are acting with an AI system.
Key Dates & Enforcement:   TRAIGA was signed into law in June 2025 and takes effect on January 1, 2026. With no private right of action, the Act can only be enforced by the Texas Attorney General. The Act requires the Attorney General to create an “online mechanism” on their website where consumers can submit complaints of potential violations. If the Attorney General determines a violation has occurred, there is a 60-day cure period. If the violation continues after this period, the Attorney General may bring a claim for, among other things:
  • an injunction;
  • a civil penalty for curable breaches between $10,000 and $12,000;
  • a civil penalty for uncurable breaches between $80,000 and $200,000; and
  • a civil penalty for each day of continued violation between $2,000 and $40,000.
 

California CCPA Draft Regulations

Overview: On July 24, 2025, the California Privacy Protection Agency (CCPA) board voted 5-0 to finalize Draft Regulations to the California Consumer Privacy Act (CCPA). The CPPA sent the rulemaking package to the Office of Administrative Law, which has 30 days to approve the regulations. For a deeper dive on the CCPA Draft Regulations, please see our post here. Key Provisions:
  • Automated-decisionmaking (ADMT): Businesses must inform consumers with a pre-use notice and provide opt-out rights when AI or automated tools influence “significant decisions,” including those about employment, education, housing, healthcare, financial or lending services, and similar areas.
  • Risk Assessments: Organizations engaging in high-risk data processing (such as the decisions covered in ADMT, above) must conduct risk assessments before beginning processing, and must update them regularly, including within 45 days of any material change of the system. For more information about conducting an AI Risk Assessment, you can check out our post here.
  • Cybersecurity Audits: Businesses meeting certain thresholds must undergo annual, evidence-based audits carried out by a “qualified, objective, independent professional.” The audits must rely on specific evidence (as opposed to assertions by the business management), and all information related to the audit should be kept for a minimum of five years after completion.
Key Dates & Enforcement: Compliance with these Draft Regulations will be required once they are approved by the Office of Administrative Law. The deadlines include:
  • ADMT Regulations: January 1, 2027
  • Privacy Risk Assessments: December 31, 2027
  • Cybersecurity Audits:
    • For businesses with $100+ million in annual gross revenue: April 1, 2028.
    • For businesses between $50 million and $100 million in annual gross revenue: April 1, 2029.
    • For businesses with less than $50 million in annual gross revenue: April 1, 2030.

Other Laws to Consider

Along with the more far-reaching laws provided above, there are additional laws that businesses may want to consider when building, implementing, or otherwise engaging with AI tools or systems.
  • Utah’s Artificial Intelligence Policy Act
    • Effective as of May 2024, this Act mandates certain disclosures when businesses use generative AI to interact with consumers. This applies specifically to “regulated professions,” where the provider shall make the disclosure prominently, regardless of whether it is obvious the person is interacting with an AI system or not.
  • New York City’s Local Law 144 (and other AI employment regulations)
    • Signed in 2021, this law applies to employers and employment agencies in New York City that use “automated employment decision tools” to screen candidates or employees. It requires that an independent bias audit be conducted within one year of using the AI tools. For more information on AI in employment, see our article on AI In the Workplace: Legal Considerations for Leadership Teams.
  • California’s AI Transparency Law (SB 942)
    • Effective January 1, 2026, this law applies to “covered providers” – those offering generative AI systems with over 1 million monthly users in California. These providers must provide: 1) a free, public AI detection tool; and 2) certain disclosures as a label or embedded within their content.
  • California’s Data Transparency Law (AB 2013)
    • Effective January 1, 2026, developers of generative AI systems must post a disclosure on their website including documentation used to train the AI system. This documentation includes high-level summary of datasets used in the development of the AI system – the sources or owners of the datasets, how they further the purpose of the AI system, the number of datapoints in the datasets, and more.

Key Takeaway

As lawmakers race to keep up with the breakneck speed of AI implementation, guidance is quickly becoming enforcement. While specific requirements between these laws vary, the common thread is clear: covered entities are expected to understand, document, and justify their AI systems’ design, data, and impact. Additionally, organizations utilizing AI should consider building responsible AI governance into their operations. By incorporating these governance processes into everyday systems and – similar to those for privacy and cybersecurity – organizations may proactively protect against legal, ethical and operational risk when implementing AI.
0
Image depicting the flag of Texas, which is blue, white, and red, with a lone white star.

Overview of the Texas Responsible AI Governance Act

AI & Algorithms, United States , ,
In June 2024, Texas Governor Abbott signed HB 149 for the Texas Responsible Artificial Intelligence Governance Act (TRAIGA, or the Act) into law, which will go into effect on January 1, 2026. With this law, Texas joins California, Colorado and Utah in implementing AI-specific state laws. While TRAIGA was originally provided a comprehensive AI framework, the final version has been significantly pared down. With narrow substantive provisions, TRAIGA focuses on harms caused by AI, and the Act regulates – or completely bans – certain uses of these systems. Businesses that develop or deploy AI systems in Texas should consider measures for compliance with the Act. This may include reviewing system uses and updating the system’s documentation with details the Texas Attorney General (AG) may need to investigate a complaint. Scope & Applicability TRAIGA applies broadly to private sector companies if they provide AI-generated content or services to Texas residents, even if they are located outside the state of Texas. Additionally, government agencies interacting with the public fall squarely within the scope of the Act. Substantive Provisions
  • Prohibited Uses for all AI. TRAIGA prohibits certain uses of AI systems for both the public and private sector, including intentionally inciting self-harm, violence or crime; infringing on an individual’s rights; or unlawfully discriminating. The Act also prohibits deploying AI systems that intentionally generate illegal content, as well as child sexual abuse material or sexually explicit chat systems that impersonate children.

Notably, accidental or disparate impact alone is most likely not enough to violate TRAIGA, as there most likely must be a purposeful intent to discriminate using the AI system.

  • Public Sector: While the scope of TRAIGA includes both public and private entities, there are additional requirements for Texas governmental agencies.
    • Prohibited Uses: Prohibited uses include, among other things, social scoring and uniquely identifying an individual using biometric data, with limited exceptions.
    • Transparency Requirements: Governmental agencies must, among other things, provide conspicuous notice to consumers that are interacting with the AI system – even if this would be obvious to the user; and are prohibited from gathering biometric data without the individual’s informed consent if doing so would infringe that individual’s rights.
Enforcement With no private right of action, TRAIGA can only be enforced by the Texas AG. The Act requires the attorney general to create an “online mechanism” on the AG’s website where consumers can submit complaints of potential violations. If the AG investigates a complaint, developers and deployers of the AI systems may be required to provide information including, but not limited to, a description of the:
  • purpose, intended use, deployment context, and associated benefits of the AI system;
  • categories of data used as inputs and of the outputs produced by the system;
  • types of data used to train the system;
  • evaluation criteria of the performance of the system;
  • known limitations of the system; and
  • post-deployment and user safeguards (e.g., oversight, use and learning processes established to address issues).
If the AG determines a violation has occurred, there is a 60-day cure period. If the violation continues after this period, the AG may bring a claim for, among other things:
  • an injunction;
  • a civil penalty for curable breaches between $10,000 and $12,000;
  • a civil penalty for uncurable breaches between $80,000 and $200,000; and
  • a civil penalty for each day of continued violation between $2,000 and $40,000.
Safe Harbor Notably, TRAIGA states that a defendant may not be found liable for violations if the defendant substantially complies with the most recent version of the Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (AI RMF Framework) published by the National Institute of Standards and Technology (NIST) or another recognized risk management framework for AI systems. Sandbox Program Under TRAIGA, the Texas Department of Information Resources is required to establish a sandbox program. This program would enable developers and deployers of AI systems to obtain legal protection and limited market access. The AG may not file or pursue charges against a program participant for violations of TRAIGA that occur during the testing period. Effective Date The Act goes into effect on January 1, 2026.
0
An image of the flag of Europe, which consists of twelve golden stars forming a circle on a blue field.

EDPB Opinion on AI Models and GDPR Principles: Key Takeaways

AI & Algorithms, EEA & UK, EU, GDPR , , , , ,
In December 2024, the European Data Protection Board (EDPB) issued an Opinion in response to a request from the Irish supervisory authority, focusing on the application of GDPR principles in the context of AI models. The Irish supervisory authority posed three specific questions:
  1. When and how can an AI model be considered “anonymous”?
  2. What is the appropriateness of legitimate interest as a legal basis for AI deployment and development?
  3. What are the consequences of unlawful processing of personal data on subsequent operations of the AI model?
Through its answers, the EDPB provided key guidance on how AI models interact with fundamental rights to privacy and data protection established in the GDPR.

Anonymous AI Models

According the EDPB, “[f]or a model to be anonymous, it should be very unlikely 1) to directly or indirectly identify individuals whose data was used to create the model, and 2) to extract such personal information from the model through queries.” While anonymous data can help mitigate privacy concerns, it does not automatically make the AI model completely exempt from GDPR compliance. When a model is claimed to be anonymous, supervisory authorities will evaluate the claims of anonymity on a case-by-case basis, considering “all the means likely to be used” by the controller or a user. The Opinion states that supervisory authorities should review the documentation provided by the controller when assessing if the model is truly anonymous. The EDPB outlines methods that the controller may use to demonstrate anonymity, which may include: 1) reducing the amount of personal data used during training, 2) taking steps to ensure this data cannot be identified, and 3) utilizing technical safeguards to prevent data extraction from the AI model using prompts or queries. Key Takeaway: If a business claims an AI model relies on anonymous data, the claims of anonymity should be substantiated on a case-by-case basis with sufficient evidence and documentation. To do this, businesses with allegedly anonymous AI models may need to implement technical measures to limit the collection of data, reduce the likelihood of data being identifiable, protect against that data being extracted by users during deployment, and create documentation capable of demonstrating these efforts.

Legitimate Interest as a Legal Basis

Under the GDPR, a legitimate interest may constitute a legal basis for companies to process personal data when they have a justifiable reason to do so (beyond obtaining consent). However, the legitimate interest should be balanced against the data subject’s rights and interests, which requires careful consideration and justification when processing information from data subjects. The Opinion provides a framework to assess if a legitimate interest can be a valid legal basis for processing personal data in AI development and deployment. The framework is comprised of a three-step test:
  1. Identify the legitimate interest pursued by the controller;
  2. Assess the necessity of the processing for purposes of the legitimate interest; and,
  3. Balance the legitimate interests against the rights and freedoms of the data subjects.
When conducting this test, the controller should be careful to identify an interest that is lawful, clearly articulated, and non-speculative. For example, a legitimate interest may be to develop an AI model’s conversational agent or to improve threat detection in an information system. The controller should also adhere to GDPR data minimization principles, which state that the processing activities must be proportionate and in line with only what is necessary to achieve the legitimate interest. Finally, controllers should conduct a nuanced balancing test. This test considers the unique circumstances of each case, which may include the data subject’s interest in retaining control over their data, personal benefits, or socioeconomic interest. The Opinion notes, the more precisely an interest is defined in relation to the purpose of the processing, the more precise the estimation of benefits and risks will be. By employing this framework, developers and deployers should be able to decrease the likelihood that their AI models are disproportionately infringing on individual privacy rights and better align their AI practices with GDPR requirements. Key Takeaway: The three-step analysis, according to the Opinion, is crucial to improving compliance for organizations relying on legitimate interest as a legal basis for processing in AI development or deployment. Organizations relying on legitimate interest in this AI context should review their processing activities to determine whether they are proportionate, transparent, and aligned with GDPR principles—like data minimization—to justify the reliance on legitimate interest as a legal basis for processing.

Consequences of Unlawful Processing

The Opinion notes that supervisory authorities enjoy discretionary powers to investigate and assess violations, and they can choose appropriate remedial measures based on the context of the case. However, the EDPB also provides guidance for the supervisory authorities, based on three scenarios.
  1. In the first scenario, personal data is retained in the AI model. The Opinion states that supervisory authorities will need to consider the surrounding circumstances of the AI model to determine if the development and deployment phases of the model involve different legitimate purposes for processing. If so, each should be examined separately.
  2. In the second scenario, personal data is retained in the model and is processed by another controller during deployment. In this instance, the supervisory authorities should determine if the deploying controller conducted an appropriate assessment to demonstrate accountability with Articles 5(1)(a) and 6 of the GDPR. This assessment should show that the AI model was not developed by unlawfully processing personal data.
  3. In the final scenario, a controller unlawfully processes personal data to develop the AI model, and then anonymizes the data before processing it in the context of deployment. The Opinion states that, if it can be demonstrated to the supervisory authorities that the deployment of the AI model does not entail the processing of personal data, then the GDPR does not apply. Therefore, the unlawfulness of the initial processing in development should not impact the deployment operation of the model.
While supervisory authorities do have substantial discretion in oversight of processing activities, the scenarios highlighted by the EDPB show that the development and deployment phases, while connected, may need to be evaluated independently. Key Takeaway: Organizations should proactively ensure compliance at both the development and deployment stages of an AI model. Supervisory authorities will likely use the above examples as guidance, emphasizing the important of demonstrating lawful practices through each stage of the model. The EDPB’s Opinion is an important guide for organizations navigating the intersection of AI and data privacy law. By addressing issues around anonymous AI models, legitimate interest, and lawful processing in development and deployment stages, the Opinion emphasizes responsible AI development. As AI technologies continue to advance, businesses should be aware of the ways supervisory authorities are overseeing their AI models. The insights provided by the EDPB provide a foundation to help businesses to advance and develop new AI models, while also helping to safeguard and protect the rights of individuals.
1 2 3 4