0
Flag of California, depicting a large brown bear beside a red star, above the words "California Republic."

California: New AI laws in California – roundup of the 2025 legislative session

This article was originally published by OneTrust DataGuidance on November 24, 2025 and can be found on the DataGuidance website here.

California introduces comprehensive AI laws focusing on transparency, children’s safety, healthcare, antitrust, and law enforcement.

California has taken an aggressive stance towards artificial intelligence (AI) legislation and will likely set the standard for other US states. Back in 2024, Governor Newsom vetoed comprehensive AI safety legislation under bill SB 1047 and advised caution on regulations for this nascent and important technology. This year, Governor Newsom pressed ahead with a full slate of new AI laws. The reasons for this change in approach are many, including but not limited to the lack of federal AI legislation, the growing concern over children’s interactions with AI, especially sexualized content, and harmonization with more stringent requirements in the EU and elsewhere.

This year’s legislative session set records for the number and scope of new AI laws. For the roundup this year, Lily Li, of Metaverse Law Corporation, breaks down the new AI laws by scope and sector, noting where this may add on to existing California legislation and rulemaking from 2024-2025.

General AI safety, transparency, and risk assessments

  • SB 53: Transparency in Frontier Artificial Intelligence Act (Wiener) – Starting in January 2026, California will require large frontier AI developers to publish a framework detailing how they incorporate safety, security, and testing standards into their AI models. SB 53 also creates a mechanism for AI developers and the public to report critical safety incidents, and protects internal whistleblowers who report risks posed by frontier AI models. The law establishes significant penalties for companies that fail to comply, with fines of up to $1 million per violation.
  • AB 316: Artificial Intelligence defenses (Krell) – This amends California’s Civil Code. If a party to a lawsuit develops, modifies, or uses AI, this law prohibits them from asserting as a defense that the AI autonomously caused the harm.
  • AB 853: California AI Transparency Act (Wicks) – This bill expands the existing AI Transparency Act and modifies the effective date from January 1, 2026, to August 2, 2026. The California AI Transparency Act requires covered generative AI developers to provide an AI-detection tool to assess whether image, video, or audio content is created or altered by generative AI. This bill adds to the existing law by requiring large online platforms to embed provenance data into generated content. Starting January 1, 2028, users will also have the option to include latent disclosures on ‘capture devices’ such as cameras, video recorders, and other recorders.

This new California approach to AI transparency and safety legislation needs to be read in conjunction with the following existing laws.

  • California Privacy Protection Agency’s (CPPA’s) recently approved Cyber, Risk, ADMT, and Insurance Regulations – The CPPA’s most recently updated 127-page regulation package contains requirements governing cybersecurity audits, risk assessments, and automated decision-making technology. AI developers and systems that process personal information and meet certain California privacy thresholds will now face new cybersecurity audit and risk assessment requirements. In addition, automated and significant decisions concerning the provision or denial of financial or lending services, housing, education enrollment or opportunities, employment or independent contracting opportunities or compensation, or healthcare services will trigger significant notice, opt-out, and risk assessment requirements.
  • AB 2013: AI Training Data Transparency Act (Irwin-2024) – Passed last year, this law will require covered generative AI developers to publish online a high-level summary of the datasets used in the development of the generative AI system or service, including but not limited to whether personal information or copyrighted information is included in the training data. The law is scheduled to go into effect on January 1, 2026.

Children’s safety, age verifications, and companion chatbots

  • SB243: Companion Chatbots (Padilla) – This law applies to chatbots that provide human-like interactions and are capable of sustaining relationships across multiple interactions. Beginning July 1, 2027, developers of these ‘companion chatbots’ will need to develop and report protocols addressing suicidal ideation and self-harm to regulators and the public. The law requires AI disclosures, referrals to suicide hotlines or crisis text lines, and break reminders. SB 243 further requires developers to institute reasonable measures to prevent the chatbot from producing visual material of sexually explicit conduct or directly stating that the minor should engage in sexually explicit conduct. The legislation includes a private right of action to individuals who suffer ‘an injury in fact’ with statutory damages of $1,000 per violation, or actual damages if greater.
  • AB 1043 – Digital Age Assurance Act (Wicks) – Starting January 1, 2027, operating systems and covered application stores will be required to obtain age data from users and pass on age bracket data to developers when users download and launch an application.
  • AB 56: Social Media Warning Law (Bauer-Kahan) – Starting January 1, 2027, covered social media platforms will need to display a warning label to minors the first time a user accesses the platform each day, after three hours of active use, as well as once per hour of cumulative active use after that. The warning label must say ‘The Surgeon General has warned that while social media may have benefits for some young users, social media is associated with significant mental health harms and has not been proven safe for young users.’
  • AB 621: Deepfake pornography (Bauer-Kahan) – This amends California’s Civil Code and expands protections against deepfake pornography. The law explicitly provides a cause of action against individuals who create or disclose deepfake pornography if they know, or reasonably should know, that the depicted individual was a minor and also provides a cause of action against individuals who knowingly facilitate or recklessly aid or abet the creation or disclosure of such nonconsensual deepfake pornography. The bill confirms that a minor cannot consent to the creation or distribution of deepfake pornography.

California’s approach to AI and children has a long and complicated history, and these new laws should be read in conjunction with the following laws on the books.

  • California Age Appropriate Design Code (Wicks) – This law was signed on September 15, 2022, and was scheduled to go into effect on July 1, 2024. Modeled after the UK Age Appropriate Design Code, this law requires businesses to conduct impact assessments, provide Privacy by Default, estimate the age of all users, and restrict dark patterns. The law was enjoined in March 2025, but is being appealed by the California Attorney General.
  • Protecting Our Kids from Social Media Addiction Act (Skinner-2024) – This law is scheduled to go into effect on January 1, 2027, and prohibits covered social media platforms from providing addictive feeds to minors without verifiable parental consent. The law has so far escaped a constitutional challenge, but may face other court challenges prior to the effective date.

Healthcare AI and chatbots

  • AB 489: Health care professions: deceptive terms or letters: artificial intelligence (Bonta) – This law prohibits AI systems from falsely indicating or implying possession of a medical license or certificate through advertising, marketing, or other functionality. AB 489 also makes AI developers directly subject to the healthcare professional licensing board or enforcement agency if they develop such a system. Each use of a prohibited term, letter, or phrase shall constitute a separate violation.

California’s approach to AI in healthcare also needs to be read in conjunction with the following laws and guidance.

  • Legal Advisory on the Application of Existing California Law to Artificial Intelligence in Healthcare – In January 2025, California Attorney General Rob Bonta issued this advisory, setting forth California’s existing consumer protection, civil rights, competition, and data privacy laws governing healthcare AI.
  • SB 1120: Physicians Make Decisions Act (Becker-2024) – This law prohibits covered healthcare service plans from denying, delaying, or changing healthcare services based, in whole or in part, on medical necessity using AI, algorithms, or other software tools. Such determinations shall require a physician or licensed healthcare professional and review of individual circumstances. This law also requires written policies and procedures governing such determinations.
  • AB 3030: Artificial Intelligence in Health Care Services (Calderon – 2024) – This law applies to health facilities, clinics, physicians’ offices, or other health group practices that use generative AI for communications about patient clinical information. Under this bill, generative AI, which pertains to clinical information, must include:
    • a disclaimer that indicates the communication was generated by AI at the beginning of the interaction; and
    • clear instructions on how the patient can contact the appropriate person.

Antitrust and pricing discrimination

  • AB 325: Cartwright Act violations (Aguiar-Curry) – This amends California’s existing antitrust law, the Cartwright Act, to explicitly cover ‘common pricing algorithms.’ The law prohibits:
    • the use or distribution of a ‘common pricing algorithm’ as part of a contract, combination in the form of a trust, or conspiracy to restrain trade or commerce; or
    • coercion to set or adopt a recommended price or term, recommended by the common pricing algorithm for the same or similar products or services.

Complaints shall not be required to allege facts tending to exclude the possibility of independent action.

Law enforcement use of AI

  • SB 524 Law Enforcement Agencies (Arreguín) – SB 524 requires law enforcement to disclose if an official report was written either fully or in part using AI, as well as retain the first draft created by AI and an associated audit trail that, at minimum, identifies both the officer who used AI to create a report and the video and audio footage used to create a report, if any. SB 524 also prohibits AI vendors from sharing, selling, or otherwise using information, except as provided in the bill (e.g., troubleshooting, bias mitigation, quality control, legal purposes, etc.).

Employment and bias

While Governor Newsom vetoed SB 7, the No Robo Bosses Act, the Governor’s veto letter pointed to the CPPA’s ADMT regulations as addressing some of the bill’s requirements. Per Governor Newsom, SB 7 is ‘partially covered’ by these regulations, as they ‘allow employees and independent contractors to better understand how their personal data is used by automated decision technology.’ In addition, the California Civil Rights Council’s recently promulgated regulations state that California’s antidiscrimination laws apply to AI workplace tools. These regulations address another concern raised in SB 7, which sought to prohibit ADS systems from inferring a worker’s protected status.

0
Image of a cellular phone with the ChatGPT app open.

Overview: The EU General-Purpose AI Code of Practice

Why Do We Need a Code of Practice?

On August 2, 2025, the general-purpose AI (GPAI) provisions of the EU AI Act went into effect. GPAI models (including models that support most generative AI, like ChatGPT), now face certain obligations in the EU, including requirements around transparency, copyright and systemic risk. However, the EU AI Act is a framework: it defines obligations but leaves technical details to harmonized standards and codes of practice. While this approach sets certain expectations and allows the EU AI Act to remain technology-neutral, it also leaves questions about how businesses substantially comply with the EU AI Act. To bridge this gap, a multi-stakeholder group drafted the General-Purpose AI Code of Practice (GPAI Code). On August 1, 2025, the European Commission issued a formal opinion confirming the GPAI Code is an “adequate tool” to help demonstrate compliance with the EU AI Act. Why is the Code significant? This opinion signals that organizations who adopt the GPAI Code may be able to demonstrate good-faith efforts to comply with the relevant provisions of the EU AI Act –  according to the Commission’s website: “The Code of Practice helps industry comply with the AI Act legal obligations…of general-purpose AI models.” In its opinion, the Commission notes that the Code provides actionable commitments and reporting mechanisms, especially for high-risk models. Additionally, the Commission emphasized that the Code provides a practical framework to demonstrate regulatory compliance. Following this endorsement, providers of GPAI models can voluntarily sign the Code, which “will reduce their administrative burden and give them more legal certainty than if they proved compliance through other methods.” Still, signatories should be aware that the Code explicitly states that adherence to the Code does not necessarily constitute evidence of compliance with the EU AI Act.

What is a General-Purpose AI Model?

A GPAI model is a component of an AI system with a wide range of possible uses, whether intentional or unintentional. It is important to note that these models are not systems in themselves but are part of AI systems. Additional elements, like user interfaces, are necessary to make these models fully operational systems. Under Article 3(63) of the EU AI Act, a GPAI model includes those trained on a “large amount of data using self-supervision at scale.”  They can be applied across sectors or tasks, usually without substantial modification, meaning GPAI models “can be integrated into a variety of downstream systems or applications.” Recital 98 of the EU AI Act states that the generality of the model can also be determined by the number of parameters, and “models with at least a billion parameters…should be considered to display significant generality and to competently perform a wide range of distinctive tasks.” GPAI models are sometimes called “foundation” or “frontier” models, and while they may include large language models (LLMs), they can also process audio, physical, textual or visual data, powering systems like DALL-E, GPT-4, Gemini, LaMDA, SEER, ALIGN, and more.

How are general-purpose AI models regulated?

Under the EU AI Act, the chapter on GPAI both addresses generative AI and outlines some of the most stringent requirements under the Act. However, all requirements for GPAI under the EU AI Act are directed to providers as opposed to deployers. Providers of GPAI models have a range of obligations under the EU AI act, both directly to supervising authorities and onward to AI providers who integrate the GPAI models into their systems. Obligations of Providers of GPAI Models If a provider places a GPAI model on the EU market, or integrates such a model into its own AI system on the EU market, it must:
  • Prepare and maintain technical documentation for regulators. This should include at least a general description of the GPAI model, including the tasks it’s designed to perform and the types of systems in which it can be integrated; acceptable use policies; and information on training process.
  • Prepare and maintain documentation for downstream providers. This should include information that allows the downstream AI system providers to comply with their own obligations under Article 53(1)(b). Similar to the technical documentation, this includes but is not limited to a general description of the model, and a description of its elements and development process.
  • Prepare an EU copyright policy. This policy should establish a means to comply with EU regulations on copyright and related rights.
  • Prepare and publish a summary of training content. Using the template provided by the AI Office, providers of GPAI must share a comprehensive summary of AI training information. This should allow stakeholders to exercise their rights by informing them of the information used to train the GPAI model.
  • Cooperate with relevant authorities and appoint an authorized representative. Providers must also cooperate with relevant authorities, and if they are established outside the EU, appoint an authorized representative located in the EU.
It is notable that under Recital 85, the EU AI Act states that GPAI systems “may be used as high-risk systems by themselves or be components of other high-risk systems.” Therefore, the providers of GPAI systems must work closely with providers of high-risk AI systems to ensure compliance with any requirements of high-risk systems under the Act. Obligations of Providers of GPAI Models with Systemic Risk What does “systemic risk” mean? GPAI models with systemic risk include models that reasonably pose foreseeable negative effects relating to major accidents, disruption of critical sectors, serious consequences to public health and safety, public and economic security, democratic processes, and the dissemination of false or discriminatory content, or other similar effect. Under Article 51(1) of the EU AI Act, a GPAI model will be classified as having systemic risk if:
  • It has high impact capabilities, or
  • It is designated by the Commission to have high impact capabilities based on the criteria in Annex XIII (i.e., the number of parameters in the model, the size of the data set, the amount of computation used to train the model, etc.).
What are the additional obligations for these models? In addition to the requirements for all GPAI models, those with systemic risk have additional obligations related to:
  • Model evaluation, assessment, and mitigation of systemic risks;
  • Incident management and reporting; and
  • Cybersecurity protections and technical documentation.
Because there are differences in the obligations between GPAI systems generally and GPAI systems with systemic risk, this classification procedure should be noted by providers of GPAI systems; it is essential to understand where each GPAI model falls, and what requirements the model has under the EU AI Act. According to Article 52(6), a list of GPAI models with systemic risk will be published and updated by the European Commission, but it has not been published at the time of writing.

What is the General-Purpose AI Code of Practice?

While not legally binding, providers of GPAI models can use the Code of Practice to demonstrate compliance with their obligations under the EU AI Act. The Code consists of three chapters on 1) transparency, 2) copyright, and 3) safety and security. The first two chapters apply to all providers of general-purpose AI models, providing a way to demonstrate compliance with obligations under Article 53 of the AI Act. The final chapter applies only to general-purpose AI models with systemic risk under Article 55 of the AI Act. Chapter 1: Transparency Among other things, this chapter requires signatories to create and maintain documentation for all GPAI models distributed within the EU for up to ten years. There are exceptions for models that are free, open-source, and do not pose systemic risk. When completing this documentation, signatories must use a standard Model Documentation Form, which includes information on licensing, technical specifications, training data, and other parameters of the GPAI model. The Code encourages publication of this information to promote transparency. Chapter 2: Copyright This chapter requires signatories to create and maintain a copyright policy that complies with the EU’s legal standards. This includes, but is not limited to, ensuring that data collected by web crawling is lawfully accessible, and certain websites flagged for copyright infringement are avoided. Importantly, signatories must designate a contact for copyright holders to submit complaints, along with a process for handling those complaints. Chapter 3: Safety & Security (GPAI with systemic risk only) One of the main elements of this chapter is the requirement for signatories to develop a state-of-the-art Safety and Security Framework before releasing any GPAI model categorized as posing a systemic risk. Additionally, systemic risks should be identified and inventoried, and before progressing with development or deployment, the signatories should weigh the relative risks and determine if they are acceptable, among other requirements.

What’s next?

The Code will be monitored and reviewed at regular intervals by the AI Office, and may be updated in response to emerging risks, technological developments, or incidents involving general-purpose AI models.
0
Image of a computer circuit board with "AI" written on one of the chips.

AI Updates: An Overview of the Legal Landscape

As AI continues to advance, so do regulatory efforts. During the 2024 legislative session, 45 states along with Puerto Rico, the Virgin Islands, and Washington D.C. all introduced AI bills. With the legislative session for 2025 wrapping up, we are seeing similar tends this year. As new legal requirements emerge, organizations across the U.S. and EU may face overlapping – yet not identical – regulations that touch on issues of bias, safety, privacy, and transparency. Additionally, these laws may categorize the same AI system differently in different jurisdictions, requiring a nuanced approach to navigating these laws. Keeping this in mind, this article provides a brief overview of a handful of these laws. The practical takeaway? Businesses operating in the U.S. or EU should be aware of their legal requirements. Additionally, these organizations may want to consider a programmatic, auditable, and documented approach to AI governance, which may allow the business to map their AI controls to multiple legal frameworks.

Converging Themes

While details of AI laws differ across jurisdictions, trends seem to be converging on risk-based classification, transparency requirements, and enforcement efforts. Regulators are moving toward risk-based classification. This means AI uses are categorized according to their use case (and the risk associated with that use case). As seen in the EU AI Act, the Colorado AI Act, and TRAIGA, systems may be prohibited or classified by risk. High-risk systems tend to have stricter governance, testing and documentation requirements. Another shared theme is transparency. Laws including the EU AI Act, Colorado AI Act, Utah AI Policy Act, may require covered entities to tell people when AI is in use, while other laws may require the developer or deployer to explain the logic behind certain outputs, and provide consumers with a methods of contesting certain decisions, or opt out of certain types of decisionmaking entirely. The California AI Transparency Act and the EU AI Act may also require labeling of certain AI-generated content. Finally, enforcement is sharpening. The EU AI Act comes with regulatory teeth, with fines of the higher of €35,000,000 or 7% global annual turnover for violation of prohibited practices. In the U.S., state attorneys general and regulators have been active in monitoring AI missteps, including consumer protection and privacy violations. For example, attorneys general in Massachusetts and Oregon have issued advisories on how consumer protection laws apply to AI, while Texas Attorney General Ken Paxton reached the first-of-its-kind settlement in a healthcare generative AI investigation.

The European Union Artificial Intelligence Act (EU AI Act)  

Overview: The EU AI Act is the world’s first comprehensive AI regulation and sets a high-water mark for governance expectations. The Act is technology neutral and uses risk-based classification to sort AI systems into risk-tiers, each with escalating obligations. Key Provisions:
  • Prohibited systems include cognitive behavioral manipulation, most real-time biometric identification, and systems used for social scoring. These systems are considered to pose an unacceptable risk to safety or fundamental rights.
  • High-risk systems include hiring tools, biometric identification, and critical safety technology. They must undergo conformity assessments, maintain technical documentation, and ensure human oversight.
  • Limited-risk systemsinclude chatbots, deepfake generators, and public facing generative AI. These systems have transparency obligations to ensure users understand they are interacting with AI.
  • Minimal-risk systems include AI-enabled spam filers, grammar checkers, and basic AI in video games. These systems have no specific obligations under the Act, but best practices are encouraged.
Key Dates & Enforcement:
  • February 2, 2025: Prohibitions on certain AI systems and requirements on AI literacy start to apply.
  • August 2, 2025: Rules on general practice AI models, governance, confidentiality, and penalties start to apply.
  • August 2, 2026: The remainder of the AI Act (except for Article 6(1)) applies.
The Act will be enforced by European AI Office and national market surveillance authorities. Non-compliance with the prohibition of AI practices is subject to an administrative fine of up to €35,000,000 or up to 7% worldwide annual turnover, whichever is higher. Non-compliance with other provisions shall be subject to administrative fines of up to €15,000,000 of up to 3% of its total worldwide annual turnover, whichever is higher.

Colorado: Consumer Protections for Artificial Intelligence Act (CO AI Act)

Overview: Enacted in May 2024, the CO AI Act was the first far-reaching AI law in the United States. This Act primarily focuses on high-risk AI systems, including but not limited to those which influence “consequential decisions” – those impacting areas such as employment, education, housing, healthcare, finance, insurance, legal services, and essential government services. Key Provisions: Developer and deployers must both exercise “reasonable care” to protect consumers from known or reasonably foreseeable risks of algorithmic discrimination. For both, this may include providing notice to the Colorado Attorney General within 90 days of becoming aware of new discrimination risks.
  • Developers. There is a rebuttable presumption that the developer used reasonable care if they disclose, among other things:
    • reasonably foreseeable uses and known inappropriate or harmful uses of the AI system (including of algorithmic discrimination) and the measures taken to mitigate them;
    • the intended purpose, benefits, uses and outputs of the AI system; and
    • high-level summaries of the data types used to train the AI system, including data governance measures.
  • Deployers must also exercise reasonable care to protect consumers from any known or reasonably foreseeable risks of algorithmic discrimination. Similarly, there is a rebuttable presumption that the deployer used reasonable care if they complete the following, among other things:
    • a risk-management program that considers the NIST AI Risk Management Framework (AI RMF) or another similarly recognized risk management framework with substantially similar requirements (for more information about conducting an AI Risk Assessment, you can check out our post here);
    • an impact assessment, that includes the purpose, use cases, deployment context, and an analysis of whether it poses any foreseeable risks of discrimination, along with steps taken to mitigate those risks;
    • notice to consumers when certain systems are being used that include the system purpose, contact information, and options to opt-out of AI processing for that purpose, correct personal information used in the decisionmaking process, and appeal the decisionmaking process.
  • Disclosure should be clear. Regardless of risk level, any AI system that is directly interacting with Colorado consumers must disclose that it is an AI system, unless that would be obvious to a reasonable person.
Key Dates & Enforcement: While this law was originally set to take effect in 2026, Colorado Governor Polis called a special legislative session to address budget issues, taking place on August 21. The impact of SB24-05 (Consumer Protections for AI) is on the agenda, which may result in a delayed enforcement deadline and substantive changes to the law’s provisions. Violations are treated as deceptive trade practices under Colorado’s Consumer Protection Act, subject to enforcement by the Colorado Attorney General and penalties of up to $20,000 per violation.

Texas Responsible AI Governance Act (TRAIGA)

Overview: While TRAIGA originally provided a comprehensive AI framework, the final version has been significantly pared down. With narrow substantive provisions, TRAIGA focuses on harms caused by AI, and the Act regulates – or completely bans – certain uses of these systems. TRAIGA applies broadly to private sector companies if they provide AI-generated content or services to Texas residents, even if they are located outside the state of Texas. Additionally, government agencies interacting with the public fall squarely within the scope of the Act. You can read more about TRAIGA at our blog post covering the Act here. Key Provisions:
  • Prohibited AI For Public and Private Sectors include but are not limited to intentionally inciting self-harm, violence or crime; infringing on an individual’s rights; or unlawfully discriminating (with purposeful intent). The Act also prohibits deploying AI systems that intentionally generate illegal content, as well as child sexual abuse material or sexually explicit chat systems that impersonate children.
  • Prohibited AI uses for the Public Sector include but are not limited to social scoring and uniquely identifying individuals with biometric data (with limited exceptions).
  • Transparency Requirements for Public Sector may require governmental agencies to, among other things, provide conspicuous notice to consumers that they are acting with an AI system.
Key Dates & Enforcement:   TRAIGA was signed into law in June 2025 and takes effect on January 1, 2026. With no private right of action, the Act can only be enforced by the Texas Attorney General. The Act requires the Attorney General to create an “online mechanism” on their website where consumers can submit complaints of potential violations. If the Attorney General determines a violation has occurred, there is a 60-day cure period. If the violation continues after this period, the Attorney General may bring a claim for, among other things:
  • an injunction;
  • a civil penalty for curable breaches between $10,000 and $12,000;
  • a civil penalty for uncurable breaches between $80,000 and $200,000; and
  • a civil penalty for each day of continued violation between $2,000 and $40,000.
 

California CCPA Draft Regulations

Overview: On July 24, 2025, the California Privacy Protection Agency (CCPA) board voted 5-0 to finalize Draft Regulations to the California Consumer Privacy Act (CCPA). The CPPA sent the rulemaking package to the Office of Administrative Law, which has 30 days to approve the regulations. For a deeper dive on the CCPA Draft Regulations, please see our post here. Key Provisions:
  • Automated-decisionmaking (ADMT): Businesses must inform consumers with a pre-use notice and provide opt-out rights when AI or automated tools influence “significant decisions,” including those about employment, education, housing, healthcare, financial or lending services, and similar areas.
  • Risk Assessments: Organizations engaging in high-risk data processing (such as the decisions covered in ADMT, above) must conduct risk assessments before beginning processing, and must update them regularly, including within 45 days of any material change of the system. For more information about conducting an AI Risk Assessment, you can check out our post here.
  • Cybersecurity Audits: Businesses meeting certain thresholds must undergo annual, evidence-based audits carried out by a “qualified, objective, independent professional.” The audits must rely on specific evidence (as opposed to assertions by the business management), and all information related to the audit should be kept for a minimum of five years after completion.
Key Dates & Enforcement: Compliance with these Draft Regulations will be required once they are approved by the Office of Administrative Law. The deadlines include:
  • ADMT Regulations: January 1, 2027
  • Privacy Risk Assessments: December 31, 2027
  • Cybersecurity Audits:
    • For businesses with $100+ million in annual gross revenue: April 1, 2028.
    • For businesses between $50 million and $100 million in annual gross revenue: April 1, 2029.
    • For businesses with less than $50 million in annual gross revenue: April 1, 2030.

Other Laws to Consider

Along with the more far-reaching laws provided above, there are additional laws that businesses may want to consider when building, implementing, or otherwise engaging with AI tools or systems.
  • Utah’s Artificial Intelligence Policy Act
    • Effective as of May 2024, this Act mandates certain disclosures when businesses use generative AI to interact with consumers. This applies specifically to “regulated professions,” where the provider shall make the disclosure prominently, regardless of whether it is obvious the person is interacting with an AI system or not.
  • New York City’s Local Law 144 (and other AI employment regulations)
    • Signed in 2021, this law applies to employers and employment agencies in New York City that use “automated employment decision tools” to screen candidates or employees. It requires that an independent bias audit be conducted within one year of using the AI tools. For more information on AI in employment, see our article on AI In the Workplace: Legal Considerations for Leadership Teams.
  • California’s AI Transparency Law (SB 942)
    • Effective January 1, 2026, this law applies to “covered providers” – those offering generative AI systems with over 1 million monthly users in California. These providers must provide: 1) a free, public AI detection tool; and 2) certain disclosures as a label or embedded within their content.
  • California’s Data Transparency Law (AB 2013)
    • Effective January 1, 2026, developers of generative AI systems must post a disclosure on their website including documentation used to train the AI system. This documentation includes high-level summary of datasets used in the development of the AI system – the sources or owners of the datasets, how they further the purpose of the AI system, the number of datapoints in the datasets, and more.

Key Takeaway

As lawmakers race to keep up with the breakneck speed of AI implementation, guidance is quickly becoming enforcement. While specific requirements between these laws vary, the common thread is clear: covered entities are expected to understand, document, and justify their AI systems’ design, data, and impact. Additionally, organizations utilizing AI should consider building responsible AI governance into their operations. By incorporating these governance processes into everyday systems and – similar to those for privacy and cybersecurity – organizations may proactively protect against legal, ethical and operational risk when implementing AI.
0
Image depicting the flag of Texas, which is blue, white, and red, with a lone white star.

Overview of the Texas Responsible AI Governance Act

In June 2024, Texas Governor Abbott signed HB 149 for the Texas Responsible Artificial Intelligence Governance Act (TRAIGA, or the Act) into law, which will go into effect on January 1, 2026. With this law, Texas joins California, Colorado and Utah in implementing AI-specific state laws. While TRAIGA was originally provided a comprehensive AI framework, the final version has been significantly pared down. With narrow substantive provisions, TRAIGA focuses on harms caused by AI, and the Act regulates – or completely bans – certain uses of these systems. Businesses that develop or deploy AI systems in Texas should consider measures for compliance with the Act. This may include reviewing system uses and updating the system’s documentation with details the Texas Attorney General (AG) may need to investigate a complaint. Scope & Applicability TRAIGA applies broadly to private sector companies if they provide AI-generated content or services to Texas residents, even if they are located outside the state of Texas. Additionally, government agencies interacting with the public fall squarely within the scope of the Act. Substantive Provisions
  • Prohibited Uses for all AI. TRAIGA prohibits certain uses of AI systems for both the public and private sector, including intentionally inciting self-harm, violence or crime; infringing on an individual’s rights; or unlawfully discriminating. The Act also prohibits deploying AI systems that intentionally generate illegal content, as well as child sexual abuse material or sexually explicit chat systems that impersonate children.

Notably, accidental or disparate impact alone is most likely not enough to violate TRAIGA, as there most likely must be a purposeful intent to discriminate using the AI system.

  • Public Sector: While the scope of TRAIGA includes both public and private entities, there are additional requirements for Texas governmental agencies.
    • Prohibited Uses: Prohibited uses include, among other things, social scoring and uniquely identifying an individual using biometric data, with limited exceptions.
    • Transparency Requirements: Governmental agencies must, among other things, provide conspicuous notice to consumers that are interacting with the AI system – even if this would be obvious to the user; and are prohibited from gathering biometric data without the individual’s informed consent if doing so would infringe that individual’s rights.
Enforcement With no private right of action, TRAIGA can only be enforced by the Texas AG. The Act requires the attorney general to create an “online mechanism” on the AG’s website where consumers can submit complaints of potential violations. If the AG investigates a complaint, developers and deployers of the AI systems may be required to provide information including, but not limited to, a description of the:
  • purpose, intended use, deployment context, and associated benefits of the AI system;
  • categories of data used as inputs and of the outputs produced by the system;
  • types of data used to train the system;
  • evaluation criteria of the performance of the system;
  • known limitations of the system; and
  • post-deployment and user safeguards (e.g., oversight, use and learning processes established to address issues).
If the AG determines a violation has occurred, there is a 60-day cure period. If the violation continues after this period, the AG may bring a claim for, among other things:
  • an injunction;
  • a civil penalty for curable breaches between $10,000 and $12,000;
  • a civil penalty for uncurable breaches between $80,000 and $200,000; and
  • a civil penalty for each day of continued violation between $2,000 and $40,000.
Safe Harbor Notably, TRAIGA states that a defendant may not be found liable for violations if the defendant substantially complies with the most recent version of the Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (AI RMF Framework) published by the National Institute of Standards and Technology (NIST) or another recognized risk management framework for AI systems. Sandbox Program Under TRAIGA, the Texas Department of Information Resources is required to establish a sandbox program. This program would enable developers and deployers of AI systems to obtain legal protection and limited market access. The AG may not file or pursue charges against a program participant for violations of TRAIGA that occur during the testing period. Effective Date The Act goes into effect on January 1, 2026.
0
A minimalistic picture of a human brain being digitized into technological lines that also look like a human brain.

What is an AI risk assessment? And how is one conducted?

AI is ubiquitous, and organizations are adopting AI solutions at a rapid pace. Findings from the first nationally representative survey in the US on generative AI use suggest that “U.S. adoption of generative AI has been faster than adoption of the personal computer and the internet.” With this proliferation comes legal risk, and AI risk assessments are essential tools for organizations to understand the risks and the legal requirements that come with AI adoption. Like privacy risk assessments, AI risk assessments aim to identify, evaluate and mitigate potential risks associated with systems or processes. Because AI can introduce unique challenges—including algorithmic bias, transparency issues, and accountability concerns – the assessment should be tailored to the unique elements of the AI system being implemented. Below is a general overview on how to conduct an AI risk assessment. While the scope and specific frameworks of each risk assessment will vary, it is essential to maintain a structured, systematic approach to ensure the system is being evaluated thoroughly.

Determine Which Laws Apply

To begin any risk assessment, the first step is to determine which laws, regulations, and standards apply. For AI systems, these laws may include, but are not limited to, AI-specific laws, sector-specific laws, and state privacy laws. How to identify applicable laws Begin by identifying the jurisdictions where the AI system will be deployed, accessed, or will otherwise impact individuals. Then, assess which sectors the AI system will be operating in (e.g., finance, employment, healthcare) and whether any AI-specific or general laws apply to the system or its use. Applicable AI-specific laws may include, but are not limited to:
  • California Training Data Transparency Act. In effect on January 1, 2026, this law requires documentation about any generative AI system available to consumers in California. This documentation must be posted on the developer’s website and includes, among other things, a summary of the datasets used in the development of the system, the source of the datasets, how these datasets further the AI system’s intended purpose, and a description of the types of data points within the data sets.
  • California AI Transparency Act. In effect on January 1, 2026, this law covers providers with generative AI systems that are accessible in California and have over one million monthly users. Under this law, covered entities are required to make an AI-detection tool at no cost to users of the AI system. The law also requires the covered entity must provide an optional and mandatory embedded disclosure for all outputs, among other things.
  • Colorado Artificial Intelligence Act. Enacted in 2024, this Act includes parameters around “high-risk” AI systems—those which make, or are a substantial factor in making, consequential decisions. This Act is designed to protect against algorithmic discrimination and imposes obligations relating to transparency and disclosures, risk analysis and mitigation, and impact assessments for both developers and deployers.
  • Utah Artificial Intelligence Policy Act. Enacted in early 2024, this Act requires providers of generative AI systems to ensure that the system discloses whether the user is talking with a generative AI system. In some instances, this disclosure must be made at the beginning of the interaction with the user.
  • Illinois Human Rights Act. In effect on January 1, 2026, amendments to the Illinois Human Rights Act will address the use of AI systems, specifically in employment contexts. The Act currently prohibits discrimination for protected classes in Illinois, and the amendments to the Act will expand its scope to include employment discrimination resulting from the use of AI. For more about this Act, visit our previous article here.
  • EU AI Act. The EU AI Act entered into force on August 1, 2024, but its provisions are phased into effect over time. Under this Act, AI systems are categorized into one of three risk levels: unacceptable, high and low. While AI systems with unacceptable risk are prohibited under this Act, those models classified as high or low risk are subject to additional transparency, risk, and safety obligations.
Additional privacy laws & standards Data protection and privacy laws and regulations, like the California Consumer Privacy Act (CCPA) or General Data Protection Regulation (GDPR), should be taken into consideration, because AI systems frequently process personal or sensitive data. For an overview of the current US state comprehensive privacy laws, visit our previous article here. In addition to identifying applicable laws, it is also helpful to understand emerging standards and ethical guidelines for responsible AI, such as those from ISO, IEEE, or NIST. Although not legally binding, these frameworks can provide best practices to align the AI system or processes with industry standards.

Choose Your Framework

After understanding the legal requirements that apply to your AI system, your organization should select a risk assessment framework that aligns with the type of AI system being implemented and your organization’s goals. Because AI is still relatively new, frameworks are still in development. However, there are a handful of frameworks currently available, which include, but are not limited to:
  1. NIST AI Risk Management Framework. This framework – and its accompanying playbook – was developed by the National Institute of Standards and Technology (NIST) and is designed to “increase the trustworthiness of AI systems, and to help foster the responsible design, development, deployment, and use of AI systems over time.” Because the NIST framework addresses risks to organizations, people, and society in general, it offers a flexible approach that can be used across various industries.
  2. ISO/IEC 42001:2023. This framework focuses on AI management system standards across all types of AI applications and contexts, and offers organizations guidance on creating, deploying, and monitoring AI systems. This standard is particularly useful for organizations seeking international recognition for their AI governance practices, and covers areas including responsible AI, reputation management and user trust, managing AI-specific risks, and innovating within the ISO/IEC framework.
  3. CNIL Self-Assessment Guide for Artificial Intelligence (AI) Systems. This framework offers organizations an analysis grid to assess the maturity of their AI systems in light of the GDPR. Published by the CNIL, the French data protection authority, this framework outlines general aspects of data protection law as well as specific elements that should be more thoroughly reviewed in the context of AI. Because this assessment focuses on the GDPR, it is best for organizations seeking compliance with European data protection and AI laws.
Regardless of the framework, any organization implementing an AI system or process should conduct an assessment using a structured approach. Not only will this approach help provide a more comprehensive assessment, but it will enable greater consistency with each iteration of the assessment, allowing the organization to more effectively compare risks and manage accountability.

Identify AI Stakeholders

Identifying relevant stakeholders in the organization’s AI system or process ensures that all relevant perspectives and concerns are considered. In turn, this helps provide a more thorough, well-rounded assessment. Who are stakeholders? A stakeholder is anyone who is affected by, has an interest in, or has control over an AI system. Key groups often include developers, engineers, product owners or managers, compliance teams, organizational leadership teams, and users. How to identify stakeholders To identify relevant stakeholders for an AI system or process, start by analyzing the AI system’s lifecycle. Consider who is involved in each phase, from design and development to deployment. For example, developers and engineers play vital roles in understanding technical implications throughout the lifecycle, while leadership teams can help guide the intended purpose and evolution of the system. Users should also be considered, as they can provide use-case examples after deployment and feedback on their interactions with the system. Additionally, it is essential to include a diverse range of stakeholders. Balancing differing priorities, such as ensuring fairness, reducing bias, and operational efficiency, will help address potential risks more comprehensively. A range of perspectives can help uncover blind spots, build trust, and ensure that the AI system aligns with legal standards and user expectations.

Map Your System

Mapping your AI system will help provide a clear understanding of how the AI system operates, interacts with, and impacts its environment. By accounting for system components, data flows, and dependencies, an organization can better pinpoint potential risks of bias, inaccuracies, or other issues at each stage of the AI system’s lifecycle. Outline the system  Start by outlining the AI system’s purpose and scope. Define each input, output, and process, and include algorithms, data sources, and models that the AI system relies on. Integrations with other platforms should also be considered and documented. During this process, the organization should refer back to the roles of all stakeholders to ensure each is accounted for. Define the data journey After the system’s structure is defined, trace the data journey from collection, to decision-making, to output. During this process, it is important to highlight any personal data and sensitive data. Processing of this information can lead to issues where errors, biases, or other vulnerabilities may emerge, and may implicate specific AI or other data privacy laws. Identify monitoring methods Finally, map feedback loops and other mechanisms for monitoring the system after deployment. AI systems evolve through updates and learning processes, and it is essential to understand how these changes can expose additional risks. By creating a detailed data map, the organization can establish a comprehensive foundation to carry out the remainder of the risk assessment in a thorough manner.

Set Quality and Accuracy Metrics

For any assessment, metrics must be compared to ensure the system operates as intended, delivers meaningful results, and meets stakeholder expectations. To determine these metrics, the organization should first define the goals of the AI system. Key questions to ask may include:
  • What specific problem is the AI system designed to solve?
  • What value does the AI system contribute?
  • What decisions or actions will the AI influence or automate?
  • What are the users’ needs and expectations from the system?
  • Are there specific fairness, inclusivity, or accessibility goals?
  • How should the system evolve with time or use?
The organization’s metrics should be tailored to address the answers to these and related questions. Next, consider the datasets used to train and evaluate the system. Ensuring that data is complete, consistent, and representative will help ensure the AI system reflects real-world usage. Therefore, datasets should also have metrics to ensure reliable data is being used to assess the system. The reliability of the system should also be defined. Consider metrics like error rates, false positives, and false negatives to gain insight on how AI handles instances like edge cases or unexpected inputs. Finally, user metrics can also be insightful into how well the AI system is performing. These could include satisfaction scores, task success rates, or other metrics to determine how well the AI meets user expectations. After each metric is defined, establish a threshold or benchmark for each. Continuous monitoring and regular evaluation against these standards will help ensure the AI system maintains reliability over time. For dynamic AI systems – which continuously evolve with new data or updates – assessing quality and accuracy is an ongoing process.

Assess Privacy and Cybersecurity

Privacy and cybersecurity are both deeply interconnected components of AI risk assessments. Taking steps to assess these elements helps ensure user safety – particularly when the system collects or otherwise processes personal or sensitive information. Increased Risk of Vulnerability in AI Systems AI systems can handle large amounts of data, making them targets for malicious actors and raising significant threats for privacy concerns. In an evaluation of the cyber security risks to AI by the UK’s Department for Science, Innovation and Technology, vulnerabilities from malicious actors were identified at each stage of an AI system’s lifecycle. Without robust security measures, these vulnerabilities can be more easily exploited.  However, by mitigating these vulnerabilities, organizations can enhance their security measures to better protect against a range of cyber threats. Data Protection Impact Assessments (DPIAs) Most U.S. states with comprehensive data privacy laws require organizations to conduct a data protection impact assessment or data privacy impact assessment (DPIA) for high-risk data processing activities. DPIAs are systematic evaluations that require organizations to adopt privacy-forward practices and require close interaction between privacy and cybersecurity functions. DPIAs help organizations evaluate how personal data is collected, stored, processed and shared. In the context of AI, DPIAs are essential for identifying privacy risks in the training, deployment, and maintenance phases of the AI system. In many instances, DPIAs are required in Europe and the U.S. in the case of:
  • Deployment of high-risk AI systems, as defined under the EU AI Act;
  • Evaluation of personal aspects relating to individuals based on automated processing. This includes profiling, or decisions made on an evaluation that produces legal effects, or similar impacts on a natural person;
  • Systematic monitoring of a publicly accessible area on a large scale;
  • Processing personal data that constitutes sensitive personal data;
  • Processing personal data where it could present a heightened risk of consumer harm, such as unfair or deceptive treatment; financial, physical or reputational injury to consumers; or physical or other intrusion on solitude or private affairs;
  • Processing personal data for purposes of targeted advertising; or
  • Sales of personal data.
Like frameworks for the overarching AI assessment, there are also frameworks to help conduct a DPIA, including the:
  1. NIST Risk Management Framework (RMF). This framework is designed to provide a structured yet flexible approach for managing security and privacy risks, including conducting a DPIA. Through this framework, an organization can link risk management processes at the system level and organizational level. The NIST Cybersecurity Framework can be aligned with the NIST RMF and can be implemented through NIST risk management processes.
  2. ISO/IEC 29135:2023. This document provides guidelines for the process of a privacy impact assessment, and the structure and content of a DPIA report. It is applicable to all types of organizations, regardless of size, including public and private companies, government entities, and not-for-profit organizations.
  3. ICO Sample DPIA Template. This template from the UK’s Information Commissioner’s Office provides an example of how an organization can record the DPIA process and outcome. This template should be read alongside the guidance for an acceptable DPIA set out in the European Guidelines for DPIAs.
The frameworks to conduct a DPIA are similar those used to conduct an overarching AI risk assessment. While both identify and mitigate potential risks, a DPIA will focus on personal data privacy concerns arising from or within the AI system. While NIST points out that “there is no foolproof way” to protect AI from attacks, using a DPIA to understand privacy and cybersecurity risks can help reduce damage to or by an AI system.

Review Bias

After the groundwork of the assessment has been completed, it is essential to understand the results of the assessment – specifically when it comes to bias and discrimination. Bias in an AI system occurs when a model produces unfair or skewed outcomes due to issues in the data, algorithms, or deployment of the system. These skewed outcomes pose significant ethical, legal, and regulatory risks, making a comprehensive review of bias an essential part of an AI risk assessment. Bias from Training Data & Algorithms To review bias, the organization should start by examining the data used to train the AI system. The training data helps AI systems learn to make decisions and should be carefully reviewed. This data should be representative of the context in which the AI system will operate, and issues with this dataset – such as under or overrepresentation of certain groups – can lead to discriminatory outcomes. In addition to issues with training data, the algorithms used can also introduce or amplify bias. According to a report on managing bias in AI, NIST points out that these situations “often arise when algorithms are trained on one type of data and cannot extrapolate beyond those data.” This could be due to an issue with the data itself or because of the mathematical representations of the data in the algorithms. Bias from Deployment Context After reviewing the technical elements of the AI system, bias review should also include deployment contexts. This is because even seemingly neutral or well-trained models can produce biased results if deployed in contexts the AI system was not trained for. Differences in user behavior may create unintended outcomes. To mitigate these risks, organizations should ensure datasets are diverse, representative, and regularly audited for imbalances or stereotypes. Additionally, organizations should conduct context-specific testing before deployment and implement feedback mechanisms to monitor and address bias over time.

Manage Risks

Effective risk management is the final step of conducting an AI risk assessment. Per NIST, “[a]ddressing, documenting, and managing AI risks and potential negative impacts effectively can lead to more trustworthy AI systems.” This process should be done through a proactive, iterative, and comprehensive approach to identify and assess risks – especially for systems that evolve over time. Using the steps above, organizations can conduct regular performance reviews and implement feedback loops to better pinpoint potential risks as well as their severity and likelihood of harm. After identifying risks, organizations should clearly document and communicate risk management processes to stakeholders, ensuring that system limitations and safeguards are understood. Additionally, businesses should take a collaborative approach with stakeholders to mitigate risks and help align practices with best-in-class recommendations. Key practices for managing risk include adopting policies for system oversight and adopting regular assessments to ensure ongoing compliance with laws and regulations. AI systems will never be risk-free. However, businesses can effectively use AI risk assessments to safeguard against potential harms. Through a systematic evaluation of the AI system, organizations can create more trustworthy and reliable AI systems, while ensuring compliance and protecting user privacy.
1 2 3 4 5 6 … 9