Privacy Policies and Practices: Hims & Hers Enforcement

AI & Algorithms , , , , , , , , ,

Most people are familiar with the idea of a privacy policy – the long document we often agree to when signing up for a service or purchasing goods. But what does a privacy policy actually do? In short, it outlines how a business collects, uses and shares personal information. The contents of a privacy policy may depend on the size and function of the business, the sector in which the business operates, and the jurisdictions in which the business is located or does business. 

Drafting and publishing the policy is only part of the job though. Businesses also need to also confirm that their actual practices, including the technology operating behind their website, match the promises that they make to their consumers in these policies.

The Federal Trade Commission (FTC) has long warned businesses that if they make privacy representations, they must honor them. The FTC specifically advises companies to review their privacy policies and ensure that their actual practices are consistent with those representations. As the recent FTC  lawsuit against Hims & Hers demonstrates, it is imperative that consumers are adequately informed about business practices in a clear and conspicuous manner. 

What are the risks of misalignment between business practices and consumer representations? 

In the United States, the FTC has the power to enforce the terms of privacy practices via the authority in Section 5 of the FTC Act, which prohibits unfair or deceptive advertising practices. The Commission’s landmark 1999 consent order with the web host GeoCities was the FTC’s first public settlement in the area of internet privacy, and as of 2023, the FTC has brought at least 97 internet privacy cases. 

Most recently, in July 2026, the FTC was joined by California and Utah authorities in a lawsuit against the telehealth company Hims & Hers for deceptive and unlawful privacy practices. This lawsuit alleged that the company shared customers’ sensitive health data, including medical conditions, with advertising platforms, despite implying that they keep health information private on their privacy policy. 

According to the FTC, some information was shared to third-party advertising platforms through customer lists, while other information was transmitted through third-party tracking technologies; the consumers’ health information was allegedly shared with Meta, Snap and other third parties. In its complaint, the FTC states that Hims & Hers also fails to disclose its billing practices adequately and makes it difficult for consumers to cancel their subscriptions. Christopher Mufarrige, Director of the FTC’s Bureau of Consumer Protection claims this creates a scenario where “consumers [are] unknowingly locked into recurring subscriptions and the disclosure to third parties of consumers’ most private health information without their consent.” 

This case remains pending and the allegations have not been adjudicated. Still, the lesson here is clear: a business’s sharing and selling practices must be accurately disclosed to the consumer via their privacy policy. 

This does not mean that businesses need to abandon tracking, analytics, or subscription services. Rather, this case – and many of the other enforcement actions highlighted by the FTC – emphasize an organization’s need to understand what these technologies do before describing their privacy practices to consumers and ensure that privacy policies accurately disclose these practices. 

Key Takeaways: 

Privacy policies should accurately disclose the business’s privacy practices to the consumer in a clear and conspicuous way. By reviewing both their privacy representations and the technologies behind them, an organization can take steps to ensure their privacy practices are accurate and up to date. Some of these compliance review measures may include: 

  • Inventorying pixels, cookies, analytical tools, chat-bot features, and other third-party technologies. 
  • Identifying what personal information each tool may collect or transmit, and who receives it. 
  • Paying particular attention to sensitive information and data entered into forms, portals and chat features. 
  • Reviewing vendor configurations and contractual terms governing data use.
  • Comparing actual data flows against the company’s privacy policy and other consumer-facing statements.
  • Requiring reviews before changing internal business practices that impact personal privacy.

Drafting and publishing a privacy policy may be required under certain state laws. However, this policy – like most privacy and compliance efforts – should not be treated as a one-time task. Websites, vendors and business practices change, and privacy policies should stay in alignment with these changing practices.  

AI Watermarking: Key Updates on New Transparency Rules

AI & Algorithms , , , , , , ,

Updated on October 9, 2026, in light of the passage of SB 1000: California AI Transparency Act.

As generative artificial intelligence (AI) continues to advance, it’s becoming increasingly difficult to distinguish human-created content from AI-generated material. In turn, regulators are requiring certain AI systems to disclose where and how content is created. 

Two important examples are the European Union’s AI Act and California’s AI Transparency Act. While both of these laws address AI-generated content, neither requires businesses to use the same type of disclosure in every situation. Instead, the rules distinguish between different forms of transparency, including machine-readable markings that may not be visible to the human eye. These requirements may apply to both the provider of the AI system and the deployer, or the entity that makes the AI available to consumers.

The laws are falling into place, but how do they work in practice? For businesses developing or using generative AI, the challenge is not only whether AI-generated content should be watermarked, but how different types of media should carry that watermark. 

The EU AI Act 

Transparency requirements under Article 50 of the EU AI Act became generally applicable on August 2, 2026, with a grace period until December 2, 2026, for certain products that were already on the market as of August 2, 2026. 

Among other requirements, providers of AI systems that generate synthetic text, images, audio or video must generally ensure that the output can be detected as artificially generated or manipulated. This may include machine-readable markings built into the content rather than displayed as a visible layer. 

The AI Act also creates separate disclosure requirements for certain users of AI-generated content. For example, deepfakes generally must be clearly disclosed as AI-generated or manipulated; AI-generated or manipulated text about matters of public interest may also require disclosure when it is published without human review. 

While the law provides the end goal, it does not necessarily define how to achieve it. For example, the EU AI Act was accompanied by the Guidelines on Transparency Obligations for Providers and Deployers of Certain AI Systems. These guidelines provide definitions and explain how compliance with the AI Act’s transparency obligations may be demonstrated. But still, these guidelines defer to “providers and deployers [who] can determine adequate measures themselves, while taking into account these guidelines.” In short, this means the EU AI Act sets the standard but leaves the means of meeting it to the AI provider or deployer. 

This means AI transparency can look different, both because of the media involved and because the provider or deployer can determine how best to meet these transparency standards. For example, a machine-readable watermark may help technology detect that AI was involved, while a visible or audible disclosure is meant to inform the person viewing or interacting with the content that it was AI- generated or altered. But the form and format of these transparency measures may largely be subject to the organisation’s discretion. 

The California AI Transparency Act 

California has also adopted AI-content transparency requirements through the California AI Transparency Act, which was originally enacted through SB 942 and amended through AB 853 (which went into effect on August 2, 2026) and SB 1000 (which went into effect on September 30, 2026). Among other things, the California AI Transparency Act has a requirement for 1) a disclosure verification tool, and 2) disclosures within the AI-generated media.

What is the disclosure verification tool?

The law generally requires “covered providers” – defined as any person who produces a generative AI system publicly accessibly within the geographic boundaries of California – to make a disclosure verification tool available.

This tool should be publicly accessible and allow a user to assess if an image, video or audio content (or any combination) was created or altered by a covered provider’s generative AI system. There is an exception for minor modifications of media.

The tool should also provide any “provenance data” detected in the media. This includes information “about the origin of a piece of digital content and the history of modifications of the content.”

Notably, a covered provider may satisfy the requirements regarding the tool by directing a user to a third-party tool, provided the covered provider gives certain disclosures, and the third-party tool meets certain standards.  

What disclosures does the verification tool detect? To effectuate the purpose of the disclosure verification tool, the Act requires disclosures to be included in any image, video or audio content created by the covered provider’s generative AI system, with exceptions for minor modifications.

The California AI Transparency Act distinguishes between two different types of disclosures: latent and manifest. A latent disclosure is built into the content, but it’s not readily visible to the average person. A manifest disclosure is one that a layperson can more easily see and understand. 

Per SB 1000, the latent disclosure must include certain information, including but not limited to the name or the covered provider, the generative AI system that created or altered the content, and the time and date of the creation or alteration of the content.

There are additional compliance requirements for coved providers who license generative AI systems to licensees of those systems.

Through the implementation and subsequent amendments of the California AI Transparency Act, it is evident that California’s requirements are also rapidly evolving. Businesses subject to the law may need to monitor legislative changes as lawmakers revisit how disclosure and detection measures are required under the law.

What does AI watermarking look like? 

Anthropic’s recent changes to Claude provide one example of how companies may approach these requirements. 

In August 2026, Anthropic announced that the new Claude models would include machine-readable markings in response to the EU AI Act and other similar legislative requirements. For generated text, Claude uses an imperceptible watermark based on patterns in how the model selects words. According to Anthropic’s press release, the watermark is not a visible or hidden set of characters and does not identify the person or company who generated the content. 

However, these tools are far from perfect. Anthropic explains that a text watermark can disappear or weaken; they describe it as not “foolproof” if content is heavily rewritten, translated or combined with other materials. File-based provenance information can also get lost if files are converted, resaved or captured by screenshot. 

This means that businesses should not treat watermarking as a perfect way to determine whether something was created by AI. Instead, it may be one tool to provide more information about where content came from and whether AI was involved. 

Can EU rules affect U.S. businesses? 

United States companies may also feel the effects of the EU AI Act even when a particular use of an AI product occurs outside Europe. 

For a company operating in multiple countries, creating different versions of the same product for each jurisdiction can become expensive and technically complicated. A business may instead choose and decide to build one version that satisfies the strictest applicable requirements and use that version more broadly.

For example, Anthropic has said that although its Claude watermarking changes were driven by the EU AI Act, it is applying them globally rather than limiting them to European users. 

This can create a spillover, where a law passed in one jurisdiction changes how an AI product operates for users everywhere else.  Similar regulatory concepts are also appearing in U.S. state AI laws. California and other states have adopted requirements involving transparency, disclosures, and AI governance that overlap with themes found in the EU AI Act, even though the laws differ in their scope and specific requirements.

As AI regulations continue to develop across different jurisdictions, companies may consider whether maintaining different disclosure mechanisms per jurisdiction makes sense. Alternatively, companies may comply with the most stringent regulations across all markets, setting a higher standard of compliance across the board. 

What should businesses take away?

Businesses do not necessarily need to approach every type of AI generated content in the same way. However, companies developing or using generative AI may want to review how their systems identify AI generated content, and whether their disclosure practices meet the requirements that might apply to them. Some steps businesses may want to consider include:

  • Identifying what types of AI-generated content the business uses or provides to consumers, including text, images, audio and video. 
  • Determining whether applicable laws require machine-readable markings or visible disclosures.
  • Understanding how watermarks or provenance information may change if content is edited or redistributed. 
  • Checking if AI-generated content from outside vendors keeps its watermark or disclosure when the business edits, downloads, or republishes it into another product. 
  • Considering whether maintaining different product versions across jurisdictions is practical.
  • Monitoring United States, EU, and other developing AI transparency requirements and laws. 

AI watermarking is still rapidly developing both technically and legally. Businesses do not necessarily need to treat every AI-generated output the same way, but they may need to increasingly understand when AI-generated content should be identified and whether their current systems can provide that transparency. For businesses using AI-generated content in marketing, customer communications, and other business activities, understanding when and how that content must be identified can help reduce compliance risks as these rules continue to develop 

0
Chicago Grand Central Looking Up

DOJ Issues Final Rule on US Bulk Sensitive Data

Data Brokers, Data Transfers, United States , , , ,
The International Emergency Economic Powers Act (IEEPA) vests the President with authority to deal with extraordinary threats to national security and foreign policy that have their source in part or in whole outside of the United States. Acting pursuant to the IEEPA, President Biden issued Executive Order 14117, “Preventing Access to Americans’ Bulk Sensitive Personal Data and United States Government-Related Data By Countries of Concern” (the EO). The EO directed the Department of Justice (DOJ or Department) to establish and implement regulations addressing threats from certain countries of concern attempting to access and exploit bulk amounts of US sensitive data, including personal and government data. On December 27, 2024, the DOJ issued the Final Rule, which went into effect on April 8, 2025. Additional compliance provisions for certain transactions take effect on October 6, 2025. The Final Rule prohibits or restricts a range of transactions involving categories of bulk sensitive personal data or government-related data between the US and countries of concern or covered persons. In assisting businesses to adapt to this comprehensive update, the DOJ provided a Fact Sheet, a Compliance Guide, and over 100 FAQs on the Final Rule, along with an Implementation and Enforcement Policy. Below are five main takeaways that US entities may want to consider in light of these regulations.
  1. Enforcement May Be More Lenient Until July 8, 2025 
The DOJ’s Implementation and Enforcement Policy, states that the Department will “target its enforcement efforts during the first 90 days to allow US persons (e.g., individuals and companies) additional time to continue implementing the necessary changes to comply with the [Final Rule].” The Department’s civil enforcement actions for violations of the Final Rule will not be a priority “so long as the person is engaging in good faith efforts to comply with or come into compliance with the [Final Rule] during that time.” However, the Department makes clear that it will “pursue penalties and other enforcement actions as appropriate for egregious, willful violations” during the delayed enforcement period.
  1. DOJ Will Consider Good Faith Efforts to Comply
While the Implementation and Enforcement Policy reflects that civil actions for violations of the Final Rule will not be a priority, this depends on the entity’s good faith effort to comply. According to this Policy, examples of evidence of good faith efforts may include, but are not limited to:
  • Conducting internal reviews of access to sensitive data.
  • Conducting internal reviews to determine whether transactions involving access to such data flows constitute data brokerage.
  • Reviewing internal datasets and datatypes to determine if they are subject to the Final Rule.
  • Conducting due diligence on potential new vendors.
  • Renegotiating vendor agreements or negotiating contracts with or transferring products or services to new vendors.
  • Adjusting employee work locations, roles or responsibilities.
  • Evaluating investments from countries of concern or covered persons.
  • Implementing the CISA Security Requirements.
  1. “Good Faith” May Include Satisfying CISA Security Requirements 
A good-faith effort to comply may be demonstrated, in part, by implementing the CISA Security Requirements, which were developed concurrently with the Final Rule pursuant to the EO. The security requirements are intended to address threats that arise when conducting restricted transactions, as detailed below. These security requirements are divided into two sections: i) organizational- and covered system-level requirements; and ii) data-level requirements.
  1. Before October 6, 2025, Determine if Your Company is Conducting Restricted Transactions
US entities engaged in restricted transactions under the Final Rule have affirmative data compliance program and audit obligations, among other obligations. In addition, the Final Rule provides that data brokerage transactions are prohibited with any foreign entity unless the US person contractually binds the foreign entity from subsequent transactions of that data with a country of concern or covered person. They must also report any known or suspected violation of this requirement.
  1. An Iterative Review Plan May be Needed for Covered Transactions 
With the Final Rule coming into effect and enforcement nearing, US companies that engage in certain data transactions or share information with third parties that may be covered persons or countries of concern should evaluate their transactions and data practices. After a thorough review of the types of information collected, who that information is shared with, and who is involved in the processing of that data, it may be helpful to adopt a compliance policy to ensure transactions are being handled appropriately in light of the Final Rule.
0
A minimalistic picture of a human brain being digitized into technological lines that also look like a human brain.

What is an AI risk assessment? And how is one conducted?

AI & Algorithms, EU, United States , , , , ,
AI is ubiquitous, and organizations are adopting AI solutions at a rapid pace. Findings from the first nationally representative survey in the US on generative AI use suggest that “U.S. adoption of generative AI has been faster than adoption of the personal computer and the internet.” With this proliferation comes legal risk, and AI risk assessments are essential tools for organizations to understand the risks and the legal requirements that come with AI adoption. Like privacy risk assessments, AI risk assessments aim to identify, evaluate and mitigate potential risks associated with systems or processes. Because AI can introduce unique challenges—including algorithmic bias, transparency issues, and accountability concerns – the assessment should be tailored to the unique elements of the AI system being implemented. Below is a general overview on how to conduct an AI risk assessment. While the scope and specific frameworks of each risk assessment will vary, it is essential to maintain a structured, systematic approach to ensure the system is being evaluated thoroughly.

Determine Which Laws Apply

To begin any risk assessment, the first step is to determine which laws, regulations, and standards apply. For AI systems, these laws may include, but are not limited to, AI-specific laws, sector-specific laws, and state privacy laws. How to identify applicable laws Begin by identifying the jurisdictions where the AI system will be deployed, accessed, or will otherwise impact individuals. Then, assess which sectors the AI system will be operating in (e.g., finance, employment, healthcare) and whether any AI-specific or general laws apply to the system or its use. Applicable AI-specific laws may include, but are not limited to:
  • California Training Data Transparency Act. In effect on January 1, 2026, this law requires documentation about any generative AI system available to consumers in California. This documentation must be posted on the developer’s website and includes, among other things, a summary of the datasets used in the development of the system, the source of the datasets, how these datasets further the AI system’s intended purpose, and a description of the types of data points within the data sets.
  • California AI Transparency Act. In effect on January 1, 2026, this law covers providers with generative AI systems that are accessible in California and have over one million monthly users. Under this law, covered entities are required to make an AI-detection tool at no cost to users of the AI system. The law also requires the covered entity must provide an optional and mandatory embedded disclosure for all outputs, among other things.
  • Colorado Artificial Intelligence Act. Enacted in 2024, this Act includes parameters around “high-risk” AI systems—those which make, or are a substantial factor in making, consequential decisions. This Act is designed to protect against algorithmic discrimination and imposes obligations relating to transparency and disclosures, risk analysis and mitigation, and impact assessments for both developers and deployers.
  • Utah Artificial Intelligence Policy Act. Enacted in early 2024, this Act requires providers of generative AI systems to ensure that the system discloses whether the user is talking with a generative AI system. In some instances, this disclosure must be made at the beginning of the interaction with the user.
  • Illinois Human Rights Act. In effect on January 1, 2026, amendments to the Illinois Human Rights Act will address the use of AI systems, specifically in employment contexts. The Act currently prohibits discrimination for protected classes in Illinois, and the amendments to the Act will expand its scope to include employment discrimination resulting from the use of AI. For more about this Act, visit our previous article here.
  • EU AI Act. The EU AI Act entered into force on August 1, 2024, but its provisions are phased into effect over time. Under this Act, AI systems are categorized into one of three risk levels: unacceptable, high and low. While AI systems with unacceptable risk are prohibited under this Act, those models classified as high or low risk are subject to additional transparency, risk, and safety obligations.
Additional privacy laws & standards Data protection and privacy laws and regulations, like the California Consumer Privacy Act (CCPA) or General Data Protection Regulation (GDPR), should be taken into consideration, because AI systems frequently process personal or sensitive data. For an overview of the current US state comprehensive privacy laws, visit our previous article here. In addition to identifying applicable laws, it is also helpful to understand emerging standards and ethical guidelines for responsible AI, such as those from ISO, IEEE, or NIST. Although not legally binding, these frameworks can provide best practices to align the AI system or processes with industry standards.

Choose Your Framework

After understanding the legal requirements that apply to your AI system, your organization should select a risk assessment framework that aligns with the type of AI system being implemented and your organization’s goals. Because AI is still relatively new, frameworks are still in development. However, there are a handful of frameworks currently available, which include, but are not limited to:
  1. NIST AI Risk Management Framework. This framework – and its accompanying playbook – was developed by the National Institute of Standards and Technology (NIST) and is designed to “increase the trustworthiness of AI systems, and to help foster the responsible design, development, deployment, and use of AI systems over time.” Because the NIST framework addresses risks to organizations, people, and society in general, it offers a flexible approach that can be used across various industries.
  2. ISO/IEC 42001:2023. This framework focuses on AI management system standards across all types of AI applications and contexts, and offers organizations guidance on creating, deploying, and monitoring AI systems. This standard is particularly useful for organizations seeking international recognition for their AI governance practices, and covers areas including responsible AI, reputation management and user trust, managing AI-specific risks, and innovating within the ISO/IEC framework.
  3. CNIL Self-Assessment Guide for Artificial Intelligence (AI) Systems. This framework offers organizations an analysis grid to assess the maturity of their AI systems in light of the GDPR. Published by the CNIL, the French data protection authority, this framework outlines general aspects of data protection law as well as specific elements that should be more thoroughly reviewed in the context of AI. Because this assessment focuses on the GDPR, it is best for organizations seeking compliance with European data protection and AI laws.
Regardless of the framework, any organization implementing an AI system or process should conduct an assessment using a structured approach. Not only will this approach help provide a more comprehensive assessment, but it will enable greater consistency with each iteration of the assessment, allowing the organization to more effectively compare risks and manage accountability.

Identify AI Stakeholders

Identifying relevant stakeholders in the organization’s AI system or process ensures that all relevant perspectives and concerns are considered. In turn, this helps provide a more thorough, well-rounded assessment. Who are stakeholders? A stakeholder is anyone who is affected by, has an interest in, or has control over an AI system. Key groups often include developers, engineers, product owners or managers, compliance teams, organizational leadership teams, and users. How to identify stakeholders To identify relevant stakeholders for an AI system or process, start by analyzing the AI system’s lifecycle. Consider who is involved in each phase, from design and development to deployment. For example, developers and engineers play vital roles in understanding technical implications throughout the lifecycle, while leadership teams can help guide the intended purpose and evolution of the system. Users should also be considered, as they can provide use-case examples after deployment and feedback on their interactions with the system. Additionally, it is essential to include a diverse range of stakeholders. Balancing differing priorities, such as ensuring fairness, reducing bias, and operational efficiency, will help address potential risks more comprehensively. A range of perspectives can help uncover blind spots, build trust, and ensure that the AI system aligns with legal standards and user expectations.

Map Your System

Mapping your AI system will help provide a clear understanding of how the AI system operates, interacts with, and impacts its environment. By accounting for system components, data flows, and dependencies, an organization can better pinpoint potential risks of bias, inaccuracies, or other issues at each stage of the AI system’s lifecycle. Outline the system  Start by outlining the AI system’s purpose and scope. Define each input, output, and process, and include algorithms, data sources, and models that the AI system relies on. Integrations with other platforms should also be considered and documented. During this process, the organization should refer back to the roles of all stakeholders to ensure each is accounted for. Define the data journey After the system’s structure is defined, trace the data journey from collection, to decision-making, to output. During this process, it is important to highlight any personal data and sensitive data. Processing of this information can lead to issues where errors, biases, or other vulnerabilities may emerge, and may implicate specific AI or other data privacy laws. Identify monitoring methods Finally, map feedback loops and other mechanisms for monitoring the system after deployment. AI systems evolve through updates and learning processes, and it is essential to understand how these changes can expose additional risks. By creating a detailed data map, the organization can establish a comprehensive foundation to carry out the remainder of the risk assessment in a thorough manner.

Set Quality and Accuracy Metrics

For any assessment, metrics must be compared to ensure the system operates as intended, delivers meaningful results, and meets stakeholder expectations. To determine these metrics, the organization should first define the goals of the AI system. Key questions to ask may include:
  • What specific problem is the AI system designed to solve?
  • What value does the AI system contribute?
  • What decisions or actions will the AI influence or automate?
  • What are the users’ needs and expectations from the system?
  • Are there specific fairness, inclusivity, or accessibility goals?
  • How should the system evolve with time or use?
The organization’s metrics should be tailored to address the answers to these and related questions. Next, consider the datasets used to train and evaluate the system. Ensuring that data is complete, consistent, and representative will help ensure the AI system reflects real-world usage. Therefore, datasets should also have metrics to ensure reliable data is being used to assess the system. The reliability of the system should also be defined. Consider metrics like error rates, false positives, and false negatives to gain insight on how AI handles instances like edge cases or unexpected inputs. Finally, user metrics can also be insightful into how well the AI system is performing. These could include satisfaction scores, task success rates, or other metrics to determine how well the AI meets user expectations. After each metric is defined, establish a threshold or benchmark for each. Continuous monitoring and regular evaluation against these standards will help ensure the AI system maintains reliability over time. For dynamic AI systems – which continuously evolve with new data or updates – assessing quality and accuracy is an ongoing process.

Assess Privacy and Cybersecurity

Privacy and cybersecurity are both deeply interconnected components of AI risk assessments. Taking steps to assess these elements helps ensure user safety – particularly when the system collects or otherwise processes personal or sensitive information. Increased Risk of Vulnerability in AI Systems AI systems can handle large amounts of data, making them targets for malicious actors and raising significant threats for privacy concerns. In an evaluation of the cyber security risks to AI by the UK’s Department for Science, Innovation and Technology, vulnerabilities from malicious actors were identified at each stage of an AI system’s lifecycle. Without robust security measures, these vulnerabilities can be more easily exploited.  However, by mitigating these vulnerabilities, organizations can enhance their security measures to better protect against a range of cyber threats. Data Protection Impact Assessments (DPIAs) Most U.S. states with comprehensive data privacy laws require organizations to conduct a data protection impact assessment or data privacy impact assessment (DPIA) for high-risk data processing activities. DPIAs are systematic evaluations that require organizations to adopt privacy-forward practices and require close interaction between privacy and cybersecurity functions. DPIAs help organizations evaluate how personal data is collected, stored, processed and shared. In the context of AI, DPIAs are essential for identifying privacy risks in the training, deployment, and maintenance phases of the AI system. In many instances, DPIAs are required in Europe and the U.S. in the case of:
  • Deployment of high-risk AI systems, as defined under the EU AI Act;
  • Evaluation of personal aspects relating to individuals based on automated processing. This includes profiling, or decisions made on an evaluation that produces legal effects, or similar impacts on a natural person;
  • Systematic monitoring of a publicly accessible area on a large scale;
  • Processing personal data that constitutes sensitive personal data;
  • Processing personal data where it could present a heightened risk of consumer harm, such as unfair or deceptive treatment; financial, physical or reputational injury to consumers; or physical or other intrusion on solitude or private affairs;
  • Processing personal data for purposes of targeted advertising; or
  • Sales of personal data.
Like frameworks for the overarching AI assessment, there are also frameworks to help conduct a DPIA, including the:
  1. NIST Risk Management Framework (RMF). This framework is designed to provide a structured yet flexible approach for managing security and privacy risks, including conducting a DPIA. Through this framework, an organization can link risk management processes at the system level and organizational level. The NIST Cybersecurity Framework can be aligned with the NIST RMF and can be implemented through NIST risk management processes.
  2. ISO/IEC 29135:2023. This document provides guidelines for the process of a privacy impact assessment, and the structure and content of a DPIA report. It is applicable to all types of organizations, regardless of size, including public and private companies, government entities, and not-for-profit organizations.
  3. ICO Sample DPIA Template. This template from the UK’s Information Commissioner’s Office provides an example of how an organization can record the DPIA process and outcome. This template should be read alongside the guidance for an acceptable DPIA set out in the European Guidelines for DPIAs.
The frameworks to conduct a DPIA are similar those used to conduct an overarching AI risk assessment. While both identify and mitigate potential risks, a DPIA will focus on personal data privacy concerns arising from or within the AI system. While NIST points out that “there is no foolproof way” to protect AI from attacks, using a DPIA to understand privacy and cybersecurity risks can help reduce damage to or by an AI system.

Review Bias

After the groundwork of the assessment has been completed, it is essential to understand the results of the assessment – specifically when it comes to bias and discrimination. Bias in an AI system occurs when a model produces unfair or skewed outcomes due to issues in the data, algorithms, or deployment of the system. These skewed outcomes pose significant ethical, legal, and regulatory risks, making a comprehensive review of bias an essential part of an AI risk assessment. Bias from Training Data & Algorithms To review bias, the organization should start by examining the data used to train the AI system. The training data helps AI systems learn to make decisions and should be carefully reviewed. This data should be representative of the context in which the AI system will operate, and issues with this dataset – such as under or overrepresentation of certain groups – can lead to discriminatory outcomes. In addition to issues with training data, the algorithms used can also introduce or amplify bias. According to a report on managing bias in AI, NIST points out that these situations “often arise when algorithms are trained on one type of data and cannot extrapolate beyond those data.” This could be due to an issue with the data itself or because of the mathematical representations of the data in the algorithms. Bias from Deployment Context After reviewing the technical elements of the AI system, bias review should also include deployment contexts. This is because even seemingly neutral or well-trained models can produce biased results if deployed in contexts the AI system was not trained for. Differences in user behavior may create unintended outcomes. To mitigate these risks, organizations should ensure datasets are diverse, representative, and regularly audited for imbalances or stereotypes. Additionally, organizations should conduct context-specific testing before deployment and implement feedback mechanisms to monitor and address bias over time.

Manage Risks

Effective risk management is the final step of conducting an AI risk assessment. Per NIST, “[a]ddressing, documenting, and managing AI risks and potential negative impacts effectively can lead to more trustworthy AI systems.” This process should be done through a proactive, iterative, and comprehensive approach to identify and assess risks – especially for systems that evolve over time. Using the steps above, organizations can conduct regular performance reviews and implement feedback loops to better pinpoint potential risks as well as their severity and likelihood of harm. After identifying risks, organizations should clearly document and communicate risk management processes to stakeholders, ensuring that system limitations and safeguards are understood. Additionally, businesses should take a collaborative approach with stakeholders to mitigate risks and help align practices with best-in-class recommendations. Key practices for managing risk include adopting policies for system oversight and adopting regular assessments to ensure ongoing compliance with laws and regulations. AI systems will never be risk-free. However, businesses can effectively use AI risk assessments to safeguard against potential harms. Through a systematic evaluation of the AI system, organizations can create more trustworthy and reliable AI systems, while ensuring compliance and protecting user privacy.
0

Upcoming Webinar: The EU AI Act: How Will It Affect Your Business?

AI & Algorithms, EU , , , , , , , ,
On Thursday, December 12, 2024 at 11am ET, DataCamp will host a live webinar titled “The EU AI Act: How Will It Affect Your Business?” The speakers are Dan Nechita, Lead Technical Negotiator for the EU AI Act on behalf of the European Parliament, and Lily Li, Data Privacy, Cybersecurity & AI Lawyer and Founder of Metaverse Law. They will cover AI governance and risk management requirements under the EU AI Act and new US AI law. Attendees will:
  • Learn about the scope and requirements of the EU AI Act and other AI legislation.
  • Understand the risk classification system and the requirements for AI literacy.
  • Learn how to comply with regulations and the consequences of non-compliance.
  Join us for this informational webinar by registering at the DataCamp website using this hyperlink.
1 2