Phishing in the Age of AI

Phishing is one of the most common practices used by hackers looking to gain access to company systems. Phishing attacks typically rely on an individual impersonating a trusted person or organization to convince the victim of the attack to click a link, provide login credentials, send money or disclose other – often sensitive – information. 

A recent settlement involving Ambry Genetics (Ambry) is a reminder that phishing attacks can create consequences beyond a single compromised account. In September 2026, the U.S. Department of Health and Human Services’ Office for Civil Rights (OCR) announced a $700,000 settlement with Ambry following a phishing incident that potentially exposed protected health information (PHI) belonging to over 200,000 people. 

The settlement highlights why businesses need to consider phishing awareness and prevention as part of their broader cybersecurity program – especially as AI can amplify the scale of phishing attacks and can make impersonation attempts more difficult to decipher. 

What does phishing look like?

According to NIST, phishing attacks occur when “the perpetrator masquerades as a legitimate business or reputable person” to gain access to personal, confidential, or other sensitive information. An example of phishing might be an employee who receives an email that appears to come from another co-worker, bank, vendor or boss asking them to reset a password, review an invoice or log into an account. The link may then lead to a fake website designed to steal information. 

Phishing may also happen through text or SMS messages and phone calls. The FBI refers to these variations as “smishing” and “vishing,” respectively, and they look like typical phishing attempts, just transacted through a different means of communication.

Attackers do not always need to break through technical security controls. Often, they just need to convince one employee to give them access through fraudulent means. 

What happened at Ambry Genetics?

Ambry Genetics is a California based genetic-testing company.  In 2020, the company discovered that an employee had compromised a company email account via a phishing attack. 

According to OCR, the attacker potentially gained access to PHI belonging to over 200,000 people. The compromised information may have contained data points such as names, addresses, birthdays, diagnoses, lab results, and financial information.

OCR’s investigation of the attack revealed more than a single phishing incident:  It uncovered that Ambry was not compliant with broader security practices required by HIPAA and its relevant rules, including questions of whether the company had conducted an adequate risk analysis, properly removed employee access after they had left the company, and assigned unique user login information to individual users. 

As part of the settlement, Ambry agreed to a two-year corrective action plan. The plan includes elements such as requirements to conduct proper risk analyses, address identified security risks, and review data protection policies and procedures, while also providing additional workplace training. 

It is important to highlight that a phishing attack can be the impetus for regulators taking action. What looks like an isolated incident may lead regulators to examine more than just how the attacker got in – they may also investigate whether the company had reasonable security practices in place before the incident occurred. 

How might AI change phishing?

The FBI has warned that criminals are using AI to create more polished and personalized phishing messages to imitate voices or images of trusted people. AI-generated audio can be used to impersonate people to make a phone call that sounds like it’s coming from a boss, co-worker, or family member. 

Because AI is generally increasing the ability to create phishing messages, traditional warning signs may be less reliable. For example, poor grammar, awkward wording or random phone numbers may be less obvious when AI can quickly generate messages and voices. 

Therefore, businesses who are providing training on how to recognize phishing may consider focusing less on whether a message simply looks or sounds legitimate, and more on whether the request itself should be independently verified. For example, if someone asks for a wire transfer, password reset or access to confidential information, employees should consider verifying that request through other means. This could include calling a known number or using a separate communication method. The FBI recommends independently confirming the identity of someone making a suspicious or unusual request. 

Key Takeaways and Security Practices to Consider 

Phishing attacks aren’t new, but the age of artificial intelligence may be changing the scope and frequency of these kinds of attacks. For example, a study by IBM shows how AI can be used to amplify both the amount of phishing attacks and the human quality of the attack itself.  With this in mind, fundamental phishing prevention is as important as ever. When considering how to lower its susceptibility to phishing emails, business may consider the following:

  • Training employees: Employees are often key targets for phishing attacks and should understand what these attacks can look like and how to report suspicious messages to information security teams within the business. 
  • Enabling MFA: Multi-factor authentication (MFA) is a process where an individual has to provide multiple forms of verification to enter an account. For example, if MFA is enabled, a stolen password alone should not give an attacker immediate access to an account. 
  • Limiting employee access: As a general rule, employees should only be able to access systems and information that is needed for their job. Typically, this access is revoked after a period of time or when it is no longer needed. 
  • Preparing for incidents: Businesses may consider having tools in place to disable compromised accounts, investigate incidents and determine if sensitive information was exposed, among other steps that may be required in the investigation process. 

Agentic AI meets Cybersecurity

AI has moved beyond simply answering questions or giving recommendations. By now, you may have heard a new(er) term floating around the internet: agentic AI. These are AI systems that can be given access to tools, company data, and applications, with the goal of performing tasks on a user’s behalf.

While this added functionality may improve efficiency, it can also create cybersecurity risks. For example, an AI system that can only draft emails inherently has different risks than one that’s able to view inboxes, see customer information and send emails on a person’s behalf. 

As businesses begin to use these agentic systems, they should consider both what information an AI agent can access and what authority an AI agent has to act on that information. 

What are key distinctions of agentic AI? 

An important difference in understanding the risk profiles between standard AI (like an LLM) and agentic AI is not just that the agent can perform more tasks. Instead, it is often an issue of access. For example, a conversation with an LLM may be a one-time input and output of that information, while the use of agentic AI may give the agentic system ongoing access and authority to operate inside company environments. This increase in access and authority has the potential to substantially increase the amount of risk that comes with this type of system, because agentic AI may be able to use its existing permissions to make decisions or perform actions without human oversight.  

Therefore, the level of access and permission an agent has can be pivotal to how risk is controlled and managed in an organization. An AI agent with narrow access and required human oversight/permission will likely present lower risks than one that is able to act independently without such restrictions. For businesses, this makes permissions, oversight and the ability to stop or limit an agent central to cybersecurity concerns.

How can businesses maintain control over AI agents? 

The National Institute of Standards and Technology’s AI Risk Management Framework (“NIST AI RMF”) provides a useful way for businesses to approach these risks. This voluntary industry standard framework organizes AI risk management into four functions: Govern, Map, Measure and Manage. Rather than just a checklist, this framework encourages organizations to establish responsibility for their AI systems, understand how they work, evaluate their risks and continue managing those risks throughout the AI lifecycle.  

This framework is flexible and can be applied to agentic AI systems. As a foundation, the principles outlined in the NIST RMF can help businesses answer practical security questions: Who is responsible for overseeing an agent? What is the agent’s intended role? What systems and sets of information does it need access to? What could happen if mistakes are made? And what measures are in place if its actions/behavior have to be limited or stopped?

Additionally, NIST has begun focusing specifically on identity and authorization risks for these AI agents, including issues like authentication and auditing. 

How can AI agents become a cybersecurity target?

Businesses may also consider attacks directed at the AI agent itself, such as prompt injection. Prompt injection is where instruction contained in a prompt (or input) is used to manipulate an AI agent’s behavior. For an AI agent, this prompt could be in an email, document or other outside content that the agent is asked to review. 

For example, pretend a business has an AI agent which is tasked at reading emails and has access to internal files. A malicious email might contain instructions attempting to persuade the AI agent to retrieve information it does not need and disclose sensitive information back to the sender of the email. This is a classic example of a prompt injection attack that deployers of AI agents should be aware of. 

NIST’s agent-security work specifically identifies prompt injection as an issue organization are recommended to consider when developing identity and authorization controls for agents. 

What safeguards should businesses look for? 

Security should not depend solely on the employee using an AI agent correctly. Instead, businesses should consider the protections that are built into the AI agent itself. 

For example, an AI agent may be designed with limited, instead of broad, access to company information by default. This way, the AI agent’s ability to access information is limited by permissions required to perform its rule. Higher-risk actions may require additional approval. For example, an agent might be allowed to draft an email independently, but sending sensitive information would require human confirmation, approval, and oversight of the agent. 

Businesses may also consider whether the agent creates records or logs of its own activity, whether different systems can be restricted, and if an agent’s access is able to be quickly suspended. These safeguards may limit damage caused by both mistakes and outside attacks that result in attempts to manipulate an agent. 

Again, the goal does not need to be to prevent AI from acting independently, but rather to ensure that its ability to act stays within clearly identified boundaries. 

Why does monitoring matter? 

Even when a business has security limits in place, it will still need to be able to identify when an AI agent behaves unexpectedly or attempts to go outside those pre-defined limits. 

A recent OpenAI incident provides a good example of why monitoring is essential.  In September 2026, an AI agent was given a search-based training task, and in completing the task, the agent found that there was a gap in internet access restrictions and used the internet to query an external chatbot. OpenAI reported that its monitoring system detected the unauthorized activity within about 15 minutes, and additional controls were added that would have blocked the access.

This incident highlights the possibility that setting permissions before deployment may not be enough to prevent AI agents from only acting within their boundaries. Put simply, businesses should also monitor what an agent does. For example, a standard review of an AI agent’s activity logs may help determine:

  • What system an agent accessed,
  • What actions it performed,
  • When that action occurred,
  • Whether sensitive information was involved, and,
  • Whether the activity stayed within an agents assigned role/access.

Businesses should remember that cybersecurity runs in both directions, and risks can include: 1) protecting an AI agent from unauthorized access, and 2) monitoring the AI agent itself for unauthorized activity. If unusual activity occurs, businesses should know how to quickly respond. 

What happens if an AI agent accesses something it should not? 

Existing cybersecurity laws may also be relevant when an AI agent interacts with systems outside of the business. 

For example, the federal Computer Fraud and Abuse Act (“CFAA”) addresses certain access to protected computers without authorization or beyond its authorized access. Courts have already begun to explore how this law applies when an AI agent performs the action.

In Amazon.com Services LLC v. Perplexity AI, Inc., Amazon argued that Perplexity’s agentic browser violated the CFAA by accessing Amazon through users’ accounts. In August 2026, the Ninth Circuit vacated a preliminary injunction against Perplexity after concluding Amazon was unlikely to show that Perplexity itself has accessed Amazon’s computers. Instead, the court viewed the user as accessing Amazon with the AI agent helping perform the actions. 

In this case, the court made clear that its decision was limited to the facts of that case and did not create a broad rule for agentic AI. The bigger takeaway is that businesses should set clear limits on where their agents can operate and what they can access.

However, businesses should not assume they are off the hook just because an AI acted independently. California Civil Code § 1714.46 says a company cannot defend a claim by arguing that the AI caused the harm on its own. This does not mean the business is automatically liable, but it highlights why companies need to keep oversight and controls in place.

What should businesses take away?

Agentic AI are changing the way businesses operate, but giving an AI system the ability to act upon company data and applications should require additional security planning and considerations before deployment. Businesses deploying agentic AI may want to consider the following:

  • Define the agent’s job: establish clear guidelines as to what the agent is supposed to do and what falls in and out of that role.
  • Limit permissions: give the agent access only to the systems it absolutely needs to operate with for its assigned tasks.
  • Keep humans in the loop: determine how best to monitor agentic AI’s actions and, where necessary, implement additional oversight. 
  • Look for built-in safeguards: evaluate whether the agent supports limited permissions, approval controls, logging and quick revocation of access. 
  • Monitor agent activity: maintain oversight to determine what the agent accessed and if it acted outside its role. 
  • Avoid giving agents access to sensitive data: avoid giving an AI agent access to health, financial, employee or highly confidential client information.
  • Protect against manipulation: consider how the agent interacts with potentially malicious emails, websites and other outside information.
  • Prepare for incidents: establish protocols and procedures to quickly disable agents if needed. 

Agentic AI changes the cybersecurity question from what AI can see to what AI can do. Businesses need to understand this and limit authority, monitor activity and be prepared to step in if something goes wrong.

Privacy Policies and Practices: Hims & Hers Enforcement

Most people are familiar with the idea of a privacy policy – the long document we often agree to when signing up for a service or purchasing goods. But what does a privacy policy actually do? In short, it outlines how a business collects, uses and shares personal information. The contents of a privacy policy may depend on the size and function of the business, the sector in which the business operates, and the jurisdictions in which the business is located or does business. 

Drafting and publishing the policy is only part of the job though. Businesses also need to also confirm that their actual practices, including the technology operating behind their website, match the promises that they make to their consumers in these policies.

The Federal Trade Commission (FTC) has long warned businesses that if they make privacy representations, they must honor them. The FTC specifically advises companies to review their privacy policies and ensure that their actual practices are consistent with those representations. As the recent FTC  lawsuit against Hims & Hers demonstrates, it is imperative that consumers are adequately informed about business practices in a clear and conspicuous manner. 

What are the risks of misalignment between business practices and consumer representations? 

In the United States, the FTC has the power to enforce the terms of privacy practices via the authority in Section 5 of the FTC Act, which prohibits unfair or deceptive advertising practices. The Commission’s landmark 1999 consent order with the web host GeoCities was the FTC’s first public settlement in the area of internet privacy, and as of 2023, the FTC has brought at least 97 internet privacy cases. 

Most recently, in July 2026, the FTC was joined by California and Utah authorities in a lawsuit against the telehealth company Hims & Hers for deceptive and unlawful privacy practices. This lawsuit alleged that the company shared customers’ sensitive health data, including medical conditions, with advertising platforms, despite implying that they keep health information private on their privacy policy. 

According to the FTC, some information was shared to third-party advertising platforms through customer lists, while other information was transmitted through third-party tracking technologies; the consumers’ health information was allegedly shared with Meta, Snap and other third parties. In its complaint, the FTC states that Hims & Hers also fails to disclose its billing practices adequately and makes it difficult for consumers to cancel their subscriptions. Christopher Mufarrige, Director of the FTC’s Bureau of Consumer Protection claims this creates a scenario where “consumers [are] unknowingly locked into recurring subscriptions and the disclosure to third parties of consumers’ most private health information without their consent.” 

This case remains pending and the allegations have not been adjudicated. Still, the lesson here is clear: a business’s sharing and selling practices must be accurately disclosed to the consumer via their privacy policy. 

This does not mean that businesses need to abandon tracking, analytics, or subscription services. Rather, this case – and many of the other enforcement actions highlighted by the FTC – emphasize an organization’s need to understand what these technologies do before describing their privacy practices to consumers and ensure that privacy policies accurately disclose these practices. 

Key Takeaways: 

Privacy policies should accurately disclose the business’s privacy practices to the consumer in a clear and conspicuous way. By reviewing both their privacy representations and the technologies behind them, an organization can take steps to ensure their privacy practices are accurate and up to date. Some of these compliance review measures may include: 

  • Inventorying pixels, cookies, analytical tools, chat-bot features, and other third-party technologies. 
  • Identifying what personal information each tool may collect or transmit, and who receives it. 
  • Paying particular attention to sensitive information and data entered into forms, portals and chat features. 
  • Reviewing vendor configurations and contractual terms governing data use.
  • Comparing actual data flows against the company’s privacy policy and other consumer-facing statements.
  • Requiring reviews before changing internal business practices that impact personal privacy.

Drafting and publishing a privacy policy may be required under certain state laws. However, this policy – like most privacy and compliance efforts – should not be treated as a one-time task. Websites, vendors and business practices change, and privacy policies should stay in alignment with these changing practices.  

AI Watermarking: Key Updates on New Transparency Rules

Updated on October 9, 2026, in light of the passage of SB 1000: California AI Transparency Act.

As generative artificial intelligence (AI) continues to advance, it’s becoming increasingly difficult to distinguish human-created content from AI-generated material. In turn, regulators are requiring certain AI systems to disclose where and how content is created. 

Two important examples are the European Union’s AI Act and California’s AI Transparency Act. While both of these laws address AI-generated content, neither requires businesses to use the same type of disclosure in every situation. Instead, the rules distinguish between different forms of transparency, including machine-readable markings that may not be visible to the human eye. These requirements may apply to both the provider of the AI system and the deployer, or the entity that makes the AI available to consumers.

The laws are falling into place, but how do they work in practice? For businesses developing or using generative AI, the challenge is not only whether AI-generated content should be watermarked, but how different types of media should carry that watermark. 

The EU AI Act 

Transparency requirements under Article 50 of the EU AI Act became generally applicable on August 2, 2026, with a grace period until December 2, 2026, for certain products that were already on the market as of August 2, 2026. 

Among other requirements, providers of AI systems that generate synthetic text, images, audio or video must generally ensure that the output can be detected as artificially generated or manipulated. This may include machine-readable markings built into the content rather than displayed as a visible layer. 

The AI Act also creates separate disclosure requirements for certain users of AI-generated content. For example, deepfakes generally must be clearly disclosed as AI-generated or manipulated; AI-generated or manipulated text about matters of public interest may also require disclosure when it is published without human review. 

While the law provides the end goal, it does not necessarily define how to achieve it. For example, the EU AI Act was accompanied by the Guidelines on Transparency Obligations for Providers and Deployers of Certain AI Systems. These guidelines provide definitions and explain how compliance with the AI Act’s transparency obligations may be demonstrated. But still, these guidelines defer to “providers and deployers [who] can determine adequate measures themselves, while taking into account these guidelines.” In short, this means the EU AI Act sets the standard but leaves the means of meeting it to the AI provider or deployer. 

This means AI transparency can look different, both because of the media involved and because the provider or deployer can determine how best to meet these transparency standards. For example, a machine-readable watermark may help technology detect that AI was involved, while a visible or audible disclosure is meant to inform the person viewing or interacting with the content that it was AI- generated or altered. But the form and format of these transparency measures may largely be subject to the organisation’s discretion. 

The California AI Transparency Act 

California has also adopted AI-content transparency requirements through the California AI Transparency Act, which was originally enacted through SB 942 and amended through AB 853 (which went into effect on August 2, 2026) and SB 1000 (which went into effect on September 30, 2026). Among other things, the California AI Transparency Act has a requirement for 1) a disclosure verification tool, and 2) disclosures within the AI-generated media.

What is the disclosure verification tool?

The law generally requires “covered providers” – defined as any person who produces a generative AI system publicly accessibly within the geographic boundaries of California – to make a disclosure verification tool available.

This tool should be publicly accessible and allow a user to assess if an image, video or audio content (or any combination) was created or altered by a covered provider’s generative AI system. There is an exception for minor modifications of media.

The tool should also provide any “provenance data” detected in the media. This includes information “about the origin of a piece of digital content and the history of modifications of the content.”

Notably, a covered provider may satisfy the requirements regarding the tool by directing a user to a third-party tool, provided the covered provider gives certain disclosures, and the third-party tool meets certain standards.  

What disclosures does the verification tool detect? To effectuate the purpose of the disclosure verification tool, the Act requires disclosures to be included in any image, video or audio content created by the covered provider’s generative AI system, with exceptions for minor modifications.

The California AI Transparency Act distinguishes between two different types of disclosures: latent and manifest. A latent disclosure is built into the content, but it’s not readily visible to the average person. A manifest disclosure is one that a layperson can more easily see and understand. 

Per SB 1000, the latent disclosure must include certain information, including but not limited to the name or the covered provider, the generative AI system that created or altered the content, and the time and date of the creation or alteration of the content.

There are additional compliance requirements for coved providers who license generative AI systems to licensees of those systems.

Through the implementation and subsequent amendments of the California AI Transparency Act, it is evident that California’s requirements are also rapidly evolving. Businesses subject to the law may need to monitor legislative changes as lawmakers revisit how disclosure and detection measures are required under the law.

What does AI watermarking look like? 

Anthropic’s recent changes to Claude provide one example of how companies may approach these requirements. 

In August 2026, Anthropic announced that the new Claude models would include machine-readable markings in response to the EU AI Act and other similar legislative requirements. For generated text, Claude uses an imperceptible watermark based on patterns in how the model selects words. According to Anthropic’s press release, the watermark is not a visible or hidden set of characters and does not identify the person or company who generated the content. 

However, these tools are far from perfect. Anthropic explains that a text watermark can disappear or weaken; they describe it as not “foolproof” if content is heavily rewritten, translated or combined with other materials. File-based provenance information can also get lost if files are converted, resaved or captured by screenshot. 

This means that businesses should not treat watermarking as a perfect way to determine whether something was created by AI. Instead, it may be one tool to provide more information about where content came from and whether AI was involved. 

Can EU rules affect U.S. businesses? 

United States companies may also feel the effects of the EU AI Act even when a particular use of an AI product occurs outside Europe. 

For a company operating in multiple countries, creating different versions of the same product for each jurisdiction can become expensive and technically complicated. A business may instead choose and decide to build one version that satisfies the strictest applicable requirements and use that version more broadly.

For example, Anthropic has said that although its Claude watermarking changes were driven by the EU AI Act, it is applying them globally rather than limiting them to European users. 

This can create a spillover, where a law passed in one jurisdiction changes how an AI product operates for users everywhere else.  Similar regulatory concepts are also appearing in U.S. state AI laws. California and other states have adopted requirements involving transparency, disclosures, and AI governance that overlap with themes found in the EU AI Act, even though the laws differ in their scope and specific requirements.

As AI regulations continue to develop across different jurisdictions, companies may consider whether maintaining different disclosure mechanisms per jurisdiction makes sense. Alternatively, companies may comply with the most stringent regulations across all markets, setting a higher standard of compliance across the board. 

What should businesses take away?

Businesses do not necessarily need to approach every type of AI generated content in the same way. However, companies developing or using generative AI may want to review how their systems identify AI generated content, and whether their disclosure practices meet the requirements that might apply to them. Some steps businesses may want to consider include:

  • Identifying what types of AI-generated content the business uses or provides to consumers, including text, images, audio and video. 
  • Determining whether applicable laws require machine-readable markings or visible disclosures.
  • Understanding how watermarks or provenance information may change if content is edited or redistributed. 
  • Checking if AI-generated content from outside vendors keeps its watermark or disclosure when the business edits, downloads, or republishes it into another product. 
  • Considering whether maintaining different product versions across jurisdictions is practical.
  • Monitoring United States, EU, and other developing AI transparency requirements and laws. 

AI watermarking is still rapidly developing both technically and legally. Businesses do not necessarily need to treat every AI-generated output the same way, but they may need to increasingly understand when AI-generated content should be identified and whether their current systems can provide that transparency. For businesses using AI-generated content in marketing, customer communications, and other business activities, understanding when and how that content must be identified can help reduce compliance risks as these rules continue to develop 

AI Price Fixing and Discrimination with Recommendations for Retailers

Next year will likely bring more lawmaking surrounding artificial intelligence algorithms used to set consumer pricing. Two themes are emerging: price fixing and price discrimination.

AI price fixing uses algorithms to coordinate pricing, which is raising increased scrutiny in the U.S. and Europe. Companies provide competitively sensitive data to third-party providers that analyze it to recommend prices for participating companies, leading to artificially inflated prices and reducing competition.

After over a year of litigation, the Department of Justice (DOJ) recently filed a proposed settlement with a revenue management software company, related to its algorithmic rental pricing software. State AGs continue to litigate against participating companies, and lawmakers have introduced bills to ban algorithmic price fixing. California has passed laws to restrict the use of common pricing algorithms, and regulators in the EU and UK are also focusing on algorithmic collusion. 

Regulation challenges include the argument that antitrust laws may have loopholes that make AI-driven pricing hard to prosecute; distinguishing collusion from companies’ legitimate responses to market conditions; and AI complexities creating difficulty proving violations.

AI price discrimination uses algorithms to set personalized prices for products or services based on the individual consumer’s behavior, demographics or perceived willingness to pay. The process is often hidden, making it hard to know who pays what. That raises fairness concerns (algorithms might exploit vulnerable groups and charge them more), prompting legal scrutiny and proposed regulations. 

The FTC is investigating AI’s role in price discrimination, while recently passed regulations and laws in California aim to ban using personal data for discriminatory pricing or make it easier to sue over algorithmic pricing. New York has enacted several price discrimination laws requiring algorithmic pricing disclosure and prohibiting gender-based pricing. 

Recommendations for retailers

While it is perfectly legal to test prices and adjust them to market conditions, retailers should be careful when reviewing and adopting AI pricing tools. The recent proposed settlement from the DOJ highlights some best practices for using these tools: 

  • Confirm that the AI tools train on publicly available pricing data or aggregated data, not the non-public and sensitive pricing strategies of competitors. 
  • Ensure that the AI tools allow prices to be lowered as well as increased within user-defined price ranges. 
  • Keep humans in the loop and check that users can reject or override price suggestions. 

In addition, businesses may consider the following best practices to lessen the risks of allegations of unlawful AI price discrimination:

  • The AI tools should not adjust pricing dynamically based on consumer profiles or other sensitive personal information (e.g., veteran status, religious beliefs, health condition). 
  • If the AI tool adjusts pricing based on location, retailers should confirm that this reflects legitimate market conditions and business concerns, rather than historic bias. 
  • Confirm that the AI tool uses de-identified and/or aggregated data, both to train its models and during deployment, so information is not linkable to an individual. 

1 2 3 4