Discover the privacy, confidentiality, and compliance challenges of AI in clinical trials, from HIPAA and FDA rules to AI vendor risks.

AI Use in Clinical Trials: Privacy, Confidentiality, and Compliance Risks

AI & Algorithms , , ,

Artificial Intelligence (AI) is becoming increasingly common in health care clinical trial operations. Sponsors, contract research organizations, research sites and vendors may use AI for many tasks such as patient recruitment, eligibility screening, and informed consent support among other things. While this can improve efficiency, these tools may also create privacy, confidentiality and compliance risks in an already highly regulated environment. 

Clinical trials often involve sensitive patient information, confidential study data, and strict documentation obligations. This means that AI use in clinical research cannot be treated like traditional workplace software. Before deploying an AI tool, organizations should understand what data the tool will access, where that data will go, whether the vendor can use it to train models, and whether the output can be explained, reviewed, and preserved.

How is AI being used in clinical trials? 

AI tools may be used throughout the clinical trial process. For example, AI can be used to help identify potential participants, screen patient eligibility, summarize medical records, draft study documents, monitor data and flag adverse events. Generative AI may also be used for other tasks in the clinical trial process such as summarizing meeting notes, clinical records or other trial related documents.

These uses can be helpful, but they also raise basic compliance questions. Before using an AI tool, organizations need to ask whether the tool fits within the trial’s legal and contractual framework. Clinical trial agreements and site agreements often control who may access trial data and how that data can be used, but those contracts may have been drafted without considering the use of AI in the trial. If an AI vendor is introduced without updating those agreements, contracts that are not AI-specific may create gaps in data access, confidentiality and regulatory accountability.

What legal obligations matter?

AI use in clinical trials must also fit within existing privacy, research and documentation obligations. HIPAA and its related rules may apply when AI vendors receive, store and analyze PHI. This means, organizations may need privacy and security safeguards before sharing their trial data with the AI vendor.

FDA electronic records requirements should also be considered as clinical trial records have to be reliable, auditable and traceable. If an AI tool creates a summary or other output without prompt logs or documented human review, it may create problems for recordkeeping and data integrity. 

Consent for the use of AI in the clinical trial process is also a key factor, especially when AI is used in recruitment processes or eligibility screenings. In these situations, participants should receive clear information about how AI is involved and how their information may be used.

Finally, confidentiality should remain at the forefront. Clinical trials do not only involve patient data, but may also involve sensitive commercial information, such as investigational product data, interim results, safety signals, and proprietary research methods. If an AI tool takes in this information for transcription, summarization, or analysis without clear confidentiality restrictions, it may create risks for sponsor confidentiality, trade secret protection, or attorney-client privilege.

Why are AI vendors a risk?

AI vendors can be one source of risk in clinical trials because they may receive and process very sensitive trial data. Standard AI vendor terms may not be enough when clinical trials involve protected health information or FDA-regulated records. Before using a specific AI vendor, organizations should review whether the vendor can retain prompts or outputs, use trial data to train its models, allow employee access to uploaded data, or store information outside approved systems.

Vendor contracts should clearly address data retention, model training, confidentiality, audit rights, security controls and human oversight. Additionally, companies should not rely only on what the vendor promises in the contract. They should verify how the vendor actually handles trial data before using the AI tool.

What should companies take away? 

For sponsors, meaning the organizations responsible for clinical trials, the main takeaway is that AI should be reviewed before it is used with clinical trial data. The organization should review what the AI tool will be used for, whether it will access PHI or confidential study information, whether the vendor can use trial data to train its models and if AI-generated outputs can become part of the trial record.

There are AI vendors that offer privacy-protective features like PHI redaction and de-identification. However, companies should not assume these features automatically make the tools compliant. Companies still need to verify how the vendor stores, processes, deletes and protects clinical trial data before deployment. 

AI may help clinical trials become more efficient, but efficiency does not replace privacy, confidentiality, consent, or regulatory accountability. Companies should update agreements, limit data exposure, verify vendor practices, document human review, and clearly disclose AI involvement where appropriate.

0

AI Chats and Law Enforcement: What Are You Sharing? 

AI & Algorithms , ,
AI chat platforms are increasingly becoming repositories of sensitive personal, professional, and legal information, and the legal frameworks governing what can be done with that information remain unsettled. This can have serious repercussions for individuals, businesses, and their advisors who happen to find themselves in the complex intersection of law enforcement and information privacy.  

What are users actually sharing?

The volume and sensitivity of information flowing into AI chat platforms go beyond what many users fully appreciate. Chatbots prompt users to provide background, context, and points of view, all of which may reveal intentions. This interface allows AI models to respond conversationally and prompt further explanation, inviting more disclosure than traditional searches. Below, we have highlighted two key reasons this leads to additional information being disclosed in this context:

The Illusion of the Advisor

Users increasingly interact with AI platforms as they would with a trusted professional, an attorney, therapist, or financial planner. However, AI chat platforms are not bound by traditional confidentiality obligations that govern licensed professionals. There is no attorney-client privilege, no therapist-patient privilege, and no fiduciary duty attached to a chatbot conversation. The sensitivity of the content does not create the protection the user may assume exists.

Agentic AI’s increased access

As the industry moves from chat interfaces to AI agents, this risk may continue to grow. Agentic AI is a tool that streamlines workflows; however, it requires broad, constant access to a user’s data across devices and applications. Major technology companies have already released early versions. As these agents become standard, the question of what an AI platform “knows” will no longer be limited to what was typed into a chat window, but may instead extend to digital communications such as email and text, documents, financial records, and location history.

What Can the Government Access?

Prosecutors and investigators have already begun seeking access to chatbot conversation histories in criminal investigations, and the legal framework governing those requests is still taking shape. However, there are a few current frameworks governing the chatbot’s permissible uses and disclosures of user intentions. 

Subpoenas and Third-Party Doctrine

Under the traditional application of the third-party doctrine, information voluntarily shared with a third-party platform has lesser protection than the Fourth Amendment typically affords. A government agency seeking chat transcripts may obtain them via subpoena without meeting the higher probable cause standard required for a warrant. The Supreme Court introduced some limits in Carpenter v. United States (2018), but its application to AI conversation logs is entirely untested.

National Security Demands

AI platforms may be subject to National Security Letters and Foreign Intelligence Surveillance Act (FISA) orders requiring disclosure of user data, with limited judicial oversight and strict non-disclosure obligations. A platform that receives such a demand often cannot notify the affected user, who has no opportunity to contest the disclosure. For businesses using AI tools for sensitive professional work, this exposure can be far-reaching and hard to foresee until it materializes. 

The Regulatory Gap

Currently, frameworks are designed for passive content-hosting platforms. However, these privacy frameworks are a poor fit for conversational AI.  

Ambiguity in Section 230 Protections

Section 230 of the Communications Decency Act shields platforms from liability for user-generated content. Whether that shield extends to AI chatbot outputs generated by the platform, not merely hosted by it, remains unresolved. A chatbot that produces a harmful response is authoring a reply, not hosting a post. Courts have not yet answered whether Section 230 immunity applies, and platforms that assume it does may find that assumption is not correct.

Consent Frameworks and Cross-border Complexity

Most AI platforms rely on broad, scroll-past consent mechanisms that regulators increasingly consider inadequate to secure meaningful consent. In the absence of comprehensive federal privacy legislation, compliance obligations vary by state and sector, and for multinational organizations, cross-border data flows through AI platforms may simultaneously implicate GDPR transfer requirements and foreign mandatory access regimes.

Key Takeaways

As AI use becomes more and more prevalent for use of everyday tasks and sensitive information alike, individuals and businesses may want to consider the following key takeaways: 
  • Establish policies governing employee use of AI chat platforms for work matters, with explicit restrictions on sharing confidential, privileged, or regulated information.
  • Review data retention and third-party sharing policies for any AI platforms in use, and update litigation hold procedures to treat AI chat logs as a discoverable data category.
  • Assess AI agent tools – those requiring broad device and application access – before deployment, with legal review of data exposure and applicable frameworks.
  • Brief leadership on the government access risk: AI chat transcripts may be subpoenaed or compelled under national security processes, often without user notification.
  • For multinational organizations, conduct a cross-border data flow analysis covering AI platform use and compliance with GDPR and analogous transfer frameworks.
When using these AI tools, it’s important to remember that the legal protections available for information shared with AI are not proportional to the information’s sensitivity or the user’s reasonable expectations. Closing that gap is, at this moment, primarily the responsibility of the user and the organizations that employ them. While legal frameworks are developing to align these interests, it is best to implement best practices early. 
0

Risks of Shared AI Workspaces and Confidentiality, Security, and Privacy Concerns

AI & Algorithms , ,
Traditionally, the relationship between a company and its outside advisors, law firms, consultants, and financial advisors has been governed by confidentiality agreements, attorney-client privilege, and codes of professional ethics. These agreements assure that these outside advisors have access only to the information necessary for the scope of the project. However, artificial intelligence is becoming a mainstay in these working relationships, dismantling that clear separation.  AI-powered productivity tools are increasingly deployed not just within a single organization, but across shared digital workspaces, the collaborative platforms where companies and their external advisors jointly draft documents, manage new projects, exchange data, and make decisions. This shift represents a fundamentally new risk landscape, one that most organizations and their advisors have not yet adequately mapped.  This post identifies the three primary risk categories that arise when AI enters these shared spaces and the key considerations to mitigate them.  

Risk 1: Confidentiality

When AI tools operate within a shared workspace, there are two primary threats to client confidentiality:  1) Cross-client training and model contamination, and  2) over-input of information.  

Cross-Client Training Model Contamination

Many AI tools learn continuously from user interactions. For example, if a law firm’s AI assistant is trained, even implicitly, on documents, queries, and outputs across multiple client engagements sharing a platform environment. In this case, client information can become embedded in the model’s behavior. The AI may begin surfacing language, structures, or strategic approaches drawn from one client’s confidential materials when assisting another.  This is an example of cross-client training contamination. 

Over-Input of Information

When processing the information above, AI tools may ask follow-up questions, or the user may want to include additional context and guidance for the tool. These prompts and the need for greater contextual clarity may drive users to input additional information, information that may not normally be shared or be strictly necessary for the task at hand. This could lead to AI tools being trained on, and potentially re-sharing, information that is not strictly necessary. 

Risk 2: Overexposure

AI processes operating across shared workspaces introduce a new failure mode: overexposure through automated workflow. When an AI agent is tasked with summarizing documents, preparing briefings, or surfacing relevant materials, it may draw on content from across the workspace without respecting the role-based and project-based permissions designed to contain that information.

Misconfiguration and Permission Gaps

AI tools in shared workspaces are typically configured by IT or platform administrators, not by the lawyers or compliance officers who understand the sensitivity of the underlying information. Permissioning structures that may be technically correct for human access often fail to account for how AI agents traverse and aggregate information. A consultant with project-scoped access to a workspace may, through the AI layer, receive synthesized summaries that draw on materials outside their authorized scope.

Role and Project Segmentation Failures

Even well-intentioned configurations can break down when AI tools are updated. For example, this could occur when team membership changes or when workspace structures evolve mid-engagement. Unlike a human employee who is subject to ongoing supervision, an AI system with broad access will continue operating at that level until it is explicitly restricted. The moment of overexposure may be difficult to trace, making the discovery of these failures especially challenging. 

Risk 3: Accountability

Who is Responsible when AI makes the decision? Professional service relationships often assign responsibilities clearly; for example, the lawyer is responsible for legal advice, the auditor for the audit opinion, and the consultant for the recommendation. These lines of responsibility are the foundation of malpractice liability, professional licensing, and regulatory compliance. However, AI tools make this division more complicated. 

The Absence of Auditable Decision Trails

Many AI tools used in professional services do not generate decision logs or explainable outputs in a meaningful sense. When a deal recommendation, a compliance conclusion, or a litigation strategy is influenced by an AI-generated analysis, there may be no record of what data the AI considered, what it weighted, or what it excluded. If the decision is later challenged in litigation, a regulatory proceeding, or a malpractice claim, the AI’s contribution cannot be reconstructed or audited.

Diffused Liability Across a Shared Platform

In a shared workspace involving the company, its law firm, its auditors, and potentially a technology platform provider, an AI-assisted error may have no clear owner. Did the AI fail because of a platform defect? Because the law firm configured it incorrectly? Because the company provided bad inputs? Because no human professional adequately reviewed the output? Engagement letters, platform terms of service, and professional liability policies may not be drafted to answer these questions.

Key Considerations in Light of these Risks

The risks described may be present in any organization that has extended its advisory relationships (law firms, consultants, and financial advisors, to name a few) into AI-enabled collaborative platforms. To minimize these risks, organizations may want to consider the following tips:  Consider…
  • Auditing shared platforms and tools currently used with outside advisors to identify any AI features, and map what data those features can access. 
  • Reviewing engagement agreements, NDAs, and platform terms of service for AI-specific confidentiality provisions. 
  • Assessing whether AI access controls in shared workspaces respect role-based and project-based information silos and construct limitations where they do not. 
  • Establishing AI decision-logging protocols with outside advisors, including requirements for human review and sign-off before AI-influenced advice is acted upon. 
  • Negotiating clear contractual allocation of liability for AI-related errors across the full advisory chain, company, advisors, and platform providers. 
  • Briefing executive leadership and the board on AI-specific risks in advisory relationships, particularly in regulated industries where privilege and data protection obligations are most acute. 
Establishing governance frameworks for AI early in advisory relationships may enable companies to reduce their own exposure and hold advisors accountable if one of the risks of use materializes. 
0

Overview of New York’s Child Data Protection Act

Children, Cybersecurity, New York, State Privacy Laws, United States , , , , ,
In June 2024, New York Governor Kathy Hochul signed the New York Child Data Protection Act (Act) into law, which will go into effect on June 20, 2025. Per the Act’s justification, “[c]hildren now live much of their lives online,” including learning, socializing, shopping. They also “make mistakes online, and they discover who they are online,” and, accordingly, they should be able to do so without the “concern of omnipresent monitoring and recording.” The Act enables this through two major provisions:
  1. if a digital service knows a user is a minor (or if the service is primarily directed to minors), it will “default to only being able to use that child’s data in a way that is strictly necessary to provide the service;” and
  2. digital services using third-party service providers must “contractually restrict those third parties from using the personal data of minors except for specified purposes” and include additional safeguards to help ensure compliance.
The Office of the New York State Attorney General has also released Implementation Guidance to clarify key questions raised in the rulemaking process.

Scope & Applicability

This Act applies only to conduct occurring in the state of New York. This means that commercial conduct that takes place outside of New York is not covered by the Act if: 1)  the user was outside of the state or 2) no data collected while the user was in the state was used.
  • Covered Users. The Act imposes restrictions on processing information of “covered users.” This includes users of websites, online services, or connected devices (the “Websites”) who are: 1) actually known by the operator to be a minor (under 18), or 2) who are using Websites primarily directed to minors.
  • Operator. An operator is defined as any person who offers Websites, who alone – or jointly with others – controls the purposes and means of processing personal data. Notably, one who acts as both a controller and processor shall comply with obligations for both roles, depending on the purposes and means of processing personal data.
  • Personal data. This definition includes any data that identifies or could be reasonably linked, directly or indirectly, with a specific natural person or device.

Substantive Provisions

Processing Restrictions. The Act provides that, among other things, an operator shall not process the personal data of a covered user collected through the Sites, unless one of the following applies:
  1. the user is 12 or younger, and processing is permitted under COPPA;
  2. the user is 13 or older and the processing is “strictly necessary”; or
  3. the user is 13 or older and the processor has received informed consent.
Strictly Necessary Processing. The term “strictly necessary” includes, among other things, processing that is required to:
  • Provide or maintain a specific product or service requested by the covered user;
  • Conduct the operator’s internal business operations (excluding those that relate to marketing, advertising, research and development, providing products or services to third parties, pr prompting covers users to use the Site when it is not in use); and
  • Identify and repair technical errors that impair functionality.
According to the Implementation Guidance, processing that is “strictly necessary” to provide a process or service required by a covered user depends on the “expectations of a reasonable covered user,” similar to the guidance provided under the CCPA regulations. The Guidance also clarifies that business operations “shall not include any activities relating to marketing, advertising, research and development, [or] providing products or services to third parties.” Informed Consent. If the information being processed is not “strictly necessary,” the operator will need informed consent, through either: 1) a device communication or signal, or 2) an informed consent request. A request for informed consent should, among other things:
  1. be made separately from any part of the transaction.
  2. clearly and conspicuously state that the processing is not strictly necessary, and consent is not mandatory to continue using the Websites.
  3. clearly present an option to refuse to provide consent as the most prominent option.
Additionally, the user should be able to revoke consent at any time as easily as they provided it.

Enforcement

The New York Attorney General may bring an action or special proceeding to enjoin any violation of this Act, and to obtain civil penalties of up to $5,000 per violation. Further, the Act gives the New York Attorney General authority to issue rules and regulations ad necessary, and according to the Implementation Guidance, the Office of the Attorney General intends to issue these rules. The Implementation Guidance also states that, until such rules are finalized, the Office of the Attorney General will exercise discretion in pursuing enforcement actions, taking good-faith compliance efforts of covered businesses into account.

Effective Date

The Act goes into effect on June 20, 2025.
0
Chicago Grand Central Looking Up

DOJ Issues Final Rule on US Bulk Sensitive Data

Data Brokers, Data Transfers, United States , , , ,
The International Emergency Economic Powers Act (IEEPA) vests the President with authority to deal with extraordinary threats to national security and foreign policy that have their source in part or in whole outside of the United States. Acting pursuant to the IEEPA, President Biden issued Executive Order 14117, “Preventing Access to Americans’ Bulk Sensitive Personal Data and United States Government-Related Data By Countries of Concern” (the EO). The EO directed the Department of Justice (DOJ or Department) to establish and implement regulations addressing threats from certain countries of concern attempting to access and exploit bulk amounts of US sensitive data, including personal and government data. On December 27, 2024, the DOJ issued the Final Rule, which went into effect on April 8, 2025. Additional compliance provisions for certain transactions take effect on October 6, 2025. The Final Rule prohibits or restricts a range of transactions involving categories of bulk sensitive personal data or government-related data between the US and countries of concern or covered persons. In assisting businesses to adapt to this comprehensive update, the DOJ provided a Fact Sheet, a Compliance Guide, and over 100 FAQs on the Final Rule, along with an Implementation and Enforcement Policy. Below are five main takeaways that US entities may want to consider in light of these regulations.
  1. Enforcement May Be More Lenient Until July 8, 2025 
The DOJ’s Implementation and Enforcement Policy, states that the Department will “target its enforcement efforts during the first 90 days to allow US persons (e.g., individuals and companies) additional time to continue implementing the necessary changes to comply with the [Final Rule].” The Department’s civil enforcement actions for violations of the Final Rule will not be a priority “so long as the person is engaging in good faith efforts to comply with or come into compliance with the [Final Rule] during that time.” However, the Department makes clear that it will “pursue penalties and other enforcement actions as appropriate for egregious, willful violations” during the delayed enforcement period.
  1. DOJ Will Consider Good Faith Efforts to Comply
While the Implementation and Enforcement Policy reflects that civil actions for violations of the Final Rule will not be a priority, this depends on the entity’s good faith effort to comply. According to this Policy, examples of evidence of good faith efforts may include, but are not limited to:
  • Conducting internal reviews of access to sensitive data.
  • Conducting internal reviews to determine whether transactions involving access to such data flows constitute data brokerage.
  • Reviewing internal datasets and datatypes to determine if they are subject to the Final Rule.
  • Conducting due diligence on potential new vendors.
  • Renegotiating vendor agreements or negotiating contracts with or transferring products or services to new vendors.
  • Adjusting employee work locations, roles or responsibilities.
  • Evaluating investments from countries of concern or covered persons.
  • Implementing the CISA Security Requirements.
  1. “Good Faith” May Include Satisfying CISA Security Requirements 
A good-faith effort to comply may be demonstrated, in part, by implementing the CISA Security Requirements, which were developed concurrently with the Final Rule pursuant to the EO. The security requirements are intended to address threats that arise when conducting restricted transactions, as detailed below. These security requirements are divided into two sections: i) organizational- and covered system-level requirements; and ii) data-level requirements.
  1. Before October 6, 2025, Determine if Your Company is Conducting Restricted Transactions
US entities engaged in restricted transactions under the Final Rule have affirmative data compliance program and audit obligations, among other obligations. In addition, the Final Rule provides that data brokerage transactions are prohibited with any foreign entity unless the US person contractually binds the foreign entity from subsequent transactions of that data with a country of concern or covered person. They must also report any known or suspected violation of this requirement.
  1. An Iterative Review Plan May be Needed for Covered Transactions 
With the Final Rule coming into effect and enforcement nearing, US companies that engage in certain data transactions or share information with third parties that may be covered persons or countries of concern should evaluate their transactions and data practices. After a thorough review of the types of information collected, who that information is shared with, and who is involved in the processing of that data, it may be helpful to adopt a compliance policy to ensure transactions are being handled appropriately in light of the Final Rule.
1 2 3 … 5