0

AI vendor management – human programming for machine learning

Machine learning and artificial intelligence (AI) have permeated the supply chain. The reasons are apparent. Low cost and efficiency are an easy sell in today’s economy, with rampant inflation in the supply chain and tight labor markets. Yet, the economic motivation for AI must be tempered by human (or human-programmed) review of AI systems. Rules are necessary to ensure that the fundamental privacy and moral rights of individuals are protected. From data input to disaster recovery, AI vendor management ensures both the protection of businesses and the broader society. In an Insight article written by Lily Li, Founder of Metaverse Law for Data Guidance, Lily discusses data minimization for AI vendors, algorithmic bias and disgorgement, considerations for AI terms and conditions, and business continuity and disaster recovery considerations for AI. Click here to continue reading.
0
Image of computer circuitry in a harsh red tint.

The Risks of LLMs and Generative AI

[Modified version originally published as International Insights Article: Privacy implications for organizations using generative AI, by Lily Li, on OneTrust DataGuidance, June 2023.] Well, the cat is out of the bag – or at least the chat is. Generative AI and large language models (“LLMs”) are here to stay. From philosophical conversations between the dead to Murakami-inspired artworks for downtown LA, the possibilities of user-friendly AI are limitless. Regulators are scrambling to enforce existing legislation and enact new legislation to contain this trend. But, like all enforcement, it will take time. As a result, many companies are moving quickly to adopt and deploy these tools, testing the legal and ethical boundaries of AI. To stay competitive, companies should not wait for data protection regulators to play cat-and-mouse games with these nascent technologies. Instead, companies need to be proactive and adopt strategies to implement transparent and trustworthy AI – not just to avoid lawsuits and regulatory fines – but to protect their data and their brands. Companies also need to be able to account for the data they input into their generative AI or LLM algorithms, or else risk destruction of these algorithms altogether. In this article, we’ll discuss the latest privacy and security risks from generative AI and LLMs, a few of the existing privacy laws that apply to these technologies, and the potential for algorithmic disgorgement or deletion in response to privacy violations.   Social Engineering and Identity Verification Generative AI has clearly passed the Turing test. From all outward appearances, companies and their employees cannot tell the difference between human-generated and AI-generated text. This makes it easier for traditional phishing emails and other scams to look legitimate to readers — making it far more likely for employees to click on malicious links and download malware. Going one step further, generative AI can create realistic identities. From resumes to cover letters, online social media profiles to sample work product, these tools can improve a threat actor’s ability to pass itself off as a well-rounded individual, bypassing normal screening tools and even HR processes. In this era of remote work, it is easy to imagine malicious actors getting onboarded and hired due to their made-up “skills” and turning into insider threats once they gain access to company systems. This risk increases for companies that rely on virtual assistants and employees, where there are even fewer external validations of identity. While companies often rely on phishing training and cyber insurance to mitigate traditional cyber-attacks, this is not enough going forward. Many cyber insurance policies exclude social engineering attacks, exclude activities involving managers or other high-level employees, or confine social engineering and phishing attacks to technological attacks and not traditional identity theft, crime, and fraud. Consequently, companies should consider AI-based email filtering systems and EDR/MDR systems to combat sophisticated phishing attacks. Security awareness training should extend beyond phishing training and include identification verification and reporting of suspicious activity across the organization. Companies should also consider HR and other vendor onboarding policies to include in-person vetting or other external validation for recruiting and outsourcing.   Privacy and DSAR Risks
  • Is Processing of Personal Data for Generative AI Lawful?
Large language models, and similar machine learning tools, have a privacy problem. All these systems rely on processing vast quantities of public and sometimes proprietary data to generate responses and analysis. Absent further safeguards, these inputs will likely contain personal data. Which then begs the question, where does this data come from and is the processing lawful? This question came to a head recently in Italy, where data protection authorities issued a temporary ban on ChatGPT,[1] citing OpenAI’s failure to provide transparent notices regarding how it processes the personal data of users and data subjects (required under Articles 12, 13, and 14 of the GDPR). More importantly, the authorities found no legal basis under Article 6 of the GDPR for the collection and processing of personal data to train OpenAI’s algorithms. Impacted data subjects did not consent to the processing and, reading between the lines, OpenAI’s legitimate interest was an insufficient basis for processing given the: (i) failure to provide notice; (ii) inability to correct and delete data; and (iii) heightened privacy risks for children due to the lack of age verification techniques. OpenAI subsequently addressed Italy’s concerns in sufficient detail to resume services,[2] but it remains unclear whether other data protection regulators in the EU will also confront OpenAI over the GDPR’s transparency and lawful bases requirements. If businesses utilize generative AI and LLMs, they should be prepared to provide compliant privacy notices to data subjects, and either obtain their explicit consent or conduct a legitimate interest analysis prior to submitting any personal data to AI or LLM platforms. These data privacy risks also exist in the United States. The California Consumer Privacy Act (CCPA), as amended by the California Privacy Rights Act (“CPRA”), also requires businesses to provide transparent privacy notices and privacy rights to individuals. In addition, CPRA has imported the GDPR concepts of data minimization and proportionality. Personal data processing needs to be “reasonably necessary and proportionate to achieve the purposes for which the personal information was collected or processed, or for another disclosed purpose that is compatible with the context in which the personal information was collected.”[3] Consequently, companies should be wary of taking existing datasets containing personal information and running them through generative AI systems, if this use runs contrary to the expectations of data subjects when they originally submitted the data. Companies may need to re-evaluate their privacy notices and provide further notices regarding AI processing. Furthermore, both GDPR and the CPRA (and similar US state laws) require covered organizations to give individuals the right to opt out of automated processing or automated decision-making, including profiling.[4] While California lawmakers have yet to issue regulations concerning automated decision-making, it will likely align with GDPR concepts. This means that individuals will have the right to opt-out of AIs making decisions that have legal effects, such as those surrounding employment, housing, or access to services and benefits. So, for those who are wondering, you can’t have chatbots all the way down — eventually, there needs to be a human decisionmaker at the end of the line.  
  • Who Owns the Data? Privacy Rights to Correct and Delete
Generative AI and LLMs also call into question the ownership and control of personal data. GDPR, CCPA, HIPAA, and GLBA, among other regulations, require covered entities to obtain contractual commitments with vendors that process personal data, PHI, or NPI on their behalf.[5] By giving company personal data to an AI system absent formal review, companies may be in violating these laws, trading away the privacy of their customers, and giving up valuable IP to third parties. To combat this problem, companies should always read the terms and privacy policies of any new AI and LLM tools to confirm, as an initial step:
  • The company owns all content provided to the AI system and any output generated by the AI
  • The AI provider will provide appropriate technical and organizational measures to protect personal data
  • The AI provider will maintain the confidentiality of data and limit use of the data to those purposes disclosed by the AI provider (and similarly, disclosed by the company to the relevant data subjects)
  • The AI provider will assist the company in responding to privacy requests, including those that require correct and deletion of personal data
  • The AI provider has appropriate data transfer mechanisms in place if personal data will cross borders
Assuming the generative AI or LLM terms and privacy policies cover the items above, the company may need to negotiate additional clauses under GDPR, CCPA, HIPAA, and GLBA depending on whether regulated data is provided to these platforms. If these contractual commitments do not exist, then companies should consider policies prohibiting the disclosure of personal or proprietary data — or else risk unauthorized access or even public disclosure of this information. Even if the terms and privacy policies guarantee the confidentiality of data, companies should still validate whether the generative AI or LLM model appropriately de-identifies or anonymizes personal data or proprietary data when it improves its language models. One of the most concerning issues with generative AI is its inexplicability — often the programmers creating the model do not even understand how the AI is generating its output. Thus, even if a data subject submits a deletion or correction request, it is unclear whether this request will be propagated through the model to remove/amend information that was previously fed into the model. Consequently, companies should test any generative AI or LLM model to confirm whether identifiable data is output from the model, based on test inputs. Finally, even if a company does not input personal information into a generative AI or LLM platform, employees may be tempted to use these platforms to research or create media about a known individual. Unfortunately, generative AI regularly creates false information about individuals. At best, this may trigger notification to data subjects under Article 14 of the GDPR “from which source the personal data originate, and if applicable, whether it came from publicly accessible sources” — so they are aware of the processing and can exercise any privacy rights. At worst, publication of this personal data may be grounds for a defamation lawsuit. Once again, companies need to implement robust identity verification and external validation of AI output concerning personal data.  
  • Children’s Privacy
The impact of generative AI and LLM products on children will be tremendous, given the ease and accessibility of chatbots, and the vast potential for personalized education, gaming, and social services. Companies operating in this space should pay close attention to children’s privacy rules that may impact their use or provision of generative AI and LLM products and services. California’s Age-Appropriate Design Code, modeled after the UK’s Age appropriate design code, for instance, requires data protection impact assessment and a “high level” of privacy for online providers of services, products, or features that are “likely to be accessed by children.”[6] This law covers children under the age of 18. In addition, COPPA – a US federal privacy law – requires clear and conspicuous privacy notices and affirmative consent by parents prior to collection of personal information from children under 13. Companies that offer products and services that may be attractive to children will need to implement these heightened privacy requirements, or in the alternative, implement robust age-gating techniques.   Regulatory Enforcement and Algorithmic Disgorgement Once an AI system is trained on bad data, can it be saved? According to the U.S. Federal Trade Commission (FTC) – perhaps not. While there is currently no comprehensive federal legislation in the United States governing privacy or AI, the FTC does have the ability to regulate “unfair and deceptive acts or practices in or affecting commerce.”[7] The FTC has interpreted its enforcement power to include unfair and misleading practices regarding the collection and use of personal data – including, for example, actions against Cambridge Analytica for harvesting of Facebook user data, and against GoodRx Holdings for its unauthorized disclosures of consumers’ personal health information to Facebook, Google, and other companies.[8] The FTC’s scrutiny of privacy and security practices extends to AI. In January 2021, the FTC entered a settlement order with photo storage service, Everalbum, over allegations that it deceived consumers about its use of facial recognition technology.[9]  While Everalbum allegedly represented that it would not apply facial recognition to users’ content unless they opted-in, it applied facial recognition technology by default for most users without any ability to turn this feature off. As part of the settlement order, the FTC required Everalbum to delete all facial recognition models or algorithms developed with Everalbum users’ photos or videos. More recently, the FTC required algorithmic destruction in an action against WW International, Inc., formerly known as Weight Watchers, and a subsidiary called Kurbo, Inc.[10] According to FTC Chair Lina Khan, “Weight Watchers and Kurbo marketed weight management services for use by children as young as eight, and then illegally harvested their personal and sensitive health information….Our order against these companies requires them to delete their ill-gotten data, destroy any algorithms derived from it, and pay a penalty for their lawbreaking.” Thus, AI companies face potential deletion or disgorgement of their algorithms if they collect personal data in an unfair or deceptive manner. While it may be tempting to amass larger and larger datasets to build the best algorithms, companies that rely on improper collection of data may find themselves bereft of their most valuable intellectual property.   Move Deliberately and Create Things Generative AI and LLMs do not operate in a vacuum. They derive from the voices, both inspired and insipid, from all corners of the world wide web. And they create fabulous and fabulously weird content. We encourage companies to take advantage of generative AI and LLMs to create the next generation of personalized education, medicine, and creative exploration. At the same time, we encourage companies to be mindful of the existing rules that protect our privacy, so that transparent and trustworthy AI can be the foundation of these new creations.  
[1] https://www.garanteprivacy.it/web/guest/home/docweb/-/docweb-display/docweb/9870847 [2] https://www.garanteprivacy.it/home/docweb/-/docweb-display/docweb/9881490#english [3] Cal. Civ. Code Section 1798.100(c) [4] GDPR, Article 22; Cal. Civ. Code Section 1798.185(a)(16) [5] See, e.g., GDPR, Article 28; Cal. Civ. Code Section 1798.140(ag)(1); 45 CFR Section 164.504(e)(Business Associate requirements under HIPAA) [6] Cal. Civ. Code Section 1798.99.31(a) [7] 15 U.S.C. Sec. 45(a)(1) [8] See https://www.ftc.gov/news-events/topics/protecting-consumer-privacy-security/privacy-security-enforcement for a list of FTC enforcement actions concerning privacy and cybersecurity [9] https://www.ftc.gov/news-events/news/press-releases/2021/01/california-company-settles-ftc-allegations-it-deceived-consumers-about-use-facial-recognition-photo [10] https://www.ftc.gov/news-events/news/press-releases/2022/03/ftc-takes-action-against-company-formerly-known-weight-watchers-illegally-collecting-kids-sensitive
0
A green speech bubble depicted on a dark background.

Chatbot Contracts: Enforcing TOS Agreements in Computer-Generated Conversations

[Although the rise of generative AI and large language models may seem novel, regulation of chatbots extends back years. To demonstrate, here is an article originally published by Metaverse Law’s founder and president, Lily Li, in the Spring 2017 Orange County ABTL Report.] Humanity has long imagined self-aware computers that can pilot our vehicles, purchase goods, and even sing songs for us, whether as the malevolent Hal in 2001: A Space Odyssey or the spunky Samantha in Her. Though fully sentient artificial intelligence is still science fiction (as far as we know), computer software has become “smart” enough to converse with us through text-based services like Facebook messenger, WhatsApp, or WeChat, or voice-operated services like Amazon’s Alexa or Apple’s Siri. As more e-commerce transactions are completed via these “chatbots” or “chatterbots” and away from browser-based websites, this begs the question: Will courts enforce the Terms of Service for chatbot contracts when the terms no longer appear on the same page – or even the same medium – as the transaction itself?
The Rise of Chatbots Consumer appetite for on-demand goods and services continues to grow, but at the same time, consumers are consolidating their online attention on a limited number of platforms. For social media and messenger services, this means Facebook. In 2016, 79% of online users were on Facebook, with 76% checking in daily. (Pew Research Center, Social Media Update 2016) Facebook’s Messenger had approximately 1 billion users, with WhatsApp and WeChat following closely behind. (Economist.com, “Bots, the next frontier”, April 9, 2016.) On the e-commerce and voice front, Amazon reigns supreme. Amazon accounted for 53 percent of all online sales growth in the United States in 2016, capitalizing on sales of its popular Echo and Echo Dot devices. (Slice Intelligence 2016). In light of these trends, e-retailers are increasingly leaving their own websites and apps, and developing custom, conversational chatbots to sell through these platforms. Internet Contracts 101: Mutual Assent and Notice The majority of e-commerce sales are regulated by online Terms of Service (“TOS”), also known as Terms and Conditions or Terms of Use (“TOU”). These internet contracts usually contain arbitration, forum, and venue provisions that govern the conduct of litigation. As a threshold matter, courts will only enforce these TOS if they find mutual assent to their provisions. In other words, consumers must be put on reasonable notice of online TOS, then provide objective outward manifestations of their agreement to the contract. Long v. Provide Commerce, Inc., 245 Cal.App.4th 855, 862 (2016). Courts have generally found mutual assent in “clickwrap” or “clickthrough” contracts, where the consumer clicks on an “I agree” or similar box or button, in tandem with a presentation of the TOS. In re Facebook Biometric Info. Privacy Litig., 185 F. Supp. 3d 1155, 1166 (N.D. Cal. 2016) (upholding California choice-of-law provision where plaintiffs clicked a box affirming they had read and agreed to the TOS, or where a separate plaintiff clicked a “Sign Up” button, with language immediately below stating that clicking the button constituted assent to the TOS). In contrast, courts are more hesitant to find mutual assent in situations where a link to the TOS appears on the online platform, but consumers do not affirmatively “click” to agree to those provisions. Compare Nguyen v. Barnes and Noble Inc., 763 F.3d 1171, 1178-1179 (9th Cir. 2014) (conspicuous hyperlink on every webpage not enough to demonstrate assent, where users were not prompted to take affirmative action) with Small Justice LLC v. Xcentric Ventures LLC, 99 F.Supp.3d 190, 197-98 (D. Mass 2015) (court distinguishes Nguyen and enforces TOS, where, in addition to hyperlink on each page, TOS were visible before the “continue” button on the final screen). For these “browsewrap” contracts, courts will analyze the conspicuousness of the TOS on the page, in context with the rest of the site or application, to determine whether “a reasonably prudent Internet consumer [is] on inquiry notice of the browsewrap agreement’s existence and contents.” Long, 245 Cal.App.4th at 123 (2016) (declining to impose TOS where hyperlink appeared in light green font on a page with light green background); see also Lee v. Intelius Inc., 737 F.3d 1254, 1257 (9th Cir. 2013) (TOS written in small, light grey print, next to a misleading “YES” button, caused customer confusion and was designed to deceive). Chatbots via Messenger: More of the Same Existing precedent on internet contracts is well equipped to handle text-based chatbots, and courts should be favorable to TOS presented conspicuously through such services. These chatbots have the ability to fashion contracts analogous to “clickwrap” or “clickthrough” agreements, by featuring conspicuous hyperlinks to online terms in a messenger window, and requiring consumers to affirmatively click to agree, type “YES” or “I Agree”, or words to that effect. The guided nature of text-based chatbots should in fact promote the enforceability of their TOS in court. Unlike a normal browser window, which may hide terms amidst other content, a messenger window limits consumer attention to a single step-by-step process. If done properly, consumers cannot proceed directly to an online shopping cart and bypass the terms completely. Instead, consumers can be required to outwardly manifest their assent to the TOS by typing or clicking for each transaction – a process favored by the courts. See Nguyen, 763 F.3d at 1177. Of course, by relying on third-party messenger platforms, chatbot services need to remain vigilant and ensure that TOS remain visible to consumers. In-messenger advertisements, large swathes of text, or strange fonts or colors imposed by a third-party platform may hide terms and render them unenforceable. For instance, in Specht v. Netscape Communications Corp., 306 F.3d 17, 23-30 (2d Cir. 2002), the court refused to enforce a software download TOS where consumers had the ability to click a “Download” button for free software, and consumers had to scroll down the page below the “Download” button to access a link to the TOS. Since the link was essentially subsumed under a “Download” splash screen, consumers had no inquiry notice of the TOS. Id. Similarly, consumers have all faced scenarios where third-party applications create splash screens above the content on websites, such as survey notices, advertisements, and videos, which may obscure small chatbot windows. Furthermore, chatbot services need to be aware of the TOS of third-party messenger platforms, which often require incorporation of specific licensing, privacy, and usage agreements within the chatbot terms. Here, clear access and delineation between these two competing sets of TOS is key, as the courts may refuse to enforce TOS where there is confusion as to which TOS apply, or refuse to enforce TOS that are only accessible through a series of pages and links. See Specht, 30 F.3d at 23-30; see also Cvent, Inc. v. Eventbrite, Inc. 739 F.Supp.2d 927 (E.D Va. 2010) (refusing to enforce TOS, where it was one of a series of links, and TOS page consisted of more links to other TOS). Voice Recognition – Hello World! For now, voice-based chatbots still rely on written TOS provided during online account sign up, which are subject to the same notice and assent requirements discussed above. Thus, when the TOS change for an underlying voice-activated device – or the third-party chatbot using such a device – consumers need to review, and generally provide affirmative assent, on a separate platform or application from the voice-activated service. Courts have often refused to enforce updated TOS, absent such express notice and affirmative assent from consumers, prior to ongoing use of an online service. See Douglas v. United States District Court, 495 F.3d 1062, 1066 (9th Cir. 2007) (court refuses to enforce arbitration agreement in revised TOS, holding that “[p]arties to a contract have no obligation to check the terms on a periodic basis to learn whether they have been changed by the other side”); Diverse Elements, Inc. v. Ecommerce, Inc., 5 F.Supp.3d 1378, 1381 (“[p]arties can…provide for modification in the contract and subsequently modify the contract with no new and independent consideration [Cite]…[t]his principle does not, however, allow parties to reserve the unfettered right to amend contracts without notice and at any unspecified time”); but see Klein v. Verizon Communications, Inc., 920 F.Supp.2d 670, 680-684 (E.D. Va. 2013) (upholding Verizon’s TOS where they provided that notice of revisions could be given by email, and new arbitration provisions were in fact provided by email). The ongoing requirement for consumers to access a separate device or application and “accept” new and revised TOS may become more onerous over time, however, as consumers move towards pure voice services through dozens (if not hundreds) of providers. Indeed, the whole impetus behind voice-based chatbots, as opposed to text-based solutions, is consumer desire for 24/7 on-demand services without the need to login or access physical devices. Consequently, courts will increasingly face scenarios where notices of new TOS or amended TOS are provided solely by voice. The chatbot will ask users to verbally agree to updated TOS, and then provide the terms separately by email or other text-based application. In these situations, it is not practicable to expect consumers to sit through an audio recitation of the TOS prior to purchase. Nor can TOS be provided concurrently with the verbal agreement, like “clickthrough” contracts, as there is no hyperlink, scroll-through, or pop-up window to view (absent VR/AR applications). Thus, in a pure voice paradigm, consumers will give – and will generally want to give – assent before they have an opportunity to review terms, if they review them at all. At first blush, this situation may appear to completely defeat the notice and mutual assent requirements for contract formation. Early case law surrounding “shrinkwrap” agreements, however, suggests that at least in certain jurisdictions, courts may still enforce these contracts. In ProCD, Inc. v. Zeidenberg, 86 F.3d 1447, 1451 (7th Cir. 1996), for example, Judge Easterbrook of the Seventh Circuit enforced the terms of a software license that was visible to plaintiff only after he had purchased a consumer package and downloaded the software. In enforcing this “shrinkwrap” agreement (named after the plastic cellophane around software boxes), the court noted that “[t]ransactions in which the exchange of money precedes the communication of detailed terms are common,” and quoted examples such as airline tickets, concert tickets, and standard warranties with consumer products. Id. at 1451. The court also recognized situations where “[a] customer may place an order by phone in response to a line item in a catalog or a review in a magazine…[t]here is no box; there is only a stream of electrons, a collection of information that includes data, an application program, instructions, many limitations…, and the terms of sale.” Id. at 1451-52. Judge Easterbrook reaffirmed this position in Hill v. Gateway 2000, Inc., 105 F.3d 1147, 1149 (7th Cir. 1997), by enforcing an arbitration agreement shipped in a computer box, where the consumer ordered the computer by phone and had the opportunity to return the computer in 30 days. The court noted, “[i]f the staff at the other end of the phone for direct-sales operations such as Gateway’s had to read the four-page statement of terms before taking the buyer’s credit card number, the droning voice would anesthetize rather than enlighten many potential buyers. Others would hang up in a rage over the waste of their time.” Id. The Seventh Circuit’s adoption of “order by phone now, see terms later” in ProCD and Hill seem like apt analogies for voice-based chatbots, where consumers verbally assent to an order, then view written terms at a later time. These cases, and their progeny, thus provide potential bases for enforcing TOS agreements for voice chatbots, so long as consumers have a reasonable opportunity to rescind the terms or refund the transaction later. See O’Quin v. Verizon Wireless,256 F.Supp.2d 512, 516 (M.D. La. 2003) (“[s]everal other federal and state courts have come to similar conclusions under similar factual scenarios [to Hill and ProCD], which were all premised on the consumer having the opportunity to return the product in order to avoid any term or condition that he found to be unacceptable”). Not all jurisdictions recognize the reasoning in Hill and ProCD, however. See Specht, 150 F.Supp.2d at 592; Klocek v. Gateway, Inc., 104 F.Supp.2d 1332, 1337 (D. Kan. 2000); Arizona Retail Sys., Inc. v. Software Link, Inc., 831 F.Supp. 759 (D.Ariz. 1993) (license agreement shipped with computer software not part of agreement). The Tenth Circuit, for instance, has stated outright that Kansas law rejects the reasoning of ProCD, holding that “a seller’s later-arriving written contract constitutes at most only a proposal to modify a preexisting oral contract, and […] a buyer’s assent to the proposed modification won’t be inferred simply from the buyer’s continuing the preexisting oral contract.” Howard v. Ferrellgas Partners, L.P., 748 F.3d 975, 982 (10th Cir. 2014). Consequently, chatbot providers must tread carefully before offering pure voice-based TOS agreements. Chatbots and Policy: Keeping it Simple Smart chatbots have immense potential to make consumers’ lives easier. Instead of navigating through endless webpages, dense text, and the inevitable clickbait ads, chatbots can provide an intuitive, conversational platform for e-commerce. Given the many consumer benefits of chatbot technology, everyone will benefit from clear case law governing the enforceability of chatbot contracts, and prior “clickthrough” and “shrinkwrap” doctrines provide useful guidance for the courts. *Disclaimer* This article is not legal advice or legal opinion, and the contents are intended for general informational purposes only. Circumstances may differ from situation to situation. All legal and other issues must be independently researched.
0

Metaverse Law’s Lily Li to guest star on Threat Watch podcast to discuss risks of ChatGPT, generative AI, and LLMs

Near the end of 2022, generative AI models became something of a sensation. Art-based models like Midjourney, DALL-E, and Stable Diffusion threw the art world into a panic, prompting companies to ban AI-generated art.[1] Models like ChatGPT—and its underlying GPT-3.5 and GPT-4 LLMs—seemingly invaded every social sphere, from academia[2] to big tech,[3] and prompted many to start asking, “Will AI replace us?”[4] Given all this buzz around generative AI and LLMs, it’s only natural to consider the IT and security risks stemming from these emerging technologies. Afterall, there have been numerous recorded instances of actors using ChatGPT to build malware,[5] to improve malware,[6] to send phishing emails,[7] and more. To discuss these topics, Metaverse Law’s founder Lily Li will join host Dr. Rebecca Wynn on BrightTALK’s Threat Watch podcast to discuss the many issues, risks, and concerns arising out of the use of AI. WHAT: Metaverse Law’s founder Lily Li will join host Dr. Rebecca Wynn on the Threat Watch podcast to discuss AI, chatbots, LLMs, and more. WHEN: March 30, 2023 — 12:00 pm ET WHERE: Online (with free registration) TOPICS:
  • Data leaking and misuse in the AI supply chain.
  • Data transfer issues resulting from the use of AI.
  • IT and cyber security concerns.
  • Social engineering stemming from AI.
  • And more!
Whether you are currently using or thinking about using AI in your business, you do not want to miss Lily’s discussion on the risks and issues arising from this technology.
[1] https://brushwarriors.com/art-websites-that-ban-ai/ [2] https://www.tidio.com/blog/ai-in-education/ [3] https://www.zdnet.com/article/how-to-use-chatgpt-to-write-code/ [4] https://www.forbes.com/sites/robtoews/2021/02/15/artificial-intelligence-and-the-end-of-work/?sh=75edd9c456e3 [5] https://www.hackread.com/chatgpt-blackmamba-malware-keylogger/ [6] https://blog.checkpoint.com/2023/02/07/cybercriminals-bypass-chatgpt-restrictions-to-generate-malicious-content/ [7] https://research.checkpoint.com/2022/opwnai-ai-that-can-save-the-day-or-hack-it-away/
0
Graphic depicting the three phases of analysis under NIST's AI RMF: map, measure, manage.

Creating trustworthy AI and reducing liability with NIST’s AI Risk Management Framework

On January 26, 2023, the National Institute of Standards and Technology (NIST) released the first version of the Artificial Intelligence Risk Management Framework (AI RMF).[1] The AI RMF is a voluntary resource meant to help organizations “manage the many risks of AI and promote trustworthy and responsible development and use of AI systems.”[2] To support the goal of the AI RMF, NIST supplemented its release with a companion NIST AI RMF Playbook,[3] AI RMF Explainer Video,[4] an AI RMF Roadmap,[5] AI RMF Crosswalk,[6] and statements from organization and individuals interested in the success of the AI RMF.[7] Together, these resources provide organizations with a comprehensive toolbox for identifying and managing AI risks. Given the growing regulatory interest in scrutinizing AI, these resources — although voluntary to use — provide important insights into what regulators may or may not want to see in AI products, services, and systems.

Background

The US Federal Government has long recognized the need for AI regulation. In 2016, the National Science and Technology Council produced a report stating that “the approach to regulation of AI-enabled products to protect public safety should be informed by assessment of the aspects of risk.”[8] In 2018, President Donald Trump signed a law establishing the National Security Commission on Artificial Intelligence to consider how to defend against AI threats and promote AI innovation.[9] In 2019, following Executive Order 13859,[10] the White House’s Office of Science and Technology Policy released guidance detailing the ten principles that Federal agencies should consider when determining how to regulate AI.[11] In response, NIST released a position paper, which called for US agencies to create globally relevant, non-discriminatory AI standards. Recognizing that AI has the potential to transform every sector of the US economy and society, Congress passed the National AI Initiative Act of 2020, which established the National Artificial Intelligence Initiative (NAIA), and directed NIST to “develop voluntary standards for artificial intelligence systems.”[12] On July 29, 2021, NIST issued a Request for Information to Help Develop an AI Risk Management Framework,[13] in which NIST asked individuals, groups, and organizations to submit comment on the goals of the AI RMF and on how those goals should be achieved. On October 15, 2021, NIST published a summary analysis of those comments,[14] and on December 13, 2021, the agency published a concept paper incorporating input from the initial Request for Information.[15] NIST released a draft AI RMF on March 17, 2022,[16] but, based on comments received during a NIST workshop held that same month,[17] the agency released a modified second draft on August 18, 2022,[18] and held another workshop in October 2022.[19] Four months later, NIST released the first version of the AI RMF.

Seven characteristics of trustworthy AI

As a flexible framework designed to adapt to a wide range of systems, products, and organizations, the AI RMF does not prescribe specific technical requirements that must be satisfied before an AI is considered trustworthy. Instead, the AI RMF provides a list of characteristics that must be balanced “based on the AI system’s context of use.”[20] These characteristics are:
  1. Valid and reliable
    1. Valid: Confirmation, though the provision of objective evidence, that the requirements for the AI’s specific intended use or application have been fulfilled. (ISO 9000:2015.)
    2. Reliable: The ability of AI system to perform as required, without failure, for a given time interval, under given conditions, including the entire lifetime of the system. (ISO/IEC TS 5723:2022.)
    3. Accurate: The closeness of the AI system’s results of observations, computations, or estimates to the true values or the values accepted as being true. (ISO/IEC TS 5723:2022.)
    4. Robust / Generalized: The ability of an AI system to maintain its level of performance under a variety of circumstances, which includes performing in ways that minimize potential harm to people if it is operating in an unexpecting setting. (ISO/IEC TS 5723:2022.)
  2. Safe
    1. AI systems should not, under defined conditions, lead to a state in which human life, health, property, or the environment is endangered. (ISO/IEC TS 5723:2022.)
  3. Secure and resilient
    1. Secure: AI systems should maintain confidentiality, integrity, and availability through protection mechanisms that prevent unauthorized access and use. (NIST Cybersecurity Framework and Risk Management Framework.)
    2. Resilient: AI systems, as well as the ecosystems in which they are deployed, should withstand unexpected adverse events or unexpected changes in their environment or use — or if they can maintain their functions and structure in the face of internal and external change and degrade safely and gracefully when this is necessary. (ISO/IEC TS 5723:2022.)
  4. Accountable and transparent
    1. Transparent: Information about an AI system and its outputs should be available to individuals interacting with such a system, regardless of whether they are even aware that they are doing so, and be tailored to the role or knowledge of AI actors or individuals interacting with or using the AI system.
    2. Accountable: AI systems should incorporate actionable redressability related to AI system outputs that are incorrect or otherwise lead to negative impacts.
  5. Explainable and interpretable
    1. Explainable: The AI system should describe how the AI system functions, with descriptions tailored to individual differences such as the user’s role, knowledge, and skill level.
    2. Interpretable: The AI system should communicate a description of why an AI system made a particular prediction or recommendation. (“Four Principles of Explainable Artificial Intelligence” and “Psychological Foundations of Explainability and Interpretability in Artificial Intelligence.”[21])
  6. Privacy-enhanced
    1. Privacy values such as anonymity, confidentiality, and control should guide choices for AI system design, development, and deployment, but privacy-enhancing technologies (PETs) may be needed to support privacy-enhanced AI design.
  7. Fair — with harmful bias managed
    1. Fair: AI systems should incorporate equality and equity by addressing issues such as harmful bias and discrimination, which includes taking into consideration cultural context and demographic differences.
    2. With harmful bias managed: AI systems should consider and manage three major categories of AI bias:
      1. Systemic: Bias found in the AI datasets, the organizational norms, practices, and processes across the AI lifecycle, and the broader society that uses the AI system.
      2. Computational and statistical: Bias found in AI datasets and algorithmic processes, and often stems from systematic errors due to non-representative samples.
      3. Human-cognitive: Bias relating to how an individual or group perceives AI system information to make a decision or fill in missing information, or how humans think about purposes and functions of an AI system.

Practical benefit of complying with the AI RMF

As a voluntary framework, the AI RMF does not mandate compliance with its principles; however, as the NIST Cybersecurity Framework demonstrates, voluntary compliance may help shield an organization from legal risks. The NIST Cybersecurity Framework offers a risk-based approach to cybersecurity and a methodology for developing a comprehensive information security program. Like the AI RMF, the Cybersecurity Framework is voluntary, and therefore does not form the basis for any regulatory action. Yet, if a cybersecurity incident occurs, an organization that has implemented the Cybersecurity Framework can use their adherence in their favor. For example, if a regulator alleges the organization was negligent in its cybersecurity practices, the organization can rebut the allegations by demonstrating that its program was designed in accordance with the Cybersecurity Framework and therefore was reasonably designed to counter foreseeable risks. Compliance with the AI RMF may produce similar benefits, given that NIST created the AI RMF for AI industry stakeholders to “cultivate trust in the design, development, use, and evaluation of AI technologies and systems in ways that enhance economic security and improve qualify of life.”[22]
[1] https://www.nist.gov/itl/ai-risk-management-framework [2] https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf [3] https://pages.nist.gov/AIRMF/ [4] https://www.nist.gov/video/introduction-nist-ai-risk-management-framework-ai-rmf-10-explainer-video [5] https://www.nist.gov/itl/ai-risk-management-framework/roadmap-nist-artificial-intelligence-risk-management-framework-ai [6] https://www.nist.gov/itl/ai-risk-management-framework/crosswalks-nist-artificial-intelligence-risk-management-framework [7] https://www.nist.gov/itl/ai-risk-management-framework/perspectives-about-nist-artificial-intelligence-risk-management [8] https://obamawhitehouse.archives.gov/sites/default/files/whitehouse_files/microsites/ostp/NSTC/preparing_for_the_future_of_ai.pdf [9] https://www.govinfo.gov/content/pkg/COMPS-15483/uslm/COMPS-15483.xml; https://www.nscai.gov/ [10] https://trumpwhitehouse.archives.gov/presidential-actions/executive-order-maintaining-american-leadership-artificial-intelligence/ [11] https://www.whitehouse.gov/wp-content/uploads/2020/01/Draft-OMB-Memo-on-Regulation-of-AI-1-7-19.pdf [12] https://www.congress.gov/bill/116th-congress/house-bill/6216 [13] https://www.federalregister.gov/documents/2021/07/29/2021-16176/artificial-intelligence-risk-management-framework [14] https://www.nist.gov/system/files/documents/2021/10/15/AI%20RMF_RFI%20Summary%20Report.pdf [15] https://www.nist.gov/system/files/documents/2021/12/14/AI%20RMF%20Concept%20Paper_13Dec2021_posted.pdf [16] https://www.nist.gov/system/files/documents/2022/03/17/AI-RMF-1stdraft.pdf [17] https://www.nist.gov/news-events/events/2022/03/building-nist-ai-risk-management-framework-workshop-2 [18] https://www.nist.gov/system/files/documents/2022/08/18/AI_RMF_2nd_draft.pdf [19] https://www.nist.gov/news-events/events/2022/10/building-nist-ai-risk-management-framework-workshop-3 [20] https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf [21] https://www.nist.gov/artificial-intelligence/ai-fundamental-research-explainability [22] https://www.nist.gov/itl/ai-risk-management-framework/ai-risk-management-framework-faqs
1 4 5 6 7