The Use of Artificial Intelligence Systems

SCOTT A. WHITE COMMISSIONER OF INSURANCE STATE CORPORATION COMMISSION BUREAU OF INSURANCE July 22, 2024 P.O. BOX 1157 RICHMOND, VIRGINIA 23218 1300 E. MAIN STREET RICHMOND, VIRGINIA 23219 TELEPHONE: (804) 371-9741 www.sec.virginia.gov/boi Administrative Letter 2024-01 TO: All Companies Licensed to Conduct the Business of Insurance in Virginia and All Interested Parties RE: The Use of Artificial Intelligence Systems The Virginia Bureau of Insurance ("Bureau") reminds all companies licensed to conduct the business of insurance in the Commonwealth ("Insurers") that decisions, conduct, or actions impacting consumers that are made or supported by advanced analytical and computational technologies, including Artificial Intelligence ("Al") Systems (as defined below), must comply with all applicable insurance laws and regulations. This includes, but is not limited to, those laws and regulations that address unfair trade practices, unfair claim settlement practices, and unfair discrimination. This Administrative Letter sets forth the Bureau's expectations as to how Insurers will govern and manage the risk from the development, acquisition, and use of Al technologies, including Al Systems. This Administrative Letter also advises Insurers of the type of information and documentation that the Bureau may request during an investigation or examination of any Insurer regarding its use of such technologies and Al Systems. SECTION 1: INTRODUCTION, BACKGROUND, AND LEGISLATIVE AUTHORITY Background Al is transforming the insurance industry. Al techniques are deployed across all stages of the insurance life cycle, including product development, marketing, sales and distribution, underwriting and pricing, policy servicing, claim management, and fraud detection. Al may facilitate the development of innovative products, improve consumer interface and service, simplify and automate processes, and promote efficiency and accuracy. However, Al, including Al Systems, can present unique risks to consumers, including the potential for inaccuracy, unfair discrimination, data vulnerability, and lack of transparency and explainability. Insurers should take actions to understand and minimize these risks. The Bureau encourages the development and use of innovation and Al Systems that contribute to safe and stable insurance markets and ensure that citizens of the Commonwealth are provided with access to adequate and reliable insurance protection. However, the Bureau expects that decisions made and actions taken by Insurers using Al Systems will comply with all applicable state and federal laws and regulations. Administrative Letter 2024-01 July 22, 2024 Page 2 of 10 The Bureau recognizes the Principles on Artificial Intelligence that the NAIC adopted in 2020 as an appropriate source of guidance for Insurers as they develop and use Al Systems. Those principles emphasize the importance of the fairness and ethical use of Al; accountability; compliance with state laws and regulations; transparency; and a safe, secure, fair, and robust system. These foundational principles should guide Insurers in their development and use of Al Systems and underlie the expectations set forth in this Administrative Letter. Legislative Authority The regulatory expectations and oversight considerations set forth in Section 3 and Section 4 of this Administrative Letter are grounded in the laws and regulations of the Commonwealth. Unfair Trade Practices/Claim Settlement/Discrimination. Insurers are expected to adopt practices, including governance frameworks and risk management protocols, that are designed to ensure that the use of Al Systems does not result in: 1) unfair trade practices, as described in Chapter 5 of Title 38.2 of the Code of Virginia ("Code"); 2) unfair claim settlement practices, as described in Section 38.2-510 of the Code and 14 VAC 5-400 of the Virginia Administrative Code; or 3) unfair discrimination, as described in Sections 38.2-508, 38.2-508.1, and 38.2-508.2 of the Code. Actions taken by Insurers in the Commonwealth must not violate these provisions, regardless of the methods the Insurer used to determine or support its actions. Corporate Governance. Insurers are required to report on governance practices and provide a summary of the Insurer's corporate governance structure, policies, and practices pursuant to the requirements of Article 5.2 of Chapter 13 of Title 38.2 of the Code and 14 VAC 5- 265 of the Virginia Administrative Code. These requirements apply to elements of the Insurer's corporate governance framework that address the Insurer's use of Al Systems to support actions and decisions that impact consumers. Rating. Insurers must comply with all insurance laws and regulations regarding rates, rating plans, rating rules, practices and standards (referred to in this Administrative Letter as the "Rating Laws"). For example, the Rating Laws found in Title 38.2 of the Code and Title 14 of the Virginia Administrative Code mandate that insurance rates are not excessive, inadequate, or unfairly discriminatory, and do not discriminate based on protected classes. The Rating Laws apply regardless of the methodology that the Insurer used to develop rates, rating rules, and rating plans. That means that an Insurer is responsible for ensuring that rates, rating rules, and rating plans developed using Al techniques and Predictive Models that rely on data and Machine Learning do not result in rates or practices that violate the Rating Laws. Market Conduct. Pursuant to Sections 38.2-200, 38.2-515, 38.2-1317.1, and 38.2-1317.2 of the Code, among other provisions, an Insurer's conduct in the Commonwealth, including its use of Al Systems to make or support actions and decisions that impact consumers, is subject to examination and investigation, including market conduct actions. Section 4 of this Administrative Letter provides guidance on the kinds of information and documents that the Bureau may request in an Al-focused examination or investigation, including a market conduct action. SECTION 2: DEFINITIONS For the purposes of this Administrative Letter the following terms are defined: Administrative Letter 2024-01 July 22, 2024 Page 3 of 10 "Adverse Consumer Outcome" refers to a decision by an Insurer that adversely impacts the consumer in a manner that violates applicable law or regulation. "Algorithm" means a clearly specified mathematical process for computation; a set of rules that, if followed, will give a prescribed result. "Al System" is a machine-based system that can, for a given set of objectives, generate outputs such as predictions, recommendations, content (such as text, images, videos, or sounds), or other output influencing decisions made in real or virtual environments. Al Systems are designed to operate with varying levels of autonomy. "Artificial Intel ligence (Al)" refers to a branch of computer science that uses data processing systems that perform functions normally associated with human intelligence, such as reasoning, learning, and self-improvement, or the capability of a device to perform functions that are normally associated with human intelligence such as reasoning, learning, and self-improvement. This definition considers machine learning to be a subset of artificial intelligence. "Degree of Potential Harm to Consumers" refers to the severity of adverse economic impact that a consumer might experience as a result of an Adverse Consumer Outcome. "Generative Artificial Intel ligence (Generative Al)" refers to a class of Al Systems that generate content in the form of data, text, images, sounds, or video, that is similar to, but not a direct copy of, pre-existing data or content. "Machine Learning (ML)" refers to a field within artificial intelligence that focuses on the ability of computers to learn from provided data without being explicitly programmed. "Model Drift" refers to the decay of a model's performance over time arising from underlying changes such as the definitions, distributions, and/or statistical properties between the data used to train the model and the data on which it is deployed. "Predictive Model" refers to the mining of historic data using algorithms and/or machine learning to identify patterns and predict outcomes that can be used to make or support the making of decisions. "Third Party" means an organization other than the Insurer that provides services, data, or other resources related to Al. SECTION 3: REGULATORY GUIDANCE AND EXPECTATIONS Decisions made, and conduct and actions taken by Insurers using Al Systems must comply with all legal and regulatory requirements. Compliance with these requirements is mandated regardless of the tools and methods Insurers use to make such decisions and support their conduct and actions. In the absence of proper controls, Al has the potential to increase the risk of inaccurate, arbitrary, capricious, or unfairly discriminatory outcomes for consumers and violate legal requirements. Therefore, it is important that Insurers adopt and implement controls specifically related to their use of Al that are designed to understand and eliminate the risk of Adverse Consumer Outcomes. Administrative Letter 2024-01 July 22, 2024 Page 4 of 10 All Insurers are expected to develop, implement, and maintain a written program (an "AIS Program") for the responsible use of Al Systems that make or support decisions, conduct, and actions of the Insurer. The Bureau recognizes that robust governance, risk management controls, and internal audit functions play a core role in understanding and mitigating risk. Insurers' due diligence of decisions, conduct, and actions driven by Al Systems will lessen the risk of violating unfair trade practice, unfair claim settlement, and unfair discrimination laws and other applicable legal requirements. The Bureau also strongly encourages the development and use of verification and testing methods to identify errors and bias in Predictive Models and Al Systems, as well as the potential for unfair discrimination in the decisions and outcomes resulting from the use of Predictive Models and Al Systems. The controls and processes that an Insurer adopts and implements as part of its AIS Program should be reflective of, and commensurate with, the Insurer's own assessment of the degree and nature of risk posed to consumers by the Al Systems that it uses, considering: (i) the nature of the decisions being made, informed, or supported using the Al System; (ii) the type and Degree of Potential Harm to Consumers resulting from the use of Al Systems; (iii) the extent to which humans are involved in the final decision-making process; (iv) the transparency and explainability of outcomes to the impacted consumer and regulator; and (v) the extent and scope of the Insurer's use or reliance on data, Predictive Models, and Al Systems from Third Parties. Similarly, controls and processes should be commensurate with both the risk of Adverse Consumer Outcomes and the Degree of Potential Harm to Consumers. As discussed in Section 4, the decisions, conduct, and actions made as a result of an Insurer's use of Al Systems are subject to the Bureau's examination to determine compliance with all applicable existing legal requirements governing the conduct of the Insurer. AIS Program Guidelines 1.0 General Guidelines 1.1 The AIS Program should be designed to eliminate the risk that the Insurer's use of an Al System will result in Adverse Consumer Outcomes. 1.2 The AIS Program should address governance, risk management controls, and internal audit functions. 1.3 The AIS Program should vest responsibility for the development, implementation, monitoring, and oversight of the AIS Program and for setting the Insurer's strategy for Al Systems with senior management accountable to the Board of Directors ("Board") (or similar body) or an appropriate committee of the Board. 1.4 The AIS Program should be tailored to and proportionate with the Insurer's use and reliance on Al and Al Systems. Controls and procedures should be focused on the elimination of Adverse Consumer Outcomes and the scope of the controls and procedures applicable to a given Al System use case should reflect and align with the Degree of Potential Harm to Consumers. 1.5 The AIS Program may be independent of or part of the Insurer's existing Enterprise Risk Management program. The AIS Program may adopt, incorporate, or rely upon, in whole or in part, a framework or standards developed by an official third-party standard organization, such Administrative Letter 2024-01 July 22, 2024 Page 5 of 10 as the National Institute of Standards and Technology ("NIST") Artificial Intelligence Risk Management Framework. 1.6 The AIS Program should address the use of Al Systems across the insurance life cycle, including areas such as product development and design, marketing, use, underwriting, rating and pricing, case management, claim administration and payment, and fraud detection. 1.7 The AIS Program should address all phases of an Al System's life cycle, including design, development, validation, implementation (both systems and business), use, on-going monitoring and testing, updating and retirement. 1.8 The AIS Program should address the Al Systems used with respect to regulated insurance practices whether developed by the Insurer or a Third-Party vendor. 1.9 The AIS Program should include processes and procedures providing notice to affected consumers that Al Systems are in use and provide access to appropriate levels of information based on the phase of the insurance life cycle in which the Al Systems are being used. 2.0 Governance The AIS Program should include a governance framework for the oversight of Al Systems used by the Insurer. Governance should prioritize transparency, fairness, and accountability in the design and implementation of the Al Systems, recognizing that proprietary and trade secret information must be protected. An Insurer may consider adopting new internal governance structures or rely on the Insurer's existing governance structures; however, in developing its governance framework, the Insurer should address the following items: 2.1 The policies, processes, and procedures, including risk management and internal controls, to be followed at each stage of an Al System life cycle, from proposed development to retirement. 2.2 The requirements adopted by the Insurer to document compliance with the AIS Program policies, processes, procedures, and standards. Documentation requirements should be developed with Section 4 of this Administrative Letter in mind. 2.3 The Insurer's internal Al System governance accountability structure, such as: a) The formation of centralized, federated, or otherwise constituted committees comprised of representatives from appropriate disciplines and units within the Insurer, such as business units, product specialists, actuarial, data science and analytics, underwriting, claims, compliance, and legal. b) Scope of responsibility and authority, chains of command, and decisional hierarchies. c) The independence of decision-makers and lines of defense at successive stages of the Al System life cycle. d) Monitoring, auditing, escalation, and reporting protocols and requirements. Administrative Letter 2024-01 July 22, 2024 Page 6 of 10 e) Development and implementation of ongoing training and supervision of personnel. 2.4 Specifically with respect to Predictive Models: the Insurer's processes and procedures for designing, developing, verifying, deploying, using, updating, and monitoring Predictive Models, including a description of methods used to detect and address errors, performance issues, outliers, bias, or unfair discrimination in the insurance practices resulting from the use of the Predictive Model. 3.0 Risk Management and Internal Controls The AIS Program should document the Insurer's risk identification, mitigation, and management framework and internal controls for Al Systems generally and at each stage of the Al System life cycle. Risk management and internal controls should address the following items: 3.1 The oversight and approval process for the development, adoption, or acquisition of Al Systems, as well as the identification of constraints and controls on automation and design to align and balance function with risk. 3.2 Data practices and accountability procedures, including data currency, lineage, quality, integrity, bias analysis and minimization, and suitability. 3.3 Management and oversight of Predictive Models (including algorithms), including: a) Inventories and descriptions of the Predictive Models. b) Detailed documentation of the development and use of the Predictive Models. c) Assessments such as interpretability, repeatability, robustness, regular tuning, reproducibility, traceability, model drift, and the auditability of these measurements where appropriate. 3.4 Validating, testing, and retesting as necessary to assess the generalization of Al System outputs upon implementation, including the suitability of the data used to develop, train, validate and audit the model. Validation can take the form of comparing model performance on unseen data available at the time of model development to the performance observed on data post-implementation, measuring performance against expert review, or other methods. 3.5 The protection of nonpublic information, particularly personal information and privileged information, including unauthorized access to the Predictive Models themselves. 3.6 Data and record retention. 3.7 Specifically with respect to Predictive Models: a narrative description of the model's intended goals and objectives and how the model is developed and validated to ensure that the Al Systems that rely on such models correctly and efficiently predict or implement those goals and objectives. Administrative Letter 2024-01 July 22, 2024 Page 7 of 10 4.0 Third-Party Al Systems and Data Each AIS Program should address the Insurer's process for acquiring, using, or relying on: (i) Third-Party data to develop Al Systems; and (ii) Al Systems developed by a Third Party, which may include, as appropriate, the establishment of standards, policies, procedures, and protocols relating to the following considerations: 4.1 Due diligence and the methods employed by the Insurer to assess the Third Party and its data or Al Systems acquired from the Third Party to ensure that decisions made or supported from such Al Systems that could lead to Adverse Consumer Outcomes will meet the legal requirements imposed on the Insurer itself. 4.2 Where appropriate and available, the inclusion of terms in contracts with Third Parties that: a) Provide audit rights and/or entitle the Insurer to receive audit reports by qualified auditing entities. b) Require the Third Party to cooperate with the Insurer with regard to regulatory inquiries and investigations related to the Insurer's use of the Third Party's product or services. 4.3 The performance of contractual rights regarding audits and/or other activities to confirm the Third Party's compliance with contractual and, where applicable, regulatory requirements. SECTION 4: REGULATORY OVERSIGHT AND EXAMINATION CONSIDERATIONS The Bureau's regulatory oversight of Insurers includes oversight of an Insurer's conduct in the Commonwealth, including its use of Al Systems to make or support decisions that affect consumers. Regardless of the existence or scope of a written AIS Program, in the context of a rate filing, examination, investigation, inquiry, or market conduct action, an Insurer can expect to be asked about its development, deployment, and use of Al Systems, or any specific Predictive Model, Al System or application and its outcomes (including Adverse Consumer Outcomes) from the use of those Al Systems, as well as any other information or documentation deemed relevant by the Bureau. Insurers should expect those inquiries to include the Insurer's governance framework, risk management, and internal controls (including the considerations identified in Section 3 of this Administrative Letter). In addition to conducting a review of any of the items listed in this Administrative Letter, the Bureau may also ask questions regarding any specific Predictive Model, Al System, or its application, including requests for the following types of information and/or documentation: 1. Information and Documentation Relating to Al System Governance, Risk Management, and Use Protocols 1.1. Information and documentation related to or evidencing the Insurer's AIS Program, including: Administrative Letter 2024-01 July 22, 2024 Page 8 of 10 a) The written AIS Program b) Information and documentation relating to or evidencing the adoption of the AIS Program c) The scope of the Insurer's AIS Program, including any Al Systems and technologies not included in or addressed by the AIS Program. d) How the AIS Program is tailored to and proportionate with the Insurer's use and reliance on Al Systems, the risk of Adverse Consumer Outcomes, and the Degree of Potential Harm to Consumers. e) The policies, procedures, guidance, training materials, and other information relating to the adoption, implementation, maintenance, monitoring, and oversight of the Insurer's AIS Program, including: i. Processes and procedures for the development, adoption, or acquisition of Al Systems, such as: (1) Identification of constraints and controls on automation and design. (2) Data governance and controls, any practices related to data lineage, quality, integrity, bias analysis and minimization, suitability, and Data Currency. ii. Processes and procedures related to the management and oversight of Predictive Models, including measurements, standards, or thresholds adopted or used by the Insurer in the development, validation, and oversight of models and Al Systems iii. Protection of nonpublic information, particularly personal information and privileged information, including unauthorized access to Predictive Models themselves. 1.2 Information and documentation relating to the Insurer's pre-acquisition/pre-use diligence, monitoring, oversight, and auditing of data or Al Systems developed by a Third Party. 1.3 Information and documentation relating to or evidencing the Insurer's implementation and compliance with its AIS Program, including documents relating to the Insurer's monitoring and audit activities respecting compliance, such as: a) Documentation relating to or evidencing the formation and ongoing operation of the Insurer's coordinating bodies for the development, use, and oversight of Al Systems. b) Documentation related to data practices and accountability procedures, including data lineage, quality, integrity, bias analysis and minimization, suitability, and Data Currency. c) Management and oversight of Predictive Models and Al Systems, including: Administrative Letter 2024-01 July 22, 2024 Page 9 of 10 i. The Insurer's inventories and descriptions of Predictive Models, and Al Systems used by the Insurer to make or support decisions, conduct, and actions that can result in Adverse Consumer Outcomes. ii. As to any specific Predictive Model or Al System that is the subject of investigation or examination: (1) Documentation of compliance with all applicable Al Program policies, protocols, and procedures in the development, use, and oversight of Predictive Models and Al Systems deployed by the Insurer. (2) Information about data used in the development and oversight of the specific model or Al System, including the data source, provenance, data lineage, quality, integrity, bias analysis and minimization, suitability, and Data Currency. (3) Information related to the techniques, measurements, thresholds, and similar controls used by the Insurer. d) Documentation related to validation, testing, and auditing, including evaluation of Model Drift to assess the reliability of outputs that influence the decisions made based on Predictive Models. Note that the nature of validation, testing, and auditing should be reflective of the underlying components of the Al System, whether based on Predictive Models or Generative Al. 2. Third-Party Al Systems and Data In addition, if the rate filing, investigation, or examination concerns data, Predictive Models, or Al Systems collected or developed in whole or in part by Third Parties, the Insurer should also expect the Bureau to request the following additional types of information and documentation. 2.1 Due diligence conducted on Third Parties and their data, models, or Al Systems. 2.2 Contracts with Third-Party Al System, model, or data vendors, including terms relating to representations, warranties, data security and privacy, data sourcing, intellectual property rights, confidentiality and disclosures, and/or cooperation with regulators. 2.3 Audits and/or confirmation processes performed regarding Third-Party compliance with contractual and, where applicable, regulatory obligations. 2.4 Documentation pertaining to validation, testing, and auditing, including evaluation of Model Drift. The Bureau recognizes that Insurers may demonstrate their compliance with the laws that regulate their conduct in the Commonwealth in their use of Al Systems through alternative means, including through practices that differ from those described in this Administrative Letter. The goal of the Administrative Letter is not to prescribe specific practices or to prescribe specific documentation requirements. Rather, the goal is to ensure that Insurers in the Commonwealth Administrative Letter 2024-01 July 22, 2024 Page 10 of 10 are aware of the Bureau's expectations as to how Al Systems will be governed and managed and of the kinds of information and documentation about an Insurer's Al Systems that the Bureau expects an Insurer to produce when requested. As in all cases, examination, investigations, and market conduct actions may be performed using procedures that vary in nature, extent, and timing in accordance with regulatory judgment. Work performed may include inquiry, examination of company documentation, or any of the continuum of market actions described in the NAIC's Market Regulation Handbook. These activities may involve the use of contracted specialists with relevant subject matter expertise. Nothing in this Administrative Letter limits the authority of the Bureau to conduct any investigation, examination, or enforcement action relative to any act or omission of any Insurer. Any questions concerning this Administrative Letter may be addressed to the Bureau's Division of Innovative Solutions & Strategies at lnnovativeSolutions_Strategies@scc.virginia. gov. Cordially, Scott A. White Commissioner of Insurance