• The Colorado Division of Insurance has issued a final regulation requiring life insurers authorized to do business in Colorado to implement AI governance and risk-management measures that are reasonably designed to prevent unfair discrimination in the use of AI models that use external consumer data and information sources. The regulation has undergone two comment periods, and the final version becomes effective on November 14, 2023.
• In this Debevoise In Depth, we discuss the key requirements of the final regulation, highlight the most notable changes from the previous version and discuss how companies can best prepare for compliance. Most significantly, the final regulation now requires insurers to remediate any detected instances of unfair discrimination… continue reading on the Debevoise Data Blog here.

On September 21, 2023, the Colorado Division of Insurance (the “DOI”) released its Final Governance and Risk Management Framework Requirements for Life Insurers’ Use of External Consumer Data and Information Sources, Algorithms, and Predictive Models (the “Final Regulation”). As discussed below, the Final Regulation (which becomes effective on November 14, 2023) reflects several small changes from the previous version of the regulation that was released on May 26, 2023 (the “Draft Regulation”). A redline reflecting these changes can be found here.

The most substantive change is the requirement that insurers must remediate any detected unfair discrimination. This change is especially significant in light of the DOI’s release of its draft regulation on Quantitative Testing for Unfairly Discriminatory Outcomes for Algorithms and Predictive Models Used for Life Insurance Underwriting (the “Draft Testing Regulation”) on September 28, 2023, which requires insurers to estimate the race and ethnicity of all proposed insureds that have applied for life insurance coverage and then conduct detailed quantitative testing of models that use external consumer data and information sources (“ECDIS”) for potential bias. The Testing Regulation provides that certain results of that prescribed testing methodology will be deemed to be unfairly discriminatory and thereby require the insurer to “immediately take reasonable steps . . . to remediate the unfairly discriminatory outcome . . .” We will be writing much more about our concerns over the Draft Testing Regulation in the coming weeks.

In this Debevoise Data Blog Post, we discuss the Final Regulation, how it differs from the Draft Regulation, and what companies should be doing now to prepare for compliance.

Overview of the Revised Regulation

Like the Revised Regulation, the Final Regulation requires life insurers that are authorized to do business in Colorado to implement AI governance and risk management measures that are reasonably designed to prevent unfair discrimination in the use of ECDIS, as well as algorithms and predictive models (“Models”) that use ECDIS, in insurance practices.

Definition of ECDIS

The Final Regulation expands what is meant by ECDIS by adding “biometric data” to the definition that appeared in the Revised Regulation:

ECDIS means, for the purposes of this regulation, a data or an information source that is used by a life insurer to supplement or supplant traditional underwriting factors or other insurance practices or to establish lifestyle indicators that are used in insurance practices. This term includes credit scores, social media habits, locations, purchasing habits, home ownership, educational attainment, licensures, civil judgments, court records, occupation that does not have a direct relationship to mortality, morbidity or longevity risk, consumer-generated Internet of Things data, biometric data, and any insurance risk scores derived by the insurer or third party from the above listed or similar data and/or information sources. Section 4(C).

The Final Regulation also adds the following definition for “Internet of Things”:

“Internet of Things” means, for the purposes of this regulation, networks of physical objects embedded with sensors, software, and other technologies for the purposes of collecting, transmitting, and exchanging data over the Internet. This definition does not apply to devices that require direct human intervention for data collection and exchange.

The Final Regulation does not include a definition of “Traditional Underwriting Factors,” but that term is defined in the Draft Testing Regulation to include:

• information provided by or on behalf of the individual to whom the information relates in response to questions on the application for insurance, including medical information, family history, and disability;
• occupational information, based on actuarially sound principles, that has a direct relationship to mortality, morbidity, or longevity risk;
• behavioral information related to a specific individual, including motor vehicle records and criminal history of a non-juvenile felony conviction, that has a direct relationship to mortality, morbidity, or longevity risk;
• MIB data;
• prescription drug history;
• income, tax, assets, or other elements of a specific person’s financial profile provided on an application for insurance by the applicant; or
• digitized or other electronic forms of the information listed above.

Addition of a Remediation Requirement

The core obligation of the Final Regulation remains focused on discrimination with respect to race, but adds a remediation requirement (changes from the Draft Regulation are reflected below with underlining for new text and strikethroughs for removed text):

Life insurers that use ECDIS, as well as algorithms and predictive models that use ECDIS in any insurance practice, must establish a risk-based governance and risk management framework that facilitates and supports policies, procedures, and systems, designed to determine whether the use of such ECDIS, algorithms, and predictive models potentially result in unfair discrimination with respect to race, and remediate unfair discrimination, if detected. Section 5(A).

As we will discuss more in our upcoming Debevoise Data Blog post on the Draft Testing Regulation, a requirement to remediate unfair discrimination may seem unobjectionable, but when it is combined with (1) a rigid quantitative testing methodology for assessing bias, and (2) a requirement to estimate the race and ethnicity of all life insurance applicants using BIFSG, which introduces a significant error rate into the analysis, there is substantial risk that there will be findings of unfair discrimination that merely reflect noise in the data, rather than any actual discrimination. If that “noise discrimination” must nonetheless be remediated, there is a risk that the application of the Final Regulation, along with the Draft Testing Regulation as currently drafted, will in some cases actually cause companies to treat customers unfairly.

The term “unfair discrimination” remains defined by Section 10-3-1104.9, C.R.S. as:

[T]he use of one or more external consumer data and information sources, as well as algorithms or predictive models using external consumer data and information sources, that have a correlation to race, color, national or ethnic origin, religion, sex, sexual orientation, disability, gender identity, or gender expression, and that use results in a disproportionately negative outcome for such classification or classifications, which negative outcome exceeds the reasonable correlation to the underlying insurance practice, including losses and costs for underwriting.

Governance and Risk Management Obligations

The Final Regulation did not add or remove any components of the governance and risk management framework, and made only slight changes to the existing components (again, reflected below with underlining for new text and strikethroughs for removed text).

• Guiding Principles (largely unchanged). Insurers must have documented governing principles that provide guidance for ensuring that ECDIS (and Models that use ECDIS) are designed, developed, used, and monitored in a matter that is well-suited for achieves effective oversight and management and do not lead to are reasonably designed to prevent unfair discrimination. Section 5(A)(1).
• Board Oversight (largely unchanged). The board of directors or appropriate board committee must oversee the governance structure and risk management framework. Section 5(A)(2).
• Senior Management Accountability (largely unchanged). Senior management must be responsible and accountable for “setting and monitoring the overall strategy” on the use of ECDIS and AI models that use ECDIS. This includes establishing clear lines of communication, delegated decision-making authority, and regular reporting to senior management regarding ECDIS risks. Section 5(A)(3).
• Cross-Functional Governance Group (largely unchanged). Insurers must establish a documented cross-functional ECDIS, algorithm, and predictive model governance group that is composed of representatives from “key functional areas” including legal, compliance, risk management, product development, underwriting, actuarial, data science, marketing, and customer service, as applicable. Section 5(A)(4).
• Policies (largely unchanged). Insurers must have written documented policies, and processes, and procedures, including assigned roles and responsibilities, for the design, development, testing, deployment, use, selection and oversight of vendors), and ongoing monitoring of ECDIS and algorithms that use ECDIS to ensure that they are documented, tested, and validated. Section 5(A)(5).
• Training (largely unchanged). Insurers’ policies and procedures must include an ongoing internal supervision and training program for relevant personnel on the responsible and compliant use of ECDIS that addresses issues. Section 5(A)(5).
• Consumer Complaints and Inquiries (largely unchanged). Insurers must establish documented processes for addressing consumer complaints and inquiries about the use of ECDIS and models that use ECDIS. Such policies and procedures must provide consumers with sufficiently clear information the information necessary so that consumers can take meaningful action in the event of an adverse decision made based on the use of ECDIS and models that use ECDIS. Section 5(A)(6).
• Risk Assessments and Prioritization (largely unchanged). Insurers must establish a documented rubric for assessing and prioritizing risks associated with the deployment of ECDIS, as well as models that use ECDIS, in insurance practices with appropriate reasonable consideration given to insurance practices’ consumer impact. Section 5(A)(7).
• Vendor Risk Management (changed). Insurers that use third-party vendors for their ECDIS and models that use ECDIS remain responsible for ensuring compliance with the requirements in the Revised Regulation and must establish a process for the selection and oversight of these vendors. Insurers may satisfy requests for documentation and information by third-party vendors providing the requested documents or information directly to the Division on behalf of the insurer. Whether insurers will actually be willing to let their data or model vendors make document productions directly to the DOI relating to the insurers’ AI compliance remains to be seen. Section 5(B).

Revised Documentation Obligations

Aside from the additional documentation requirements noted above (which are largely just clarifications, rather than new obligations), there are a few changes to the documentation obligations between the Draft Regulation and the Final Regulation, which are reflected below:

• Inventory of AI Models (unchanged). Insurers are required to maintain a documented up-to-date inventory, which includes version control, of all utilized ECDIS, as well as models that use ECDIS, a detailed description of each, their purposes, and the outputs generated through their use. The inventory is limited to AI models that use ECDIS. Section 5(A)(8).
• Documentation of Material Changes (unchanged). Insurers are required to maintain documentation that explains any material changes in the inventory, as well as the rationale for the changes. Section 5(A)(9).
• Bias Assessments (unchanged). Insurers must have a description of any testing conducted to detect unfair discrimination resulting from the use of ECDIS and models that use ECDIS, including the methodology, assumptions, results, and steps taken to address unfair discriminatory outcomes. Section 5(A)(10).
• Monitoring (largely unchanged). Insurers must document ongoing monitoring regarding the performance of AI models that use ECDIS, including account for model drift. Section 5(A)(11).
• Vendor Selection (unchanged). Insurers must document the process used for selecting external vendors that supply ECDIS or AI models that use ECDIS. Section 5(A)(12).
• Annual Reviews (largely unchanged). Insurers must conduct regular documented comprehensive annual reviews of the governance structure and risk management framework and make appropriate updates to the required documentation to ensure its accuracy. Section 5(A)(13).

Changes to Certification and Compliance Deadlines

Insurers using ECDIS, and Models that use ECDIS, now have until June 1, 2024 to provide a report to the DOI summarizing the progress made towards implementing the requirements of the Revised Regulation. The deadline in the Draft Regulation had been six months from the effective date. And those insurers now have until December 1, 2024 to submit a report summarizing compliance. The deadline in the Draft Regulation had been one year from the effective date.

As in the Draft Regulation, the Final Regulation requires that the report summarizing compliance must be submitted annually. There are several changes in the requirements relating to the content of the report including:

• the title and qualifications of each individual responsible for ensuring compliance, along with the specific requirement for which that individual is responsible are required, but the Final Regulation adds that the names of each individual may also be provided but are unnecessary to comply with this requirement;
• a signature of an officer attesting to compliance with the Revised Regulation is required;
• in the event an insurer is unable to attest to compliance with this regulation, the insurer must submit to the DOI a corrective action plan; and
• the Final Regulation adds that the report shall be no more than 10 (ten) pages, including an executive summary, and address Sections 5(A)1 through 5(A)(13). Section 6(B).

Takeaways

• Gap Analysis & Road Map. Insurers should consider conducting a gap analysis between the requirements in the Final Regulation and their current AI and data governance and compliance program. After the gap analysis, insurers should consider developing a road map to compliance. For some companies that are covered by the Final Regulation, it may take significant time and resources to fully implement these requirements, and so they may want to start early. And even companies that are not subject to the Final Regulation may consider conducting a gap analysis in anticipation that these rules, or similar ones, could be adopted by other regulators in the coming years or could come to be considered best practices for AI governance and compliance programs.
• Risk Assessment. The Final Regulation requires that insurers develop a rubric to assess and prioritize risks. Insurers should consider creating a list of high-risk factors uses to identify what are the high-, medium-, or low-risk use cases for ECDIS. Those criteria can then be used to identify the highest-risk uses of ECDIS for prioritization and help create the road map to compliance.
• Cross-Functional Group. The Final Regulation calls for the creation of a cross-functional group. Determining which representatives from “key functional areas” should be in the group, how often the group should meet, what resources it needs, to whom it will report, how it will make decisions, and how its decisions will be implemented are all complicated considerations that will take time and discussion.
• Budget. Many components of obligations in the Final Regulation could require some companies to significantly increase their compliance budgets and secure additional resources.

To subscribe to the Data Blog, please click here.

The Debevoise Artificial Intelligence Regulatory Tracker (“DART”) is now available for clients to help them quickly assess and comply with their current and anticipated AI-related legal obligations, including municipal, state, federal, and international requirements.