Key Takeaways:

• On July 11, 2024, the New York State Department of Financial Services adopted Insurance Circular Letter No. 7 regarding the Use of Artificial Intelligence Systems and External Consumer Data and Information Sources in Insurance Underwriting and Pricing.
• The final version of Circular largely adopts the language of the January 2024 Proposed Insurance Circular Letter on these issues but also includes a few key updates.
• In this blog post, we discuss details of the Final Circular as well as any material changes from the Proposed Circular.

---

On July 11, 2024, the New York State Department of Financial Services (the “NYDFS”) adopted Insurance Circular Letter No. 7 regarding the Use of Artificial Intelligence Systems and External Consumer Data and Information Sources in Insurance Underwriting and Pricing (the “Final Circular”). The Final Circular largely adopts that language of the January 2024 Proposed Insurance Circular Letter on these issues (the “Proposed Circular” or “PCL”), which we discussed previously here and here. However, NYDFS has made a few key updates in the Final Circular as a result of comments it received on the PCL. The Final Circular imposes significant obligations on insurers using artificial intelligence systems (“AIS” or “AI systems”) or external consumer data and information sources (“ECDIS” or “external data”) and is likely to affect the AI regulatory landscape beyond New York State and perhaps beyond the insurance sector. The Final Circular clarifies that, unlike the Colorado insurance AI law, it applies to AI systems regardless of the use of ECDIS. A redline of the Final Circular against the Proposed Circular can be found here.

The Final Circular builds on and expands the applicability of NYDFS’s 2019 Insurance Circular Letter No. 1 (the “2019 Letter”) (here) and clarifies the 2019 Letter’s disclosure and transparency obligations. The 2019 Letter was limited to the use of external consumer data and information sources for underwriting life insurance and focused on risks of unlawful discrimination that could result from the use of ECDIS and the need for consumer transparency regarding its use. The Final Circular incorporates the general obligations from the 2019 Letter while adding more detailed requirements, expanding the scope beyond life insurance, and adding significant governance and documentation requirements.

Legal Status of a NYDFS Circular Letter

The NYDFS’s circular letters do not change applicable law or regulations. However, the NYDFS uses circular letters to provide guidance to the insurance industry regarding how it interprets existing law and regulations, to address issues and industry practices that it finds require changes and to clarify its expectations of the industry. The Final Circular provides the NYDFS’s guidance regarding its views on existing laws (such as those prohibiting unfair discrimination) and regulations (such as those requiring a corporate governance framework, internal audit functions, and recordkeeping) in relation to the use of AI and ECDIS. Circular letters also often indicate the NYDFS’s enforcement priorities. Additionally, courts historically have given some deference to the NYDFS’s interpretations of the laws that it is empowered to enforce (e.g., insurance, banking, and financial services laws) and its own regulations.

In this next section, we will discuss the details of the Final Circular as well as any material changes from the Proposed Circular.

Key Updates and Elements of the Final Circular

Scope

The PCL covered the NYDFS’s expectations with respect to insurers authorized to write insurance in New York State, licensed fraternal benefit societies, and the New York State Insurance Fund, that use ECDIS or AIS. The Final Circular extends the scope to also cover Article 43 corporations (non-profit medical and dental indemnity, or health and hospital service corporations) and health maintenance organizations.

The Final Circular defines AIS as “any machine-based system designed to perform functions normally associated with human intelligence, such as reasoning, learning, and self-improvement, that is used — in whole or in part — to supplement traditional medicalhealth, life, property or casualty underwriting or pricing, as a proxy for traditional medicalhealth, life, property or casualty underwriting or pricing, or to establishidentify ‘lifestyle indicators’ that may contribute to an underwriting or pricing assessment of an applicant for insurance coverage.” (Redlined against the Proposed Circular).

ECDIS is defined to include “data or information used — in whole or in part — to supplement traditional medical, property or casualty underwriting or pricing, as a proxy for traditional medical, property or casualty underwriting or pricing, or to establishidentify ‘lifestyle indicators’ that may contribute to an underwriting or pricing assessment of an applicant for insurance coverage.” The Final Circular also provides that ECDIS “does not include an MIB Group, Inc. member information exchange service, a motor vehicle report, prescription drug data, or a criminal history search.” (Redlined against the Proposed Circular).

It appears the use of the term “medical” for the ECDIS definition, which was changed to “health, life” for AIS definition, was an updating oversight as “medical” insurance is not a category under the New York Insurance Law (“Insurance Law”).

Unlike the Colorado Division of Insurance Governance and Risk Management Framework Requirements for Life Insurers’ Use of External Consumer Data and Information Sources, Algorithms, and Predictive Models (“Colorado Governance Regulation”) (here), the Final Circular’s definition of ECDIS does not include an enumerated list of factors that qualify as “lifestyle indicators,” such as social media habits, purchasing habits, home ownership, and educational attainment. Further, the scope of the Final Circular is broader than the Colorado Governance Regulation in that it applies to all types of insurance (rather than just life, at least for now, in Colorado), but narrower in that it applies to only underwriting and pricing, whereas the Colorado Governance Regulation applies to “other insurance practices,” which may include marketing and claims management.

One very important clarification in the Final Circular is that it applies even if ECDIS is not used to operate an AI model. The opening section of the Final Circular states that “it is the intent of the Department to cover AIS utilization and models regardless of whether those AIS leverage ECDIS.”

The Final Circular also appears to resolve another ambiguity that appeared in the Proposed Circular regarding whether its application is limited to consumer insurance lines or also applies to some forms of commercial insurance. For example, when discussing the prohibition of unlawful discrimination in the underwriting process, the Final Circular also references Insurance Law § 2303, which applies to unfairly discriminatory rates for property and casualty insurance. It therefore appeared that the NYDFS was focused on insurance products with a direct consumer impact. Responding to comments on the Proposed Circular that commercial property/casualty and group life insurance products may not always have a direct consumer impact, the opening section of the Final Circular states:

The final Circular Letter maintains the expectation that insurers take an appropriate, risk-based approach to utilizing ECDIS and AIS. It is up to insurers to determine the appropriate sufficiency thresholds and standards of proof based on the product and the particular use of ECDIS or AIS. It is noted by the Department that in certain circumstances commercial property/casualty and group life insurance may not have a direct consumer impact, and in other cases it could. For example, commercial property/casualty insurance is issued to sole proprietors.

Fairness and Proxy Assessment

According to the Final Circular, insurers are obligated under existing laws to establish that their use of ECDIS or AIS is not unfairly discriminatory, is supported by generally acceptable actuarial practices, and is based on reasonably anticipated experience. With respect to proxy discrimination, the Final Circular clarifies that insurers should evaluate the extent to which ECDIS that is being used for underwriting and pricing are correlated with (i.e., are a proxy for) status in any protected classes that may result in unfair discrimination. Some may read this clarification as expanding the concept of proxy variables to include inputs that are correlated to a protected class, regardless of whether they are independently predictive. We, however, view the inclusion of the phrase “that may result in unfair discrimination” as limiting the concept of proxy discrimination to the use of inputs that draw their predictive power largely from their association with a protected class, but would not apply to inputs that may be correlated to a protective class but are shown to be independently predictive through testing that controls for that protected class.

Insurers also bear the burden of ensuring that ECDIS or AIS provided by vendors complies with anti-discrimination laws, and insurers cannot rely solely on a vendor’s claim of non-discrimination.

Quantitative Testing

The Final Circular specifies that insurers should assess the use of ECDIS or AIS for any disproportional adverse effects in underwriting or pricing on similarly situated insureds or insureds of a protected class, which is notable as it goes beyond merely testing for effects on protected classes. Accordingly, the Final Circular should be read along with NYDFS Circular Letter No. 6 (2023) (here), relating to “Unfair and Unlawful Discrimination in the Sale of Life Insurance and Annuities in the Individual Market and Certain Group Markets,” which discussed the NYDFS’s concerns regarding similarly situated consumers receiving different terms for the same policies.

The NYDFS defined the quantitative assessment in the following step-by-step process:

• Step 1: Assess whether the use of ECDIS or AIS produces disproportionate adverse effects in underwriting or pricing for similarly situated insureds, or insureds of a protected class. This assessment should be conducted for any protected class where membership in such protected class either may be determined using data available to the insurer or may be reasonably inferred using accepted statistical methodologies.

• If there is no prima facie showing of a disproportionate adverse effect, then the insurer may conclude its evaluation.

• If there is a prima facie showing of such disproportionate adverse effects, then the insurer should continue to Step 2.

• Step 2: The insurer should assess whether there is a legitimate, lawful and fair explanation or rationale for the differential effect.

• If no legitimate explanation can account for the differential effect, the insurer should modify its use of such ECDIS or AIS and evaluate the modified use beginning with Step 1 of this quantitative assessment.

• Step 3: If a legitimate explanation can account for the differential effect, then the insurer should conduct a search for a less discriminatory alternative variable or methodology that would reasonably meet the insurer’s legitimate business needs.

• If a less discriminatory alternative exists, then the insurer should modify its use of ECDIS or AIS accordingly and reevaluate the modified use beginning with Step 1 of this quantitative assessment.

• If no less discriminatory alternative exists, the insurer should conduct ongoing model risk management consistent with the Circular’s Governance and Risk Management requirements in Section III, and repeat Step 3 at least annually.

Compared to the Colorado Division of Insurance Draft Regulation on Quantitative Testing for Unfairly Discriminatory Outcomes for Algorithms and Predictive Models Used for Life Insurance Underwriting (“Colorado Draft Testing Regulation”) (here), the Final Circular offers much more flexibility to insurers in performing their quantitative assessments, and encourages insurers to use multiple statistical metrics in evaluating data and model outputs, including:

• Adverse Impact Ratio: Analyzing the rates of favorable outcomes between protected classes and control groups to identify any disparities.

• Denials Odds Ratios: Computing the odds of adverse decisions for protected classes compared to control groups.

• Marginal Effects: Assessing the effect of a marginal change in a predictive variable on the likelihood of unfavorable outcomes, particularly for members of protected classes.

• Standardized Mean Differences: Measuring the difference in average outcomes between protected classes and control groups.

• Z-tests and T-tests: Conducting statistical tests to ascertain whether differences in outcomes between protected classes and control groups are statistically significant.

• Drivers of Disparity: Identifying variables in AIS that cause differences in outcomes for protected classes relative to control groups. These drivers can be quantitatively computed or estimated using various methods, such as sensitivity analysis, Shapley values, regression coefficients, or other suitable explanatory techniques.

Qualitative Assessment

In addition to any quantitative analysis, an insurer’s comprehensive assessment should involve a qualitative assessment of unfair or unlawful discrimination that includes being able to explain how the insurer’s AIS operates and the logical relationship between the ECDIS and an insured or potential insured individual’s risk.

Testing Frequency

The Final Circular provides that testing should be administered prior to putting AIS into production and at least annually thereafter, as well as whenever material updates or changes are made to either the ECDIS or AIS.

Testing Documentation

The Final Circular provides that an insurer should document the processes and reasoning behind its testing methodologies and analysis for unfair or unlawful discrimination, and should be prepared to make such documentation available to the NYDFS upon request.

AI Risk and the Threshold for Quantitative Testing

The Final Circular notes that an insurer may deploy ECDIS and AIS in a variety of ways, and that the NYDFS recognizes that there is no one-size-fits-all approach to managing data and decisioning systems. Therefore, insurers “should take an approach to developing and managing their use of ECDIS and AIS that is reasonable and appropriate to each insurer’s business model and the overall complexity and materiality of the risks inherent in using ECDIS and AI.”

A risk-based approach should allow for some uses of ECDIS or AIS in underwriting or pricing to be deemed sufficiently low risk so as not to be subject to detailed quantitative testing, but that possibility is arguably foreclosed by the Final Circular’s requirements relating to quantitative testing described above.

Testing Data Availability

The Final Circular provides that an insurer should evaluate and, where possible, not use ECDIS or AIS for underwriting or pricing purposes unless the insurer can establish that the data source or model, as applicable, does not use and is not based in any way on any class protected pursuant to Insurance Law Article 26, which includes race, color, creed, national origin, disability, sex, marital status, being a victim of domestic violence, and past lawful travel.

In response to this language in the Proposed Circular, we raised the question as to how insurers would test for discriminatory impact on protected classes for which the insurer does not collect any data. The Final Circular clarifies in three separate places that there is no expectation that insurers collect additional data to conduct its fairness analysis:

• “This assessment should be conducted for any protected class where membership in such protected class either may be determined using data available to the insurer or may be reasonably inferred using accepted statistical methodologies[;]”

• “There is no expectation that insurers collect additional data from, or about, individuals to perform exemplary analysis[;]” and

• “Whether ECDIS correlates with a protected class may be determined using data available to the insurer or may be reasonably inferred using accepted statistical methodologies. If such correlations are identified, insurers should consider whether the use of such ECDIS is required by a legitimate business necessity.”

This is consistent with Circular Letter No. 5 (1964) (here), the subject of which NYDFS describes in its posted list of Circular Letters as “Ban on inquiries as to race, color, creed or national origin in insurance industry reports and forms to effectuate [Insurance Law §] 2606” (here).

Accordingly, it appears that the Final Circular is adopting the same approach as the Colorado Draft Testing Regulation and limiting the scope of its testing requirements to discrimination on the basis of race (and perhaps sex/gender), which can arguably be estimated using some combination of first name, last name, and location of home address. Indeed, the Colorado Draft Testing Regulation specifically requires the use of Bayesian Improved First Name Surname Geocoding (“BIFSG”) to estimate certain categories of race of insureds for testing purposes.

Transparency, Notice and Consumer Redress

Where an insurer is using ECDIS or AIS, the insured must provide a notice that discloses: (i) whether the insurer uses AIS in its underwriting or pricing process, (ii) whether the insurer uses data about the person obtained from external vendors, and (iii) that such person has the right to request information about the specific data that resulted in the underwriting or pricing decision, including contact information for making such request. Similarly, if an applicant will not be approved for insurance under an underwriting process utilizing ECDIS or AIS because of specific ECDIS data, the Final Circular requires the insurer to provide the applicant with a process to review for accuracy those data that resulted in the applicant not qualifying for the ECDIS or AIS-based underwriting process.

Insurers should also be prepared to respond to consumer complaints and inquiries about their use of AIS and ECDIS by implementing procedures to receive and address such complaints. Insurers must maintain any records of complaints regarding AIS or ECDIS and be prepared to make such records available to the NYDFS upon request.

Governance

According to the Final Circular, insurers are obligated to have a corporate governance framework that is appropriate for the nature, scale, and complexity of the insurer and provides appropriate oversight of the use of ECDIS and AIS.

Board and Senior Management Oversight
The Final Circular notes that the board of directors is obligated to oversee the insurer’s use of ECDIS and AIS and ensure that an effective governance framework is carried out. If certain board duties are delegated, as permitted, then the Final Circular provides that appropriate lines of reporting must be in place as well as regular, detailed reporting to the board. Further, the Final Circular explains that senior management should be responsible for the “day-to-day implementation” of ECDIS and AI systems, which includes:

• Establishing adequate policies and procedures;

• Assigning competent staff;

• Overseeing model risk management;

• Ensuring effective challenge and independent risk assessment;

• Reviewing internal audit findings; and

• Taking prompt remedial action when necessary.

Senior management should also ensure that all relevant operation areas are appropriately engaged, such as through a cross-functional management committee with representatives from key function areas, including legal, compliance, risk management, product development, underwriting, actuarial, and data science, as appropriate.

Auditing
The Final Circular expounds on insurers’ existing internal audit obligations under 11 NYCRR § 89.16. According to the Final Circular, insurers should ensure the internal audit function is appropriately engaged with the insurer’s use of ECDIS and AIS consistent with its financial, operational, and compliance risk. Such auditing should assess the overall effectiveness of the AIS and ECDIS risk management framework, which may include:

• Verifying that acceptable policies and procedures are in place and are appropriately adhered to;

• Verifying records of AIS use and validation to test whether validations are performed in a timely manner;

• Assessing the accuracy and completeness of AIS documentation;

• Evaluating the processes for establishing and monitoring internal controls;

• Assessing supporting operational systems and evaluating the accuracy, reliability, and integrity of ECDIS and other data used by AIS;

• Assessing potential biases in the ECDIS or other data that may result in unfair or unlawful discrimination against insureds or potential insureds; and

• Assessing whether there is sufficient reporting to the board or other governing body and senior management to evaluate whether management is operating within the insurer’s risk appetite and limits for model risk.

Policies and Procedures
The Final Circular provides that insurers using ECDIS or AIS should have written policies and procedures consistent with the Proposed Circular and which are approved annually by the board or senior management, including:

• Clearly defined roles and responsibilities;

• Monitoring and reporting requirements to senior management; and

• A training program for relevant personnel on the lawful use of ECDIS and AIS with an accountability mechanism for ensuring that all relevant personnel complete regular training in a timely manner.

Documentation
The Final Circular explains that insurers are obligated to maintain comprehensive documentation of all ECDIS and AIS use, whether developed internally or supplied by third parties. Insurers should be prepared to provide such documentation to the NYDFS upon request. The Final Circular indicates that comprehensive documentation may include:

• A description of the process for identifying and assessing risks associated with an insurer’s use of ECDIS and AIS;

• A description of associated internal controls designed to mitigate such identified risks;

• An up-to-date inventory of all AIS implemented for use, under development for implementation, or recently retired;

• A description of how each AIS operates, including any ECDIS or other inputs and their sources, the purpose and products for which the AIS is designed, actual or expected usage, any restrictions on use, and any potential risks and appropriate safeguards;

• A description of the process for tracking changes of an insurer’s use of ECDIS and AIS over time, including a documented explanation of any changes, associated rationale for such changes, and parties responsible for the approval of such changes;

• A description of the process for monitoring ECDIS and AIS usage and performance, including a list of any previous exceptions to policy and reporting;

• A description of testing conducted at least annually to assess the output of AIS models, including drift that may result from the use of machine learning or other automated updates; and

• A description of data lifecycle management process, including ECDIS acquisition, storage, usage and sharing, archival, and destruction.

Risk Management
The Final Circular provides that insurers are permitted to manage the risks of AIS either within an existing enterprise risk management function, as required by the Insurance Law, or separately as part of an independent program. Regardless of an insurer’s choice, the Final Circular specifically provides that insurers should:

• Manage the relevant risks at each stage of the AIS life cycle and consider risk from individual AIS models and in the aggregate;

• Include standards for model development, implementation, use, and validation;

• Promote independent review and effective challenge to risk analysis, validation, testing, development, and other processes related to ECDIS and AIS; and

• Have competent and qualified personnel to execute and oversee AIS risk management with appropriate means of accountability.

Third-Party Vendors
The Proposed Circular provided several obligations with respect to third-party vendors, including that compliance cannot be delegated to a vendor who is providing the ECDIS or the AIS. The Final Circular adds a contracting requirement, providing that insurers must:

• Retain responsibility for understanding any tools, EDCIS, or AIS used in underwriting and pricing for insurance that were developed by third-party vendors ensuring compliance with all applicable regulations;

• Develop written standards, for the use of ECDIS and AIS developed by a third-party vendor;

• Implement procedures for reporting any incorrect information to third-party vendors for further investigation and update, as necessary;

• Develop procedures to remediate incorrect information from their AIS that the insurer has identified or has been reported to a third-party vendor; and

• Include terms in contracts with third-party vendors, where appropriate and available, that: (i) provide audit rights or entitle the insurer to receive audit reports by qualified auditing entities, and (ii) require the third-party vendor to cooperate with the insurer regarding regulatory inquiries and investigations related to the insurer’s use of the third-party vendor’s product or services.

Enforcement

The NYDFS may audit and examine an insurer’s use of ECDIS and AIS, including within the scope of regular or targeted examinations pursuant to Insurance Law § 309, or a request for special report pursuant to Insurance Law § 308.

What Steps Can Insurers Take?

Insurers that are subject to the guidance provided in the Final Circular should consider the following steps:

• Examine the requirements of the Final Circular and map the various stated obligations against existing policies and procedures for use of ECDIS and AIS, in order to identify potential gaps. In particular, the testing requirements may require significant investments to operationalize.

• Assess whether any additional resources, in terms of personnel or budget will be needed to meet the obligations set forth in the Final Circular, and begin the process of securing those resources.

This publication is for general information purposes only. It is not intended to provide, nor is it to be used as, a substitute for legal advice. In some jurisdictions it may be considered attorney advertising.