Guidance 

☐ Interpretation     ☐ Approach     ☑ Information     ☐ Decision

No. AU0137INF Active

Download a copy in PDF format

 

 

Purpose

This Information Guidance is intended to articulate foundational and sound practices for operational risk management (ORM) in the rating and underwriting of automobile insurance.[1]

The purpose of this Guidance is to promote just, reasonable and accurate rates. It will also support fair treatment for consumers engaged in the underwriting process.

The degree of adoption of this Guidance should be a function of a proportionality principle, meaning that it is commensurate with the nature (including business model), size, complexity and risk profile of the insurer.

Strategy for reforming regulation of auto insurance rates and underwriting

This Information Guidance on Operational Risk Management has been formulated as part of FSRA’s broader strategy to reform the regulation of rates and underwriting in Ontario’s auto insurance sector. Building on FSRA’s Standard Filing Guidance and Rule on Unfair and Deceptive Acts or Practices, this Guidance is intended as a further and important step in the direction of transitioning to principles-based and outcomes-focused regulation.

Although this Information Guidance does not create new obligations, it is intended to outline the path forward to a future state in which ORM becomes an obligation and enables expedited rate change processes for adherent entities. To this end, FSRA plans to transition from Information Guidance to a combination of Interpretation and Approach Guidance as work in support of rate and underwriting regulation reform advances.

In this future state, ORM Interpretation Guidance would identify ORM requirements in certain sections of FSRA’s UDAP Rule and thereby create compliance obligations, while Approach Guidance would explain how regulated entities would be assessed and granted access to streamlined processes for changing rates. In this manner, FSRA’s planned second phase of ORM Guidance will serve to facilitate greater insurer accountability for fairness in rates and underwriting while enhancing regulatory effectiveness.

Scope

This Guidance affects insurance companies writing automobile insurance in Ontario.[2]

Rationale and background

Through targeted consultations with auto insurers and analysis of information collected during the review of rate filings, FSRA has identified gaps in operational risk management and model governance practices in the industry.

Examples include:

  • Lack of robust three lines of defence, governance and control throughout the model lifecycle (e.g., missing independent second line for model review or model approval function), which may lead to consumers being priced inaccurately.
  • Lack of process to understand the impact to individual consumers from the use of models, including machine learning models, which may lead to unfair discrimination and constitute unfair or deceptive acts or practices (“UDAP” or “UDAPs”) prescribed by the Unfair or Deceptive Acts or Practices Rule (the “UDAP Rule”).
  • Lack of operational risk management process to identify, mitigate and report underwriting and/or rating errors,[3] which may lead to undetected errors and incorrect premiums charged to consumers.

FSRA has identified what it regards as sound practices for operational risk management and model governance practices through jurisdictional scans and stakeholder consultation[4] as a mechanism to address the gaps.

Identifying and promoting sound operational risk management and model governance practices achieves FSRA’s statutory objects.[5]

  • contribute to public confidence in the regulated sectors
  • monitor and evaluate developments and trends in the regulated sectors
  • promote high standards of business conduct
  • protect the rights and interests of consumers
  • foster strong, sustainable, competitive and innovative financial services sectors

FSRA anticipates that adopting the ORM framework will help insurers achieve more effective compliance with the applicable requirements under the Insurance Act and the Automobile Insurance Rate Stabilization Act, including meeting obligations of s. 439 of the Insurance Act as it relates to s. 4(1)(i)-(ii), s. 9(1)(v), s. 9(1)(ii) and s. 9(1)(iv) of the UDAP Rule.[6]

FSRA also anticipates operational resilience[7] as an outcome that benefits from effective management of operational risk. ORM activities such as risk identification and assessment, risk mitigation (including the implementation of controls) and the monitoring of risks and control effectiveness work together to minimize operational disruptions and their impact[8].

Principles

FSRA’s Rate Regulation Principles are foundational to its approach to auto insurance rate regulation, and have been central to the design of this guidance.

In developing a principles-based and outcomes-focused approach to managing operational risk in rating and underwriting, FSRA was guided by all six principles with a particular emphasis on Consumer Focus.

Operational risk management framework

Operational risk defined and other terminology

Operational risk is defined as the risk of loss resulting from inadequate or failed internal processes, people and systems or from external events.

FSRA’s concerns about operational risk are driven by the negative consumer outcomes and breaches of applicable law that can occur if operational risk (including model risk) is not well managed. For these types of risks, the focus is on ensuring appropriate governance and controls are in place to identify and manage risks, rather than managing every activity in detail.

It is FSRA’s view that rating and underwriting activities are subject to operational risks that can be managed in a similar manner.

As such, for the purposes of this Guidance:

  • The term “ORM” refers specifically to operational risk management for auto insurance rating and underwriting activities.
  • The term “ORM Framework” refers to the policies, procedures and any related documents that outline how the company manages its operational risk in its auto insurance rating and underwriting activities.
  • The term “Senior Management” refers to the insurer’s Senior Management, including the Chief Executive Officer, the Chief Financial Officer, the Chief Risk Officer, Chief Compliance Officer, the Executives responsible for rating and underwriting, and any other employee formally identified by the insurer’s Board of Directors.

The operational risk management cycle

Managing operational risk in an insurer’s processes under an ORM Framework generally follows a consistent cycle, termed here as the ORM Cycle.

This section outlines the ORM Cycle and associated steps observed by FSRA that should be taken to manage an operational risk in rating/underwriting, including specific considerations.

  1. Risk identification: An insurer’s ORM Framework should outline how it will ensure that operational risks are identified in a timely manner. The specific tools used to identify operational risk will depend on a range of relevant factors, particularly the nature (including business model), size, complexity and risk profile of the insurer — examples of identification tools include surveys, workshops, registers and questionnaires.
  2. Risk assessment: An insurer’s ORM Framework should outline how it will assess the materiality of identified operational risks in a consistent manner. The insurer should be able to articulate its inherent and residual risk, where inherent risk is defined as the risk level prior to accounting for existing controls or risk responses, and residual risk is defined as the risk level after accounting for existing controls/responses.
  3. Risk prioritization and mitigation: Based on the risk level identified through the assessment process, the insurer can rank any new risk against its pre-existing risks for prioritization purposes and, therefore, can determine how the risk should be managed (e.g., accept, reduce, share, avoid) to align with its risk appetite. The insurer’s ORM Framework should outline the processes and controls by which risks are prioritized and managed.
  4. Risk monitoring and reporting: An insurer’s ORM Framework should outline how it will monitor risks being managed, how it will report risk levels to relevant stakeholders and how it will address risks that fall outside acceptable levels. When a risk falls outside acceptable levels, the insurer’s ORM Framework should outline how action plans will be established to bring risks within acceptable levels, including appropriate escalations — if needed, to Senior Management and the Board of Directors.

The ORM Cycle should occur on an ongoing basis for existing processes and on an ad-hoc basis as required for new product introductions, projects or changes contemplated to existing products and processes.

On a periodic basis, the insurer should also review all high-risk areas (even those that are appropriately mitigated within acceptable levels) to have a full understanding of the significant or emerging risks to ensure its operational risk is within its risk appetite.

Operational risk management foundational practices

For the ORM Cycle to be applied effectively to address the risks in rating/underwriting, FSRA has observed that the ORM Framework should, at a minimum, include the following foundational practices:

  1. define an operational risk appetite specific for rating and underwriting of automobile insurance
  2. clearly define roles and responsibilities with robust accountability mechanisms
  3. have data governance in place
  4. be maintained on an ongoing basis

Each of these foundational practices are outlined below.

1) Defining operational risk appetite for rating and underwriting

To ensure that operational risks are managed in a consistent manner throughout the organization over time, insurers should develop and maintain a comprehensive risk appetite statement for operational risks in rating and underwriting of automobile insurance.

The risk appetite is a statement, or series of statements, that describe(s) the entity’s attitude towards risk-taking. More specifically, it articulates the nature and types of operational risk that the insurer is willing or expected to assume to achieve its business objectives.

The operational risk appetite statement should be succinct, clear and include a measurable component (limits/thresholds). The purpose of having a measurable component is to indicate the level of operational risk that is considered acceptable within the insurer. The limits/thresholds may also serve to indicate the level at which operational risk events, near misses or cumulative patterns are considered necessary for escalation to Senior Management and the Board of Directors (in some cases, separate reporting thresholds may be established).

In formulating a risk appetite statement, insurers may consider elements such as: changes in the external environment; material changes in business or activity volumes; the quality of the control environment; the effectiveness of risk management or mitigation strategies; the insurer’s operational risk event experience; and the frequency, volume or nature of risk appetite limit/threshold breaches.

In the case of insurers with lower operational risk profiles, risk appetite can be evidenced by the use of reporting/escalation thresholds for material operational risk events.

2) Clearly defined roles, responsibilities and accountability mechanisms

For an ORM Framework to be effective, it is essential that all participating stakeholders and their respective responsibilities are clearly documented and defined, and that adequate accountability mechanisms are established. To achieve these outcomes, an appropriate governance structure and a Three Lines of Defence model, explained below, should be implemented.

Governance structure

To instill a strong, organization-wide risk management culture, Senior Management and the Board of Directors should play an active role.

As the stakeholder ultimately responsible for the activities of an insurer, the Board of Directors should also have responsibility for the insurer’s ORM Framework. This includes ensuring it has a sound understanding of the insurer’s operational risks and whether the ORM Framework is operating as expected, including ensuring that independent risk management functions exist and are effective.

Senior Management should be responsible for establishing and maintaining the policies and processes that operationalize the ORM Framework, embedding appropriate accountability mechanisms, such as the Three Lines of Defence model discussed below, throughout the organization. Issues should be escalated to Senior Management and the Board of Directors when necessary.

Three lines of defence

Appropriate accountability in the management of operational risk is essential. A “Three Lines of Defence” structure is one way to achieve such accountability. For illustrative purposes, the roles and responsibilities of each of the three lines are described below. Note that insurers determining what is considered an appropriately robust structure may consider factors such as size, ownership structure, complexity of operations, corporate strategy and risk profile.

First line of defence

The business line – the first line of defence – has ownership of risk whereby it acknowledges and manages the inherent and residual operational risk that it incurs in conducting its activities. The first line of defence is responsible for the ORM Cycle as per the insurer’s ORM Framework, including following its Risk Appetite, policies/processes, reporting, monitoring, etc. The first line of defence may develop a quality assurance process to assist it in fulfilling its risk ownership responsibilities.

Second line of defence

The second line is an independent function, such as compliance or risk management, that should provide an effective challenge and oversight of first line’s activities. The second line is responsible for the design and implementation of the ORM Framework to ensure that operational risk is properly managed and within the insurer’s risk appetite. The second line’s review should include, at a minimum, a review of:

  • Reproducibility: An understanding of first line’s processes and procedures and the ability to independently trace the first line’s decision-making. This necessarily implies that first line should have current, accurate and complete documentation that can be reviewed by the second line.
  • Soundness: An objective and independent review of whether the first line’s management of operational risk is conceptually sound. If the second line considers the first line’s management of operational risk inadequate or incomplete, then feedback should be provided on how findings can be remediated.

Third line of defence

The third line of defence is administered by the internal audit function, providing independent assurance to Senior Management and the Board of Directors on the effectiveness of the insurer’s ORM Framework with respect to rating and underwriting.

The third line of defence should be separate and independent from both the first and second, and it should provide an objective review/test of the insurer’s ORM controls, processes, systems, and of the effectiveness of the first and second line of defence functions.

The third line of defence is best placed to observe and review ORM more generally within the context of the insurer’s overall risk management and corporate governance functions. Objective review and testing coverage should be sufficient in scope to verify the ORM Framework has been implemented as intended and is functioning effectively.

3) Having data governance in place

Effective decision-making is contingent on the quality of data. An insurer’s ORM Framework should outline how its data governance practices apply in its ORM practices, ensuring that data used is appropriate, accurate, complete and timely. This should include items such as:

  • Data quality assessments: Insurers should identify characteristics that data should possess to produce credible estimates. Then, based on their criteria, data should be verified through fitness-for-use assessments. Data quality should be monitored on a regular basis to ensure fitness of use.
  • Identification of data problems/opportunities: Timely identification and resolution of problems, including opportunities for making improvements in data processes, with a goal to increasing the quality of existing and future data.
  • Identification of data limitations: Insurers should identify all known limitations in their data and consider items such as why, despite limitations, the data is appropriate for use, special monitoring considerations for such data, etc.
  • Data ownership: In addition to characteristics regarding data, it is critical that each data source has a designated owner to instill accountability for data quality.

4) Maintenance of the operational risk management framework

Maintenance of the ORM Framework is important from an auditability standpoint and for business continuity purposes (i.e., to enable sustainable operation in the long term). The review functions play an essential role in identifying opportunities for improvement in the ORM Framework.

Sound practices observed by FSRA include:

  • Training: Implementing and maintaining an ORM Framework will require a thorough understanding from staff of their roles and responsibilities. Insurers should outline in their ORM Frameworks the policies/processes used to ensure that staff are adequately trained on an ongoing basis. The ORM Framework should also outline how it reviews the adequacy of its training processes.
  • Documentation: Insurers should ensure they have current, accurate and complete documentation of their entire ORM Framework. This includes items/documents like a risk registry, risk appetite statements, model risk management policies, model documentation, key decisions, process documentation, interactions among the three lines of defence, use of key risk indicators, etc. Insurers should also ensure they log the materialization of any operational risks or near misses as well as any exercises used to learn from these events (e.g. root cause analysis). Any information relevant to the creation and maintenance of the ORM Framework and in decision-making throughout the ORM Cycle should be documented.
  • Periodic reviews: Changes in operational, market or other conditions may render certain policies, processes or other elements inadequate, thereby requiring revision. An insurer’s ORM Framework should outline how it would monitor the appropriateness of all elements of its framework and, if deemed necessary, how the insurer would adjust them. This includes but is not limited to training, documentation, risk appetite statement and governance structures.

Model risk management

The quantitative nature of models[9] allows for more precise management than traditional operational risks while also posing additional risks due to their complexity. These risks are particularly pronounced in advanced analytical techniques used in rating and underwriting, such as machine learning and artificial intelligence, and also pose risks in interpretability and explainability. FSRA has outlined how ORM practices may be applied for model risk management in Appendix 1.

Application to other existing obligations

Other areas these ORM practices may be applied include, but are not limited to:

  • Third-party products and services: Insurers may decide to use third-party data, use tools created by external parties, or hire a third-party to completely develop products for its own use. Working with third parties poses additional risks and insurers should take reasonable measures to ensure oversight of use of third-party data or services. Insurers will retain their regulatory obligations if they use third parties and, therefore, insurers, not vendors, hold accountability for consumer outcomes.
  • Protection of personal information: Insurers face various obligations from a confidentiality and privacy standpoint, e.g., the Federal government’s Personal Information Protection and Electronic Documents Act (PIPEDA) and the proposed Consumer Privacy Protection Act through the pending Bill C-11. Applying practices such as a Three Lines of Defence structure can assist insurers in meeting these obligations.

Effective date and future review

This Guidance became effective on September 20, 2022 and will be reviewed no later than September 20, 2025.

About this guidance

This document is consistent with FSRA’s Guidance Framework. As Information Guidance, it describes FSRA’s views on certain topics without creating new compliance obligations for regulated persons.

Appendix 1: Model risk management

This appendix outlines foundational and sound practices specific to managing models used in:

  • Ratemaking: Any model used in the process of estimating future costs associated with the transfer of risk in insurance or other risk-transfer mechanisms[10]. This includes estimation of future costs in total as well as by the underlying levels that comprise the estimate of future cost.
  • Risk classification systems: Any model used to assign risks to groups based upon the expected future cost or benefit of the coverage or services to be provided.
  • Underwriting: Any model used to determine whether a risk should be written, renewed or cancelled.

Applying the operational risk management cycle to managing model risk

FSRA has observed how the principles-based nature of ORM practices can be applied effectively in Model Risk Management (MRM).

When insurers establish and maintain an effective internal ORM Framework, including controls such as the Three Lines of Defence model, this can minimize the risk that models are not developed and implemented in a fair and sound manner.

FSRA acknowledges insurers may already have standalone Model Risk Management frameworks. The practices outlined in this guidance should be adopted having regard to the proportionality principle, considering materiality of the models, as well as the size and complexity of the insurer.

Risk identification

The development of any model should be addressed by the insurer’s ORM Framework. Insurers should have a model inventory, enabling a comprehensive view of all models being used by the insurer.

Risk assessment

For each model identified, the insurer should assess the operational processes/controls in place used to manage model risk. Going through this exercise will allow the insurer to articulate the inherent operational risk in its modeling activities and the quality of risk management applied, from which it can assess the residual operational risk.

When an insurer has strong operational processes/controls in place then it should be able to mitigate the following model risks throughout the modelling process:

  • Inputs: Data issues such as inaccurate, inappropriate, insufficient, incomplete data, and misuse and/or misunderstanding of data.
  • Processing/computation: Model development issues such as flawed hypotheses and/or assumptions/judgment, inappropriate specifications, coding or calculation errors, unstable models, etc.
  • Outputs: Model implementation and monitoring issues such as inappropriate, improper or unintended usage, erroneous implementation (e.g., rating errors), misinterpretation of the model results, and inadequate monitoring and/or controls.

By understanding whether controls in place are adequate and whether they are being effectively applied, the insurer can then decide how to deal with an identified risk.

Risk prioritization and mitigation

An insurer can determine how a risk should be managed based on the level of risk determined through assessment. If this process differs for models, then it should be specifically addressed in the insurer’s ORM Framework.

Risk monitoring and reporting

Monitoring and reporting processes[11] enable insurers to identify whether the operational risk posed by a model remains within acceptable risk levels and whether the escalation of an issue is necessary, e.g., to the Senior Management and/or the Board of Directors.

Model risk management foundational practices

FSRA expects Model Risk Management should, at a minimum, include:

  1. clearly defined model materiality
  2. three Lines of Defence being applied throughout the model lifecycle
  3. model Approval Function
  4. process to assess model fairness

1) Clearly defined model materiality

The insurer’s ORM Framework should outline a process to assess and classify the materiality of models (e.g., complexity of model and financial impact) and also outline the corresponding governance requirements depending on the model materiality. Both quantitative and qualitative measures, when possible, should be considered.

The degree of governance/control in place for each step of the model lifecycle should at least be commensurate to the model’s materiality.

2) Three lines of defence being applied throughout the model lifecycle

Roles of Three Lines of Defence in model risk management include the following:

  • the first line of defence (e.g., model owner, developer, user[12], etc.) is responsible and accountable for managing the model risk
  • the second line of defence should be broadly comprised of two core functions:
    • Independent model review: This function is responsible for the independent vetting and validation[13] of models, providing conceptual and technical reviews of models developed and maintained, i.e., the full lifecycle of a model is reviewed (from inputs/data to outputs/monitoring).[14]
    • Model risk management: This function is responsible for establishing policies that explain the insurer’s Model Risk Management practices and maintaining the model governance framework. Examples include, among others: establishing policies on how model reviews will be prioritized, maintaining an inventory of models, maintaining an inventory of current, accurate and complete documentation,[15] tracking model findings, etc.
  • the third line of defence should perform independent periodic review and/or audit of internal model oversight to assess adherence to established policies and procedures

Based on the materiality of the model, the Three Lines of Defence should be adequately applied, and the corresponding documentation should be current, accurate and complete, in each step of the model lifecycle, which generally include development stage, implementation stage, monitoring/review stage for new/revised models:

  • During model development, first line should identify an economic or business rationale for developing a new or revised model. It should ensure documentation related to the model development process is comprehensive and addresses the modelling techniques adopted, and any assumptions/approximations employed. The second line’s review of the first line’s analysis and conclusions should be sufficiently independent and thorough.
  • During implementation, the insurer should ensure the approved model is implemented for its intended purpose and the implemented model reconciles with the intended model. Insurers should perform pre-implementation and post-implementation tests to mitigate the risk of errors and document their processes.
  • During monitoring, the model should account for emerging data to ensure it is still appropriate for use. Models should be subject to a periodic review, or a review might be initiated in instances where there has been a material event and/or change (e.g., changes in underlying business environment; changes in the size or scope of a business line; deterioration in book of business; changes in mix of business; deterioration in model performance; results of model audits).

3) Model approval function

To ensure clarity and accountability in deciding which model is sent for implementation, a Model Approval Function (MAF) should be implemented for the purpose of approving new/revised models for operational use.

The MAF may be a senior accountable person or a standalone internal committee or a function incorporated into an existing internal committee — insurers may decide what is appropriate for them, depending on the size and complexity of the insurer. The MAF should review all relevant materials as a basis to make its decision — this includes items such as the model’s results, materials from second line’s review, documentation, identified findings, etc. The ORM Framework should outline how it establishes the MAF.

Models approved for use by the MAF should satisfy all applicable legislative requirements and regulatory guidance. The MAF should also understand how any other model(s) may have materially influenced the development of an implemented model. The MAF should be presented with not only the model intended for implementation, but also details about the other models that influenced the one intended for implementation.

Depending on the size and complexity of the insurer, along with the materiality of the model being reviewed, it may be acceptable for the roles of model reviewer and approver to be combined, provided that the risk of potential conflict of interest is mitigated and independence of the MAF is maintained. The insurer’s policies/processes should outline how it handles this situation.

4) Process to assess model fairness

Insurers should have processes and tools to ensure there is no unfair discrimination in models used for rating and underwriting, throughout the modelling process:

  • Inputs: Insurers must ensure they are not using prohibited variables. Other examples of considerations include, among others: processes and controls that ensure the ethical use of data, enable insurers to detect errors and/or bias and mitigate the impacts, where possible.
  • Processing/computation: The goal of modelling should not be only to maximize predictive performance but to do so subject to a fairness constraint. Examples of considerations include, among others: whether there is alternative specification of the model, which has less adverse impact on a customer group but still achieves the level of predictive power/strength of quality.
  • Outputs: Insurers should ensure they have implemented measures that let them assess and track fairness of model outputs. Examples of considerations include, among others: a process/tool to ensure that the model’s outputs are aligned with its fairness objectives and other objectives, a process in place to detect unintended model use and unintended harms to individuals or groups during model monitoring and review.

Interpretability and explainability of AI/ML models

The complexity and automated nature of artificial intelligence (AI) and machine learning (ML) models can amplify the risk of models being developed in an unfair manner. To mitigate this risk, insurers should establish tools that enable the interpretability and explainability of AI/ML models, which FSRA has defined as follows:

  • Interpretability: The ability to understand a model’s soundness (e.g., understanding its mechanics, the model results and whether the results meet the model developer’s objectives).
  • Explainability: The ability to convey the model’s results and its drivers to stakeholders not involved in the development of the model (e.g., consumers, business partners, FSRA, etc.).

Explainability is particularly important as providing consumers with clear information enables them to make informed decisions, which is consistent with the goals of FSRA’s Fair Treatment of Consumers Guidance. FSRA acknowledges that the degree of explanation required may vary by stakeholders.[16]

Insurers should outline in their ORM Frameworks the processes/controls used to develop and operationalize these tools, ensuring they are used effectively and sustainably for AI/ML models.

Effective Date: September 20, 2022


1 Information Guidance: Indicative of FSRA views on certain topics without creating new compliance obligations for regulated persons. Refer to FSRA’s guidance framework and types of guidance.
2 FSRA acknowledges how the outlined foundational and sound practices may be applied across different products/business lines or implemented at the enterprise level by P&C insurers.
3 FSRA is developing Guidance to set out regulatory requirements to auto insurers regarding the regulatory reporting and resolution of rating and underwriting errors.
4 Sound Operational risk management practices identified in this Guidance are consistent with those promulgated by the Committee of Sponsoring Organizations (COSO) and International Organization for Standardization (ISO) and with guidelines set out by Office of the Superintendent of Financial institutions (OSFI) (namely B-10). Model risk management practices outlined in appendix 1 are consistent with OSFI guidelines E-23 and E-25.
5 FSRA Objects (see section 3): Financial Services Regulatory Authority of Ontario Act, 2016, S.O. 2016, c. 37, Sched. 8
6 See UDAP Rule for details: Unfair or Deceptive Acts or Practices
7 Operational resilience can be defined as the ability of a financial institution to deliver critical operations through disruption. Refer to Basel Committee on Banking Supervision, Principles for Operational Resilience, March 2021
8 Basel Committee on Banking Supervision, Revisions to the Principles for the Sound Management of Operational Risk, March 2021
9 A model generally refers to a methodology, system and/or approach that applies mathematical/statistical/actuarial techniques as well as theoretical and (professional) judgmental assumptions (including but not limited to actuarial assumptions) to process input data in order to generate quantitative estimates.
10 Definition source: Property / Casualty Ratemaking

11 Examples of Key Risk Indicators (KRIs) for model risk monitoring and reporting could include:

  • Number of models assessed with high residual risk
  • Number of models with significant deterioration in model performance
  • Number of models deemed unsuitable for their given purpose by the independent review
  • Number of overdue model reviews
  • Number of rating and underwriting errors and their impacts
  • Number of overdue reporting of rating and underwriting errors

12 Model owner is the first-line unit(s)/individual(s) responsible for the model selection, coordinating model development, initial testing, ongoing monitoring, outcomes analysis, administering changes and documentation. The model owner could also be the model developer who is responsible for designing, developing, evaluating and documenting models or model user who relies on the model’s outputs as a basis for making business decisions.
13 “Vetting” and “validation” are both review exercises used to determine the appropriateness of a model. To distinguish between both, “vetting” is used to evaluate whether a model is appropriate to place in production whereas “validation” refers to exercises used to ensure that a model in production is still appropriate for use.
14 The practices should be adopted having regard to the proportionality principle, considering materiality of the models, as well as the size and complexity of the insurer. For small insurers with low operational risk exposures and/or models with low materiality, independent model review may be achieved through independent peer review in the first line function.
15 For example, decisions and key related information around model approvals and implementation processes should be adequately documented.
16 Different stakeholders (e.g., Modellers, Reviewers, Regulators, Brokers/agents, Consumers) may need different level of details about an AI/ML model. Refer to the Transparency section of the “Auto Insurance Data and Analytics Strategy Technical Advisory Committee Report: Fair Treatment of Consumers in Uses of Big Data Analytics in Auto Insurance” for details.