Model Risk Management Patrick Ferrell - AVP Nathan Schlindwein Sr. - - PowerPoint PPT Presentation

Model Risk Management

Patrick Ferrell - AVP Nathan Schlindwein– Sr. Auditor IASA April 27, 2017

Overview

• Company background
• Why the focus?
• Challenges
• Internal Audit’s role
• Implementation plan

2

RLI Profile

• Specialty Property/Casualty Insurance company

serving “niche” or underserved

• Traded on NYSE (RLI) – Sox compliant
• Operates primarily in the United States with
• ver 40 locations and more than 950

employees

• 2016 Financial Status
• Gross Written Premium of $875M
• Assets of $2.8B
• Consistently outperforms industry

profitability

3

Products We Offer

4

Underwriting Profit

5

RLI has achieved 21 straight years of a combined ratio* below 100

Model Risk - Defined

• The possibility of financial loss, incorrect

business decision, misstatement of external financial disclosures or damage to the company’s reputation arising from:

– Possible errors in the model design – Misapplication of model, or model results, by model users – Errors in data inputs or assumptions – Incomplete processing – Unauthorized changes

6

What is MRM?

• Definition: Model risk management formalizes the

approach to the design, implementation, use and governance of key models within the business

• Should be part of a broader ERM framework and report

to high level within company

• A robust MRM can mitigate risks and is becoming a vital

component of ERM and corporate governance

• Disclosure requirements - To date, NAIC is calling for

disclosure of model validation within ORSA; no guidance on expectations (leading to range of emerging practices)

7

Why the Focus on Model Risk Management (MRM)?

• Emphasis began with the banking industry
• During financial crisis, unexpected losses and incorrect

management decisions arose because management didn’t understand the intended purpose of the model

• North American CRO Council issued a paper in 2012 outlining

eight core principles for strong model risk management

• Factors increasing the importance of modeling and need to appropriately

validate models include:

• Growth of products requiring complex valuation models
• Regulator and rating agency expectations
• Critical models insurers use may not be subject to internal control testing
• r external audit
• Range of emerging validation practices

8

Why the Focus at RLI?

• Our Audit Committee and executive

management began asking what we were doing to mitigate this risk at RLI

9

Challenges for RLI

• Ownership
• Model definition
• Implementation

10

Challenge – Ownership

• Model risk management is a cross-company initiative involving

multiple departments and potential interdependencies – upstream and downstream processes

• Requires a cross-functional coordination with consistent

application of model risk ranking and control/documentation requirements

• Requires a broad knowledge of all departments and their

potential use of models

• RLI Solution: Internal Audit facilitates but does not own Model Risk

Management; Creation of a (cross-functional) Model Risk Governance Committee comprised of senior-level management

11

MRM for Insurers

12

* Source: PwC, “Insurance Model Risk Management Maturity Framework and Diagnostic Tool”, January 2014

Model Risk Governance Committee

• Consists of:

– President & COO – SVP, Risk Services – VP, CFO – VP, Corp Development (in charge of ERM) – AVP, IAS (ex-officio member)

• Responsibilities include:

– Approval and ownership of Model Risk Management Policy (along with any changes thereafter) – Approval for any changes to policy document or changes in MRM process as a whole

13

Roles and Responsibilities

• Model owner – Works with dept or product VP and

responsible for:

– Development of inventory of models used in their area – Risk ranking of each model – Documentation and testing of applicable controls on an annual basis

• Department or Product VP – in addition to the above,

also responsible for:

– Annual attestation regarding completeness of model inventory and operating effectiveness of controls around each model’s risk(s) – Reporting inventory and testing results to MRM Facilitator annually

14

Internal Audit’s Role

• Model Risk Management (MRM) Facilitator (currently

IAS) – responsible for:

– Maintenance of policy document – Gathering of model information company-wide and aiding departments in identifying higher-risk models – Facilitating update and attestation process annually – Reporting corporate model risk inventory and results of testing to Model Risk Governance Committee annually – Assist departments in identifying and designing appropriate controls and monitoring procedures

15

Challenge - Model definition

• “What is a model?”

– Every spreadsheet? – Complex calculations? – Statistical component?

• No right answer, but significant impact on

resources needed to implement effective MRM program

16

RLI’s definition

• A “model” consists of three components:

– An input component, which delivers assumptions and data to the processing component – A processing component, which transforms inputs into estimates – A reporting component, which translates estimates into useful business information

17

Model risk characteristics

• Key (higher risk) models are defined by the following

characteristics:

– Are key drivers of important decisions – Involve external communication or reporting (financial reporting, rating agencies, reinsurers, regulators) – Financial statement balances and/or disclosures rely upon the model and the financial statement balances are significant – The model is complex due to nature of algorithm or volume of inputs – The model results are not predictable or cannot benchmarked to another model

• Non-key (lower risk) models are identified by the

following:

– Used for general business decisions and model outputs are not directly recorded or disclosed in f/s – Financial statements or disclosures which rely upon the model are not significant

18

Implementation

• Creation and approval of Model Risk

Management Policy

• Creation of risk ranking and control criteria
• Development of model risk ranking template

and supplemental documentation worksheet

19

Risk Ranking and Control Criteria

• Criteria to be considered when evaluating individual model risks:

– Expertise of the user – Expertise of the model creator – Level of automation – Level of change control – External reporting – Likelihood and severity of error

• Criteria to consider when establishing and documenting mitigating

controls:

– Reconciliation – Secondary review – User access control – System edit controls – Independent validation

20

Example: Model Risk Template

21

Example: Model Risk Template

22

Example: Model Risk Template

23

Right-size Risk Weightings

• Majority of RLI’s models are owned by the Risk

Services Department

– Met with owner of Risk Services model to discuss Key and Non-key model risk rankings – Made adjustments to risk weightings based on discussion

24

Example: Risk Ranking Guidance

25

Example: Risk Ranking Guidance

26

Annual Assessment & Attestation Process

• Model Risk Management SharePoint site

– Maintains Inventory – Tracks Assessment and Attestation

27

Questions?