Managing Model Risk A Practitioner Perspective Douglas Gardner - - PowerPoint PPT Presentation

Managing Model Risk – A Practitioner Perspective

Douglas Gardner October 5, 2011

• Wells Fargo has a “hybrid” approach to model risk management, relying on multiple

teams

• Each line of business (LOB) or corporate function is responsible for managing all its

risks, including model risk which involves both model governance (e.g., model inventory, risk ranking, documentation, approvals, issue tracking) and technical

• validation. Model developers are responsible for practicing sound model

development procedures and documentation.

• Corporate Risk (CR) sets policies, oversees LOB model risk governance and

validation, and acts as a validation center of excellence for LOBs that choose to

• utsource validation.
• Internal Audit reviews compliance with policy at both the LOB and CR levels. It now

includes model SMEs in order to effectively review technical model requirements. For LOB reviews, there is close cooperation with CR regarding policy requirements and interpretation.

• Model risk committees at both the corporate and LOB level are employed to oversee

model risk management.

• Modeling seminars are used to disseminate modeling knowledge and model risk

management best practices.

1

Wells Fargo’s Approach

• OCC 2011-12 is the culmination of previous regulatory guidance, regulatory capital

requirements , industry best practices and previous examiner feedback – although a significant enhancement and framed somewhat differently , nothing new came as a large surprise.

• Challenge for most institutions likely lies with the details outlined in the new guidance

and the emphasis on several topics that have not received as much attention previously

• Board of Directors’ responsibilities
• Assessing aggregate model risk
• Detailed guidance around model development
• Detailed guidance around ongoing monitoring of models
• Vendor guidelines for model validation
• Continuous nature of process
• “Model risk is too important to be left just to quants”

2

Assessment of OCC 2011-12

Regulatory Expectations for Managing Model Risk

Mark Levonian Mark Levonian

Office of the Comptroller of the Currency

October 2011

New regulatory guidance New regulatory guidance

• OCC Bulletin 2011‐12 (April 2011)

Supervisory Guidance on Model Risk Management p y g

• Replaces OCC Bulletin 2000‐16

Replaces OCC Bulletin 2000 16

• Developed jointly with Federal Reserve

Developed jointly with Federal Reserve

• Available at www occ treas gov
• Available at www.occ.treas.gov

2 Levonian ‐ OCC ‐ October 2011

Risk should be managed Risk should be managed

• Using models within any business brings

potential for adverse consequences from p q decisions based on models that are incorrect

• r misused
• r misused
• Managing any risk requires:
• Managing any risk requires:

– Identifying the sources of risk – Assessing the risk – Mitigating or controlling the risk g g g

3 Levonian ‐ OCC ‐ October 2011

Some principles Some principles

• Effective governance and controls needed

Effective governance and controls needed

• Consider materiality
• Model development process matters
• Validation a core element but not the only
• Validation a core element, but not the only

element

– Is model performing as expected given design

• bjectives and business use?

– Validation depends on context of use

• “Effective challenge” is key

Effective challenge is key

4 Levonian ‐ OCC ‐ October 2011

What makes “challenge” effective? What makes challenge effective?

Incentives

Compensation structure Functional independence p Organizational culture T h i l t i i

Effective Challenge

Competence

Technical training Critical thinking Practical experience

Influence

Stature within organization Reporting lines

Influence

Reporting lines Management support

5 Levonian ‐ OCC ‐ October 2011

Governance and controls Governance and controls

• Model risk management (including validation)

is a process, not a one‐time event p ,

• Need for strong policies, procedures, testing

l d ibili i b l

• Roles and responsibilities must be clear

– Ownership, controls, and compliance

• Documentation is part of managing

B d f di t d i t l dit h

• Boards of directors and internal audit have a

role to play

6 Levonian ‐ OCC ‐ October 2011

www.occ.treas.gov/news‐issuances/bulletins/2011/bulletin‐2011‐12.html

7 Levonian ‐ OCC ‐ October 2011

Mark Levonian Mark Levonian

Office of the Comptroller of the Currency

Mark Le onian@occ treas go Mark.Levonian@occ.treas.gov

8 Levonian ‐ OCC ‐ October 2011

OCC 2011-12 Guidance on Model Risk Management

Creating a Comprehensive Model Risk Management Framework October 5, 2011

1

Roles and Responsibilities

MODEL OWNER Ensure models are properly developed, documented, tested, implemented and used. Ensure models have undergone appropriate validation and approval processes. Provide all necessary information for validation activities. Address model problems as they rise. MODEL VALIDATION/ MODEL RISK Responsible for overall Model Risk Program. Independent validation with focus on “effective challenge”. Maintain Model Inventory, Issues database, and provide reporting on Model Risk to Sr. Management and BoD. INTERNAL AUDIT Assess overall effectiveness of and compliance with model risk management framework for individual models and in aggregate.

• SR. MANAGEMENT AND BoD

Set bank’s model risk appetite. Responsible for ensuring a strong model risk management framework is in place. Reviews periodic model risk reporting.

Areas of Focus to Implement Guidance

• Create overall governance structure

– Model approvals – Issues escalation – Reporting on model risk

• Model Development Documentation and Testing
• Model Implementation Documentation and

Testing

• Model Change Management Process

– Change in model structure or assumptions – Change in use

2

Areas of Focus to Implement Guidance

• Ongoing model monitoring

– Performance monitoring – Risks and limitations, external conditions – Process risks- data and systems

• Model Validation

– Conceptual soundness – Vendor models – Outcomes analysis- backtesting, benchmarking, stress testing, sensitivity testing

• Policies and Procedures

3

Key Challenges

• Ensuring everyone understands and accepts

their role in model risk management

• Staffing
• Education
• Quantification of Model Risk

4