Ottmar Bongers Chicago, 17 th Oct. 2012 | 06.10.2012 | Seite - - PowerPoint PPT Presentation
The 2012 Moody's Analytics Risk Practitioner Conference The Validity of Validation Ottmar Bongers Chicago, 17 th Oct. 2012 | 06.10.2012 | Seite 1 List of Disclaimer 1. All characters/financial institutions in this presentation are
The 2012 Moody's Analytics Risk Practitioner Conference The Validity of Validation
Ottmar Bongers Chicago, 17th Oct. 2012
List of Disclaimer
are fictitious, any resemblance to real persons/financial institutions, living or dead is purely coincidental.
BaFin but are just the presenter’s personal view.
The Validity of Validation
Models in the financial crisis
enormous uncertainty
e.g. ABS, CDOs
not really a good track record lessons learned ?
Regulatory Changes for Internal Models (Pillar 1)
capital increase for market risk by factor X More Internal Models, more capital, more need for validation!
Changes in the Industry on Models ?
But for how long ? More model use, more challenge, more need for validation!
The Validity of Validation
2.1 Scope & Objectives of Model Validation 2.2 Validation Process 2.3 Documentation – Report - Action
2.1 Scope & Objectives of Model Validation
Model Validation is … Fit for purpose? … set of processes and activities … to verify that models are performing as expected, in line with their design objectives and business uses … to identify potential limitations and assumptions and access their possible impact * * FED/OCC, Supervisory Guidance on Model Risk Management, 4.4.2011
2.1 Scope & Objectives of Model Validation
EC regulatory capital
VaR/ST per silo:
MR CR OpR BR LR..
valuation models enterprise risk- and capital models risk models per risk type
parameter estimates (e.g. IRB)
pricing + basis for risk models
Objects of Model Validation
2.1 Scope & Objectives of Model Validation
F O R E C A S T
risk model: much more than a formula
exposure data historic/market data exp. data exp. data proxies
risk drivers R I S K M O D E L model design parameter risk measures VaR Stress Test
2.1 Scope of Model Validation
model design model input : parameter/assumptions/ exposure data / market data processing reporting
Comprehensive Validation
risk analysis / model coverage use of the model results
2.1 Scope of Model Validation Objectives of comprehensive model validation:
efficient model based risk/capital management processes
2.2 Model Validation Process
risk analysis/model coverage per risk type/per business lines only validation of model design: for model development only for model audit only model input: no periodic review no data quality check processing: no quality metric no measurement of process quality model results: no assessment of model risk no escalation threshold Limited validity of validation !
2.2 Model Validation Process
risk analysis/model coverage: established periodic process per risk type and across silos/business lines: assessment of changes in products, clients, exposures, risk profile, market conditions validation of model design: model input: processing: model results: assessment of design/parameter/assumptions based on results of risk analysis and model use tolerance level for data and process quality measurement of key process indicators assessment of model risk well defined level of comfort for model risk escalation thresholds
2.3 Documentation – Report - Action
documentation: lack of governance, policies and procedures lack of documented validation approaches lack of documented validation results validation report: tons of statistical tests information overkill no assessment of materiality no assessment of impact on model use no recommendations action: none, at least not triggered by report Validation process neither efficient nor reliable for senior management nor auditable for internal audit!
2.3 Documentation – Report - Action Documentation
2.3 Documentation – Report - Action Validation Report
Efficient & documented processes & actionable reports are a prerequisite for the validity of a Comprehensive Validation !
The Validity of Validation
3.1 Objectives, Roles & Responsibilities 3.2 Aspects of EC Validation 3.3 Documentation – Report - Action
3.1 Objectives, Roles & Responsibilities
What purpose?
Without objective no validation ! EC-Model fit for purpose ?
3.1 Objectives, Roles & Responsibilities
not transparent
no improvements - no quality
3.1 Objectives, Roles & Responsibilities
cross silo risk analysis / risk management
Without “clients”, mandate, resources and effective challenge no validity of validation ! “effective challenge” is key criteria
The Validity of Validation
3.1 Objectives, Roles & Responsibilities 3.2 Aspects of EC Validation 3.3 Documentation – Report - Action
3.2 Aspects of EC Validation: Credit Risk
risk analysis: products, markets, clients model design/ e.g. credit spread – migration – default risk model coverage: trading/banking book market data: e.g. migration matrices parameter: IRB parameter PD, LGD, CCF asset correlation, R² exposure: e.g. industry mapping, CCR model based EaD * recalibration
* BCBS, Sound practices for backtesting counterparty credit risk models, 12/2010
in line with use: going/gone concern
3.2 Aspects of EC Validation: Credit Risk
fat tail transparency impact on EC involvement of Business Model risk management instruments
3.2 Aspects of EC Validation: Market Risk
risk analysis: products: proxies - full reval, risk factor mapping markets: segments RF, liquidity holding period strategies: pricing factors, risk factors model design: credit spread – migration risk holding period, transformation x days to 1 year model coverage: risks not in EC, estimates market data: quality? representative? stale? exposure: 2nd order risk risk factor mapping quality of daily production across asset classes
3.2 Aspects of EC Validation: Market Risk
production
understand your p/l industrialized production Model risk management instruments
3.2 Aspects of EC Validation: Market Risk
fat tail transparency Needed: combined know how of markets, products, p/l, mapping, market data & model.
3.2 Aspects of EC Validation
Model risk, uncertainties and risks not in EC should be, transparent, accounted for & managed.
3.3 Documentation - Report - Action
Validation Report is key for EC model risk management. what should be done what was done
The Validity of Validation
Timely model adjustments can be a competitive advantage, a failure however can have disastrous consequences.
assessment relative absolute
Evolution in use of economic capital
High validity of EC validation can only be achieved by strong commitment of Board and Senior Management.
The Validity of Validation
Approval process for Internal Models
Internal Models Banks in Germany
Ongoing prudential model supervision
Risk proportionale intensity of prudential model supervision.
* : selected banks
Comprehensive Validation: „Self-Purification“ for
appropriate models & effective model use analysis RM requirements validation: fit for purpose? adjustment
use
The Validity of Validation
The Validity of Validation: Conclusions
The Validity of Validation: Conclusions
for Managing the Business.
CRO FORUM, Internal Model Admissibility, April 2009
The Validity of Validation: Conclusions
ature
anagement