Performance evaluation as a driver for change Health Quality - - PowerPoint PPT Presentation

▶
performance evaluation as a driver for change health
SMART_READER_LITE
LIVE PREVIEW

Performance evaluation as a driver for change Health Quality - - PowerPoint PPT Presentation

Jan 25, 2023 120 likes •748 views

Taking the pulse of Ontario: Performance evaluation as a driver for change Health Quality Transformation 2012 Walter P Wodchis Toronto, October 22, 2012 Leveraging the Culture of Performance Excellence in Ontario s Health System HSPRN is

slide-1
SLIDE 1

Taking the pulse of Ontario:

Performance evaluation as a driver for change

Health Quality Transformation 2012 Walter P Wodchis

Toronto, October 22, 2012 Leveraging the Culture of Performance Excellence in Ontario’s Health System

HSPRN is an inter-organization Network funded by the Ontario Ministry of Health and Long Term Care

slide-2
SLIDE 2

Some thoughts

  • Healthcare providers are doing their best

to provide optimal care in a severely challenged health system.

  • Challenges include fiscal constraint but

are primarily brought on by a severe lack

  • f coordination and communication

between providers.

2

slide-3
SLIDE 3

Ontario Leads in Performance Measurement

  • Ontario Hospital Report

 Public reporting on hospital performance 1999-2008

  • Ontario Hospital Association
  • Health Quality Ontario

 QMonitor: Annual Report on the Health System  Long Term Care Website Public Reporting

  • Ministry of Health and Long Term Care
slide-4
SLIDE 4

Audiences & Purposes for Performance Measurement

Measurement Groups

  • MOHLTC
  • LHINs
  • Health Quality

Ontario

  • Associations

(OHA)

  • CIHI
  • (Accreditation)
  • Collaboratives

/Research Groups

Audiences

  • Public
  • System

Managers

  • Providers
  • Payers
  • Regulators
  • Research

Community

Purposes

  • Accountability
  • Transparency
  • Market Function
  • Quality

Improvement & Benchmarking

  • New Knowledge
slide-5
SLIDE 5

Hospital Report Research Collaborative

slide-6
SLIDE 6

Hospital Report Research Collaborative

slide-7
SLIDE 7

Hospital Report Research Collaborative

slide-8
SLIDE 8

Hospital Report Research Collaborative

slide-9
SLIDE 9

MOHLTC Access

slide-10
SLIDE 10

MOHLTC Access

slide-11
SLIDE 11

myhospitalcare.ca (OHA. c2009-)

slide-12
SLIDE 12

myhospitalcare.ca (OHA. c2009-)

slide-13
SLIDE 13

Health Quality Ontario

slide-14
SLIDE 14

Health Quality Ontario

slide-15
SLIDE 15

Health Quality Ontario

slide-16
SLIDE 16

Health Quality Ontario

slide-17
SLIDE 17

Health Quality Ontario

SEARCH CRITERIA

INDIVIDUAL HOME RESULTS

Close Print

Falls

Incontinence Pressure Ulcers Restraints

Falls

Percentage of residents who had a recent fall.

By Home By Location By Health Topic Search: Showing 1 to 10 of 633 entries First Previous 1 2 3 4 5 Next Last Home Name Falls 2010-11 Falls 2011-12 A R GOUDIE EVENTIDE HOME(SAL. ARMY) N/A 18.4 AFTON PARK PLACE LONG TERM CARE COMMUNITY 19.1 11.8 ALBRIGHT GARDENS HOMES N/A 11.7 ALEXANDER PLACE 17.9 16.1 ALGOMA MANOR-HOME FOR THE AGED 11.4 13.1 ALGONQUIN NURSING HOME 8.3 12.1 ALLENDALE N/A 16.6 ALMONTE COUNTRY HAVEN N/A 18.0 ANSON PLACE CARE CENTRE 10.3 10.0 ARBOUR CREEK LONG-TERM CARE CENTRE 7.0 6.2

LTC Individual Results - Search By Health Topic http://www.hqontario.ca/en/reporting/ltc/modal/searchByHealt...

slide-18
SLIDE 18

Health Quality Ontario

slide-19
SLIDE 19

Canadian Institute for Health Information & Statistics Canada

slide-20
SLIDE 20

Hospital Report Research Collaborative

slide-21
SLIDE 21

Ontario

  • Ontario has a rich history and tradition of

performance measurement.

  • There continue to be some gaps in

performance measurement (e.g. primary care).

  • We have great data sources – we need to

do a better job of using them and particularly linking them together.

slide-22
SLIDE 22

How successful are these initiatives?

  • Performance measurement alone cannot

change the system

  • But the impacts & effects of changes to

the system cannot be known without performance measurement

  • Performance measurement can also direct
  • pportunities for improvement
slide-23
SLIDE 23

Where should we be heading?

  • We currently have quite a bit of

performance data on hospitals, long term care, home care

  • We have very little performance

information about primary and other physician care.

  • We have almost no information on health

system performance.

slide-24
SLIDE 24

24

Measurement that follow patients

Acute (ED, IP, SDS) LTC Rehab / CCC / Sub-acute Care CCAC

Patient Flow Patient Rebound

Home Care Specialist Care Primary Care Pharmacy

slide-25
SLIDE 25

Population-based Measurement and Management

  • We need to understand, track and report on

the performance of the health system for individuals

  • Triple Aim:

 Population health  Patient experience (including access, quality, and patient reported outcomes & experience)  Cost control

  • Achieving the Triple Aim requires a new

perspective on health system performance measurement and improvement

slide-26
SLIDE 26

Getting there

  • There are costs associated with performance

measurement – who benefits (who should pay)?

 Patients benefit from better care and information.  Public benefits from increased accountability.  Payers and regulators benefit from increased accountability / assessment of value/return.  Government benefits from ability to choose best areas to focus

  • n to improve.

 Providers benefit from more efficient and better quality of care (better care is why they’re in this – intrinsic value).

  • Value can be measured by the opportunity cost of not

doing anything (missed opportunities for improvement) and/or doing the wrong thing by focusing on the wrong areas.

slide-27
SLIDE 27

www.interrai.org Twitter: @interRAI_Hirdes

The Future of Performance Measurement in Long Term Care and Home Care:

It’s Not What It Used to Be John P. Hirdes, PhD

Professor, School of Public Health and Health Systems Ontario Home Care Research and Knowledge Exchange Chair University of Waterloo

27

slide-28
SLIDE 28

www.interrai.org

Agenda

  • State of the art information systems for nursing homes and

home care circa 1987

  • Use of interRAI instruments in Canada
  • Clinical practice
  • Quality
  • Integrated information systems
  • What worked?
  • What needs improvement

Twitter: @interRAI_Hirdes 28

slide-29
SLIDE 29

www.interrai.org

Butterworths Series on Individual and Population Aging

Series published between 1986-91 About 15 books published by leading gerontologists of the time Gerontology’s “coming of age” in Canada WF Forbes

  • Founding President Canadian Association and

Ontario Gerontology Association

  • Vice-President of Gerontological Society of America
  • Mentor and PhD Supervisor

Canadian Journal on Aging Special Issue on 25th Anniversary of series Twitter: @interRAI_Hirdes 29

slide-30
SLIDE 30

www.interrai.org

State of the Art in LTC Circa 1986

  • No national data for LTC beyond age and sex
  • Had to cite small pilot studies for basic clinical information
  • Called for implementation of standardized assessment

systems

  • But concerned that introduction of computers into LTC may be

difficult

  • Limited conceptualization of quality measurement
  • Focus on survey based methods
  • Could not conceive of QI based methods because standardized

clinical information was far-fetched at the time

  • Worried about adequacy of evidence for informing

placement into long term care

Twitter: @interRAI_Hirdes 30

slide-31
SLIDE 31

www.interrai.org

The Future of LTC

  • Authors worried about efforts to make more comfortable

nursing homes without

  • improving clinical services
  • emphasizing on interventions to prevent decline or support

rehabilitation

  • Drew analogy to National Foundation for Infantile Paralysis

in1940-50s investment into basic research on polio rather than creating better iron lungs

Twitter: @interRAI_Hirdes 31

slide-32
SLIDE 32

www.interrai.org

The Future of LTC?

  • “… one might argue that
  • ld age homes and vast

support structures to care for the helpless aged represent the iron lungs

  • f gerontology. They

maintain the elderly, yet do not address the underlying problems.”

Forbes et al., 1987

Twitter: @interRAI_Hirdes 32

slide-33
SLIDE 33

www.interrai.org

Implementation & Testing of interRAI Instruments 1996

Solid symbols – mandated or recommended by govt; Hollow symbols – research/evaluation underway

RAI 2.0 RAI-HC RAI-MH interRAI CMH interRAI ESP interRAI PC interRAI ID interRAI ED/AC interRAI CA interRAI CHA interRAI AL interRAI LTCF interRAI SQoL

Twitter: @interRAI_Hirdes 33

slide-34
SLIDE 34

www.interrai.org

Implementation & Testing of interRAI Instruments in Canada

Solid symbols – mandated or recommended by govt; Hollow symbols – research/evaluation underway

RAI 2.0 RAI-HC RAI-MH interRAI CMH interRAI ESP interRAI PC interRAI ID interRAI ED/AC interRAI CA interRAI CHA interRAI AL interRAI SQoL

DB

34 Twitter: @interRAI_Hirdes

slide-35
SLIDE 35

www.interrai.org

Applications of interRAI’s Assessment Instruments:

One assessment … multiple applications

Assessment Care Plan Outcome Measures Quality Indicators Resource Allocation Prevent Gaming Evaluate Best Practices Case-mix Single Point Entry Patient Safety Quality Improvement Public Accountability Accreditation

35 Twitter: @interRAI_Hirdes

slide-36
SLIDE 36

www.interrai.org

Beyond the “Iron Lungs of Gerontology”:

Using Evidence to Shape the Future of Nursing Homes in Canada

  • Hirdes, Mitchell, Maxwell &

White, Canadian Journal on Aging, 2011

  • Provides a national profile of

LTC residents

  • Most comprehensive clinical

profile of LTC in Canada to date

Twitter: @interRAI_Hirdes 36

slide-37
SLIDE 37

www.interrai.org Twitter: @interRAI_Hirdes 37

slide-38
SLIDE 38

www.interrai.org Twitter: @interRAI_Hirdes 38

slide-39
SLIDE 39

www.interrai.org

Triggering rates for two multi-level interRAI Clinical Assessment Protocols (CAPs), by prov/terr & setting

20 40 60 80 100 % Clients triggered Moderate Risk High Risk

Falls CAP

CCAC (HC) SH (CHA) LTC (2.0)

Mood CAP

CCC (2.0) CCAC (HC) SH (CHA) LTC (2.0) CCC (2.0)

slide-40
SLIDE 40

www.interrai.org

10 20 30 40 50 ON LTC ON CCC MB LTC SK LTC BC LTC Risk Adjusted Facility QI Rate 90th Percentile 10th Percentile Median

Twitter: @interRAI_Hirdes 40

Distribution of Risk Adjusted Facility Rates for ADL Decline QI (ADL7D) by Province and Facility Type, Canada 2009-10

Better

slide-41
SLIDE 41

www.interrai.org

Health Quality Ontario, LTC Public Reporting Website 2012, Provincial Results Display

41

slide-42
SLIDE 42

www.interrai.org

0% 5% 10% 15% 20% 25% 30% 35% 40% 45%

Percent of Residents Facility

Percent of residents who fell in the last 30 days, by Ontario LTC home

Source: Q4 1112, CCRS, CIHI; graph prepared by Health Quality Ontario

10th Percentile, 8.5% Median, 13.7% 90th Percentile, 18.8%

42

slide-43
SLIDE 43

www.interrai.org

0.0% 5.0% 10.0% 15.0% 20.0% 25.0% Yukon (n=4) Nova Scotia (n=6) Newfoundland and Labrador (n=7) Manitoba (n=38) British Columbia (n=274) Ontario (n=637) Indicator Rate

Percent of residents who fell in the last 30 days, by province

Source: Q4 1112, CCRS, CIHI; graph prepared by Health Quality Ontario The box-plots show the location of the 10th, 25th, median, 75th, and 90th percentiles.

43

slide-44
SLIDE 44

www.interrai.org

Taking Stock of What We’ve Achieved

Twitter: @interRAI_Hirdes 44

slide-45
SLIDE 45

www.interrai.org

What Worked?

  • Education
  • Continuous, multidimensional training strategy is essential
  • In-person, webinars, eLearning, competency testing, conferences
  • Well-done in some sectors
  • Feedback
  • System will fail if it seems like the data go into a “black hole”
  • Different cycle times for different audiences
  • Clinicians  immediate
  • Managers  monthly
  • General public  annual
  • Benchmarking
  • Sharing data allows you to observe yourself through a different lens

Twitter: @interRAI_Hirdes 45

slide-46
SLIDE 46

www.interrai.org

What Worked? cont’d

  • IT systems
  • Usability, reports, real time access
  • Tremendous variability across vendors
  • Partnerships
  • CIHI – interRAI partnership made national implementation feasible
  • Provincial and local partnerships supported innovation
  • Research
  • Strong research support by some provincial ministries have

facilitates major interRAI innovations

Twitter: @interRAI_Hirdes 46

slide-47
SLIDE 47

www.interrai.org

What Needs to Improve?

  • Clinical use in some sectors
  • Exchange of interRAI data between partner agencies
  • Privacy sometimes used as an excuse
  • Power and trust interferes with person’s interests
  • Better use of longitudinal information
  • Integrated Assessment Record project has considerable potential
  • Need a longitudinal view WITHIN and BETWEEN settings
  • Integration concept not a “guaranteed sell” in every sector
  • Need to move to new suite and new CAPs
  • Changing the ENTIRE system is a challenge!
  • Engaging the person and family
  • Human nature seeks path of least resistance

Twitter: @interRAI_Hirdes 47

slide-48
SLIDE 48

www.interrai.org

Thank you!

Questions? Comments?

Twitter: @interRAI_Hirdes 48

slide-49
SLIDE 49

Canadian Institute for Health Information

49

  • Dr. Jeremy Veillard

Vice President of Research and Analysis Canadian Institute for Health Information

slide-50
SLIDE 50

Enhancing Ontario’s Capacity For Change: The Power of Performance Measurement, Reporting and Improvement

50

slide-51
SLIDE 51

Why Is It Important To Measure Health System Performance?

Ideally, health system performance measurement and reporting can ensure that:

  • Health systems have a strategic direction focused on

improving health outcomes and clearly communicated to system players

  • Policy decisions are informed by appropriate intelligence

with regard to health and health system problems and their determinants

  • Healthy public policies are promoted across all aspects
  • f government
  • Relationships between all health stakeholders are regulated

in a context of transparency and accountability, which is an important condition for performance improvement

51

slide-52
SLIDE 52

Health System Performance Measures Can be Used for Different Purposes

52

Formative Punitive Internal Continuous Quality Improvement Internal Evaluation External Accreditation Performance and Practice Benchmarking P4P Blaming and Shaming

Nature of expected actions Source of control

Source Modified from Boland & Fowler 2000.

slide-53
SLIDE 53

Is the Ontario Context Supportive

  • f Health System Performance

Measurement and Public Reporting?

  • A clear interest for more comparable performance

measurement, in a supportive way

  • Perception of an “indicators chaos”: need to focus on core

measures meeting the needs of various audiences

  • ECFA supportive of the use of evidence in decision-making

and alignment between performance measurement and various policy levers (incentives, funding, QIP)

  • An increasing interest in innovation and practice

benchmarking, not only performance benchmarking

53

slide-54
SLIDE 54

A Positive Approach to HSP Reporting Should Integrate Performance Measurement and Peer Learning

Objectives Types of activities Type of Benchmarking Type of measures

  • Transparency

Report on a set of standardized, comparable indicators on HSP Public Reporting Outcomes

  • Transparency
  • Performance

Improvement

  • Cooperation

Integrated performance reporting with business intelligence capabilities to drive performance benchmarking Performance Benchmarking Outcomes Process

  • Transparency
  • Performance

Improvement

  • Cooperation
  • Capacity

building On the basis of practice benchmarking activities, support best practices transfer and innovation sharing between jurisdictions Practice Benchmarking Outcomes Process Structure

Provincial policy makers Regional Health Authorities Public HOSP LTC Others PHC

54

slide-55
SLIDE 55

Unleashing the Power of Performance Measurement in the Ontario Context

Building blocks required:

  • A unifying health system performance measurement system aligned

with government priorities within a broader framework

  • A suite of public performance scorecards meeting the needs of the

general public, of policy makers and of system managers

  • An integrated analytical environment responsive to the analytical

needs of system managers

  • A shared analytical and research agenda focused on performance

improvement priorities

  • Initiatives to support capacity building for system managers across

the health system

55

slide-56
SLIDE 56

Strong Foundations in Ontario To Harness The Power of Performance Measurement

  • ECFA has all elements necessary to drive performance

improvement in a supportive way

  • HQO has a mandate to focus the performance measurement

agenda and has a unifying performance framework

  • A tradition for public reporting on health system performance

with some level of comfort by system actors

  • Access to some of the best data in the world in comparable ways

through CIHI with new data and benchmarking capabilities

– CHRP – PHC VRS – CCRS

56

slide-57
SLIDE 57

Strong Foundations in Ontario To Harness The Power of Performance Measurement (cont’d)

  • Access to various analytical environments through which system

managers can drill down through the data (CIHI, ICES, CCO)

  • Relevant analytical and research work (ICES, CIHI, HQO, CCO,

Academia, etc…)

  • Some initiatives to support capacity building for system

managers through ECFA implementation

57

slide-58
SLIDE 58

Some Progress Can Still be Made in Ontario with the Data

  • Some of the best quality data in the world

– Vital Statistics; administrative data and clinical registries; patient experience surveys; population surveys; electronic medical records

  • Still important data gaps to fill

– Efficiency and productivity measurement; primary health care (VRS well implemented in Ontario); measurement of integration of care in general

  • Hard to integrate the data for technical (less and

less true) and cultural reasons (health care versus population health and social determinants of health)

58

slide-59
SLIDE 59

Noticeable Critics About The Performance Management Paradigm in Health Care

  • Risk of focusing on the wrong performance metrics and
  • f inducing wrong behavior changes
  • Quantitative indicators not always the most appropriate

to understand how the system performs

  • Difficulty to get actors to take their responsibilities for

shared accountability measures (e.g. ALC, readmissions rates)

  • Interdependencies between the different dimensions
  • f health system performance still poorly understood
  • You cannot manage the performance of a complex

system with a small number of performance metrics

59

slide-60
SLIDE 60

The Path Forward for Ontario

  • Regular public reporting on state of health system

performance in ways meeting needs of various audiences (public engagement)

  • System managers to access integrated

analytical environment linking permanently larger range

  • f databases yet still respecting privacy
  • An agenda to integrate the data at two levels:

across the continuum of care; patient and population health perspectives

  • Improve the timeliness of the data

60

slide-61
SLIDE 61

The Path Forward for Ontario (cont’d)

  • Focus on capacity building in the system and support the

transition from performance benchmarking to practice benchmarking (structure peer learning networks)

  • Greater research and analytical focus on key systems’

priorities: quality of care, health system outcomes, patient safety, value for money

  • Mobilize key enablers (e.g. HQO, ICES, CIHI) to

focus their efforts on needs for performance improvement in Ontario

  • Engage the public and patients in this work

61

slide-62
SLIDE 62

62

Thank you