Building & Improving a Performance Management System Public - - PowerPoint PPT Presentation

Building & Improving a Performance Management System Public Health Foundation

Utah Department of Health Amanda McCarty, MS, MBA

…PHF Mission:

We improve the public’s health by strengthening the quality and performance of public health practice

Innovative Solutions. Measurable Results.

www.phf.org

2

Wor Works ksho hop p Ob Objec jective tives

– Learn to use a PM System model based on Quality Improvement (QI) principles, the Essential Services of Public Health, and the domains of PHAB to structure a PM system for your public health agency – Practice using parts of the model so you can help agency program managers develop goals, objectives, and performance measures in ways that support improving health outcomes – Target engagement of key stakeholders to help make PM system development successful

3

Agenda

• 1. Turning Point-compatible PM System Model

based on Quality Improvement (QI) Cycles

• 2. Parts of a QI-based PM System:

– Aligning goals and objectives for a selected program – Drafting performance measures for a selected program

• 3. PM System Development as QI: Design,

Deployment, Assessment, & Improvement

– Analyzing stakeholders to engage in system development

• 4. Action Planning

4

Overview: Context The Turning Point Framework

5

• Performance Standards: Organizational or system expectations

to improve public health practices based on internal or external goals or benchmarks

• Performance Measures: Clearly defined indicators for collecting

data to assess achievement of standards

• Reporting of Progress: Documenting and analyzing results vs.

expectations and communicating such information as feedback to guide future performance improvement decisions

• Quality Improvement: A process to manage change and improve

performance in public health policies, programs, or infrastructure based on standards, data, and reports

The Four Major Parts of Turning Point

6

The Turning Point Framework

– A good description of what you need for performance management but does not provide guidance on developing an

• rganization-wide Performance Management System

– Questions this model leaves unanswered are: – How do we select standards & measure against them? – What process do we use to determine what programs or practices to improve? – How do we make this a “system” to manage our

• rganization, not just more things to do?

– How do we do it in the PH context, e.g., incorporating SHA/CHA, SHIP/CHIP, MAPP, Strategic Planning, and Accreditation? 7

Performance Management & Performance Management Systems

Performance Management: Using performance information to help make better decisions. Performance Management System: Using performance information on a regular basis as part of a continually repeated cycle of performance monitoring, analysis, and improvement, in which measured results are fed back into decision making to improve future performance.

8

QI “PDCA” Cycle Meets the PM System Definition

9

Check Act Do Plan

Sometimes called “PDSA” for “Plan-Do-Study-Act”

Develop improvement plans & targets Implement improvements & collect data Monitor, analyze, & interpret data Decide on further improvements As applied to specific processes & practices

Organization-wide PM Systems

– Applies a PM cycle to management and governance systems throughout the organization, usually involving:

• Planning: strategic, policy, operational, & financial (e.g., budget)
• Operations: program & policy implementation
• Monitoring & analysis of performance at all levels: from strategic

to operational (sometimes to employee level)

• Decision making to sharpen plans & strategies and improve

performance at all levels

– Can incorporate PH context such as SHA/CHA, SHIP/CHIP, the essential services of PH or domains of PHAB – Can be viewed as a large-scale “systemic” QI PDCA Cycle

10

QI Framework for a Public Health Organization-wide PM System

11

Plan

SHIP/CHIP (e.g., MAPP) Strategic Plans Operating Plans Financial Plan (Budget)

Do

Program & Policy Implementation Operational QI

Check

Performance Monitoring & Reporting for the Population & Organization (incl. SHA/CHA, MAPP) Analysis of Results

Act

Performance-based Decision Making Decisions on Changes to Improve Results

P D C A

Expectations Data Information Direction

Engagement of:

• Leaders & managers
• Employees
• Partners
• Other stakeholders

Info Technology Backbone

QI Framework for a PH PM System Includes All Four Turning Point Quadrants

12

Plan

SHIP/CHIP (e.g., MAPP) Strategic Plan Operating Plans Financial Plan (Budget)

Do

Program & Policy Implementation Operational QI

Check

Performance Monitoring & Reporting for the Population & Organization (incl. SHA/CHA, MAPP) Analysis of Results

Act

Performance-based Decision Making Decisions on Changes to Improve Results

P D C A

Expectations Data Information Direction TP: Performance Standards & Performance Measures TP: Quality Improvement TP: Reporting of Progress

Plan

SHIP/CHIP (e.g., MAPP) Strategic Plan Operating Plans Financial Plan (Budget)

At least 4 types of plans should be aligned: they should mutually support each other SHIP/CHIP STRATEGIC PLAN OPERATING PLANS (“Business Plan,” “Service Plan,”

• r “Performance Plan”)

BUDGET

13

Focuses on strategic change & efforts to support SHIP/CHIP Covers all programs or

• rganizational units

14

Views from Different Parts of the System

30,000 ft. 10,000 ft. Ground 20,000 ft.

Vision & Mission SHIP/CHIP with Health Outcome Priorities Strategic Plan with Priority Change Goals Performance Budget with Negotiated Targets

SHA/CHA & Public Health Policy Priorities Altitude

Operating Plan with Objectives, Performance Measures, Improvement Plans, & Initial Targets Programs, Services, Projects & Initiatives, Performance Monitoring & Improvement

Alignment of Plans Across the Organization

– The PHAB domains (including the Essential Services of PH) provide a way to align all

• perating plans of a PH agency

– In addition to alignment of the SHIP/CHIP and strategic plan – The operating plans of all programs, projects, or

• rganizational units must be:
• Aligned with the public health agency’s mission

and goals

• Consistent (or not inconsistent) with the

strategic plan

15

Goal

• al Alig

Alignme nment nt is is NOT the the Sam Same e as as Goal

• al Pr

Profu

• fusion

sion

Top Level Strategy Map

17

Zoom in: Maps for programs or

• rganizational

units Zoom in again: Measures, targets, timeframes for program objectives Structures the performance mgt. system

Casc Cascad ading T ing The he Str Strate tegy Ma y Map by p by Pr Prog

• gram

am Or Or Unit Unit

Questions to ask your organization

Who are the stakeholders in Performance Management? What important things happened over the past 12-18 months and how did the team perform? What important issues does the team face? Are there things that the organization/work group/program is doing that it should not be doing or that it could modify? Are there things that the team is neglecting to do that it should do? What things could the team do that would help you perform better?

Measurement Problem

– Top strategic goals & measures may be outcome- focused:

• Reduce number of smokers in our service area by 20,000/year

– But as deployment goes down to programs and

• rganizational units the measures often become more
• utput or process focused, e.g.,
• Number of stop smoking clinics held
• Number of pamphlets handed out at a Stop Smoking Fair

– A disconnect occurs:

• Is it credible that doing well on these program measures will

achieve the desired outcome?

19

Solving the Measurement Problem

– Help program staff understand that they need to:

• Use evidence to show that improving results on their output or

process measures really does drive outcomes, or …

• Develop new measures and targets that have an evidence-base,
• r …
• Use the PDCA cycle to develop their own evidence.

– Ultimately, program measures

• Can be of any type (e.g., output, process, timeliness, customer

satisfaction, intermediate or “participant” outcomes)

• So long as improvement in the program measures logically

contribute to improving outcomes, i.e.:

• Improving program measures drives better outcomes

20

Or is a “Measurement Problem” Really a “Strategy Design Problem”? – Program measures may be inadequate to drive

• utcomes because the program cannot adequately

address an issue on its own – Many public health issues can only be adequately addressed by multiple programs and partners – Efforts of all partners must be strategically-aligned and measurement of outcomes and drivers must reflect efforts and accomplishments of all partners

21

Highest Level Outcomes

22

Population Outcomes Place Outcomes

23

Lagging & Leading Indicators

Population & place outcomes tend to “lag” … So you need short term or intermediate actions that lead to

enable mid-course corrections to your strategy.

Some intermediate actions are “participant outcomes” or other “intermediate outcomes”

24

Performance Action to Outcomes

Chains of Success in a Performance Management System

Program Action Plans Long term Population

• r Place

Outcomes Program activities, including short term process outcomes & intermediate outcomes

Environmental Health Enforcement & Investigation Example

25

Goal: Minimize EH Risks & Disparities in Risk Objective: Enforce Environmental Health Codes Performance measure: Number establishments in top safety tier for its type Performance measure: % inspections on schedule (including re-inspections) Outcome measure: Total outstanding EH violations Outcome measure: Number EH-related illnesses & injuries per 1,000 residents Outcome measure: Number food borne illnesses per 1,000 residents Objective: Develop Policies that Incentivize Compliance Performance measure: % fee & fine schedules updated & approved by BOH Objective: Engage the Community to Reduce Need for Enforcement Performance measure: Number of targeted establishments that participate in training

Tools for Implementing a PM System

Establishing Agency, Division, Office or Program Goals & Objectives Goal 1 Goal 2 Goal 3 Objective 1 for Goal 1 Objective 1 for Goal 2 Objective 1 for Goal 3 Objective 2 for Goal 1 Objective 2 for Goal 2 Objective 2 for Goal 3 Objective 3 for Goal 1 Objective 3 for Goal 2 Objective 3 for Goal 3

Objective Alignment Exercise

For your group’s program:

1. For each box at the top, enter an organizational or “health status” goal that the group’s program is responsible for reach, or aligns with the program. 2. For each organizational or “health status” goal, identify one or more objectives, especially outcomes, and enter them in the spreadsheet under that goal. – Objectives are

• Specific
• Measurable
• Achievable
• Results-Oriented
• Time-bound

27

Example

Establishing Measures for Objectives/Activities % of eating estblishments inspected at least 1x every 12 months % of eating establishments that pass inspection # of eating establishments the fail re-inspection after first failure Associated Measures Associated Measures Associated Measures Objective 1 for Goal 1 Objective 1 for Goal 2 Objective 1 for Goal 3 Enforce EH codes and laws Goal 1 Goal 2 Goal 3 Improve EH outcomes and eliminate disparities

Key Attributes of a Performance Measure

Validity... a valid measure is one that captures the essence of what it professes to measure. Reliability... a reliable measure has a high likelihood of yielding the same results in repeated trials, so there are low levels of random error in measurement. Responsiveness ... a responsive measure should be able to detect change. Functionality... a functional measure is directly related to objectives. Credibility... a credible measure is supported by stakeholders. Understandability... an understandable measure is easily understood by all, with minimal explanation. Availability... an available measure is readily available through the means on hand.

Measure Alignment Exercise

Associated Measures Associated Measures Associated Measures Objective 2 for Goal 1 Objective 2 for Goal 2 Objective 2 for Goal 3 Investigate and contain EH hazards # of confirmed new food borne illnesses per quarter # of qualified homes given a home lead testing kits per quarter

PHAB Domains Alignment Exercise

Domain 1 Monitoring Health Status to Identify and Solve Community Health Problems 2 Diangosing and Investigating Health Problems and Health Hazards in the Community 3 4 5 6 7 8 9 10 Research for New Insights and Innovative Solutions to Health Problems Essential Service Description Informing, Educating and Empowering People About Health Issues Mobilizing Community Partnerships and Action to Identify and Solve Health Problems Developing Policies and Plan that Support Individual and Community Health Efforts Enforcement of Laws and Regulations that Protect Health and Ensure Safety Linking People to Needed Personal Health Services and Assuring the Provision of Health Care When Otherwise Unavailable Assuring a Competent Public and Personal Health Care Workforce Evaluating the Effectiveness, Accessibility and Quality of Personal and Population-Based Health Service

PHAB Domains Alignment Exercise

Domain Domain Domain Objective 1 for Goal 1 Objective 1 for Goal 2 Objective 1 for Goal 3 Enforce EH codes and laws Associated Measures Associated Measures Associated Measures % of eating estblishments inspected at least 1x every 12 months 6 % of eating establishments that pass inspection # of eating establishments the fail re-inspection after first failure 1 2

Data Alignment Exercise

For each measure, consider the following: Data Considerations Objective 1 for Goal 1 Objective 1 for Goal 2 Objective 1 for Goal 3 Where are we going to get the data? How are we going to get the data? What is the data source? What is the data frequency? Data Notes Associated Measures Associated Measures Data Notes Associated Measures

Data Frequency

For your group’s program: Identify any performance measures for which you think you should get data more often than quarterly for

• perations management. Enter the frequency (e.g.,

monthly, weekly, daily) under “Data Frequency”

34

Final Spreadsheet

Operating Plan Goals, Objectives, & Measures for:

Accreditation

Goal 1:

Objective or Activity: Enforce EH codes and laws PHAB Domain Previous Period Current Period Target

& No. Periods QI Plan Notes

Improve EH

• utcomes and

Objective or Activity: Investigate and contain EH hazards 3

173.00 100.00 80.00

2

2.00 3.00 3.00

State Health Department

% of eating estblishments inspected at least 1x every 12 months % of eating establishments that pass inspection Measures: # of eating establishments the fail re-inspection after first failure

Objectives & Performance Measures

Measures:

95% 95%

6

quarterly

# of confirmed new food borne illnesses per quarter # of qualified homes given a home lead testing kits per quarter 6 1 2

Trend Direction Support & Documentation

0.00 0.00 3.00

6 quarterly

Frequency

72% 68% 100%

6

quarterly

96%

1 3 2 2 1 1

Trend Line of a Measure vs. Target

99% 98% 98% 84% 80% 88% 99%

Percent inspections on schedule (including re-inspections)

36

Target = 95% Actual

95% 99%

PLA PLAN

– Performance Management System functionality:

• Align operations with strategy (in SHIP/CHIP & strategic plan)
• Guide day-to-day operations & encourage regular, frequent QI
• Help identify improvements needed at all levels
• Support decisions to improve performance & strategy

– PM system development usually led by a design or implementation team:

• Staffed with “owner” (member of leadership), “cross section” of users

(Program & Division Staff), IT staff, consistent with PHAB 9.2.1 A:

• A current, functioning PM committee or team.
• Seeks broad engagement from stakeholders, as in PHAB 9.1.1 A:
• Engage staff at all organizational levels in establishing … a PM system.
• Defines purpose of the PM System & budget or resources to run it
• Decides whether to use the approach and spreadsheets provided here,

revise them, or use a different model 37

Performance Management Team

– Design/Implementation team transitions into an ongoing “PM SYSTEM TEAM” to:

• Decide on software, hardware, user operating guidance
• Does this system meet your needs? Does it need changed?
• Guide initial deployment & system improvement over time
• Ensure the PM system has adequate ongoing resources to run

effectively: so it meets user needs on time, all the time.

• Meets PHAB Measure 9.1.2 A on an ongoing basis:
• A current, functioning PM committee or team.

– “System Ownership” group that:

• Has full accountability for the system
• Makes sure the system meets the needs of its users to continually

improve performance

• Entertains & decides on user requests for changes
• Spend significant time focused on the system long-term

38

DO DO: De Deplo ploy y th the e Sy Syst stem em

P D C A

D E P L O Y M E N T F E E D B A C K

39

DO DO: Dep Deploy loy th the e Sys Syste tem – For successful PM system deployment, assess stakeholders and stakeholder groups for:

• Expected level of their support for system development and

deployments (e.g., what barriers, if any, are they associated with?)

• Their level of influence: How badly you need them for the PM

system to succeed

– Involve stakeholders differently based on that assessment.

40

Concluding Discussion, Assignment, Follow-up

Discussion:

• Wrap Up
• Questions?
• Reflections on the day?

Next Steps:

• Engage staff at all organizational levels
• Develop agency-wide Goals, Objectives & Measures
• For all measures, determine & enter an “Annual Target.” As

possible, Break down into quarterly targets & enter in those fields.

Fo Foll llow

• w-up

up Sch Sched edule ule

• Post-Workshop Webinar
• Discuss Implementation & Maintenance

41