Increasing On-Time Resident Task Completion Melissa Wheeler, M.A., - - PowerPoint PPT Presentation

Increasing On-Time Resident Task Completion

Melissa Wheeler, M.A., C-TAGME Residency Education Coordinator, Orthopaedic Surgery 11/27/18

Agenda

• Define Problem and Clarify Scope
• Identify What is Important to Customer
• Establish Baseline Metrics and Verify Measurement System
• Visualize Current State and Identity Primary Root Cause(s)
• Identify Solutions Addressing Root Cause(s)
• Define Risks with Implementation and Risk Mitigation Plan
• Implement New Process and Develop Control Plan for Long Term Sustainability
• Lessons Learned
• Next Steps
• Questions and Answers

2

3

Define

Problem Statement From the period of April 2018 – June 2018 38% (62 out of 163

• pportunities) of resident tasks were not completed on time and were

therefore, delinquent. This is bad because, these tasks are requirements for ACGME, Medical Education, and/or the Ortho Program. If resident tasks are not completed on time it can result in the Resident being dismissed from the residency program. Goal Statement/Financial Impact The primary goal is to decrease the resident task delinquency rate from 38% to 23% by 9/30/18. Business Case Risk Reduction – By increasing the on-time completion rate of residency tasks, we are effectively reducing the risk of Residents being dismissed from the program. CMS pays Summa Health approximately $103,000 annually per resident so if they were dismissed from the program Summa would run the risk of losing those dollars. Additionally, by increasing the on-time completion rate we are increasing Resident and Program Coordinator job satisfaction. This in turn, ensures the sustainment of the program by effectively reducing the time spent remediating poor performance and reducing the risk of resident dismissal. Project Timeline 5/23/18 5/31/18 6/31/18 7/1/18 9/30/18 Project Scope Current Scope: Process Starts – Orientation for New Ortho Residents Process Ends – Checkpoint Evaluation Team Roles Project Sponsor — Dr. Junko Team Lead — Melissa Wheeler Process Engineer — Ashley Sedorovich Core Team Members — Melissa Maddie and Scott Gelman Subject Matter Experts — Ortho Attendings and Residents

Project Charter

Project: Increasing the On-Time Completion of Required Resident Tasks

4

Control Improve Analyze Measure Define

SIPOC Analysis

Supplier Input Process Output Customer

 ACGME Graduate Medical

Education

Orthobullets Ortho Program  Too many requirements  Not enough time  Duty Hour Restrictions  Personal motivation  Don’t remember to do it  Tedious & time

consuming

 Don’t see the value  Duty Hour reporting  Case Logs  Orthobullets Milestone

exam

 Orthobullets test

completion

 Orthobullets skill masters  Orthobullets evaluations  Other ACGME

requirements

 Other Graduate Medical

Education Requirements

 Total delinquency rate  Residents Attendings Residency Director Residency Coordinator

5

Orientation for New Ortho Residents Residents reminded to complete requirements If deadline passed, reminded again and notify Residency Director Resident pulled from surgical duty until task is completed Checkpoint Evaluation

Voice of the Customer

Customer VOC Affinity CTQ

Coordinator I am constantly having to remind residents to complete their tasks. I use multiple avenues to remind them and if they still don’t then I involve the program director. I literally have placed myself in front of the exit so they can’t leave until a task is completed. Ensuring Residents complete tasks 1. % of tasks completed on time 2. Time spent following-up with Resident Adam Domico Tasks don’t get completed because they are low on the priority list of things to do Managing priorities Amount of time to dedicate to non- patient care duties Jason Boyd I have survey fatigue, you can’t even buy a shirt without someone wanting a survey about your experience and if it’s not part of my routine, I will likely forget to do it Too many surveys # of surveys being requested James Ohliger I wait until I get the 14 days past due reminder from NI for duty hours. Trigger needed to remember to complete tasks Frequency of reminders sent Tim Jay I like to wait until I have a lot to enter (i.e. case logs or duty hours) because it takes 6 minutes to get all set up so I don’t want to do it daily, that’s a waste of time. Lengthy process to enter information Amount of time to enter information into the system Kyle Petersen Time/convenience of task, example, if I open a survey for example and it’s quick, I’ll just do it on my phone, but if it’s going to take 10-15 minute, I’ll probably save it for later but may forget about it because the e-mail is “read” now. Just remembering I need to do it and/or having the time to do it. Process for documenting requirements Evaluate and address non-value added activities in workflow

6

Kano Model

7

Degree of CTQ achievement Customer Satisfaction

“Must Be” “Delighter” “Linear” Base Quality Performance Quality Excitement Quality

Logging Tasks Amount of time spent following- up with Residents Completing Patient Charts Request evaluations Complete milestone exam Resident Reminders Increasing # of completed tasks Decreasing # of reminder requests Accountability Board Reduction in the number of required tasks

8

Measure

Data Collection Plan

Data Collection Plan Measure Measure Type Data Type Data Source? How? How Often? Target Sample Size Contact? Operational Definition Delinquent Case Logs X Discrete ACGME Case Log System Data is self- reported by Residents into each system and then manually extracted by Program Coordinato r. Monthl y Collecting full data set Melissa Wheeler Delinquent if more than 1 month behind. Delinquent Duty Hours X Discrete New Innovation Resident Management System Twice per month Delinquent if more than 5 weeks behind or past the 15th of the following month for the previous month. Delinquent Test Master X Discrete Orthobullets Website Delinquent if previous month not completed. Delinquent Total Tests X Discrete Delinquent if 5 or less tests not completed. Automatically delinquent if milestone exam is not complete. Delinquent Skill Master X Discrete Delinquent if less than 12 are completed Delinquent Evaluations X Discrete Delinquent if less than 1 are completed. Task Delinquency Rate Y Discrete Manual Calculation The number of delinquent tasks divided by the total

9

10

Measurement System Drilldown

Defined as the number of monthly required tasks needed to be completed by the orthopaedic residents, which are identified on the tracking board.

Measurement: % of tasks completed on time by residents

Rotation & Time Scheduled (vacation, Nights, research) Amount of tasks each month varies

11

Before Pictures –

Requirements listed throughout 15 page Resident Guidelines

Number Residents Not Completing Tasks by Task Type

April – June 2018

12

10 20 30 40 50 60 Test Master Total Tests Skill Master Evaluations Duty Hours Case Logs

Count Task Type

Not Delinquent Delinquent

16.6% 23.9% 43.9% 22.7% 35.0% 35.0% *Percent delinquent displayed above the bars on graph

13

Analyze

Activity Flow Chart

14

New Resident Orientation ACGME New Innovations Orthobullets Residency Coordinator logs in to task platforms to review current progress and provide update Present

• verview of

Resident guidelines Discuss program expectations for completing tasks on time Complete set up of Resident accounts in

• nline

platforms Show Residents how to use online platforms for documenting task completion Residents to verify online accounts work Residents track tasks Halfway through the month? Halfway or more towards monthly goal Update Excel tracking sheet Update Excel tracking sheet and email Resident with reminder to track Y N N Residents continue progress on monthly tasks Y 1st Monday

• f Month

Y Monthly goal completed? Y ACGME New Innovations Orthobullets Residency Coordinator logs in to task platforms to review current progress and provide update Update excel sheet with task completion data (reported

• ut in 6 month evaluation

# of Times Delinquent Update Excel Tracking Sheet and send email to alert Resident that tasks are not completed and need to be finalized Update Excel Tracking Sheet and send email to alert Resident that tasks are not completed and need to be finalized Completed by mid month check? N >1 Send second alert requesting for tasks to be documented by end of month Task completed by end of month? N Email program director regarding delinquency status copy Resident Tasks Completed by Mid- Month Check? Dismissal from Program 1 N Y N Y Y N

15

Brainstorm – Potential Root Causes

• Too many requirements
• Not enough time
• Duty Hour Restrictions
• Personal motivation
• Don’t remember to do it
• Tedious & time consuming
• Don’t see the value
• Platforms are not easily accessible
• Multiple platforms are required to

complete the tasks

• No hard stop in place that makes residents

complete tasks – only verbal reminders

• No easy way to visualize all that needs to be

completed

• Task list is not visually represented to

residents

• Tasks can change monthly
• No incentive to complete tasks
• No accountability structure if the tasks

are not completed

16

Increase On-Time Completion

• f Resident

Tasks

Machine Manpower Methods

No Accountability structure No incentives Don’t see the value Don’t remember Tedious & time consuming Duty Hour restrictions Platforms not easily accessible

Ishikawa Diagram

Some tasks change monthly Not Enough time Too Many Requirements Tasks not visually represented Personal Motivation Only verbal reminders –no hard stop Multiple platforms required Unclear Instructions

Ishikawa Diagram –Results from Resident Multi-Vote

17 2 4 6 8 10 12

Number of Votes Reason for non-completion

Results of Multi-Vote -Ishikawa Diagram

18

Improve

19

Brainstorm - Solutions

• Visual display board
• Incentives for completing tasks
• Corrective action plan for not completing

tasks

• Not sharing what the plan is so that

residents do not weigh the cost-benefit

• Team Challenges for peer to peer

accountability

• Ortho Team Trophy
• Getting all tasks on one platform

Risk Identification and Mitigation Plan

20

Solutions Risks Mitigation

Visual display board If not updated consistently, could lose it's effectiveness Cross train Melissa Maddie on how to update the board in my absence. Put reoccurring monthly calendar reminders on both of our calendars to keep up on timeliness. Residents may not want their progress publicly known Fully explain the expectations and new process to them so they are aware in advance if they are delinquent, it will be public Incentives for completing tasks Costly to maintain Set a budget and do annual incentives for overall completion vs. regular incentives monthly If residents don't value the incentive, it won't help Find out what they do value/want Corrective action plan for not completing tasks Residents may weigh the cost-benefit of the punishment vs. completing the task Don't explicately state the outcome for non- performance and leave it at the discresion of the program director Inconsistency with implementation and restrictions if trying to go outside formal corrective action plan in terms of negative reinforcement Add completion rates to their annual reviews to help ensure follow through with non-performers Team Challenges for peer to peer accountability & utilize ortho trophy Depends on fully engaged chief residents to motivate their teams Only implement plan if chief residents are fully engaged Getting all tasks on one platform Not an option at this point in time ACGME requires case logs to be input into their system, Med Ed. Requires us to use NI and report to them on paper, and Orthobullets doesn't have the capacity to integrate with either of these systems.

21

After Pictures – Ortho Task Master Visual Board

• Each column

represents tasks that need to be completed monthly

• Green writing means

tasks are compliant

• Red writing means

tasks are not compliant

• Extra columns built

for additional task or department team building events

• Schedules for

rotation, conferences, and call posted on right edge

• f board

Activity Flow Chart

22

ACGME New Innovations Orthobullets Residency Coordinator logs in to task platforms to review current progress and provide update New Resident Orientation Present

• verview of

Resident guidelines Present

• verview of

accountability board Discuss program expectations for completing tasks on time Complete set up of Resident accounts in

• nline

platforms Show Residents how to use online platforms for documenting task completion Residents to verify online accounts work Residents track tasks Halfway through the month? Halfway or more towards monthly goal Document Resident progress on accountability board in green Document Resident progress on accountability board in red Y N N 1st Monday

• f Month

Residents continue progress on monthly tasks Monthly goal completed? Accountability board updated in red Updated accountability board in green Take picture of board and update excel sheet with task completion data (reported

• ut in 6 month evaluation

Y Y N ACGME New Innovations Orthobullets Residency Coordinator logs in to task platforms to review current progress and provide update

23

Control

Communication/Training Plan

Functional Group New Knowledge and Skill Requirement Estimated Length / Format Owner Start Date Completion Date

Ortho Residents *Understanding requirements/expectations *Items tracked on the board *How often the board will be updated 30 minutes Melissa 6/29/18 6/29/18 Research Director *How to update the board *Location of board data *Update frequency 15-20 minutes Melissa 6/29/18 7/13/18 Ortho Attendings *Resident expectations related to participation in tasks (evaluations) Email Melissa Next Faculty Meeting (August) 10/23/18 Program Director *Purpose of board *How to use board *Setting resident expectations *Incentives for completing tasks/Accountability for not completing tasks 15-20 minutes Melissa 6/22/18 6/29/18

24

Number Residents Not Completing Tasks by Task Type

July – September 2018

25

10 20 30 40 50 60 Test Master Total Tests Skill Master Evaluations Duty Hours Case Logs

Count Task Type

Not Delinquent Delinquent

8.7% 22.8% 31.6% 22.8% 8.8% 7.0% *Percent delinquent displayed above the bars on graph

Comparison of Task Completion Rate Pre vs. Post Implementation

26

Task Baseline Delinquency Rate Post Implementation Delinquency Rate Statistically Significant Test Master 17% 9% No Total Tests 24% 23% No Skill Master 44% 32% No Evaluations 23% 23% No Duty Hours 35% 9% Yes Case Logs 35% 7% Yes Total Delinquency Rate 38% 20% YES!

Control

Checks Reaction Plan Board is updated ½ way through the month

Anything that hasn’t been completed or ½ way to their goal will be in red.

Milestone Exam –Continue to check previous month(s) progress

Don’t erase previous month red X until completed – a resident could have multiple X’s in this column. More than 3 months behind, send e-mail to resident & PD.

Skill Masters

0 for one month = e-mail with PD copied reminding of expectation 3 months delinquent of out 6, e-mail with PD copied.

Evaluations

2 months delinquent = e-mail with PD copied 4 months delinquent = meeting with PD

Case Logs & Duty Hours

Date of last log written on board next to red

• X. 3 months or more delinquent = e-mail

with PD copied.

Total Tests

3 months delinquent out of 6, e-mail with PD copied.

27

Lesson Learned and Next Steps

28

• Consistency & Accountability are key
• Change process isn’t always easy, always have the 10% outliers
• Originally I was measuring time on task. Time was the same but overall job

satisfaction has greatly improved.

• Next Steps – Incorporate challenges & continue to make it fun!
• how to help them build routine

Survey to grads Voice of Customer Post Board: “Wow! I’m really behind, I’m going to go get caught up right now!” – PGY 1 resident