WELCOME 2019 EXAMINERS! Examiner Training Day 3 1 Welcome Back! - - PowerPoint PPT Presentation

WELCOME 2019 EXAMINERS!

Examiner Training – Day 3

1

Welcome Back!

• Review Ground Rules
• Quench any burning issues
• Debrief Quiz
• Introductions/Icebreaker

Master Examiner Introductions

• Name
• Company
• If you were the 8th

dwarf, what would your name be?

3

Master Examiners…

• Have served for a minimum of 3 years on examiner

teams, AND

• Have been either a scorebook editor or team leader,

AND

• Have demonstrated mastery of the Framework
• Serve TNCPE in leadership roles, AND
• Required to only attend Day 3 of the training

course…

• Earn an individual TNCPE membership

4

Summary of Key Changes

• New TNCPE staff
• New Level 1 approach
• New Framework

– “Questions” not “requirements”

• No feedback-ready comments at IR
• Even more table discussions in training
• New Examiner Inbox
• New rules regarding “the applicant”

– Renewed focus on meeting them where they are – Basic/Overall/Multiple rationale

5

COMMENT SELECTION – MEET THEM WHERE THEY ARE

“What is the organization in business to do? Will your comment help them do it better?”

• Dr. Susan Williams, former TNCPE Judge

Examiner Role

• Remember TNCPE’s mission: “to drive organizational

excellence across Tennessee and the Southeast”

• Meet the applicant where they are . . .
• . . . and help them move forward
• Select comments that are insightful and help the

applicant take the next appropriate step

7

Who Are TNCPE’s Applicants?

8

5 10 15 20 25 30 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 Healthcare Education Government Manufacturing Service Nonprofit

Number of Applicants by Size

9

18 13 20 23 25 31 33 31 27 17 3 7 10 15 12 12 8 4 10 1 3 5 2 3 1 4 2 5 2 2 2 1 3 5 2 3 2 3 2 3 10 20 30 40 50 60 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 More than 1,500 FTEs 501-1,500 FTEs 100-500 FTEs Less than 100 FTEs

Applicants by Level

4 6 11 20 12 21 7 7 7 1 3 3 5 14 6 7 6 4 5 3 5 5 9 1 8 5 6 6 19 12 14 13 10 11 14 17 26 26 10 7 10 20 30 40 50 60 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 Level 1 = Org. Profile Level 2 = Basic req'ts Level 3 = Overall req'ts Level 4 = Multiple req'ts

Stair Steps to Excellence

11

Level 1 Process

• Performed by master examiner “coaches”
• Begins with an Org Profile workshop
• Applicant submits Org Profile to Master

Examiner for feedback

– Final version submitted to TNCPE

• Master Examiner returns for Bridge

Workshop

– Questions in 7 Categories, similar to site visit

• Applicant receives abbreviated final feedback

report

Level 2 Process

• Standard Organizational Profile

– Maximum 5 pages

• Application addresses basic item questions for

all 17 items

– Maximum 15 pages

• Examiners conduct one day site visit
• Feedback report addresses overall item

questions

13

Level 2 Workshop

• Level 2 Applicants…

– Receive a ½ day workshop following receipt of their feedback report – Intended to help them prioritize their OFIs – Conducted by TNCPE contractor – Included in site visit fee

14

Stair Steps to Excellence

15

Other Unique Circumstances

• Small business applicants
• Multi-site applicants
• Government agency applicants
• Franchisees or divisions/departments of larger

corporations

• Applicants with field offices out of state
• Mature applicants applying at Level 1
• New applicants applying at Level 4

16

How Will You Know?

• Key Factors!!
• A key factor is an attribute of an organization or its

environment that influences the way the organization

• perates and the key challenges it faces. Examiners

will use key factors to focus their assessments on what is important to the applicant.

• Where do I find key factors?
• How and where do I record key factors?

17

Pay Particular Attention to . . .

• P.1a(3) Workforce Profile

– Are there more employees than examiners?

• P.1a(4) Facilities/Locations (Assets)

– How many? Are they in or out of state? Do they each operate independently?

• P.1a(5) Legal and Regulatory Requirements

– Do regulations dictate their ability to innovate processes?

• P.1b(1) Organizational Structure and Governance System

– Are they owned by someone else? Does the owner control the strategic plan, HR processes, or system of measurement?

• P.1b(3) Suppliers and Partners

– Are they dependent on anyone else for key requirements? Do they have control of key suppliers or not?

18

Opportunities to Discover “Unique” Situations

• Organizational Profile/Key Factors
• Team Leader and Applicant calls
• Opening Site Visit Meeting

19

Pretend for a Moment…

• LOTS is located in Tennessee with a headquarters

in Jackson

• 25 employees are decentralized over TN, NC, and

SC

• LOTS submitted a Level 2 TNCPE application for

the first time this year

Which key factors would change? How would this impact your review of LOTS? How would the site visit change?

20

A Great Comment

• Is insightful

– Tells the applicant something it didn’t know

• Provides meaningful feedback

– Validates progress – Clearly describes key gaps and their significance to guide future improvement – Helps senior leaders allocate resources

• Is easily understood

– Without reading the application – Plain English please!

21

Prioritizing OFI Comments

Remember the levels for item questions:

• If the organization has OFIs at the basic questions

level THAT ARE IMPORTANT, focus your writing here

– Be sure to specify what is keeping them from meeting the next level (overall questions)

• If the organization answers the basic item questions,

focus your OFI at the overall level

• Remember: just because the applicant applied at

Level 4 does not mean it is ready for feedback on the multiple questions for every item!

22

Item 2.2 – Which Level?

• LOTS does not systematically implement its
• strategy. (BASIC)
• LOTS lacks a method for allocating financial

resources to key action plans. (OVERALL)

• LOTS’ approach to funding action plans does

not effectively manage risk. (MULTIPLE)

23

Prioritizing OFIs 2

Remember your evaluation factors:

• Process:

– If no approach is described, focus your OFI here – If an approach is present, move on to deployment/ learning/ integration

• Results:

– If no results are presented for an area of importance, focus your OFI here – If results are present, consider trends/ comparisons/ segmentation/ integration

24

Item 3.2 – Which Would You Choose?

• LOTS lacks a method for determining

customer satisfaction. No survey data are collected.

• LOTS does not deploy customer data to its

suppliers and partners.

25

Table Exercise

• Review the assigned item in

the consensus scorebook for LOTS.

• Rewrite 1 OFI comment to

focus on the basic level question.

• Rewrite a 2nd OFI comment to

focus on a different evaluation

• factor. Note the factor.
• Consider new key factors!
• Talk about how this would

change your score.

26

Table Item 1 4.2 2 3.2 3 4.2 4 3.2 5 4.2 6 3.2 7 4.2 8 3.2

Writing Style

• Write in third person

– “LOTS . . .” instead of “You . . .”

• Write in the present tense

– “LOTS reviews . . .” instead of “LOTS has reviewed . . .”

• Write in the active voice

– “The leadership team reviews . . .” instead of “Reviews are performed . . .”

Be polite!

27

What to Avoid

• Multiple messages in one comment

– Don’t put an OFI in a strength comment!

• Prescriptive comments (i.e., don’t tell the applicant how to

close gaps)

• Comments on how the application is written
• Personal opinions or judgments

– NO: superior, outstanding, good, poor, great – YES: systematic, fact-based, fully deployed, sustained improvement

• Using the same “so what” for every comment

– . . . may help LOTS achieve its vision . . . achieve its vision . . . achieve its vision . . . .

• Jargon – industry or quality-related

28

Exercise

• Move to your assigned table
• Review and edit your assigned table’s comment(s)

– Is the comment Criteria-based? – Is it insightful/useful? – Do you understand it? What would improve it? – Does it contain a topic sentence, examples, and “so what”?

• Agree on one learning from each table

29

Resources Available Online

• Considerations for Unique TNCPE Applicants
• Linkage Between Org Profile and Process

Categories

• Sample Comments
• Scorebook Editor Tip Sheet

30

31

Sample Consensus Results OFI

7.2a(2) Comment: Results are not reported relative to the customer lifecycle. No measures are included to reflect the aspects of the customer life cycle of pre- donation, procurement, allocation and post- donation that are identified in Figure 3.1-1, Voice of the Customer. Understanding all aspects of the customer life cycle may help LOTS pursue the strategic opportunity of customer satisfaction.

32

ETHICS AND CONFLICT OF INTEREST

“People forget how fast you did a job, but they remember how well you did it.” Howard W. Newton

Applicant Expectations

• Integrity
• Confidentiality
• Professional conduct
• Respect for intellectual property

34

Rules of Conduct

• Established to maintain the confidentiality of ALL

award applicant information

• Ensure the fairness of the examination process
• Aligned with Baldrige standards for state awards
• A pledge by the Board of Examiners to uphold

professional principles

• Pertains to the entire Board of Examiners, including

Judges

35

Potential Conflicts

• Employer

– Past, present or future

• Previous evaluation of applicant

– Within last 3 years

• Key suppliers, customers, or competitors

– Those critical to your organization – Roughly 5% or greater of your business

• Financial holdings

36

True or False

No one can serve on the Board of Examiners if s/he has not attended the three-day training course.  True  False

37

True or False

The team leader is the only examiner that may contact the applicant after the site visit to clarify responses as we complete the final report.  True  False

38

True or False

To meet critical team deadlines, it is acceptable for a friend or colleague who is a trained examiner to assist me in editing my observations and/or feedback-ready comments.  True  False

39

True or False

When doing the independent review, I should visit the applicant organization’s website and social media sites for additional information.  True  False

40

True or False

My service on the 2019 Board of Examiners ends

• nce the Panel of Judges meets and the awards

are announced.  True  False

41

True or False

Cell phones should not be brought to the applicant’s facilities during the site visit.  True  False

42

True or False

At the conclusion of my site visit, TNCPE will automatically reimburse me for the appropriate site visit per diem.  True  False BONUS: 2019 site visit per diem is $________.

43

True or False

The conference and awards banquet is the proper time to approach and congratulate the organization I reviewed.  True  False

44

True or False

Only organizations headquartered in Tennessee may apply for TNCPE awards.  True  False

45

True or False

The amount of time I should expect to spend evaluating my applicant, including site visit, is 45-50 hours.  True  False

46

Multiple Choice

• If upon arrival at the site visit, an Examiner

suspects s/he may have a conflict of interest, s/he should:

a) ask the applicant contact person if s/he objects to their participation b) refrain from participating in site visit questioning and turn focus to closing meeting c) immediately call the TNCPE office

47

Multiple Choice

If your assigned applicant approaches you with an

• ffer of employment:

a) you may accept only if it has been 3 or more years since your site visit b) you may accept only if it has been 5 or more years since your site visit c) you may accept, provided the applicant sends a signed waiver to the TNCPE office d) you may not accept

48

Fill in the Blank

If a second-year examiner is unable to fulfill his/her team obligations, TNCPE will send an invoice for the examiner training fee in the amount of $___________.

49

When In Doubt…

KEEP CALM

AND

CALL TNCPE

Your Commitment

• As a member of the TNCPE Board of

Examiners, will you uphold the Rules of Conduct and Code of Ethical Standards?

• Please sign and return the Code of Conduct
• Welcome to the 2019 Board of Examiners!

51

PUTTING IT ALL TOGETHER

Independent Review | Consensus | Site Visit

How Am I related to Baldrige/TNCPE?

53

Applicant Assessment Process

54

Self assess

Apply Examiner review & site visit Award Receive Feedback Report Address OFIs

2 3 4 5 1

Examiner Evaluation Cycle

Complete Pre-work & Training

Independent

Review Consensus Review Site Visit Review Finalize Feedback Report

55

June & July August

Review for Conflict Log on to the system Review the Framework Read the Application Create Key Factor List

Click “Done”! Prepare Consensus items Participate in Consensus meeting Prepare Site Visit issues Complete Site Visit Prepare Final Scorebook

12 process items

• Criteria
• Key Factors
• Analyze: PROCESS

(ADLI)

• 6-10 Observations
• Score

5 results items

• Criteria
• Key Factors
• Analyze: RESULTS

(LeTCI)

• 6-10 Observations
• Score

IR Review

56

Review – Identify Key Factors

57

• What is a “key factor”?
• Where do I find key factors?
• How and where do I record key factors?
• How many should I have?

A key factor is an attribute of an organization or its environment that influences the way the organization operates and the key challenges it faces. Examiners will use key factors to focus their assessments on what is important to the applicant.

58

12 Process Items

• Criteria – Read the Criteria item questions
• Key Factors – Select 4-6 most relevant key factors from master list
• Analyze the Application – Read relevant section of the application to

identify processes and gaps; analyze using process evaluation factors (A/D/L/I)

• 6-10 Observations– Select and outline 6-10 strengths and OFIs
• Score the item

5 Results Items

• Criteria – Read the Criteria item questions
• Key Factors – Select 4-6 most relevant key factors from master list
• Analyze the Application – Read relevant section of the application; group

results and identify gaps; analyze using results evaluation factors (Le/T/C/I)

• 6-10 Observations– Select and outline 6-10 strengths and OFIs
• Score the item

Item Evaluation Steps

59

Review – Insightful Process Observations

• Identify gaps between the Criteria questions and

the applicant’s response. Are these gaps important to the applicant?

• Look for key factors not addressed. Did the

applicant…

– Address all customer and stakeholder requirements? – Consider all employee groups and segments? – Set measures for all relative goals/objectives? – Address all strategic challenges?

60

Review – Insightful Results Observations

• Identify gaps between the processes described in

Categories 1-6 and the results in Category 7. Are these gaps important to the applicant?

• Use the applicant results tracker
• Group results charts to provide new information

by…

– Common benchmark or comparison – Area to address – Integration with process item

61

Assignment

Examiner Item 1 7.4 2 7.5 3 2.1 4 4.1 5 5.2 6 6.2

62

Exercise – Independent Review

• Review the Criteria questions for your assigned item
• Using the process item worksheet or the results item

worksheet: (Process Item: pg. 199-203; Results Item: pg. 205- 209)

– Select 4-6 relevant key factors; consider what you are expecting to see (5-10 min.) – Read LifeBridge’s response (15-20 min.) – Analyze (30 min.)

• Note key processes [OR groups of key results] and any gaps
• Assess processes/results using A-D-L-I or Le-T-C-I
• Consider B/O/M
• Outline 6-10 strength and OFI observations

– Score your assigned item (5 min.)

– Do NOT write feedback-ready comments yet

63

64

Examiner Evaluation Cycle

Complete Pre-work & Training

Independent

Review Consensus Review Site Visit Review Finalize Feedback Report

65

June & July August September

Review for Conflict Log on to the system Review the Framework Read the Application Create Key Factor List

Click “Done”! Prepare Consensus items Participate in Consensus meeting Prepare Site Visit issues Complete Site Visit Prepare Final Scorebook

12 process items

• Criteria
• Key Factors
• Analyze: PROCESS

(ADLI)

• 6-10 Observations
• Score

5 results items

• Criteria
• Key Factors
• Analyze: RESULTS

(LeTCI)

• 6-10 Observations
• Score

Consensus Review

66

67

68

69

70

71

Preparing an Item for Consensus Meeting

• 1. Review the Criteria questions
• 2. Assign 4-6 key factors
• 3. Read applicant’s response
• 4. Review team members’ Independent Reviews

SYNTHESIZE

Synthesize team members’ IR into 6-10 comment topics Develop rationale: B/O/M, evaluation factors, and initials

WRITE

Draft 6-10 feedback comments based on your chosen topics Use TEST

SCORE

Score the item based

• n your consensus

review

REVISE

Revise comments based on feedback from your backup Provide feedback to your partner

PREPARE

Prepare to present your items during the team’s consensus meeting

74

Remember…

• It is your job to select the MOST insightful 6–

10 pieces of information

• Not all Criteria questions are equally

important to every applicant!

• Tell the applicant something it doesn’t

already know…

75

Summary of Consensus Changes

• Use the applicant’s name (LOTS) in your

comments

• Put strength and OFI comments in order of

importance!

• List initials AND B/O/M in your rationale
• TEST can be spelled

– STE – ETS – Or any other variation!

Exercise – Consensus Part 1

• Move to a table with others who completed the same

item

• Using your independent reviews, work as a team to

select 4-6 key factors for your item

• As a team, select 6-10 comment topics (strengths and

OFIs)

• What is your rationale for each comment? (initials,

B/O/M, evaluation factors)

• As a team, score your assigned item

77

Table Assignment

Examiner Item 1 7.4 2 7.5 3 2.1 4 4.1 5 5.2 6 6.2

78

Exercise – Consensus Part 2

• Select one OFI comment topic from your team’s list
• Independently write your assigned feedback-ready

comment on the worksheet provided (pg. 211-212)

• Return to your original table
• Get feedback on your comment from someone at

your table (your backup)

• Revise your comment based on the feedback

79

Comment Writing Tips

Is the comment Criteria-based? Is it insightful? T = Start with the “topic sentence” or main point E = Give 1 – 2 supporting examples from the application S = Explain the “so what” (relevance) to the applicant by linking the comment to a key factor T = Try it out loud!

80

Examiner Evaluation Cycle

Complete Pre-work & Training

Independent

Review Consensus Review Site Visit Review Finalize Feedback Report

81

June & July August September September & Early October

Review for Conflict Log on to the system Review the Framework Read the Application Create Key Factor List

Click “Done”! Prepare Consensus items Participate in Consensus meeting Prepare Site Visit issues Complete Site Visit Prepare Final Scorebook

12 process items

• Criteria
• Key Factors
• Analyze: PROCESS

(ADLI)

• 6-10 Observations
• Score

5 results items

• Criteria
• Key Factors
• Analyze: RESULTS

(LeTCI)

• 6-10 Observations
• Score

Site Visit Review

82

2018 Site Visit Strengths

. . . the team was very thorough and worked diligently to understand our complex organization. They were professional but very approachable. . . . The visit helped to re-establish some of the gaps that we recognized during the application phase, and we look forward to the feedback report to see

• ther areas that we need to address as an
• rganization.

83

2018 Site Visit OFIs

. . . We received the site visit schedule and several pages of requested documents and broadly worded topics just a week prior to the visit. That made full preparation difficult. . . . One examiner stated his questions too harshly to frontline staff. He was a bit condescending with his approach to several of the staff involved with the frontline staff focus group that the examiners requested. We had to ask the team lead for the questions that the examiners had established for the site visit, and they were received two days prior to the site visit. …With being a government agency, some of the thought process and questions can be very difficult to answer or connect to. It can be like fitting a square peg in a round hole. …Sometimes the team asked questions about things outside our control that we can't impact and added no value for us.

84

Site Visit Video

85

Site Visit Steps

• Complete SVI form in Scorebook Navigator first

– Documents, interviewees – Site Visit Issue only (verify/clarify questions) – The team leader sends these lists to the applicant 10-14 days before site visit

• Then complete a SVI worksheet in Word for each issue
• The applicant will receive your site visit issues

(verify/clarify statements) and requested documents but not your prepared A-D-L-I questions or strategy

86

Exercise – Site Visit

• Write a site visit issue based on the comment you

wrote in the last exercise (verify/clarify)

• Complete the Scorebook Navigator form (pg. 214)
• Complete a Site Visit Issue Worksheet with questions

you will ask the applicant (pg. 213) – A-D-L-I or – Le-T-C-I

87

Site Visit Findings

• Use your SVI Worksheet (Word) to take notes

during site visit

• Team leader will collect these when complete

(and save in case there are questions)

• Document your evidence/findings in the

Scorebook Navigator form with a conclusion:

– OFI remains – Delete OFI – OFI becomes a strength

• Update your comment to match the findings

88

FINALIZING AWARD RECOMMENDATIONS

“The secret to success is constancy of purpose.” Benjamin Disraeli

89

Award Recommendation Score Summary Key Themes

90

Key Themes

• Review key themes before you leave the post-

site visit meeting

• Do they appropriately reflect the applicant’s

maturity?

• Do they tell the applicant what critical few

things it must do to move to the next level?

• Are they traceable to the item comments?

91

Scoring Your Applicant

• Review item scores and complete the Score Summary

Worksheet

– Are item scores supported by the item comments? – Are scores stratified over all of the items?

• Do not change scores in Scorebook Navigator!

– Indicate on worksheet if score would move up or down by scoring range

• Select a scoring band for the applicant

– Does the scoring band reflect the applicant’s maturity? – Does the scoring band align with the key themes?

92

Score Summary Worksheet

• Developed by Panel of Judges
• Completed by team on post-site visit day
• Captures changes in score by item from

consensus to site visit

• Conditional formatting to help show where

new scores may land and tie to scoring bands

• Covered in team leader webinar and JIT

webinar

93

SCORNG SUMMARY FOR APPLICANT # 1709 Item Number Strengths (+) Strengths (++) OFIs (-) OFIs (- - ) Strengths (+) Strengths (++) OFIs (-) OFIs (- -) Comments KT a/b 7 7 KT c/d 5 3 1.1 55% 39 3 2 4 50% 35 65% 46 3 2 3 Deleted 1 OFI 1.2 45% 23 4 3 1 30% 15 45% 23 3 2 1 1 strength merged with another strength; deleted 1 OFI 2.1 45% 20 4 4 50% 23 65% 29 6 1 1 1 Moved 1 strength from 4.1 to 2.1; changed 2 OFIs to strengths; changed 1 OFI to (- -) 2.2 50% 20 3 3 50% 20 65% 26 2 1 2 1 Changed 1 strength to ++; changed 1 OFI to (- -). 3.1 30% 12 3 1 2 50% 20 65% 26 5 3 Added 2 strengths; deleted 1 OFI; added 1 OFI; no more (-

• ) OFIs

3.2 35% 16 3 2 1 50% 23 65% 29 3 2 1 4.1 40% 18 3 2 3 1 30% 14 45% 20 2 1 2 Moved 1 (++) strength to 2.1; deleted 1 strength; delete 1 (- ) OFI and 1 (- - ) OFI. 4.2 60% 27 4 2 2 1 50% 23 65% 29 3 2 1 1 Moved 1 strength to 5.2; deleted 1 OFI 5.1 65% 26 4 3 70% 28 85% 34 4 1 Deleted 3 OFIs; added 1 OFI 5.2 60% 27 5 1 2 50% 23 65% 29 5 2 2 Moved 1 strength from 4.2 and made it a (++); deleted 1 OFI; added 1 OFI 6.1 40% 18 3 5 30% 14 45% 20 3 1 1 Deleted 3 OFIs;changed 1 OFI to a (- - ) 6.2 45% 18 4 4 50% 20 65% 26 4 2 Deleted 2 OFIs 263 255 338 7.1 35% 42 3 2 1 30% 36 45% 54 3 2 1 Deleted 1 OFI and added 1 OFI 7.2 25% 21 2 4 10% 9 25% 21 2 3 1 Changed 1 OFI to (- - ) 7.3 40% 34 3 3 30% 26 45% 38 3 2 Deleted 1 OFI 7.4 20% 16 2 2 2 30% 24 45% 36 2 2 2 7.5 40% 32 3 2 50% 40 65% 52 3 1 Deleted 2 OFIs and added 1 OFI 145 134 202 408 389 539 Scoring Bands (Calculated) Approach 4 3 5 Results 2 2 3 Scoring Bands (Team Recommendation) Approach 5 Results 3 Consensus Site Visit Score Scoring Range

Table Exercise

• Review the Room Scoring Summary as a

class

• What process and results band is LOTS in

after consensus?

• As a table, review the scoring band

descriptors and determine which most accurately describes LOTS after site visit

• Process
• Results

TNCPE Recognition Levels

Level 4 Level 3 Level 2 Principles

Continue to improve and build upon outstanding results and excellent systems Commitment and practice Beginning to commit and implement

Processes

Exhibit processes that serve as role models for others, with some at or near “best-in- class.” Demonstrate significant progress in building systematic processes Put in place some key process improvements

Effectively Address

Multiple questions Overall questions Basic questions

Results

Demonstrate management excellence with superior results over time, directly attributable to a systematic well-deployed improvement approach. Results for some key processes show improvement that is directly attributable to a systematic approach. Process improvements are directly attributable to a fact-based improvement

• process. Results may or

may not be evident; however, a measurement system should be in place.

Practices

Outstanding examples of high- performance organizations Many practices from which

• ther organizations can

learn and grow.

96

Level 1 organizations are . . .

. . . interested in adopting and applying performance improvement principles

97

98

Level 2 Award Recommendation

ABC is in the early stages of developing systematic approaches to address the basic questions of the process categories, as evidenced by: listening to customers through XX system; conducting an annual Strategic Planning retreat; recruiting and hiring of new employees; emergency preparedness and prevention planning; and the existence of processes that promote ethical standards of conduct. While there are multiple processes in place, many are not fully deployed and/or lack the means to establish cycles of learning that could lead to continuous improvements for the organization. ABC collects large amounts of data but is still in the early stages of determining how to effectively analyze this data for use in strategic planning. Segmentation of data and important comparison results that would allow the

• rganization to benchmark itself against competitors in

a rapidly changing healthcare environment are missing.

99

Level 3 Award Recommendation

XYZ has demonstrated significant progress in building systematic processes that effectively address the overall questions of each item . . . It has established documented processes for most of the identified key work systems. These processes are deployed and aligned throughout the organization appropriately to meet the key business needs of the organization. Some of the processes remain informal as it is still a small company at this

• point. Organizational learning has mostly been the result of the

new owner and is now in the early stages of being transitioned to the entire workforce.

100

Level 3 Recommendation (cont.)

Results for many of the key processes show improvement that is directly attributable to a systematic improvement approach. There are 26 KPIs that are managed. What the company lacks in maturity is results and depth of addressing key multiple questions, including segmentation and comparisons to companies offering similar products. There are many examples from which other organizations can learn and grow, such as the application of multiple technologies to run and grow the business, which is unique for this industry and company size.

101

Level 4 Award Recommendation

ABC is an outstanding example of a high-performing organization. This high performance starts with the leadership system and with the senior leaders who demonstrate an uncommon commitment to continuing improvement and Baldrige principles. Their approaches for strategic planning and innovation are “best in class,” and their processes for ensuring patient-centered care and excellence are role

• model. ABC has demonstrated management excellence with superior

results over time, including increasing market share and national top decile performance in numerous healthcare measures. These results are directly attributable to a systematic, well-deployed improvement approach, which includes XYZ committees throughout the

• rganization, training for all team members,

and a focus on innovation.

102

Level 4 Recommendation (cont.)

The management team effectively addresses the salient multiple questions of each item and demonstrates alignment throughout the organization. Patients and families are at the center of each system, including leadership, strategic planning, operations, and workforce systems. ABC continues to improve and build upon its

• utstanding results and excellent systems as demonstrated by its

favorable trends and results.

103

Note: Level 4 applicants complete an HRO survey that is shared with the Panel of Judges

Table Exercise

• As a group, reach consensus on the process and results Scoring

Bands for LifeBridge using the descriptions (words)

• As a group, reach consensus on your Award level recommendation

for the Panel of Judges.

– LifeBridge submitted a Level 4 application

• On the poster provided, write your team’s recommendation.

Include supporting comments.

– Use language from the Recognition Level descriptions in the Framework booklet – Give examples to support your recommendation (refer to LOTS’ Key Themes) – If appropriate, tell what was missing that held the team back from a higher Award level recommendation

104

Components of the Feedback Report

• Cover letter

– Includes examiner hours

• Preparing to read your feedback report
• Key Themes

– Executive summary of feedback (strengths and OFIs) – Includes scoring bands pre- and post-site visit

• Item assessments

– Comments (strengths and OFIs) linked to key factors – Scoring range by item

• Appendix

– Stages of assessment process – Scoring guidelines – Scoring band descriptors

105

Panel of Judges

• A subset of the Board of Examiners
• Trained in the Criteria and required to stay current
• Identify potential conflicts and recuse themselves

from discussing these applicants

• Meet twice per year

– August/September: Check and Act, Plan – November: Do

106

2019 Panel of Judges

Elaine Boyd TDEC Nashville

Doug Brock Kendall Electric Chattanooga Chris Gray Griffin Mechanical Dickson Jim Hill John Deere (retired) Winter Haven, FL Melody Poole North Mississippi Health Services Holly Springs, MS Lavona Russell Cat Financial (retired) Nashville Renee Tew Maury Regional Medical Center Columbia

107

Role of Panel of Judges

• Determine Award levels
• Review each award application evaluation, including

– Application – Site visit issue documentation in Scorebook Navigator – Final scorebook

• Ensure consistency in determination of Award levels
• Validate the integrity of the Award Program
• Ensure equitable treatment of all applicants

108

Scorebook Editors

Support the team leader and team by:

– Consolidating key factors – Drafting key themes – Ensuring that comments are Criteria-based, actionable and serve the best interests of the applicant – Ensuring correct grammar and consistent writing style – Incorporating feedback from Panel of Judges – Working with team leader to prepare Final Scorebook and deliver to TNCPE at the conclusion of site visit

109

Recap: Timeline

• At least 10 days before site visit:

– Team leader sends SV agenda and issues to applicant

• Site Visit
• 1 week after site visit:

– Team leader (with scorebook editor) completes Final Scorebook

and notifies TNCPE

• Panel of Judges meeting
• By the first week of December:

– TNCPE sends Feedback Report to applicant

110

Upcoming Webinars

• Team Leaders (August 6)
• Independent Review – Level 3 and 4 (August 7)
• Independent Review – Level 2 (August 8)
• Consensus Review (August 22)
• Site Visit Planning (September 12)

All webinars will take place at 11:00 CDT/12:00 EDT

111

ASSESSMENT

Did you achieve your intention? Feedback for team placement. Graduation!

THANK YOU!

“One person with a commitment is worth more than 100 people who have only an interest.” Mary Crowley

113