Welcome 2018 Board of Examiners Examiner Training Day 2 Welcome - - PowerPoint PPT Presentation

Welcome 2018 Board of Examiners

Examiner Training – Day 2

Welcome Back!

• Review Ground Rules
• Quench any burning issues
• Debrief Quiz
• Icebreaker

Preparing Results Items for Consensus

“However beautiful the strategy, you should

• ccasionally look at the results.”

Winston Churchill

Review for Conflict Review the Framework Read the Application Log on to the system Create Key Factor List Click “Done”! Prepare Consensus items Participate in Consensus meeting Prepare Site Visit issues Complete Site Visit Prepare Final Scorebook

12 process items

• Criteria
• Key Factors
• Application: PROCESS
• 6-10 comments

(ADLI)

• Score
• 1 Feedback-Ready

strength and OFI

5 results items

• Criteria
• Key Factors
• Application: RESULTS
• 6-10 comments

(LeTCI)

• Score
• 1 Feedback-Ready

strength and OFI

Independent Review

4

Alignment of Item Requirements

P.1a(3) – Workforce Profile

What is your WORKFORCE profile . . . employee groups and SEGMENTS? . . . What are the

KEY drivers that engage them in achieving your MISSION and VISION. . .? What are your

• rganization’s special health and safety requirements?

5.1b(1) – Workforce Climate

HOW do you ensure workplace health, security, and accessibility for the WORKFORCE? What are your PERFORMANCE MEASURES and improvement GOALS for your workplace environmental factors? . . .

7.3a(2) – Workforce Results (Climate)

What are your current LEVELS and TRENDS in KEY MEASURES or INDICATORS of your

WORKFORCE climate, including those for WORKFORCE health, security, accessibility, and

services and benefits, as appropriate? . . .

5

GG Staff Profile

6

5.1b(1) GG ensures workplace health by providing health assessments, flu vaccinations, health screenings, coaching for wellness, and Wellness Committee activities. GG ensures workplace security through automated security devices for doors and gates that require unique codes or GG- issued entry badges, security monitoring systems, surveillance systems, and promptly removed access for terminated associates. Electronic accessibility to the GG network is provided by IT staff members who assign each employee a unique login, with password changes required quarterly. In addition, associates who need remote access to GG servers and files are provided access by the IT Department. The GG facility also has only one point of common public access to ensure workplace security. GG ensures workplace accessibility by ensuring that the facility is handicapped- accessible by meeting the International ISO Accessibility Standards, added in the 2016 cycle of improvement. Performance measures and improvement goals for workplace environmental factors are compliant with the OSHA 18001 Standards for Occupational Health and Safety. Results are indicated in Figure 5.1-3, with additional results available on-site.

7

Green Gateway Results

9

Green Gateway Results

10

Results Items

7.1 Product and Process Results 7.2 Customer Results 7.3 Workforce Results 7.4 Leadership and Governance Results 7.5 Financial and Market Results

11

Process/Results Linkages

12

Category Linkages Results 1 – Leadership 7.4 2 – Strategy 7.4 3 – Customers 7.2 4 – Measurement, Analysis, and Knowledge Management 7.1-7.5 5 – Workforce 7.3 6 – Operations 7.1 ** Sprinkled throughout categories 7.5

Results Comment Tips

• Group the graphs (in some meaningful way) and

summarize your analysis

• Relate results to the applicant’s:
• Key factors
• Strategies
• Processes/results described elsewhere
• BUT . . . comment on the results, not the process of
• btaining the results

13

Results Evaluation Factors

• Levels
• Validity or meaningful scale
• Trends *
• Rate of improvement or sustainability
• Comparisons
• Appropriateness; benchmarks
• Integration
• Importance, harmonization, segmentation

* Note: Unfavorable trends may be explained in the narrative

14

Results Comment Tips

• Comparisons – comment on:
• Performance relative to comparisons
• Appropriateness of comparisons
• The fact that comparisons are not provided
• Use comparative terms that are clear:
• Trends are favorable/unfavorable
• Results have improved/declined or are favorable/unfavorable
• Performance is better/worse
• Comment on segmentation
• Performance of specific segments
• The fact segmented data are not provided in key areas

15

Sample Consensus Results Strength

7.3a(1) Comment: The applicant's results for workforce capability show favorable levels and trends. For example, the Competency Rate (Figure 7.3-3) and Cross-Training Maturity Across Processes (Figure 7.34) demonstrate favorable trends over a 5-year period. The number of “Master” and “Novice” employees has improved since 2013, demonstrating continuous improvement in an area of strategic importance to the organization. Rationale: Modified KG's base comment. This is supported by KG and LH

• nly and ST had an OFI related to Capacity and Capability; however, the

capability measures shown should be taken at the benefit of doubt, and OFI about Capacity is not meaningful to applicant at this level with no indication that they have had capacity issues in the past.

16

Kevin Grayson & Sam Turner

Goals vs. Projections

GOALS

“Hoped for” performance In general, we do NOT evaluate performance against goals:

• We don’t know how the

applicant sets goals

• Do they set low goals or

stretch goals?

PROJECTIONS

“Expected” performance Examine relevance:

• Do projections make sense?
• Are they related to the

strategic plan?

17

18

Scoring Results Items

• 1. Consider your 6-10 comments paying special

attention to Le-T-C-I

• 2. Determine the scoring range that “best fits”

the item

• 3. Score the item

19

Assignment

Table Assignment 1 7.1 2 7.2 3 7.1 4 7.2 5 7.1 6 7.2 7 7.1 8 7.2

21

Individual Exercise

22

Set your timetable:

• Review Criteria requirements

2 min.

• Using the worksheet, identify 4-6 relevant key

factors 5 min.

• Review your homework and the team’s IR to find

commonalities and conflicts for your assigned item

• Select 6 topics (most important key messages) to

become strength and OFI comments

• Note your rationale for each comment

30 min.

• Write 2 comments (1 strength + 1 OFI)

15 min.

• Score the item

5 min.

Preparing an Item for Consensus Meeting

• 1. Review the Criteria requirements
• 2. Assign 4-6 key factors
• 3. Read applicant’s response
• 4. Review team members’ Independent Reviews

SYNTHESIZE

Synthesize team members’ IR into 6-10 comment topics Develop rationale

WRITE

Draft 6-10 feedback comments based on your chosen topics Balance strengths and OFIs

SCORE

Score the item based

• n your consensus

review

REVISE

Revise comments based on feedback from your backup Provide feedback to your partner

PREPARE

Prepare to present your items during the team’s consensus meeting

23

Exercise

• Pass your comment to the person to your right
• In the right-hand column, give feedback to your

teammate:

• Is the comment Criteria based?
• Can you understand it? What would improve it?
• What is the topic sentence?
• Are there examples from the application?
• So what?
• As a table, agree on the item score
• Agree on one key learning to share

25

26

Overall and Multiple Requirements

In Which Scoring Range…

(Le) Must the applicant’s results be responsive to the multiple requirements? (T) Are “some” adverse trends acceptable? (C) Is the applicant in the “early stages” of

• btaining comparative data?

(I) Are projections required?

28

Custom Comments Add Value

“What is the organization in business to do? Will your comment help them do it better?”

• Dr. Susan Williams, former TNCPE Judge

Examiner Role

• Remember TNCPE’s mission: “to drive
• rganizational excellence in Tennessee”
• Meet the applicant where they are . . .
• . . . and help them move forward
• Select comments that are insightful and help the

applicant take the next appropriate step

30

Who Are TNCPE’s Applicants?

31

5 10 15 20 25 30 2009 2010 2011 2012 2013 2014 2015 2016 2017 Healthcare Education Government Manufacturing Service Nonprofit

Applicants by Level

4 6 11 20 12 21 7 7 7 3 3 5 14 6 7 6 4 5 5 5 9 1 8 5 6 6 19 14 13 10 11 14 17 26 26 10 10 20 30 40 50 60 2009 2010 2011 2012 2013 2014 2015 2016 2017 Level 1 = Org. Profile Level 2 = Basic req'ts Level 3 = Overall req'ts Level 4 = Multiple req'ts

Number of Applicants by Size

33

18 13 20 23 25 31 33 31 27 3 7 10 15 12 12 8 4 10 3 5 2 3 1 4 2 5 2 2 1 3 5 2 3 2 3 2 10 20 30 40 50 60 2009 2010 2011 2012 2013 2014 2015 2016 2017 More than 1,500 FTEs 501-1,500 FTEs 100-500 FTEs Less than 100 FTEs

Some Unique Circumstances

• Small business applicants
• Multi-site applicants
• Government agency applicants
• Applicants owned by another corporation
• Applicants with field offices out of state
• Mature applicants applying at Level 1
• New applicants applying at Level 4

34

How Will You Know?

• Key Factors!!
• A key factor is an attribute of an organization or its

environment that influences the way the organization

• perates and the key challenges it faces. Examiners will

use key factors to focus their assessments on what is important to the applicant.

• Where do I find key factors?
• How and where do I record key factors?

35

Pay Particular Attention to . . .

• P.1a(3) Workforce Profile
• Are there more employees than examiners?
• P.1a(4) Locations (Assets)
• How many? Are they in or out of state? Do they each operate independently?
• P.1a(5) Legal and Regulatory Requirements
• Do regulations dictate their ability to innovate processes?
• P.1b(1) Organizational Structure and Governance System
• Are they owned by someone else? Does the owner control the strategic plan,

HR processes, or system of measurement?

• P.1b(3) Suppliers and Partners
• Are they dependent on anyone else for key requirements? Do they have

control of key suppliers or not?

36

… and to the Application Level

37

Level 4 Level 3 Excellence Level 2 Achievement Level 1 Commitment Interest Org Profile Yes Yes Yes Yes Application Focus Org Profile Basic item requirements Overall item requirements Multiple item requirements Max Length 5 pages 5 + 15 pages 5 + 35 pages 5 + 50 pages Site Visit ½ Day 1 Day+wkshp 2 Days 3 Days Feedback Focus 7 Categories Overall requirements Multiple requirements Multiple requirements

Opportunities to Discover “Unique” Situations

• Organizational Profile/Key Factors
• Team Leader and Applicant calls
• Opening Site Visit Meeting

38

Let’s Practice

Pretend that…

• 10 dealerships that sell Gateway products exist in

the Southeast

• Grass Guru (a dealership located in Chattanooga,

Tennessee) applied for the first time this year What questions do you have? What information do you need?

39

Grass Guru Key Factors

• P.1a(3) Workforce Profile: 1 owner, 2 shift managers,

15 sales associates, 1 clerical staff (part-time)

• P.1a(4) Location: Chattanooga, TN
• P.1b(1) Governance System: Grass Guru is an independent
• peration with control over its location, workforce, strategy, and

policies.

• P.1b(3) Suppliers and Partners: Two main suppliers – Gateway

and a competitor. Both suppliers conduct workforce training on products and site inspections of Grass Guru.

• P.2c Performance Improvement System: Grass Guru has not

submitted a TNCPE application before; a Gateway consultant is helping them get started.

40

Are These Comments Appropriate?

• Grass Guru’s strategic plan is deployed to staff during lunch

breaks

• Grass Guru includes input from Gateway in its strategic

planning process, but no data from its other key supplier

• Grass Guru’s employee survey data has not been segmented
• Grass Guru’s measurement system is not formal; key process

indicators are tracked on a poster board in the break room

• Grass Guru has no mechanism for providing feedback to

Gateway or its other main supplier

41

A Great Comment

• Is insightful
• Tells the applicant something it didn’t know
• Provides meaningful feedback
• Validates progress
• Clearly describes key gaps and their significance to guide

future improvement

• Helps senior leaders allocate resources
• Is easily understood
• Without reading the application
• Plain English please!

42

Prioritizing OFI Comments

Remember the levels for item requirements:

• If the organization has OFIs at the basic requirements

level, focus your writing here

• Be sure to specify what is keeping them from meeting the next

level (overall requirements)

• If the organization meets the basic item requirements,

focus your OFI at the overall level

• Remember: just because the applicant applied at Level 4

does not mean it is ready for feedback on the multiple requirements for every item!

43

Which Level?

• Grass Guru does not systematically obtain

information from its customers. (BASIC)

• Grass Guru lacks a method for determining

customer satisfaction and engagement. (OVERALL)

• Grass Guru’s approach to customer engagement

does not encompass the entire customer life cycle. (MULTIPLE)

44

Prioritizing OFIs 2

Remember your evaluation factors:

• If no approach is described, focus your OFI here
• If an approach is present, move on to deployment/

learning/ integration

• If no results are presented for an area of

importance, focus your OFI here

• If results are present, consider trends/

comparisons/ segmentation/ integration

45

Which Would You Choose?

• Grass Guru lacks a method for determining

customer satisfaction. No survey data are collected.

• Grass Guru does not deploy customer data to its

suppliers and partners.

46

Table Exercise

• Review the assigned

independent review inputs for Grass Guru. Pay special attention to the identified key factors.

• Discuss which comment would

add the most value to Grass Guru.

• On the flip chart, write an

appropriate OFI comment based on the topic(s) you

• selected. Note the

requirements level and evaluation factor(s) addressed.

47

Table Case Study 1 1 2 2 3 1 4 2 5 1 6 2 7 1 8 2

Writing Style

• Write in third person
• “The applicant . . .” instead of “You . . .”
• Write in the present tense
• “The applicant reviews . . .” instead of “The applicant has

reviewed . . .”

• Write in the active voice
• “The leadership team reviews . . .” instead of “Reviews are

performed . . .”

Be polite!

48

What to Avoid

• Multiple messages in one comment
• Don’t put an OFI in a strength comment!
• Prescriptive comments (i.e., don’t tell the applicant how to

close gaps)

• Comments on how the application is written
• Personal opinions or judgments
• NO: superior, outstanding, good, poor, high, low
• YES: systematic, fact-based, fully deployed, sustained improvement
• Using the same “so what” for every comment
• . . . may help the applicant achieve its vision . . . achieve its vision . . .

achieve its vision . . . .

• Jargon – industry or quality-related

49

Exercise

• Move to your assigned table
• Review and edit your assigned table’s comment
• Is the comment Criteria-based?
• Do you understand it? What would improve it?
• Does it contain a topic sentence, examples, and “so what”?
• Does it follow the comment guidelines?
• Agree on one learning from each table

50

Resources Available Online

• Considerations for Unique TNCPE Applicants
• Linkage Between Org Profile and Process Categories
• Comment Guidelines
• Sample Comments

51

52

Sample Consensus Results OFI

7.1b Comment: Many of the applicant's key process and work process measures have not been

• presented. For example, most metrics listed in

Figures 6.1-1 & 6.1-2 are absent. The applicant may not be able to judge its full performance without key metrics that have been designated to assess product, work process and supply-chain management performance.

53

Kevin Grayson

Key Themes

“If you just focus on the smallest details, you never get the big picture right.” Leroy Hood

Key Theme Overview

• Executive summary of feedback report for applicant’s

senior leaders

• Strategic in nature
• Summarize the most vital issues for the organization as

a whole – e.g., what must the organization do to become/remain competitive and ensure long term sustainability?

• Typically prepared by team leader or scorebook editor

with support from team members

55

What Is a Key Theme?

• A synthesis of multiple comments
• Don’t just cut and paste
• A perception or observation that:
• Recurs and is expanded upon throughout the scorebook
• Is common (cross-cutting) to more than one

item/category

• Is of high value or significant to the applicant
• May address a core value of the Criteria
• Is traceable to individual item comments

56

Key Themes Reflect

• a. The most important strengths or outstanding practices (of

potential value to other organizations) found in processes

• b. The most significant opportunities, concerns, or

vulnerabilities found in processes c. The most significant strengths (related to data, comparisons, and integration) found in results

• d. The most significant opportunities, vulnerabilities, and/ or

gaps (related to data, comparisons, and integration) found in results

57

58

Key Theme SQDCPME Scorecard 2.1 – SQDCPME used as foundation for strategic

• bjectives

5.2 – SQDCPME used for leadership and employee development 4.1 – SQDCPME used as foundation for performance metrics

+ + +

Sample Key Theme (a)

The applicant uses its cascading, balanced scorecard of Safety, Quality, Delivery, Cost, People, Maintenance, and the Environment (SQDCPME) as the foundation for its strategic objectives (Figure 2.1-3), for organizational performance metrics (Figure 4.1-3), and for the leadership and employee development phase of the Talent Management Cycle (Figure 5.2-2). Measures are reviewed and discussed by leaders during formal meetings and informal Gemba walks, and they serve as the basis for rewarding and recognizing desired workforce actions and achievements. SQDCPME performance is an input for identifying opportunities to develop workforce skills through mentoring, audits, retraining, and coaching. The deployment

• f this balanced mechanism to promote management by fact may help the

applicant create an environment for success, support its core competency

• f talent development, and assist it in achieving the goal to be the

“Best in the World.”

59

Key Themes

Customer Requirements

Applicant’s leadership team Executive summary of the Feedback Report Panel of Judges Summary of key points in the Feedback Report Other examiners Jumping off point for consensus and site visit

60

Steps in Developing Key Themes

• 1. Capture initial impressions on your first read-

through of the application

• 2. Scorebook editor or other team member

develops draft topics for key themes for team review prior to consensus meeting and discussion during consensus meeting

• 3. TL and scorebook editor finalize key themes after

item comments are finalized (after site visit)

61

Core Values as Key Themes

• Core values may become the topic of a “key theme”
• Use core values to provide value –

as the basis of a strength key theme comment

• Do not use core values to assess the applicant –

NOT as the basis of an OFI key theme comment

62

Other Sources of Key Themes

• Think horizontally and vertically
• Process items
• A-D-L-I evaluation factors
• ++ and -- comments
• Results items
• Le-T-C-I evaluation factors
• ++ and -- comments
• Scoring guidelines
• Scoring band descriptors

63

Class Discussion

Does Green Gateway demonstrate any of the Baldrige core values? What other potential key themes might be relevant for Green Gateway?

64

Table Exercise Instructions

Using the Consensus Scorebook, assist your table anchor in drafting a key theme on the Key Themes Worksheet

65

Table Key Theme Where to Look Suggested Topics 1 & 5 A Process Strengths (Cat 1-6)

• Personal Learning & Workforce Development
• Compliance, Ethics, and Safety

2 & 6 B Process OFIs (Cat 1-6)

• Missing cycles of learning for key processes
• Lack of processes to promote innovation

3 & 7 C Results Strengths (Cat 7)

• Favorable levels and trends for most key results
• Results for ethics and regulatory requirements

4 & 8 D Results OFIs (Cat 7)

• Missing results
• Missing segmentation

Assessing Levels 1 & 2

“The secret of getting ahead is getting started.”

Mark Twain

Applicants by Level

4 6 11 20 12 21 7 7 7 3 3 5 14 6 7 6 4 5 5 5 9 1 8 5 6 6 19 14 13 10 11 14 17 26 26 10 10 20 30 40 50 60 2009 2010 2011 2012 2013 2014 2015 2016 2017 Level 1 = Org. Profile Level 2 = Basic req'ts Level 3 = Overall req'ts Level 4 = Multiple req'ts

Stair Steps to Excellence

68

Level 4 Level 3 Excellence Level 2 Achievement Level 1 Commitment Interest Org Profile Yes Yes Yes Yes Application Focus Org Profile Basic item requirements Overall item requirements Multiple item requirements Max Length 5 pages 5 + 15 pages 5 + 35 pages 5 + 50 pages Site Visit ½ Day 1 Day+wkshp 2 Days 3 Days Feedback Focus 7 Categories Overall requirements Multiple requirements Multiple requirements

Level 1 & 2 Applicants

• Remember: for many, it is a BIG STEP to complete

the application and apply

• In general, Level 1 & 2 applicants:
• Have limited resources
• Are not familiar with the Framework or its terminology
• May be overwhelmed by the amount of information in

the feedback report . . . particularly the OFIs and recommendations!

69

Level 1 Specifics – through 2018

• Only submits an Organizational Profile
• pages 4-6 in the Criteria
• Maximum of 5 pages
• Accepted year-round
• Are not judged

70

Meeting the Customer’s Needs

Level 1 team leader calls applicant to:

• Set them at ease
• Be friendly and encouraging
• Ask why they are participating
• What are their expectations?
• What do they want to get out of the site visit?
• Develop site visit agenda (topics and time allotted)

to meet applicant’s needs

71

Examiners’ Level 1 Process

• 1. Independent review of Organizational Profile
• Use Level 1 Scorebook (not Scorebook Navigator)
• Identify key factors
• Look for linkage to the Criteria categories

2. Consensus review

• TL assigns category lead and backup
• Develop specific questions and key education points by

category

• Prepare examples of role-model approaches for your

categories

72

Level 1 Process

• 3. Site Visit
• Stick to your agenda!
• Be positive – confirm your desire to help them learn

and improve

• Show connections between current practices and the

Criteria

• Point out linkage between processes and results
• Be prepared to explain a “systematic” process
• Forget about your Level 3 and 4 applications
• Take good notes for your back-up

73

Level 1 Process

• 4. Document findings including:
• Key factors
• Key themes
• Assessment by category (not by item)
• Strengths
• OFIs
• Observations and recommendations

74

Do…

• Base comments on the Criteria
• Follow proper comment guidelines for strength and

OFI comments

• Use key factors to develop “so what” statements

that are relevant to the applicant

• Provide recommendations and help identify the

possible next steps

• Read it to your “next-door neighbor”

75

Example: Level 1 Strength Comment

+ Employee Individual Performance Plans align employee performance with XXX’s mission. An annual evaluation and two interim evaluations ensure that employees are achieving goals that drive overall organizational improvement.

76

Kristen Wayne

Example: Level 1 OFI Comment

− Although XXXX has a well deployed employee evaluation process, there is no evidence of succession planning or other developmental plans for potential leaders. Additional focus on learning and development may support the organization’s core competency of workforce excellence and development.

77

Jim Keiffer

Example: Level 1 Recommendation

• XXXX may want to consider a S.W.O.T. (strengths,

weaknesses, opportunities, and threats) analysis at the department level. This exercise may help senior leaders begin to identify strategic

• bjectives, core competencies, strategic

challenges, and strategic advantages specific to the

• rganization.

78

Some Exciting Changes…

• Level 1 Redesign Pilot
• Two organizations piloted a new Level 1 process in 2017
• Assigned a master examiner as coach
• Site visit occurs BEFORE the organization writes

Organizational Profile (in form of a workshop)

• Master examiner provides feedback on drafts
• Follow-up with workshop on 7 Categories to Bridge to

Level 2

• Abbreviated feedback report
• This will become our Level 1 process in 2019!

79

Level 2 Specifics

• Standard Organizational Profile
• Maximum 5 pages
• Application addresses basic item requirements for

all 17 items

• Maximum 15 pages
• Examiners conduct one day site visit
• Feedback report addresses overall item

requirements

• No recommendations or prescriptive comments!

80

Some Exciting Changes…

• Level 2 Applicants…
• Now receive a ½ day workshop following receipt of their

feedback report

• Intended to help them prioritize their OFIs
• Conducted in Jan/Feb by Jim Keiffer or Sam Turner
• Almost 100% acceptance rate in 2017
• Included in site visit fee

81

Table Exercise

• 1. Read TNCPE’s Organizational Profile
• 2. Identify three key factors that are relevant to

your assigned category

• 3. Prepare for a site visit with:
• Questions you will ask
• Education points you will share
• Recommendations to help TNCPE move forward

82

Assignment

83

Table Category 1 1 2 2 3 3 4 4 5 5 6 6 7 1 8 2

Intro to Site Visit

“What people say, what people do, and what they say they do are entirely different things.” Margaret Mead

Examiner Evaluation Cycle

Complete Pre-work & Training

Independent

Review Consensus Review Site Visit Review Finalize Feedback Report

85

2017 Site Visit Strengths

. . . I can't say enough about the team that visited my site. They were very professional yet very accommodating and

• approachable. I appreciate the way the made the visit feel

"non-threatening.“ . . . The team was excellent, provided outstanding customer

• service. The staff and I thoroughly enjoyed this site visit.

. . . I think that the team gave us ample time to explain our

• rganization, as well as asked questions that gave them a

good understanding of the organization.

86

2017 Site Visit OFIs

However:

. . . the team leader made the interviews rather uncomfortable at times as she talked down to our staff, discussed her viewpoints and philosophies at times rather than the Criteria’s, and lectured us at times . . . Meet the TNCPE guidelines for preparing for onsite (Improvement Suggestion)

87

Purpose of a Site Visit

• To learn more about the applicant
• Not just to get your questions answered!
• To clarify OFIs
• To verify role model strengths
• To allow the applicant to tell its story
• To meet the applicant’s expectations

88

Site Visit Phases

Planning On Site Post Site Visit

89

Prior to Planning Phase…

• Begin logistics as soon as teams are formed
• Dates of availability for team
• Dates of availability for applicant
• Travel arrangements
• Award cycle site visit schedules (recommended)
• September 17 – 21 (Level 2)
• September 24 – 28 (Level 3)
• October 1 – 5 (Level 4)

90

Stair Steps to Excellence

91

Level 4 Level 3 Excellence Level 2 Achievement Level 1 Commitment Interest Org Profile Yes Yes Yes Yes Application Focus Org Profile Basic item requirements Overall item requirements Multiple item requirements Max Length 5 pages 5 + 15 pages 5 + 35 pages 5 + 50 pages Site Visit ½Day+½Day 1 Day+1 Day 2 Days+1 Day 3 Days+1 Day Feedback Focus 7 Categories Overall requirements Multiple requirements Multiple requirements

92

Planning Phase

Reread Org Profile/ Key Factors to get grounded Develop site visit questions for assigned Categories Team leader sends site visit issues to applicant 1-2 weeks prior to site visit Discuss Site Visit agenda and approach during consensus meeting with team

93

On-Site Phase Opening Meeting Category interviews Review of documents Remember, after site visit, “it is not clear” means you didn’t do your job!

Opening Meeting

• Introductions are tricky
• Team Leader presents TNCPE Opening Meeting slides
• Applicant may have a presentation
• Come ready to work!

94

Site Visit Interview Tips

• If the applicant gives examples or anecdotes, ask a follow

up question

• At the end of an interview ask “Is there anything else

you’d like to tell us?”

• Use evaluation factors (A-D-L-I and Le-T-C-I) to guide

your questions

• Interview in pairs: one person conducts the interview

and the other takes notes

• Be committed to be an examiner for the entire

site visit

95

Ask Open-Ended Questions

• Tell me how it works
• How often do you do it? Who participates?
• Walk me through . . .
• Show me how . . .
• How has it changed? Have you always done it this way?
• How do you know? (vs. Do you know?)

(Be prepared to ask your questions more than one way)

96

97

Post- Site Visit Phase Finalize strengths by Category Finalize OFIs by Category Finalize recommendations by Category Finalize key themes

TNCPE Update

If you want 1 year of prosperity, grow grain. If you want 10 years of prosperity, grow trees. If you want 100 years of prosperity, grow people. Chinese proverb

History of TNCPE

99

TNCPE Culture

Purpose To strengthen Tennessee’s economy Mission To drive organizational excellence in Tennessee and

the Southeast region

Vision

To be the partner of choice in leading organizations to world-class performance excellence. Together…building a state of excellence!

Values

Customer Focus; Integrity; Respect; Collaboration; Leadership; Excellence; Innovation; Continuous Improvement

Core Competencies

Building loyal relationships with key stakeholders; Operating an effective Awards program

100

2017–2018 Highlights

• Successful partnership with North Carolina (NCAfE)

and progress toward a Southeast Alliance

• 25th Anniversary Excellence in Tennessee Conference

and Awards Banquet

• Excellent reviews on “Prioritizing Feedback” workshop

for Level 2 applicants

• Successful pilot of new approach for Level 1 applicants
• Leadership transition

101

CSFs and 2018 Goals

Relationships and Service

• Position TNCPE for the future by:
• Piloting the redesigned Level 1 award

process and evaluating its benefits

• Increasing the volume of examiner

candidates with emphasis on professional development value

• Measure member satisfaction with
• nline survey

Financial

• Sustain three major revenue streams

at current levels through CEO transition:

• Membership dues
• Conference revenue
• Award Program fees

Growth

• Explore expanding services to

Southeast region/adjacent states

• Position TNCPE for success with new

Tennessee Governor

• Engage organizations new to TN and

expand services to those currently engaged Operations

• Run day-to-day core business

effectively and efficiently:

• Regain appropriate staffing levels for

2018

• Develop ideal staffing model

for 2019

102

TNCPE Results (1993 – 2017)

• 1,469 applicants
• 642,736 Tennessee employees impacted
• 3,840 examiners trained
• 340,012 hours of service
• $34 million pro-bono consulting service to

Tennessee organizations

103

Number of Corporate Members

36 46 48 51 49 43 41 46 45 10 20 30 40 50 60 2009 2010 2011 2012 2013 2014 2015 2016 2017

Number of Members by Dues Level

6 5 5 5 4 2 2 3 3 3 3 3 2 1 1 4 3 2 3 3 12 9 10 12 11 17 17 16 18 18 5 4 3 4 5 2 4 6 8 10 12 14 16 18 20 2013 2014 2015 2016 2017 $25,000 $15,000 $10,000 $5,000 $2,500 $1,000 $500

Applicant Satisfaction w/Feedback

106

1 2 3 4 5 2009 2010 2011 2012 2013 2014 2015 2016 2017 Clarity Value to your organization Timely delivery

Applicant Satisfaction w/Examiners

107

1 2 3 4 5 2009 2010 2011 2012 2013 2014 2015 2016 2017 Professionalism Preparedness Communication

Applicant Satisfaction Overall

108

1 2 3 4 5 2009 2010 2011 2012 2013 2014 2015 2016 2017 Satisfaction with Awards Banquet We plan to participate again

Net Promoter Score

“How likely is it that you would recommend the TNCPE Award Program to others?” 0 = not at all likely → 10 = extremely likely NPS = % of responses at 9-10 minus % of responses at 0-6

109

10 20 30 40 50 60 70 80 2013 2014 2015 2016 2017 Examiners Applicants Members

TNCPE Return Applicants

110

12 14 25 35 26 33 19 17 31 10 7 13 25 21 29 16 13 27 5 10 15 20 25 30 35 40 2009 2010 2011 2012 2013 2014 2015 2016 2017 Total applicants Lv. 2, 3, 4 # return applicants

Applicants by Industry Sector

111

5 10 15 20 25 30 2009 2010 2011 2012 2013 2014 2015 2016 2017 Healthcare Education Government Manufacturing Service Nonprofit

TNCPE Total Applicants Compared to Other States

112

26 27 35 46 40 50 45 43 41

10 20 30 40 50 60 70 80 2009 2010 2011 2012 2013 2014 2015 2016 2017 TNCPE Best in Class State average

TNCPE Products and Services

• Assessment and Award Program
• Examiner Training
• Corporate and Individual Memberships
• Annual Conference & Awards Banquet
• Workshops and on-site training
• Baldrige for Beginners Workshop
• Application Writing Workshop
• Prioritizing Feedback Workshop
• Speakers Bureau
• Baldrige Express
• News and information source

113

Board of Examiners Comparison

114

147 150 175 195 196 182 155 176 176

50 100 150 200 250 300 2009 2010 2011 2012 2013 2014 2015 2016 2017 TNCPE Board of Examiners Highest # Examiners by State State Average # of Examiners

New vs. Return Examiners

115 70 75 96 100 109 89 82 88 92 77 75 79 95 87 93 73 88 98 50 100 150 200 250 2009 2010 2011 2012 2013 2014 2015 2016 2017

Return New

2018 Panel of Judges

Doug Brock Kendall Electric Chattanooga Donn Fisher MQPC Memphis Jim Hill John Deere (retired) Winter Haven, FL Cheryl Janoski Trainer/Facilitator Mills River, NC Lavona Russell Cat Financial (retired) Nashville Renee Tew Maury Regional Medical Center Columbia

116

Recruit Applicants Recruit & Train Examiners Receive & Process Applications Assign Teams Manage Assessment Process Judges Determine Award Levels Edit & Deliver Feedback Reports Present Awards & Celebrate!

Overview of TNCPE Award Cycle

117

Stair Steps to Excellence

118

Level 4 Level 3 Excellence Level 2 Achievement Level 1 Commitment Interest Org Profile Yes Yes Yes Yes Application Focus Org Profile Basic item requirements Overall item requirements Multiple item requirements Max Length 5 pages 5 + 15 pages 5 + 35 pages 5 + 50 pages Site Visit ½ Day 1 Day+wkshp 2 Days 3 Days Feedback Focus 7 Categories Overall requirements Multiple requirements Multiple requirements

Examiner Development

119

Year 1

• Orientation
• 3-day Training
• Service on a

Team

Year 2

• Orientation
• 3-day Training
• Service on a

Team

Year 3 +

• 3-day Training
• Service on a

Team

• Scorebook

Editor

Goal

• Master

Examiner

• One day of

3-day Training

• Service on a

Team

• Team Leader

Roles of the TNCPE Examiner

ard rocess

• Independent

Review

• Consensus

Review

• Site isit
• Prepare Final

Scorebook S I Worksheets

• Recommend

Award levels C m assador

• Speak to groups
• r arrange a

speaker

• Distribute

materials

• Participate in

social media

• Encourage

examiner and award applicants

• lunteer
• Conference
• Banquet
• Write articles
• Other duties as

required…

120

Let’s Stay Connected!

121

http://www.tncpe.org/

Evaluation of Day 2

+’s and Δ’s 2 key learnings to share

Tomorrow is T-shirt Day!