Welcome 2018 Board of Examiners Examiner Training Day 1 - - PowerPoint PPT Presentation

welcome 2018
SMART_READER_LITE
LIVE PREVIEW

Welcome 2018 Board of Examiners Examiner Training Day 1 - - PowerPoint PPT Presentation

Welcome 2018 Board of Examiners Examiner Training Day 1 Introductions Name Company What is your intention for examiner training? What are you famous for? 2 Team Agreement How can we make this a successful training course?


slide-1
SLIDE 1

Welcome 2018 Board of Examiners

Examiner Training – Day 1

slide-2
SLIDE 2

Introductions

  • Name
  • Company
  • What is your intention for examiner training?
  • What are you famous for?

2

slide-3
SLIDE 3

Team Agreement

How can we make this a successful training course? We will . . .

3

slide-4
SLIDE 4

Course Objectives

  • Expand your understanding of the Baldrige Excellence

Framework

  • Prepare examiners to write comments that bring value

to applicants at all levels

  • Learn more about the Scorebook Navigator examiner

software

  • Practice working as a team
  • Network and share knowledge
  • Discover opportunities for growth as an examiner

4

slide-5
SLIDE 5

Trust the Process!

I hear . . . and I forget. I see . . . and I remember. I do . . . and I understand. Ancient Chinese Proverb

5 5

slide-6
SLIDE 6

Examiner Training Improvements

  • Lots of change!
  • New TNCPE staff
  • New Score Summary Worksheet
  • New Level 2 workshop
  • Level 1 Redesign Pilot
  • Introduced in 2017 (and embraced):
  • Extra Credit slides (now “Superstar”)
  • Scorebook Navigator system

6

slide-7
SLIDE 7

TNCPE Key Concepts

  • Integrity and ethics are critical to our success
  • Meet the applicants where they are and help

them improve

  • Select the most meaningful comments for the

final feedback report

  • So what?
  • Be specific!
  • Use key factors to focus on what’s important to

the applicant

7

slide-8
SLIDE 8

Examiner Evaluation Cycle

Complete Pre-work & Training

Independent

Review Consensus Review Site Visit Review Finalize Feedback Report

8

slide-9
SLIDE 9

Review for Conflict Review the Framework Read the Application Log on to the system Create Key Factor List Click “Done”! Prepare Consensus items Participate in Consensus meeting Prepare Site Visit issues Complete Site Visit Prepare Final Scorebook

12 process items

  • Criteria
  • Key Factors
  • Application: PROCESS
  • 6-10 comments

(ADLI)

  • Score
  • 1 Feedback-Ready

strength and OFI

5 results items

  • Criteria
  • Key Factors
  • Application: RESULTS
  • 6-10 comments

(LeTCI)

  • Score
  • 1 Feedback-Ready

strength and OFI

Independent Review

9

slide-10
SLIDE 10

Pre-work Debrief: Item 1.1

Find a partner and compare your analysis

  • Similarities and differences in:
  • Key factors
  • Comments (strengths and OFIs)
  • Score
  • How did the key factors drive your analysis?
  • What was a challenge for you?
  • Agree on one key learning to share

10

slide-11
SLIDE 11

11

slide-12
SLIDE 12

Pre-Work

  • If you haven’t turned in a copy of your pre-work,

please give us a copy before we begin the next module.

  • Give us a copy of your completed pre-work

evaluation form, too.

  • Be sure and include your hours on the form.

12

slide-13
SLIDE 13

Baldrige Excellence Framework

TM Overview

slide-14
SLIDE 14
slide-15
SLIDE 15

Baldrige Is an Effective Leadership System

  • Common sense
  • Aligns the entire organization to achieve the mission and

vision and live the values

  • Identifies gaps
  • Asks really good questions. For example:
  • How do senior leaders set your organization’s vision and values?
  • How can we do it better?
  • Works for any organization!

15

slide-16
SLIDE 16

Why Baldrige?

16

slide-17
SLIDE 17

Criteria Evolution

slide-18
SLIDE 18

Core Values – What and Why?

  • Principles of performance excellence
  • Philosophical foundation of the Criteria
  • Beliefs and behaviors that are
  • Embedded in the Criteria
  • Reflected in role model organizations’ performance

excellence systems

18

slide-19
SLIDE 19

Core Values & Concepts

  • Systems perspective
  • Visionary leadership
  • Customer-focused

excellence

  • Valuing people
  • Organizational learning

and agility

  • Focus on success
  • Managing for

innovation

  • Management by fact
  • Societal responsibility
  • Ethics and transparency
  • Delivering value and

results

19

slide-20
SLIDE 20

20

slide-21
SLIDE 21

21

slide-22
SLIDE 22

Key Factor Review

  • What is a “key factor”?
  • A significant attribute of an organization that influences

the way the organization operates

  • Where do you find key factors?
  • How do you select key factors for an item?
  • Ask yourself “Will it make a difference in my

assessment?”

  • How will you use key factors?
  • To focus the assessment on what’s important to

the applicant

22

slide-23
SLIDE 23

Table Exercise

  • Read the overall requirements for

your assigned items.

  • Using the master list of key factors

for Green Gateway, select the 4 to 6 KFs that are most relevant to each of your assigned items.

  • Which multiple item requirements

are most important to this applicant based on the assigned key factors?

  • Summarize what you would expect

the see in the applicant’s response.

Table Assigned Items 1 1.1 and 1.2 2 2.1 and 2.2 3 3.1 and 3.2 4 4.1 and 4.2 5 5.1 and 5.2 6 6.1 and 6.2 7 1.1 and 1.2 8 2.1 and 2.2

23

slide-24
SLIDE 24

24

slide-25
SLIDE 25

25

slide-26
SLIDE 26

Organizational Profile

  • What is “key” to applicant?
  • Key means “major or most important;” defined as

“around the most significant five”

  • What is unique about this applicant?
  • Key challenges?
  • Governance structure?
  • Location(s) or size?

26

slide-27
SLIDE 27

27

slide-28
SLIDE 28

Category 1: Leadership

28

1.1c(1) – Creating an Environment for Success

slide-29
SLIDE 29

29

slide-30
SLIDE 30

Category 2: Strategy

Core Competencies

An organization’s areas of greatest expertise. Core competencies are those strategically important, possibly specialized capabilities that are central to fulfilling your mission or that provide an advantage in your marketplace

  • r service environment.

Work Systems

How your organization’s work is accomplished, consisting

  • f the internal work processes and external resources

you need to develop and produce products, deliver them to your customers, and succeed in your marketplace.

30

slide-31
SLIDE 31

31

slide-32
SLIDE 32

Category 3: Customers

Customer Engagement

Your customers’ investment in or commitment to your brand and product offerings... Characteristics of engaged customers include retention, brand loyalty, willingness to make an effort to do business – and increase their business – with you, and willingness to actively advocate for and recommend your brand and product offerings.

Customer Satisfaction Dissatisfaction

32

slide-33
SLIDE 33

33

slide-34
SLIDE 34

34

slide-35
SLIDE 35

Category 4: Measurement, Analysis, and Knowledge Management

4.1 Measurement, Analysis, and Improvement of Organizational Performance

a. Performance Measurement b. Performance Analysis and Review c. Performance Improvement

Now includes projecting performance and identifying opportunities for continuous improvement and innovation

4.2 Information and Knowledge Management

a. Data and Information b. Organizational Knowledge NO LONGER INCLUDES IT MANAGEMENT

35

slide-36
SLIDE 36

36

slide-37
SLIDE 37

Category 5: Workforce

37

slide-38
SLIDE 38

38

slide-39
SLIDE 39

Category 6: Operations

6.1 Work Processes

  • a. Product and Process Design
  • b. Process Management and Improvement

c. Supply-Chain Management

  • d. Innovation Management

6.2 Operational Effectiveness

  • a. Process Efficiency and Effectiveness
  • b. Management of Information Systems

c. Safety and Emergency Preparedness

39

slide-40
SLIDE 40

40

slide-41
SLIDE 41

41

slide-42
SLIDE 42

42

slide-43
SLIDE 43

Does Anyone Know…

To whom do Green Gateway’s plant managers report?

  • Check the Org Chart
  • Check the Org Profile or 1.1a(2)

43

Kelley Molavi

slide-44
SLIDE 44

Consensus Overview

“None of us are as smart as all of us.”

Ken Blanchard

slide-45
SLIDE 45

Examiner Evaluation Cycle

Complete Pre-work & Training

Independent

Review Consensus Review Site Visit Review Finalize Feedback Report

45

slide-46
SLIDE 46

Pre-Consensus

  • Team leader sets due dates for:
  • Independent review
  • Consensus review
  • Consensus meeting
  • Site visit
  • Examiners complete individual key factor lists and

independent review of every item

  • Examiners click “done”!!!

46

slide-47
SLIDE 47

47

slide-48
SLIDE 48

Pre-Consensus

  • Team leader “moves” team to consensus

in Scorebook Navigator

  • Team leader (or designee) consolidates

key factors

  • Team leader assigns item leads and backups for

consensus stage

  • Team leader sets date for consensus meeting

48

slide-49
SLIDE 49

Sample Timeline

July/August 2018 Sept./Oct. 2018

Sunday Monday Tuesday Wednesday Thursday Friday Saturday Sunday Monday Tuesday Wednesday Thursday Friday Saturday

22 23 24 25 26 27 28 2 3 4 5 6 7 8 29 30 31 1 2 3 4 9 10 11 12 13 14 15 5 6 7 8 9 10 11 16 17 18 19 20 21 22 12 13 14 15 16 17 18 23 24 25 26 27 28 29 19 20 21 22 23 24 25 30 1 2 3 4 5 6 26 27 28 29 30 31 1 7 8 9 10 11 12 13

  • 1. Receive

Assignment

  • 2. Complete

Independent Review

  • 3. Consensus

Meeting

  • 4. Begin

Site Visit

  • 5. Feedback

report due

49

slide-50
SLIDE 50

Preparing an Item for Consensus Meeting

50

  • 1. Review the Criteria requirements
  • 2. Assign 4-6 key factors
  • 3. Read applicant’s response
  • 4. Review team members’ Independent Reviews

SYNTHESIZE

Synthesize team members’ IR into 6-10 comment topics Develop rationale

WRITE

Draft 6-10 feedback comments based on your chosen topics Balance strengths and OFIs

SCORE

Score the item based

  • n your consensus

review

REVISE

Revise comments based on feedback from your backup Provide feedback to your partner

PREPARE

Prepare to present your items during the team’s consensus meeting

slide-51
SLIDE 51

Consensus Materials & Resources

  • Application
  • Baldrige Excellence Framework
  • Criteria for Performance Excellence
  • Scoring Guidelines
  • Scorebook Navigator
  • Independent reviews

51

slide-52
SLIDE 52

52

slide-53
SLIDE 53

53

slide-54
SLIDE 54

54

slide-55
SLIDE 55

55

slide-56
SLIDE 56

56

slide-57
SLIDE 57

Key Dimensions of “Process”

  • Approach or methods (A)
  • Deployment (D)
  • Evaluation, improvement and cycles of learning (L)
  • Alignment or integration with organizational needs (I)

57

slide-58
SLIDE 58

Approach refers to:

  • Methods used to carry out a process
  • Appropriateness of methods
  • Effectiveness of the methods
  • Degree to which the approach

is repeatable and based on reliable data and information (i.e., systematic)

  • “Fixin’ to” is not the same as doing!
  • The plural of “example” ≠ systematic

58

slide-59
SLIDE 59

Deployment refers to:

The extent to which…

  • Approach is applied in addressing the item requirements

relevant and important to the organization

  • The approach is applied consistently
  • The approach is used by all appropriate areas

(work units, processes, employee groups, locations, stakeholders)

59

slide-60
SLIDE 60

Learning refers to:

  • Refining the approach through cycles of evaluation

and improvement

  • Encouraging breakthrough change to the approach

through innovation

  • Sharing refinements and innovations with other

relevant work units and processes in your

  • rganization

60

slide-61
SLIDE 61

61

slide-62
SLIDE 62

Integration refers to:

The extent to which

  • The approach is aligned with organizational needs

identified in other Criteria items

  • Measures, information, and improvement systems

are complementary across processes and work units

  • Plans, processes, results, analyses, learning, and

actions, are harmonized across processes and work units to support organization-wide goals.

62

slide-63
SLIDE 63

63

slide-64
SLIDE 64

64

slide-65
SLIDE 65
slide-66
SLIDE 66
slide-67
SLIDE 67

Energizer

  • 1. Assemble cartoon frames in the correct

sequence.

  • 2. Which multiple requirement(s) of the Criteria

does the cartoon address?

  • 3. Assign roles and prepare to read the cartoon

aloud to the class.

  • 4. Report out.

67

slide-68
SLIDE 68

68

slide-69
SLIDE 69

69

slide-70
SLIDE 70

70

slide-71
SLIDE 71

71

slide-72
SLIDE 72

72

slide-73
SLIDE 73

73

slide-74
SLIDE 74

74

slide-75
SLIDE 75

75

slide-76
SLIDE 76

Preparing for Consensus

Step 1: Synthesize

slide-77
SLIDE 77

Preparing an Item for Consensus Meeting

77

  • 1. Review the Criteria requirements
  • 2. Assign 4-6 key factors
  • 3. Read applicant’s response
  • 4. Review team members’ Independent Reviews

SYNTHESIZE

Synthesize team members’ IR into 6-10 comment topics Develop rationale

WRITE

Draft 6-10 feedback comments based on your chosen topics Balance strengths and OFIs

SCORE

Score the item based

  • n your consensus

review

REVISE

Revise comments based on feedback from your backup Provide feedback to your partner

PREPARE

Prepare to present your items during the team’s consensus meeting

slide-78
SLIDE 78

What’s the Difference?

Analysis

Breaking a complex topic into smaller parts to gain a better understanding of it

Synthesis

The process of integrating materials you have gathered and then creating your own information

78

slide-79
SLIDE 79

What Should I Synthesize?

Strength and OFI comments for:

  • Approach or methods (A)
  • Deployment (D)
  • Evaluation, improvement and cycles of learning (L)
  • Alignment or integration with organizational needs (I)

Gaps

  • When a key Criteria requirement is not addressed
  • When something the applicant has identified as

important is not addressed (KF)

79

slide-80
SLIDE 80

Remember…

  • It is your job to select the MOST insightful

6–10 pieces of information . . . not necessarily those that have already been written as feedback- ready comments.

  • Not all Criteria requirements are equally important

to every applicant!

80

slide-81
SLIDE 81

A Great Comment

  • Is insightful
  • Tells the applicant something it didn’t know
  • Provides meaningful feedback
  • Validates progress
  • Clearly describes key gaps and their significance to guide

future improvement

  • Helps senior leaders allocate resources
  • Is easily understood
  • Without reading the application

81

slide-82
SLIDE 82

Finding the “Sweet Spot”

82

Relevance to Applicant (examples & key factors) Process/Results Evaluation Factors Criteria Requirements = value- added feedback

slide-83
SLIDE 83

Rationale

  • What is a “rationale?”
  • The fundamental reason or justification for something
  • Why do you need one?

83

slide-84
SLIDE 84

Sample Rationales

For proposed strengths and OFIs:

  • Key Criteria requirement
  • Important to the applicant
  • Barrier that is keeping applicant from moving to the next

scoring range or award level

  • Identified as a strength or OFI on majority of

independent review worksheets (show initials of team members)

84

slide-85
SLIDE 85

Sample Rationales

For observations not used:

  • Not important to the applicant
  • Beyond the Criteria
  • Strength conflicted with OFI comment
  • OFI conflicted with strength comment
  • Prescriptive

85

slide-86
SLIDE 86

Sample comment topics for 5.1

Item Ref. Comment Topics Rationale a(1) + PES used to assess workforce capability and capacity (A) Identified by JH (double), KM, MH, erased conflicting OFI a(2) + 12-step systematic approach for hiring, placement, and retention of employees (A, D) Identified by AY, KM, JH, key Criteria requirement b(1) + Multiple approaches for workforce health, safety, and accessibility (A) Identified by KM, AY, MH, key Criteria requirement b(1) ‒ Not clear how workplace environments differ for OSHA requirements, safety, or temporary employees (I) GAP (key factors). Identified by AY a,b ‒ Missing cycles of learning for many workforce processes (L) Addresses L, identified by AY

86

slide-87
SLIDE 87

87

slide-88
SLIDE 88

Item Ref. Comment Topics Rationale a(1) + PES used to assess workforce capability and capacity (A) Identified by JH (double), KM, MH, erased conflicting OFI a(2) + 12-step systematic approach for hiring, placement, and retention of employees (A, D) Identified by AY, KM, JH, key Criteria requirement b(1) + Multiple approaches for workforce health, safety, and accessibility (A) Identified by KM, AY, MH, key Criteria requirement b(1) ‒ Not clear how workplace environments differ for OSHA requirements, safety, or temporary employees (I) GAP (key factors). Identified by AY a,b ‒ Missing cycles of learning for many workforce processes (L) Addresses L, identified by AY

Sample comment topics for 5.1

88

slide-89
SLIDE 89

Assignments

Table Item 1 1.1 2 2.1 3 3.1 4 6.1 5 1.1 6 2.1 7 3.1 8 6.1

89

slide-90
SLIDE 90

Exercise Instructions

Step 1 (10 min.)

  • Independently review the Criteria requirements for your assigned item
  • Independently review your homework and team’s IR worksheets to find

commonalities and conflicts

Step 2 (30 min.)

  • As a table, using the poster provided . . .
  • Identify 4-6 relevant key factors from master list (< 5 min.)
  • Select 6 topics (most important key messages) to develop into

strength and OFI comments

  • Summarize your rationale for each comment

Note: Don’t write feedback-ready comments yet!

90

slide-91
SLIDE 91

91

slide-92
SLIDE 92

Preparing for Consensus

Step 2: Write Comments

slide-93
SLIDE 93

Preparing an Item for Consensus Meeting

93

  • 1. Review the Criteria requirements
  • 2. Assign 4-6 key factors
  • 3. Read applicant’s response
  • 4. Review team members’ Independent Reviews

SYNTHESIZE

Synthesize team members’ IR into 6-10 comment topics Develop rationale

WRITE

Draft 6-10 feedback comments based on your chosen topics Balance strengths and OFIs

SCORE

Score the item based

  • n your consensus

review

REVISE

Revise comments based on feedback from your backup Provide feedback to your partner

PREPARE

Prepare to present your items during the team’s consensus meeting

slide-94
SLIDE 94

Comments Should…

  • NOT exceed 75-100

words

  • NOT include dense

criteria language

  • NOT be judgmental or

prescriptive

  • NOT focus on how

well the application is written

94

  • Let the applicant know

what it does well

  • …and what it needs to

improve

  • Include examples from

the application

  • Use complete sentences
  • Contain a clear “so what”

94

slide-95
SLIDE 95

Comment Writing Tips

Is the comment Criteria-based? Is it insightful? T = Start with the “topic sentence” or main point E = Give 1 – 2 supporting examples from the application S = Explain the “so what” (relevance) to the applicant by linking the comment to a key factor T = Try it out loud!

95

slide-96
SLIDE 96

Sources of “so what”

96

  • Key factors
  • Item notes
  • Category and item commentary
  • Core Values and Concepts
  • Glossary of Terms
  • A-D-L-I progression or gaps
  • Le-T-C-I progression or gaps
  • Scoring guidelines
slide-97
SLIDE 97

Sample comment topics for 5.1

97

Item Ref. Comment Topics Rationale a(1) + PES used to assess workforce capability and capacity (A) Identified by JH (double), KM, MH, erased conflicting OFI a(2) + 12-step systematic approach for hiring, placement, and retention of employees (A, D) Identified by AY, KM, JH, key Criteria requirement b(1) + Multiple approaches for workforce health, safety, and accessibility (A) Identified by KM, AY, MH, key Criteria requirement b(1) ‒ Not clear how workplace environments differ for OSHA requirements, safety, or temporary employees (I) GAP (key factors). Identified by AY a,b ‒ Missing cycles of learning for many workforce processes (L) Addresses L, identified by AY

slide-98
SLIDE 98

5.1a(1)

Comment: Using the Performance Evaluation System (PES), the applicant demonstrates a systematic approach to assessing workforce capability and capacity needs. The PES assesses associates' skills, competencies, and certifications to produce a comprehensive Training Matrix (Figure 5.1-1). Directors validate associate capacity on an annual basis by matching time studies to job descriptions and work flow. The applicant's systematic approach for assessing workforce capability and capacity needs may help the applicant meet seasonal workforce demands and address its strategic challenge of workforce retention. Rationale: Identified by JH (double), KM, MH, erased conflicting OFI

Sample + Consensus Comment

slide-99
SLIDE 99

5.1a(2)

Comment: The applicant deploys a 12-step process for recruiting, hiring, placing, and retaining new workforce members. The collaborative process between the Human Resources (HR) Department and the organization's directors begins with HR receiving an approved position requisition. Resumes from potential candidates are solicited through recruiting agencies, and print and

  • nline advertising. Interviews and assessments are conducted in
  • rder to identify the final candidate. Newly-hired associates are

assigned a mentor and undergo 30-, 60-, and 90-day reviews to monitor progress and support retention efforts. The applicant's approach for recruiting, hiring, placing, and retaining new workforce members reinforces the organization's core competency

  • f talent development

Rationale: Identified by AY, KM, JH, key Criteria requirement

Sample + Consensus Comment

slide-100
SLIDE 100

OFI Comments

  • Before site visit, often include a phrase such as “it is

not clear”

  • Site visit is your opportunity to clarify everything that is

“not clear”!

  • Sometimes begin with “Although” or “While”
  • To highlight the gap (i.e., the approach is good but the

deployment is lacking)

  • To affirm what the applicant has told us

100

OBLIGATION

slide-101
SLIDE 101

Sample comment topics for 5.1

101

Item Ref. Comment Topics Rationale a(1) + PES used to assess workforce capability and capacity (A) Identified by JH (double), KM, MH, erased conflicting OFI a(2) + 12-step systematic approach for hiring, placement, and retention of employees (A, D) Identified by AY, KM, JH, key Criteria requirement b(1) + Multiple approaches for workforce health, safety, and accessibility (A) Identified by KM, AY, MH, key Criteria requirement b(1) ‒ Not clear how workplace environments differ for OSHA requirements, safety, or temporary employees (I) GAP (key factors). Identified by AY a,b ‒ Missing cycles of learning for many workforce processes (L) Addresses L, identified by AY

slide-102
SLIDE 102

Sample Consensus OFI Comment

5.1b(1)

Comment: It is unclear if the applicant’s processes or measures for ensuring the health, security, and accessibility of its workforce are tailored for different workplace environments. For example, there is no mention of addressing OSHA compliance or maintaining air quality (Figure P.1-7) differently in various areas of the facility such as the shop floor, shipping/receiving, or office areas. No health or security distinctions are made for different workplace environments

  • r for workforce segments, such as temporary workers (Figure P.1-

5). Consideration of the unique needs of different workplace environments may help the applicant ensure it provides a healthy workplace for all associates while effectively managing

  • rganizational resources.

Rationale: GAP (key factors), identified by AY

102

slide-103
SLIDE 103

Sample Consensus OFI Comment

5.1a,b

Comment: It is unclear if the applicant engages in systematic cycles

  • f learning related to its workforce environment processes. While

the applicant identifies examples of process changes that were implemented during cycles of improvement, it is unclear if these changes were the result of a systematic evaluation of processes, or were reactive to events or circumstances. Systematic evaluation and improvement of the organization's workforce environment processes may assist the applicant in creating new value through innovation and quality improvement. Rationale: addresses L – identified by AY

103

Amanda Yanez

slide-104
SLIDE 104

Assignments

Table Item 1 1.1 2 2.1 3 3.1 4 6.1 5 1.1 6 2.1 7 3.1 8 6.1

104

slide-105
SLIDE 105

Exercise Instructions–Step 1

  • Review the team’s work from previous exercise
  • Assign one comment to each member of your table

team (either a strength or an OFI) Independently…

  • Develop “your” outline into a feedback-ready comment
  • Provide examples to support your comment
  • So what?
  • Note item reference

105

slide-106
SLIDE 106

106

slide-107
SLIDE 107

Exercise Instructions–Step 2

Find a partner: Review each other’s comments and provide written feedback in the right-hand column:

  • 1. Does the comment:
  • Relate to Criteria requirements?
  • Identify the main point in the “topic sentence”?
  • Address relevant evaluation factor(s)?
  • Use examples from the application?
  • Convey “so what”?
  • 2. Is it actionable for the applicant?

107

slide-108
SLIDE 108

Preparing for Consensus

Step 3: Score

slide-109
SLIDE 109

Preparing an Item for Consensus Meeting

109

  • 1. Review the Criteria requirements
  • 2. Assign 4-6 key factors
  • 3. Read applicant’s response
  • 4. Review team members’ Independent Reviews

SYNTHESIZE

Synthesize team members’ IR into 6-10 comment topics Develop rationale

WRITE

Draft 6-10 feedback comments based on your chosen topics Balance strengths and OFIs

SCORE

Score the item based

  • n your consensus

review

REVISE

Revise comments based on feedback from your backup Provide feedback to your partner

PREPARE

Prepare to present your items during the team’s consensus meeting

slide-110
SLIDE 110

Important Scoring Concepts

  • Ground yourself in Criteria requirements AND

applicant’s key factors

  • Don’t average independent review scores
  • Review your comments in terms of importance to

the applicant

  • Score the item based on your consensus review

110

slide-111
SLIDE 111

Scoring Resources

  • Item format (p.2 in Framework booklet)
  • Definitions
  • Basic, overall & multiple requirements
  • A-D-L-I
  • “Alignment” vs. “integration”
  • Scoring guidelines (available in Scorebook

Navigator and on p.34 in Framework booklet)

111

slide-112
SLIDE 112

112

slide-113
SLIDE 113

Item format (page 2)

113

slide-114
SLIDE 114

Scoring Example – Item 5.1

  • Overall scoring range: 50-65%
  • Item score: 55%
  • Rationale: The applicant demonstrates effective, systematic

approaches responsive to the majority of the overall requirements, which are well-deployed across most work units. The applicant demonstrates that some of its processes have undergone cycles of improvement; however, it is not evident whether a fact-based systematic evaluation and improvement process is utilized to make these process improvements, or if they are reactive in nature. Workforce environment approaches are in the early stages of integration with overall organizational needs.

114

slide-115
SLIDE 115
slide-116
SLIDE 116

Assignments

Table Item 1 1.1 2 2.1 3 3.1 4 6.1 5 1.1 6 2.1 7 3.1 8 6.1

116

slide-117
SLIDE 117

Exercise Instructions

Individual Exercise:

1. Review your team’s 6-10 comments paying special attention to A-D-L-I 2. Determine the scoring range that “best fits” the item 3. Score the item

As a Table:

1. Share your item score and scoring rationale with your team 2. Reach consensus on the item score 3. Report out

117

slide-118
SLIDE 118

118

slide-119
SLIDE 119

What’s Missing?

What is the difference between the scoring guidelines for approach in the 70-85% range versus the 90-100% range? Answer: “well” deployed (70-85%) vs. “fully” deployed (90-100%).

119

slide-120
SLIDE 120

Consensus Meeting

“If everyone is moving forward together,

then success takes care of itself.”

Henry Ford

slide-121
SLIDE 121

The Importance of Teamwork

121

slide-122
SLIDE 122

What is Consensus?

  • Everyone can support the team’s decision.
  • When you leave the meeting will you say

“we decided” or “they decided”?

122

slide-123
SLIDE 123

Preparing an Item for Consensus Meeting

  • 1. Review the Criteria requirements
  • 2. Assign 4-6 key factors
  • 3. Read applicant’s response
  • 4. Review team members’ Independent Reviews

SYNTHESIZE

Synthesize team members’ IR into 6-10 comment topics Develop rationale

WRITE

Draft 6-10 feedback comments based on your chosen topics Balance strengths and OFIs

SCORE

Score the item based

  • n your consensus

review

REVISE

Revise comments based on feedback from your backup Provide feedback to your partner

PREPARE

Prepare to present your items during the team’s consensus meeting

123

slide-124
SLIDE 124

Purpose of Consensus

  • Resolve individual differences in:
  • Key factors
  • Strengths and OFIs
  • Scores
  • Agree on site visit issues
  • Begin to develop key themes

124

slide-125
SLIDE 125

Sample Roles

Examiner Item Lead Back-up Other 1 1.1, 1.2, 7.4 6.1, 6.2, 7.1 Team leader 2 2.1, 2.2 5.1, 5.2, 7.3 Scorebook editor 3 3.1, 3.2, 7.2 4.1, 4.2, 7.5 Back-up team leader 4 4.1, 4.2, 7.5 3.1, 3.2, 7.2 Criteria cop 5 5.1, 5.2, 7.3 2.1, 2.2 Arrange logistics 6 6.1, 6.2, 7.1 1.1, 1.2, 7.4 Time keeper

125

slide-126
SLIDE 126

Topics for Consensus Meeting

  • Changes to the language of a comment that affect:
  • The meaning of the comment
  • The A-D-L-I or Le-T-C-I assessment factors that pertain
  • The balance of strengths and OFIs
  • The score
  • Key themes
  • Comments that are not “key” or Criteria-based
  • If a “key” area to address is missed
  • Conflicts between strengths & OFIs, or comments in different

categories

  • Potential site visit issues

126

slide-127
SLIDE 127

NOT Topics for Consensus Meeting

  • Anything that does not alter the meaning of a comment
  • r the score
  • Grammatical errors
  • Typos
  • Misspellings
  • Word format changes
  • Extra words or words left out
  • Sentences or clauses that are redundant
  • Substituting acronyms for entity names, and the reverse
  • Separating the clauses of a lengthy sentence to make

shorter sentences.

127

slide-128
SLIDE 128

Exercise

128

slide-129
SLIDE 129

Tips for Consensus Meeting

  • Be prepared
  • Don’t be defensive – check your ego at the door!
  • Begin with an ice breaker
  • Set ground rules
  • Does silence mean agreement?
  • Use a round robin approach to be sure everyone’s

voice is heard

  • Only discuss areas of disagreement

129

slide-130
SLIDE 130

Do You Have a Great Meeting Space?

130

slide-131
SLIDE 131

Tips for a Virtual Consensus

Resources:

  • FreeConferenceCall.com (also has video option)
  • WebEx (through a team member)
  • GoToMeeting (through TNCPE)
  • Many other free conference call sites

Ground Rules:

  • Schedule in advance
  • Agenda is king
  • Use Round Robin frequently
  • Consider two ½ day meetings

131

slide-132
SLIDE 132

Evaluation of Day 1

+’s and Δ’s 2 key learnings per table Homework: read TNCPE

  • Org. Profile