2017 Examiner Training and Preparation Course The Journey Begins - - PowerPoint PPT Presentation

2017 Examiner Training and Preparation Course

The Journey Begins (Continues)!

Before We Begin….

Logistics

• Safety
• Location of exits
• Use of phone

(please place on silent)

• Restrooms
• Parking Lot

Parking Lot

Quality Texas Meeting Behaviors

• Be respectful
• Share openly
• Have only one conversation

at a time

• Mute phones
• Return from breaks on time
• Take care of personal needs
• Clean up after yourself
• Stay on point
• Work toward consensus
• Understand that silence

means affirmation

• Employ “ELMO” as needed
• Have fun

Ground Rules

Learning Outcomes

 Gain an understanding of Baldrige Framework; (organizational improvement)  Apply the Six-Step Independent Review Process to an Award Application  Choosing/Writing Key Factors, Strengths/OFIs, Scoring, and discussing Key Themes  Write Better Feedback Comments Using the Comment Guidelines  Improve your personal BRAND!

Course Overview

 Tuesday – Quick review of the

Baldrige Framework; Review the Organizational Profile; Determine Key Factors; Discuss Six-Step IR Process; Review Items

 Wednesday – Six-Step IR Process;

Review other Items

 Thursday – Six-Step IR Process;

Review Results Items; Review QTF Website; Scorebook Navigator Class

Introductions

• Who you are!

– Name / City

• What you do!

– Organization – Job title

• Experience with QTF or

Baldrige Criteria

• Your expectations of

this Training

About Quality Texas Laying Concrete

Quality Texas Foundation

• Ms. Lin Wrinkle

Chief Operations Officer (COO) linwrinkle@quality-texas.org; (512) 818-3901

• Dr. Mac McGuire

Chief Executive Officer (CEO) drmac@quality-texas.org; (512) 656-8946 Address: 201 Woodland Park, Georgetown, Texas 78633-2007 www.quality-texas.org

QTF Organizational Chart

Board of Directors Volunteers Training Team Judges CEO/Staff

Key Facts About Quality Texas Foundation

▪ President Reagan national concept 1987-1989; Mac Baldrige, Secretary

• f Commerce; died in rodeo accident

▪ State Concept 1990; Started 1992; Endorsed by Governor Ann Richards; accepted applications 1993 ▪ Founded in 1994 as a 501(c)3 not- for-profit corporation ▪ Full-time staff of two people; around 350 state-wide volunteers; over 1,500

• n mailing list

QTF’s Vision and Mission

Vision: Th

The e Qualit Quality y Te Texa xas s Fo Foun unda dation tion (Q (QTF TF) ) wi will ll co continu ntinue e to to be be the the pre preemine eminent nt state state p prog rogram ram ass assisti isting ng individua individuals, ls,

• rg
• rgan

anizations, and izations, and co commun mmuniti ities es wi with th co continu ntinuou

• us

s improve improvemen ment t efforts efforts.

Mission: QTF ex

QTF exists ists to to ass assist ist individua individuals, ls, o

• rga

rganiza nizations tions an and d co commun mmuniti ities es in t in the heir ir co continu ntinuou

• us

s impro improve veme ment nt effo efforts the rts thereb reby y po positi sitively vely impac impacting ou ting our r co commun mmuniti ities, state, es, state, an and na d nation tion.

Key Facts About Quality Texas Foundation

▪ Funded through

▪ Memberships/Sponsorships, ▪ Examiner Training, ▪ Customized training workshops, ▪ Application fees, ▪ Annual Conference ▪ Recognized as the most effective state program, with more Baldrige recipients than any other state (20); 1 two-time recipient; 53 state winners

QTF’s Values, CCs, and Motto

Values:

• Systematic Perspective
• Delivering Value and Results
• Personal and Organizational Learning

Core Competencies:

• Executing the Baldrige Framework
• Engaging Volunteers

Motto:

• Making a Difference Every Day!

Services We Provide

• Organizational Assessment & Feedback
• Recognition/Awards at our Annual Conference
• Examiner Training (You are here!)
• Applicant Training (how to apply) (Tng/Coaching)

– writing seminars; – assessment techniques; – site visit training; – Leadership seminars; – Strategic Planning events, and – Communities of Excellence (COE) Seminars

• Membership Opportunities with options/discounts

Why Organizations Apply

• $820:$1 ROI
• Outside evaluation of organizational performance
• Improve financials, customer/employee engagement
• Helps build a common, holistic, and systematic view
• f the organization for greater improvements
• Framework provides a common language;

standardized method to examine processes and performance; objective feedback

• Tracks progress - Get to the “next level” of

performance

• Fosters benchmarking in and out of industry

QTF Process Map

QTF Processes/Steps

Training

• Pre Examiner
• Regular
• Annually

Applications Received

Engagement (10) Commitment (20) Progress (30) Award (50)

Teams Selected Independent Review Consensus Review Site Visit Feedback Report Judges Review Board Review Award Applicant Notified

QTF Examination Process

• Training – Pre-Examiner and Regular
• Independent Review (IR) – Application;

Strengths/OFIs; Score; Complete IR Scorebook

• Consensus Review (CR) – Consensus Scorebook;

Category Champions; Prepare for Site Visit?

• Site Visit Review – If applicable; Prepare questions for
• rganization to verify/clarify application
• Feedback Report – Most valuable product
• Judge Review – Panel of Judges Review and Assess
• Board Review – Board of Directors Final Review; Calls
• Awards/Recognition at Conference

Application Levels

QTF is there for YOU and YOUR

• rganization with tailored

products and services!

QTF LEVELS

Award Next Level Award Level* Progress Recognition Commitment Recognition Engagement Recognition Pioneer Recognition

Award (Achievement) Levels 50 pages + 5 (OP) Progress Level 30 pages + 5 (OP) Commitment Level 20 pages + 5 (OP) Engagement Level 10 pages + 5 (OP) Pioneer Level 5 pages (OP) Award Next 50 pages + 5 (OP)

Examiner Path

Examiners’ Roles and Responsibilities

• Provide analysis/feedback to lead the organization to the

next level of maturity; work as a team

• Examination Process Team Members

– Examiners – Senior Examiners – Alumni Examiners – Scorebook Editor – Back-up Team Leaders – Team Leader – Process Coach

• Each team has a Subject Matter Expert in the field (Can be

any role)

Code of Ethics, Examiner Availability, and Examiner Career Path

See Sheets for Code of Ethics and Examiner Availability (please turn in to Lin Thursday am

Examiner Career Path

QTF needs YOU!

Examiner Career Path

Examiner Position

Time Experiences

Examiner

1-3 years BU TL, FBW, 1 SV, Examiner Training Each Year

Senior Examiner

4-6 years TL, 2 SV+, Examiner Training Each Year

Alumni Examiner

6+ TL, Process Coach, 3+ SV, Examiner Training Each Year Various Committees

Judge

6+ TL, Process Coach, 3+ SV, Examiner Training Each Year Various Committees

Board of Directors

6+ Corporate Sponsorship, TL, Process Coach, 3+ SV, Examiner Training Each Year Various Committees

Fellow

10+ Dedication and Continual Volunteerism for QTF Issues

Benefits of Being an Examiner

• Learn Quality Texas/Baldrige Performance

Excellence Criteria; training/experience

• Learn validated best practices; leading
• rganizations; all industry sectors
• Expand professional network; improve YOUR

brand; valuable professional credential

• Develop assessment, analysis, writing, teamwork

and leadership skills

• Give back to community/state/nation by helping
• rganizations be successful through useful

feedback (patriotism)

Conditions of Involvement

Absolute Essentials to the Credibility, Success and Prestige of the Assessment and Feedback Process

• Commit to the entire process (until

the feedback is presented to the applicant)

• Maintain Confidentiality
• Honor Time Commitments
• Represent Quality Texas
• Follow Code of Conduct; Avoid Conflicts
• f Interest

Value for the Examiner/Sponsor

• Work with a diverse team; reach consensus
• Network with quality/business experts; build

professional friendships

• Understand/apply the Baldrige Framework

to organizations; your own personal plan?

• Develop analytical/consensus-building skills
• Attend annual Texas Award Banquet and

Conference – Westin Hotel, June, 2017, Houston, TX

• Special recognition at conference

QTF EXAMINATION & FEEDBACK PROCESS

Three Steps!

QTF EXAMINATION & FEEDBACK PROCESS

Stage 1 – Independent Review – Work done on your own – (less Pioneer) Stage 2 – Consensus Review – work done with team (less Pioneer) Stage 3 – Site Visit – on site with the applicant (if Progress or Award Level); Feedback Report

Walk Through Baldrige Framework Booklet

Take a look at your Baldrige Framework Book

Baldrige Excellence Framework

• Criteria: Systems Perspective – 1
• Basic, Overall, Areas to Address – 2
• Point Values per Category/Item – 3
• Organizational Profile – 4-6
• Categories 1-7 – 7-30
• Scoring System – 31-33

Baldrige Excellence Framework

• Process Scoring Guidelines – 34
• Results Scoring Guidelines - 35
• Responding to the Criteria – 37-39
• Core Values and Concepts – 40-44
• Changes from 2015-2016 – 45-46
• Glossary of Key Terms – 47-54

Categories 1-6 Examination

How Do I Evaluate Process Items?

Process items are evaluated using four factors:

• Approach
• Deployment
• Learning
• Integration ADLI

Approach (A)

“Approach” comprises

• Methods used to carry out Process
• Appropriateness of methods vs requirements
• Effectiveness of the methods
• The degree to which the approach is repeatable and

based on reliable data/information (i.e., systematic)

• Common Language (CL): HOW do you address the

question

Deployment (D)

“Deployment” is the extent to which

• Your approach addresses item

requirements relevant and important to the

• rganization
• Your approach is applied consistently
• The approach is used by all appropriate

work units

• CL: Who needs to understand the

approach

Learning (L)

Learning comprises

• The refinement in approach;

cycles of evaluation/improvement

• The encouragement of

breakthrough change through innovation, and,

• The sharing of refinements and

innovation with other work units and processes in the organization

• CL: Are we getting smarter/better?

Integration (I)

“Integration” is the extent which

• Your approach is aligned with organizational

needs (OP/ processes)

• Your measures, information, and

improvement systems are complementary across processes and work units

• Your plans, processes, results, analyses,

learning, and actions are “harmonized” across processes/work units to support goals

• CL: Agreement/Alignment with VMVG

Category 7 Examination

Results: 45% of the Total Score Category 7 Evaluated by LeTCI

Levels – Le Trends – T Comparisons – C Integration - I

Results Evaluation Factors  LeTCI

• Levels – Current performance; meaningful measurement scale
• Trends – Numerical data; shows the direction/rate of

improvements (slope over time)

• Comparisons – Performance relative to that of other

appropriate organizations, competitors or organizations that are similar; relative to industry leaders/benchmarks

• Integration – Measures (segmentation) addressing important

performance requirements relating to customers, products/services, markets, processes, or action plans identified in OP and in process items; future performance; harmonization across processes and work units to support organization-wide goals

Segmentation

Segments can be defined by, among other things:

▪ Customers (students/families for Education; patients/families for HC) ▪ Market/Product offerings ▪ Location ▪ Workforce group (employees, tenure, admin, hourly, etc.) ▪ Size of the group in question

The applicant defines their segments (OP – P1); we assess the results of those segments

Comparisons

Comparisons can be defined by, among other things (Organizational Profile OP – P2 Competition):

• Inside the industry
• Competitive comparisons
• Outside the industry

QTF/Baldrige Examination 6 Step Process

TRUST THE PROCESS! (KEY TEACHING POINT)

OP – The Preliminary Step

• Organizational Profile, Eligibility

Forms

• Reader’s digest version of

application

• Sets stage for what the

applicant says is important

• Do not read into or take away
• Develop Key Factors from what the

applicant says is important

Step 1: Read the Criteria.

Read Baldrige Framework for Award Level QTF Pioneer, Engagement, Commitment,

• r Progress

Level Criteria

Step 2: Determine Most Relevant Key Factors

Four to Six Key Factors taken from OP, Eligibility, or from Application

Step 3: Read & Analyze the Application

Read the Application Mark as appropriate

Step 4: Identify Strengths/OFIs

Around 6 comments Strengths and OFIs

Step 5: Write Feedback Ready Comments

Remember: NERD!

N – nugget of

importance

E – evidence/example

to support comments

R – relevance to the

applicant

D – Done!

Step 6: Determine the Scoring Range and Score

Best Fit Don’t Block a Winner Tie goes to applicant

Scoring: Not like your previous education! Go to Process Scoring Page 34

UNDERSTANDING SYSTEMATIC APPROACH

UNDERSTANDING DEPLOYMENT

UNDERSTANDING IMPROVEMENT/LEARNING/INNOVATION

UNDERSTANDING INTEGRATION

Go to Results Scoring Page 35

What is different about Results Scoring?

Questions to this Point??

So let’s begin our Examination/Evaluation Independent Review

We will follow the exact same steps that you will follow

Preliminary Step: Importance of the Organizational Profile

• Reader’s digest version of the

application

• Sets the stage for what the

applicant says is important

• Frames our comments (feedback

report), strengths/OFIs, scoring, and Key Themes

• Assists with our scoring band

descriptors

• Do not read into or take away

How to Determine Key Factors

• About 18-20 KFs
• Obtained from OP, Eligibility Form, and Application
• What is important?

– OP – Eligibility Form – Application

• Do not rewrite the OP
• Walk through with me

P.1 Organizational Descriptions:

What are your key organizational characteristics?

In your response, answer the following questions:

P.1a Organizational Environment (1) Product Offerings What are your main HEALTH CARE SERVICE offerings? What is the relative importance of each to your success? What mechanisms do you use to deliver your HEALTH CARE SERVICES? (2.0, 3.0, 6.0, 7.1) (2) MISSION, VISION, and VALUES What are your stated MISSION, VISION, and VALUES? What are your organization’s CORE COMPETENCIES, and what is their relationship to your MISSION? (1.0, 2.0, 7-1-7-5)

P.1 Organizational Descriptions:

What are your key organizational characteristics?

(3) WORKFORCE What is your WORKFORCE profile? What recent changes have you experienced in WORKFORCE composition or your WORKFORCE needs? (5.0, 7.3)

• What are your WORKFORCE or employee groups and SEGMENTS,

the educational requirements for different employee groups and SEGMENTS, and the KEY drivers that engage them in achieving your MISSION and VISION? (1.0, 2.0, 5.0, 7.3)

• What are your organized bargaining units (union representation)?

What are your organization’s special health and safety requirements? (4) Assets What are your major facilities, technologies, and equipment?

6 6

P.1 Organizational Descriptions:

What are your key organizational characteristics?

(5) Regulatory Requirements What is the regulatory environment under which you

• perate? What are the KEY applicable
• ccupational health and safety regulations;

accreditation, certification, or registration requirements; industry standards; and environmental, financial, and HEALTH CARE SERVICE delivery regulations? (7.0)

P.1 Organizational Descriptions:

What are your key organizational characteristics?

• b. Organizational Relationships

(1) Organizational Structure What are your organizational structure and GOVERNANCE system? What are the reporting relationships among your GOVERNANCE board, SENIOR LEADERS, and parent organization, as appropriate? (1.0, 7.4) (2) PATIENTS, OTHER CUSTOMERS, and STAKEHOLDERS What are your KEY market SEGMENTS, PATIENT and other CUSTOMER groups, and STAKEHOLDER groups, as appropriate? What are their KEY requirements and expectations for your HEALTH CARE SERVICES, other CUSTOMER support services, and operations? What are the differences in these requirements and expectations among market SEGMENTS, PATIENT and other CUSTOMER groups, and STAKEHOLDER groups? (3.0, 7.2)

P.1 Organizational Descriptions:

What are your key organizational characteristics? (3) Suppliers and PARTNERS What are your KEY types of suppliers, PARTNERS, and COLLABORATORS? What role do they play

• In your WORK SYSTEMS, especially in producing and

delivering your KEY HEALTH CARE SERVICES and PATIENT and other CUSTOMER support services; and in enhancing your competitiveness? (2.2, 3.0, 4.0, 6.0, 7.2)

• What are your KEY mechanisms for two-way communication

with suppliers, PARTNERS, and COLLABORATORS? What role, if any, do these organizations play in contributing and implementing INNOVATIONS in your organization? What are your KEY supply-chain requirements? (3.0)

P.2 Organizational Situation:

What is your organization’s strategic situation?

P.2 Organizational Situation: What is your

• rganization’s strategic situation? In your

response, include answers to the following questions:

• a. Competitive Environment

(1) Competitive Position What is your competitive position? What are your relative size and growth in the health care industry or the markets you serve? How many and what types of competitors do you have? (1.0, 2.2, 7.0)

P.2 Organizational Situation:

What is your organization’s strategic situation?

(2) Competitiveness Changes What KEY changes, if any, are affecting your competitive situation, including changes that create opportunities for INNOVATION and collaboration, as appropriate? (1.0, 2.2, 7.0) (3) Comparative Data What KEY sources of comparative and competitive data are available from within your health care industry? What KEY sources of comparative data are available from outside your industry? What limitations, if any, affect your ability to

• btain or use these data? (1.0, 2.2, 7.0)

P.2 Organizational Situation:

What is your organization’s strategic situation?

• b. Strategic Context
• What are your KEY STRATEGIC CHALLENGES and

ADVANTAGES in the areas of HEALTH CARE SERVICES, operations, societal responsibilities, and WORKFORCE? (1.0, 2.0, 7.0)

• c. PERFORMANCE Improvement System
• What are the KEY elements of your PERFORMANCE

improvement system, including your PROCESSES for evaluation and improvement of KEY

• rganizational projects and PROCESSES? (1.0, 2.0,

3.0, 4.0, 5.0, 6.0, 7.0)

Exercise: Determine Key Factors

Activity Time (min) 1. Select a scribe, timekeeper and reporter (1) 2. Discuss at table as a group your thoughts on the key factors for this applicant from Organizational Profile Tables 1/9: P1a(1)(2); Tables 2/10: P1a(3)(4)(5); Tables 3/11: P1b(1)(2); Tables 4/12: P1b(2)(3); Tables 5/13: P2a(1)(2); Tables 6/14: P2a(3); Table 7: P2b; Table 8: P2c 20 3. Read and Highlight Key Factors from the OP and select a few to write down; Table Anchors guide this process

• 4. Record the requirements on chart paper

20 10 5. Report Out (1 minute per table) 5

55 Minutes Total

IR Key Factors (each Examiner)

• Post your own individual KF on Scorebook

Navigator

• Team Leader will post the final consolidated

KF for the team to use on Scorebook Navigator before moving forward with IR

Step 1: Read the Criteria

Exercise: Step 1 Read the Criteria

Activity Time (min) 1. Select a scribe, timekeeper and reporter (1) 2. Discuss at your table, (group) your thoughts on the key requirements for your assigned Item from Baldrige Framework Table 1: 1.1; Table 2: 1.2; Table 3: 2.1; Table 4: 2.2; Table 5: 2.2; Table 6: 3.1; Table 7: 3.2; Table 8: 4.1; Table 9: 4.2; Table 10: 5.1; Table 11: 5.2; Table 12: 6.1; Table 13: 6.2; Table 14: 5.2

• 3. Review individually your assigned Criteria Item
• 4. Record the requirements on chart paper
• 5. Report Out (1 minute per table)

20 10 10 10

50 Minutes Total

Step 2: Determine Most Relevant Key Factors for Criteria

Exercise: Step 2 Determine the Most Relevant Key Factors

Activity Time (min) 1. Select a scribe, timekeeper and reporter (1) 2. Review individually your assigned Criteria Item 15 3. Discuss at your table as a group and come to agreement on the relevant 4 to 6 key factors (from Team Leaders list) for your assigned item. Table 1: 1.1; Table 2: 1.2; Table 3: 2.1; Table 4: 2.2; Table 5: 2.2; Table 6: 3.1; Table 7: 3.2; Table 8: 4.1; Table 9: 4.2; Table 10: 5.1; Table 11: 5.2; Table 12: 6.1; Table 13: 6.2; Table 14: 5.2 4. Write your selected key factors on a flip chart for use in subsequent exercises (capture key words and phrases) 20 10 5. Report Out (1 minute per table) 5

50 Minutes Total

Step 3: Read and Analyze the Application

Exercise: Step 3 Read and Analyze the Application

Activity Time (min) 1. Select a scribe, timekeeper and reporter (1) 2. Review individually your assigned Criteria Item 15 3. Discuss at your table as a group your observations on the applicant’s response against the Criteria questions. Table 1: 1.1; Table 2: 1.2; Table 3: 2.1; Table 4: 2.2; Table 5: 2.2; Table 6: 3.1; Table 7: 3.2; Table 8: 4.1; Table 9: 4.2; Table 10: 5.1; Table 11: 5.2; Table 12: 6.1; Table 13: 6.2; Table 14: 5.2 20 4. Report Out (1 minute per table?) 5

40 Minutes Total

Step 4: Identify Strengths/OFIs

Exercise: Step 4 Identify S’s/OFI’s

Activity Time (min) 1. Select a scribe, timekeeper and reporter (1) 2. Select potential S/OFIs individually. Then, use a round robin approach to discuss individual S/OFIs identified 20 3. From this list, discuss at your table and select as a group, a total of around 4/6 potential S/OFIs. Then select 2 S/OFIs that you feel are most important. Table 1: 1.1; Table 2: 1.2; Table 3: 2.1; Table 4: 2.2; Table 5: 2.2; Table 6: 3.1; Table 7: 3.2; Table 8: 4.1; Table 9: 4.2; Table 10: 5.1; Table 11: 5.2; Table 12: 6.1; Table 13: 6.2; Table 14: 5.2

• 4. Record these on chart paper

25 5 5. Report Out (1 minute per table) 10

60 Minutes Total

Our Key Product

What Feedback Comments Should Do:

• What the applicant does well (strengths) and

what it needs to improve upon to take it to the next level OFIs (opportunities for improvement)

• Use Criteria language
• ADLI and LeTCI
• Use OP narrative and KF
• Use VMVG
• Use Scoring language

Well-Written Comments: “NERD”

• N – Nugget – Up front comment based on the Item

Criteria or a Key Factor (i.e., briefly connects to what the applicant says is important)

• E – Evidence or Example - State what the applicant

has (Strength) or is missing or not addressed (OFI)

• R – Relevance – Why is this comment important?

Include actionable “so what” (not prescriptive) information; help the applicant to improve.

• D – Done – Read to see if comment makes sense and

helps the organization understand its current status and improvement possibility

Comment Writing

Elements of a well written comment

How to Give Useful Feedback

Do:

• Base your comments on the Criteria
• Reference the evaluation factors
• Include an opening “nugget” to

give the comment significance

• Keep each comment to single issue
• Make Key Factor references
• Be accurate; check your statements
• Recognize the page limitations
• Give benefit of the doubt
• Be polite in tone
• Include a few examples
• Check spelling and grammar

Don’t:

• Be judgmental or prescriptive
• Forget linkages
• Forget the Key Factors
• Stray from the Criteria
• Use embedded S/OFI comments
• Make conflicting strength and

OFI statements

• Forget to:

– Check your score against the balance and content of comments – Check your facts – Check grammar and spelling

• Forget you’re providing a service

Step 5: Write Feedback Ready Comments

Sample Process Strength

• 1.1b By using and improving a variety of

mechanisms (Figure 1.1-2) to communicate with the workforce and community, senior leaders deliver on a key driver of workforce

• engagement. The mechanisms share and

reinforce the organization’s vision, mission, and core values. Several cycles of learning have resulted in the expanded use of social media, the addition of a county director to support communication flow from senior leaders and throughout the county, and revisions to the website to enhance transparency.

Relevance Nugget Examples

Sample Process OFI

• 2.1b(2) The applicant’s strategic objectives

and action plans (Figure 2.1-2) do not appear to address all strategic challenges. For example, financial objectives (e.g., to decrease administrative/indirect patient costs) do not align with action plans to improve collection rates and relative value units, and no action plans align with strategic challenges related to recruiting paid staff members. The lack of alignment between strategic objectives, action plans, and strategic objectives may limit the applicant’s ability to maintain its competitive position and fulfill its mission.

Relevance Nugget Examples

Sample Results Strength

• 7.2a(1) Patient and family satisfaction

results—such as aggregate patient satisfaction, satisfaction with medical services, and satisfaction with dental services (Figures 7.2-1 through 7.2-3)—have equaled

• r exceeded the top-decile level since 2013.

These results reflect the applicant’s positive competitive position and support its mission to provide easy, timely access to high-quality, safe health care services responsive to diverse cultural and socioeconomic needs, regardless of ability to pay.

Relevance Nugget Examples

Sample Results OFI

• 7.3 Results are missing for areas related to the

applicant’s strategic challenge of staff recruitment and retention. For example, results are not provided for recruitment of health care professions and physicians; for some drivers of workforce engagement, including comfort with reporting errors or unsafe acts, protection from health and safety hazards, and a flexible work schedule; and for measures of workforce safety. Without a focus on the strategic challenge of staff recruitment and retention, patients may not be able to “become the healthiest in the state.”

Relevance Nugget Examples

Exercise: Step 5 Write Feedback Ready Comments

Activity Time (min) 1. Select a scribe, timekeeper and reporter (1) 2. Draft feedback ready S/OFI comments (split table into 2 groups; 1 feedback ready comment - S and 1 OFI) Table 1: 1.1; Table 2: 1.2; Table 3: 2.1; Table 4: 2.2; Table 5: 2.2; Table 6: 3.1; Table 7: 3.2; Table 8: 4.1; Table 9: 4.2; Table 10: 5.1; Table 11: 5.2; Table 12: 6.1; Table 13: 6.2; Table 14: 5.2 3. Give feedback to the other group and then redraft the comment, using their input 4. Record the final feedback ready comment on chart paper 25 10 10

• 5. Report Out (1 minute per table)

10

55 Minutes Total

Step 6: Determine the Scoring Range and Score

Look at Scoring Guidelines in Baldrige Framework

Pages 34 Process and 35 Results

Introduction to Scoring

• Process Scoring Guidelines and Results Scoring

Guidelines

• Use Glossary for Definitions always
• Process Scores Address: Approach, Deployment,

Learning, Integration (ADLI)

• Results Scores Address:

Levels, Trends, Comparisons, Integration (LeTCI)

8.5 6.5 8.0

Exercise: Step 6 Scoring

Activity Time (min) 1. Select a scribe, timekeeper and reporter (1) 2. Using your S/OFI comments from Step 4 and the Scoring Guidelines, discuss as a group the most appropriate scoring range for your assigned Item. Table 1: 1.1; Table 2: 1.2; Table 3: 2.1; Table 4: 2.2; Table 5: 2.2; Table 6: 3.1; Table 7: 3.2; Table 8: 4.1; Table 9: 4.2; Table 10: 5.1; Table 11: 5.2; Table 12: 6.1; Table 13: 6.2; Table 14: 5.2 15 3. Next, discuss the appropriate score within that range

• 4. Record the scoring range and score on chart paper

and be prepared to discuss why you chose that score 10 10 5. Report Out (1 minute per table) 5 40 Minutes Total

Learnings?

Follow the Six Step Process

Complete for Category/Items as assigned to your table

QTF/Baldrige Examination 6 Step Process

TRUST THE PROCESS! (KEY TEACHING POINT)

Step 1: Read the Criteria.

Read Baldrige Framework for Award Level

Step 2: Determine Most Relevant Key Factors

Four to Six Key Factors taken from OP, Eligibility, or from Application

Step 3: Read & Analyze the Application

Read the Application Mark as appropriate

Step 4: Identify Strengths/OFIs

Around 6 comments Strengths and OFIs

Step 5: Write Feedback Ready Comments

Remember: NERD!

N – nugget of

importance

E – evidence/example

to support comments

R – relevance to the

applicant

D – Done!

Step 6: Determine the Scoring Range and Score

Best Fit Don’t Block a Winner Tie goes to applicant

What did we learn now?

QTF/Baldrige Examination 6 Step Process

TRUST THE PROCESS! (KEY TEACHING POINT)

Step 1: Read the Criteria.

Read Baldrige Framework for Award Level

Exercise: Step 1 Read the Criteria

Activity Time (min) 1. Select a scribe, timekeeper and reporter (1) 2. Review individually your assigned Criteria Item.

• 3. Discuss at your table, (group) your thoughts on the

key requirements for your assigned Item from Baldrige Framework Table 1/6/14: 7.1; Table 2/7: 7.2; Table 3/8: 7.3; Table 4/9: 7.4; Table 5/10: 7.5; Table 6/11: 7.3; Table 7/12: 7.4; Table 8/13: 7.5

• 4. Record the requirements on chart paper
• 5. Report Out (1 minute per table)

20 20 5 5

50 Minutes Total

Step 2: Determine Most Relevant Key Factors

Four to Six Key Factors taken from OP, Eligibility, or from Application

Exercise: Step 2 Determine the Most Relevant Key Factors

Activity Time (min) 1. Select a scribe, timekeeper and reporter (1) 2. Review individually your assigned Criteria Item 15

• 3. Discuss at your table as a group and come to agreement
• n the relevant 4 to 6 key factors for your assigned item.

Table 1/6/14: 7.1; Table 2/7: 7.2; Table 3/8: 7.3; Table 4/9: 7.4; Table 5/10: 7.5; Table 6/11: 7.3; Table 7/12: 7.4; Table 8/13: 7.5 4. Write your selected key factors on a flip chart for use in subsequent exercises (capture key words and phrases) 20 10 5. Report Out (1 minute per table) 5

50 Minutes Total

Step 3: Read & Analyze the Application

Read the Application Mark as appropriate

Exercise: Step 3 Read and Analyze the Application

Activity Time (min) 1. Select a scribe, timekeeper and reporter (1) 2. Review individually your assigned Criteria Item 15 3. Discuss at your table as a group your observations on the applicant’s response against the Criteria questions. Table 1/6/14: 7.1; Table 2/7: 7.2; Table 3/8: 7.3; Table 4/9: 7.4; Table 5/10: 7.5; Table 6/11: 7.3; Table 7/12: 7.4; Table 8/13: 7.5 20 4. Report Out (1 minute per table?) 5

40 Minutes Total

What did we learn now?

Step 4: Identify Strengths/OFIs

Around 6 comments Strengths and OFIs

Exercise: Step 4 Identify Strengths/OFIs

Activity Time (min) 1. Select a scribe, timekeeper and reporter (1) 2. Select potential S/OFIs individually. Then, use a round robin approach to discuss individual S/OFIs identified 15 3. From this list, discuss at your table and select as a group, a total of around 4/6 potential S/OFIs. Then select 2 S/OFIs that you feel are most important. Table 1/6/14: 7.1; Table 2/7: 7.2; Table 3/8: 7.3; Table 4/9: 7.4; Table 5/10: 7.5; Table 6/11: 7.3; Table 7/12: 7.4; Table 8/13: 7.5

• 4. Record these on chart paper

20 5 5. Report Out (1 minute per table) 10

50 Minutes Total

Step 5: Write Feedback Ready Comments

Remember: NERD!

N – nugget of

importance

E – evidence/example

to support comments

R – relevance to the

applicant

D – Done!

Exercise: Step 5 Write Feedback Ready Comments

Activity Time (min) 1. Select a scribe, timekeeper and reporter (1) 2. Draft feedback ready S/OFI comments (split table into 2 groups; 1 S and 1 OFI) Table 1/6/14: 7.1; Table 2/7: 7.2; Table 3/8: 7.3; Table 4/9: 7.4; Table 5/10: 7.5; Table 6/11: 7.3; Table 7/12: 7.4; Table 8/13: 7.5 3. Give feedback to the other group and then redraft the comment, using their input 4. Record the final feedback ready comment on chart paper 15 10 10

• 5. Report Out (1 minute per table)

10

45 Minutes Total

Step 6: Determine the Scoring Range and Score

Best Fit Don’t Block a Winner Tie goes to applicant

Exercise: Step 6 Scoring

Activity Time (min) 1. Select a scribe, timekeeper and reporter (1) 2. Using your S/OFI comments from Step 4 and the Scoring Guidelines, discuss as a group the most appropriate scoring range for your assigned Item. Table 1/6/14: 7.1; Table 2/7: 7.2; Table 3/8: 7.3; Table 4/9: 7.4; Table 5/10: 7.5; Table 6/11: 7.3; Table 7/12: 7.4; Table 8/13: 7.5 15 3. Next, discuss the appropriate score within that range 4. Record the scoring range and score on chart paper and be prepared to discuss why you chose that score 15 10 5. Report Out (1 minute per table) 5

45 Minutes Total

Learnings?

Scorebook Navigator

Links to the IR and consensus manuals for Scorebook Navigator are on the Quality Texas Scorebook Navigator Page

Quality Texas Foundation

Scorebook Navigator Examiner Training

Scorebook Sign On

Scorebook Sign On

• Logon

TX (Capital First Name, First Letter, Capital Last Name First Letter then the Remainder of Last Name) Example – TXKSimpson (unless you have a very common name and then your full name is spelled out)

• Password

Initial – Same as Logon +1 (TXKSimpson1) Change Password – Go to My Account (Far Right on Navigation Bar)

Changing Your Password

129

Scorebook Screen

Scorebook Screen

Stage 1 Scorebook

Scorebook Navigation Bar

 Sign Out  Scorebooks  Key Factors  Item Evaluation  Key Themes  Reports  My Account

Icons

Spell Checker Find and Replace Text Print Preview Display the Criteria Display the Application Record Action Items Team Members Save Expanded Text Entry Area Delete Current Row Move Current Row Up Move Current row Down Key Factors

Key Factors

Item Evaluation

This is the top portion of the evaluation section

• n the Item

Evaluation

• Screen. It

continues as you scroll down the page.

Key Factor Reference

After you click the Save icon, the key factor Organizational Profile Area to Address number that corresponds with your Key Factor selection appears here. Pick the appropriate key factor from the drop-down menu that appears when you click on a field in this column.

Strengths & OFI’s

Item Scoring Panel

Stage 2 Scorebook

Let’s Logon

Last Things!

• Save Often
• Write Feedback Ready Comments
• Remember ++ & --
• Remember
• Hover Help
• Ask for HELP, don’t wait and stress over

things!

Consensus Review

Improve scoring and feedback by combining Independent Reviews into team-level Consensus Review

• Team Leader will move

team to consensus in SN

• 10/12 IR scorebooks will

move into 1 CR scorebook

• Best thinking of entire

team revised and refined

What Does “Consensus” Mean?

Consensus is:

– A rationally-derived decision – Based on the contributions

• f ALL team

members – Supported by ALL team members

Composition of Consensus Teams

• Engagement – 8/10 Examiners,

(1 TL, 2 BUTLs, 1 FB Editor, 4/6 Examiners)

• Commitment – 10/12 Examiners

(1 TL, 2 BUTLs, 1 FB Editor, 6/8 Examiners)

• Progress – 10/12 Examiners (1

TL, 2 BUTLs, 1 FB Editor, 6/8 Examiners)

Composition of Consensus Team

(for Site Visit Award Level)

• 10-12 Examiners – (only 8 go to SV)

– 1 TL, – 3 BUTLs, – 2 FB Editors, – 4-6 Examiners)

• 4 Alternate Examiners go through IR & CR
• 1 Process Coach

Consensus Goals

• Overall: a feedback report detailing areas of

S/OFIs that assists your applicant to move to the next level of performance excellence.

• During Consensus Review: create a final

scorebook based on the Criteria and Organization’s Application.

• Special Note: Never do we use the

negative terms like “ding them”, “got them” in our approach on this. NEVER!

Prepare for Consensus

This stage begins with some introductory work:

• Calendar checks for calls and

timelines

• Item preferences assigned by

team leader

• The consensus planning call
• Review the Step-By-Step

Consensus Review manual on the QTF website

The Consensus Planning Call

• Review information and agenda
• Call in on time
• Adhere to ground rules
• Agree on Key Factors, Category Assignments, and

Calendar Schedule

• Clarify any questions

SCOREBOOK NAVIGATOR

Consensus Role: Category Lead or Back-up

Category Lead

• Prior to consensus call

– Complete synthesis of IR Comments – Share consensus comments with back-up/receive input – Revise comments based on back- up input; post your items – Prepare for consensus call by developing category script

• During consensus call

– Lead your item discussions (from your script)

• After consensus call

– Revise consensus comments – Post final comments

Back-up Category Lead

• Prior to consensus call:

– Review consensus comments provided by Category Lead – Provide input on comments to Category Lead

• During consensus call:

– Take notes for the Category Lead to capture comments and recommended changes – Help the Category Lead stay on time

• After consensus call:

– Share notes with Category Lead on suggested changes

• Read ALL team Strengths/OFIs (not just the two

feedback-ready comments) for each Item assigned to you

• Focus on Strengths/OFIs that best reflect the applicant’s

maturity

• Write about 6 draft consensus comments
• Note all applicable examiners’ initials
• Note why any key comments are excluded,

including outliers

• Note possible Key Themes
• Review with back-up, revise and repost
• Complete the Consensus Category Lead Script in preparation

for the consensus call

As Category Lead, You Will…

• Fulfill your responsibilities as a Category lead and a

Category Back-Up

• Everyone will serve in both of these roles
• Read and PONDER all comments from every Category
• Indicate your agreement or specific concern beside

each comment in the Notes section in Scorebook Navigator

• Make suggestions for revisions to Item drafts in

Scorebook Navigator

As a Team Member, You Will…

Conduct Consensus Meeting

• Consensus meeting held through a conference call
• All Team Members present – no starting until ALL present
• Category Lead presents comments and leads discussion
• Category Backup documents discussion when Strengths or

OFIs are changed, edited or decisions change

• Team reaches consensus on Key Factors, Strengths/OFIs

and scores for each Item and Key Themes

• Team focuses on clarity and value of Strengths/OFIs

Reaching Consensus

• Is finding a proposal acceptable enough such that
• All members have had an opportunity to

be heard

• Every member understands the decision
• Every member can actively

support the team decision

• No member opposes it
• Thumbs up or down
• Does not necessarily

imply 100% agreement

Consensus Review Scorebook

This CR Scorebook is comprised

• f comments and scores that

represents the collective knowledge and analytical skills of a team of examiners by using a systematic review process. This CR Scorebook Serves as the basis for the feedback report.

Consensus Follow-Up

Non-Site-Visit Applications

• Category Leads revise comments in Scorebook Navigator for

assigned items based on consensus discussion and

• Scorebook Editor (s) will take over clean-up using the

Feedback Report Template Site Visit Applications

• Category Leads revise comments in Scorebook Navigator for

assigned items based on consensus discussion and

• Develop Site Visit Issues and Strategies

Consensus: Success Factors

• Consensus is a “team sport” - Play nice.
• Do what you say you will do
• Be punctual for all team

commitments

• Know time constraints & keep

to them

• Prepare! Prepare! Prepare!
• Participate actively
• Use the established processes

Consensus Process Deliverables

Site Visit, A, P

• Comments

– Clear and complete – Criteria-centered – No conflicting comments

• Site Visit Worksheets
• Scores:

– Reflect applicant maturity

Non-Site Visit, E, C, P

• Feedback Comments

– Clear and complete – Criteria-centered – Contain actionable feedback – No conflicting comments

• Scores:

– Reflect applicant maturity

Consensus Call: Key Themes

• Team Leader facilitates discussion of Key Themes based
• n all Item comments and discussion notes; linkage

throughout application i.e. strategic planning, action plans, workforce issues, etc.

• Team Members arrive at agreement on Key Themes

based on application review and consensus

• Team Leader or Scorebook Editor finalizes Key Themes in

Scorebook Navigator and includes as part of team consensus report

Exercise: Consensus Process

Activity Time (min) 1. Select a scribe, timekeeper and reporter (1) 2. Walk through the consensus process at your table, using insight from the returning examiners. 20 3. Document the top three learnings about the consensus process from your table discussion

• n your flip chart.

5. Report Out (2-3 tables) 5

25 Minutes Total

Key Themes

Applicant Executive Overview

What’s a Key Theme?

• Serve as the “executive summary” of the feedback report
• Main focal point for the Panel of Judges
• Relate to one or more of the applicant’s Key Factors (issues

identified in the Organizational Profile and elsewhere in the application)

• Address requirements in multiple Categories/Items (“cross-

cutting”)

• Incorporate evaluation factor language
• Reflect the criteria Core Values

Key Themes Must…

• Originate and be traceable to multiple Item-level Strengths
• r OFIs; incorporate evaluation factor language; reflect the

Criteria core values

• Maintain consistency with the applicant’s overall score
• A more mature, high-scoring applicant

will have more key theme strengths than OFIs and vice versa.

• Be identified as the examiners proceed through the

assessment in Scorebook Navigator; will sometimes be easier to see after completion of all 7 Categories

Key Themes Respond to These Questions:

a) What are the most important strengths or outstanding practices (of potential value to other organizations) identified? b) What are the most significant opportunities, concerns,

• r vulnerabilities identified?

c) Considering the applicant’s key factors, what are the most significant strengths (data, comparisons, and linkages) found in its results? d) Considering the applicant’s key factors, what are the most significant opportunities, vulnerabilities, and/or gaps (data, comparisons, and linkages) found in its results?

A Key Theme Process Strength

Senior leaders’ focus on the applicant’s vision, mission, and values (VMV) promotes its core health care business, contributing to organizational, financial, and societal performance. The VMV create the applicant’s culture and permeate strategic planning and daily operations. Identification of key communities and their needs embeds societal responsibilities into the applicant’s strategies, strategy implementation, action plans, and daily

• perations. Examples include the Care Connection Kiosks (CCKs),

the option of English or Spanish messaging for individuals without computer access, the provision of transportation and child care, and health education outreach. To meet the vision of a healthy population, the applicant organizes its workforce into Primary Care Teams, creates personalized health plans, and sets goals for

• care. Efforts to engage the community include recruiting

volunteers and members from the communities the applicant

• serves. These activities align with the applicant’s core competencies of

patient-centered care and expertise in treating diseases prevalent in its population.

A Key Theme Process OFI

There are gaps in the applicant’s approaches to providing the range of treatment services aligned to its mission and to its core competency of expertise in treating diseases prevalent in its patient population (e.g., mental health issues, alcohol and substance abuse, obesity, diabetes, and heart disease). For example, alignment between the processes and requirements in Figure 6.1-1 and health care offerings that meet identified community needs is not evident, and some FOCUS measures do not clearly align with

• bjectives (e.g., how immunization rates and screenings

will address major health challenges. Leveraging the applicant’s core competency of expert treatment, as well as its core competency of collaborative relationships, to align services provided and measures tracked with its stated objectives may help the applicant move toward its vision of a healthier population.

A Key Theme Results Strength

AF’s results for screening and access to care, as well as customer-focused and some financial results, contribute to fulfilling the mission of providing residents easy and timely access to high-quality and safe health care services. In particular, the applicant reports good levels, beneficial trends, and favorable comparisons for screening

• utcomes and measures for access to care, with some

results exceeding the state average for community health centers (CHCs). In addition, aggregate patient satisfaction and satisfaction with medical and dental services meet or exceed the top-decile comparisons. Collectively, these results underscore the applicant’s core competencies of patient-centered care and expertise in treating diseases prevalent in its population.

A Key Theme Results OFI

The applicant does not provide some important business and financial results. Examples are missing or limited results for operating margin, fundraising revenues, cost control, and ACA impact; results for action plan outcomes; and results for the success of patient acquisition and retention mechanisms. In addition, the applicant does not report comparisons to local or regional competitors for many patient and other customer satisfaction results

• r for workforce results; nor are results provided related

to the applicant’s strategic challenge of staff

• recruitment. Tracking such business results may contribute to

ensuring financial and organizational sustainability in a rapidly changing health care environment.

Site Visits

Be prepared, be prepared, be prepared!

Site Visit and Preparation

• Attend Required Training for Site Visit with Same

Team of Examiners

• Complete Team Assignments and Attend Planning

Calls

• Arrive Day Before Site Visit
• Verify and Clarify Application and Site Visit

Issues

• Must Be On-Site Until

Site Visit Scorebook is Complete (last step of process)

Actual Site Visit

• Sunday arrival by noon (all site visit issues

discussed)

• Monday – Wednesday noon – on site
• Wednesday noon til Friday night – finish all

site visit issue resolution

• Go to bed at 10 (good sleep provides best

feedback report)

• Applicant pays for meals, travel, and hotel;

examiner reimbursed

Next Steps

• Receipt of Applications (Oct-Mar) and

Assignment of Examiner Teams

• Cycles 1-5 (detailed schedule on website)
• Site Visits (January 2017)
• Judges’ Meeting; Award Level Recipient

Recommendations

• Board of Directors’ Recipient Notification
• Feedback to Applicants
• 2017 Texas Quest for Excellence Conference

Westin Memorial City, Houston, Texas; Examiner Appreciation Event

After-Action Review

• What did you learn?
• Do you have confidence to

begin/continue the assessment stages?

• Any final questions or concerns?

Last Things…Almost!

• Be sure to complete your training surveys and

forms!

• Please contact Quality Texas if you need help.
• Let’s thank our Training Faculty!
• Please take a few moments to clean up your areas
• Have a safe trip home!
• Most of all THANK YOU FOR YOUR PATRIOTIC

WORK!

Let’s Graduate CONGRATULATIONS!

176

We appreciate YOU!

Welcome to the Quality Texas Family or Thanks for Coming Back