Presented by: Ken Phillips Phillips Associates February 21, 2018 - - PowerPoint PPT Presentation

▶
presented by ken phillips phillips associates february 21
SMART_READER_LITE
LIVE PREVIEW

Presented by: Ken Phillips Phillips Associates February 21, 2018 - - PowerPoint PPT Presentation

Jun 29, 2023 236 likes •841 views

Presented by: Ken Phillips Phillips Associates February 21, 2018 Phillips Associates 1 Agenda 1. Discover meaning of term scrap learning & its impact on wasted organization resources & lost credibility with stakeholders 2.

slide-1
SLIDE 1

Phillips Associates

1

Presented by: Ken Phillips Phillips Associates February 21, 2018

slide-2
SLIDE 2

Agenda

  • 1. Discover meaning of term “scrap learning” & its

impact on wasted organization resources & lost credibility with stakeholders

  • 2. Analyze how to build an algorithm that predicts which

learners are most & least likely to apply what they learned in a training program back on the job & which managers of the learners are likely to do a good and poor job of supporting the training

  • 3. Examine the 3-phase, 9-step Predictive Learning

Analytics methodology using data from an actual implementation

slide-3
SLIDE 3

Phillips Associates

3

Scrap learning: What is it?

slide-4
SLIDE 4

Phillips Associates

4

Term that describes the gap or difference between learning that is delivered and learning that is applied back on the job

Scrap Learning

Coined by KnowledgeAdvisors, a CEB company (now Gartner)

slide-5
SLIDE 5

Phillips Associates

5
slide-6
SLIDE 6

Phillips Associates

6

How big is the problem?

slide-7
SLIDE 7

Phillips Associates

7

Poll

In the average organization, what percent of learning that is delivered ends up as scrap?

  • A. 25%
  • B. 45%
  • C. 65%
  • D. 85%
slide-8
SLIDE 8

Phillips Associates

8

Benchmark Study 1

45%

Source: Confronting Scrap Learning CEB White Paper, 2014

slide-9
SLIDE 9

Phillips Associates

9

Benchmark Study 2

< 20%

Applied new skills back on the job

> 15%

Didn’t try to apply new skills back

  • n the job

65%

Tried applying new skills back

  • n the job, but reverted back

Source: Robert Brinkerhoff, 2004

slide-10
SLIDE 10

Phillips Associates

10

View from individual

  • rganization level
slide-11
SLIDE 11

Phillips Associates

11

According to ATD 2017 “State of the Industry Report”

Average per employee training expenditure Average number of training hours consumed per employee

$1273 34.1 = =

slide-12
SLIDE 12

Phillips Associates

12

Calculating Scrap Learning at individual

  • rganization level

$1273 X 45% = 34.1 hours X 45% = $1273 X 80% = 34.1 hours X 80% =

$573 15 $1018 27

slide-13
SLIDE 13

Phillips Associates

13

View from individual program level

(see formula on page 5 in handout)

slide-14
SLIDE 14

Phillips Associates

14

Houston, we have a problem!

Source: James Lovell, Apollo 13 flight

slide-15
SLIDE 15

Phillips Associates

15

The solution:

Predictive Learning Analytics™

slide-16
SLIDE 16

Phillips Associates

16

Definition

Methodology for peering into the future, at the conclusion of a learning program, and predicting learner outcomes and actions, with the intent of changing those outcomes and actions for the better

Predictive Learning Analytics:

slide-17
SLIDE 17

Phillips Associates

17

PLA Mission

To provide L&D professionals with a systematic, credible and repeatable process for measuring and managing scrap learning using data driven decision making

slide-18
SLIDE 18

Phillips Associates

18

PLA vs. Traditional Learning M&E

Predictive Learning Analytics Traditional Learning M&E  Focuses on individual learners  Predicts future likelihood of certain behaviors and actions  Focuses on programs or cohorts  Describes what has happened

slide-19
SLIDE 19

The PLA Methodology

Peering into the future & predicting learner

  • utcomes &

actions Changing those

  • utcomes &

actions for the better Reporting your results

slide-20
SLIDE 20

Phillips Associates

20

Phase 1: Step 1

slide-21
SLIDE 21

Phillips Associates

21

Select a Learning Program

Planned learning initiative not informal learning event Has a high profile Large number of participants are scheduled to attend (40-60 for initial Calibration Cohort)

Three Guidelines:

1. 2. 3.

slide-22
SLIDE 22

Phillips Associates

22

Phase 1: Step 2

slide-23
SLIDE 23
slide-24
SLIDE 24

Phillips Associates

24

All 12 factors are aligned with what research has found to be the 3 components of training transfer

Research sources: Baldwin & Ford 1988; Colquitt et. al. 2000; Scaduto et. al. 2008

slide-25
SLIDE 25

3 Training Transfer Components

Learning Program Design Learner Attributes Learner Work Environment

Success! Learning is applied

  • n the job

Direct Control Influence Influence

slide-26
SLIDE 26

Instructions

  • 1. Form a group of 3, 4 or 5 persons
  • 2. Keeping in mind the 3 training transfer

components, brainstorm a list of factors known to contribute to training transfer (page 1 in your handout)

  • 3. Be prepared to share your ideas with the

whole group

Example : Training transfer increases when learners have an immediate opportunity to apply what they learned in a program back on the job (Work environment)

slide-27
SLIDE 27

Phillips Associates

27

3 Training Transfer Components

Learning Program Design Learner Attributes Learner Work Environment

Success! Learning is applied

  • n the job
slide-28
SLIDE 28

Phillips Associates

28

Program Design Factors

  • 1. New information is acquired
  • 2. Program viewed as relevant to learner & his/her job
  • 3. Program viewed as important investment in one’s career

development

  • 4. Learner sees likely improvement in key department

business metric if new information learned is applied

  • 5. Learner is likely to recommend program to work

colleagues

Continued

Continued

slide-29
SLIDE 29

Phillips Associates

29

Learner Attribute Factors

  • 6. Learner is personally motivated to apply what was

learned

  • 7. Learner is confident in his/her ability to apply what was

learned

  • 8. Learner takes time to reflect on key lessons learned &

how they can help improve performance

  • 9. Learner views program as an opportunity to learn

challenging new things

Continued

Continued

slide-30
SLIDE 30

Phillips Associates

30

Work Environment Factors

  • 10. Managers actively engage learners, post-program,

regarding what was learned

  • 11. Work colleagues support learners, post-program, when

applying new things learned

  • 12. Learners have immediate opportunity to apply what

was learned

slide-31
SLIDE 31

Phillips Associates

31

Create a Survey

Convert 12 factors into survey items that reflect content of target program Incorporate survey items into an existing Level 1 evaluation or administer as a separate survey

First Then

slide-32
SLIDE 32

Phillips Associates

32

Sample Survey Items

How relevant is the (insert program name) program to you and your job? How confident are you in your ability to apply the knowledge, skills and behaviors you learned in the (insert program name) program back-on-the-job?

Not at all Relevant Extremely Relevant

1 2 3 4 6 7 5

Not at all Confident Extremely Confident

1 2 3 4 6 7 5

slide-33
SLIDE 33

Phillips Associates

33

Phase 1: Step 3

slide-34
SLIDE 34

Phillips Associates

34

Case Study

Company: Learning program: Business objective: Medical insurance company Improve operational efficiency Calibration cohort: 74 participants Continuous Process Improvement

slide-35
SLIDE 35

Phillips Associates

35

LAI Scores

Least Likely to Apply At Risk of Not Applying Most Likely to Apply

slide-36
SLIDE 36

Phillips Associates

36

MTSI Scores

Average score on factor measuring how likely manager is to be actively engaged with learner post-program regarding what was learned & is an indication of manager support for training Average LAI score for all employees reporting to same manager & is an indication of training transfer potential MTSI = difference between

  • Mgr. Average &

LAI Average

slide-37
SLIDE 37

Training Transfer Component Scores

Statistically significant differences Lowest factor score No statistically significant difference

slide-38
SLIDE 38

Phillips Associates

38

Phase 1: Step 4

slide-39
SLIDE 39

Phillips Associates

39

30 days post-program collect data from random sample of Calibration Cohort participants using either a survey or focus groups and ask 3 questions:

  • 1. % of program material applied back on job?
  • 2. Confidence level of estimate?
  • 3. Obstacles inhibiting application back on job?

Calculate Scrap Learning & ID Obstacles

slide-40
SLIDE 40

Scrap Learning Calculation

slide-41
SLIDE 41

Phillips Associates

41

Obstacles to Training Transfer

Management Policies and Procedures Communication Personal Lack of time or resources Technology Teamwork Change

(11) (10) (9) (7) (6) (4) (4) (3)

slide-42
SLIDE 42

Phillips Associates

42

Phase 2: Step 5

slide-43
SLIDE 43

Step 5: Where the Rubber Meets the Road

  • 1. Identify solutions to mitigate or eliminate obstacles

to training transfer

  • 2. Target at risk & least likely to apply learners for

reinforcement activities

  • 3. Target managers with low or negative MTSI scores

for help & support

  • 4. Manage scrap learning baseline percentage

Use data driven decision making to:

slide-44
SLIDE 44

Phillips Associates

44

Phase 2: Step 6

slide-45
SLIDE 45

Phillips Associates

45

Prediction without validation is nothing more than educated guessing at best and malfeasance at worst.

Source: Ken Phillips

slide-46
SLIDE 46

Phillips Associates

46

Phase 2: Step 7

slide-47
SLIDE 47

Phillips Associates

47

Using a new group of learners, collect data from random sample of participants using either a survey or focus groups and ask same 3 questions:

  • 1. % of program material applied back on job?
  • 2. Confidence level of estimate?
  • 3. Obstacles inhibiting application back on job?

Recalculate Scrap Learning

slide-48
SLIDE 48

Phillips Associates

48

Phase 3: Step 8

slide-49
SLIDE 49

Phase 3 Step 9

slide-50
SLIDE 50

Phillips Associates

50

Summary

The issue of scrap learning has been around forever. But, what’s different today is that with Predictive Learning Analytics™ there now is a way to measure and manage it.

Source: Ken Phillips

slide-51
SLIDE 51

Phillips Associates

51
slide-52
SLIDE 52

Phillips Associates

52

Learn more about Predictive Learning Analytics

Request our FREE ebook:

The L&D Revolution: New Rules. New Tools.

slide-53
SLIDE 53

Phillips Associates

53

 “Cut the Scrap with Predictive Learning Analytics”  “Predictive Learning Analytics: New Tools, New Rules for Increasing Training Transfer”  “Predictive Learning Analytics: A Revolutionary New Way to Evaluate Learning”  “Training Transfer: It’s an Age-Old Problem With an All- New Solution”  “Training Transfer: Managers Make the Difference”  “Identifying Obstacles to Training Transfer”  “Boost Training Transfer Using Data Driven Decision Making and Predictive Learning Analytics”

Free PLA Articles

slide-54
SLIDE 54

Phillips Associates

54

Phillips Associates ken@phillipsassociates.com (847) 231-6068 www.phillipsassociates.com 34137 N. Wooded Glen Drive Grayslake, Illinois 60030

Ken Phillips

slide-55
SLIDE 55

Phillips Associates

55

1.

Less money & time wasted on learning that is delivered but not applied back on the job

Benefits of Using PLA

2.

Increased personal credibility in eyes of business executive stakeholders

slide-56
SLIDE 56

Phillips Associates

56

Benefits of Using PLA

4.

Objective way to identify managers who do a poor job of supporting learning so that their approach can be improved

3.

More effective & efficient use of follow-up activities by targeting participants who are at risk & least likely to apply what they learned in a program back on the job

slide-57
SLIDE 57

Phillips Associates

57

Benefits of Using PLA

6.

Enhanced reputation among L&D colleagues Objective way to assess the contribution to training transfer made by each of the 3 training transfer components

5.

slide-58
SLIDE 58

Phillips Associates

58

LAI Scores

slide-59
SLIDE 59

Phillips Associates

59

Reg egis ister er by Feb.

  • b. 28

28 and d sav ave e

$1 $100 00

Cut the Scrap in Your Organization!

  • A New, Highly Interactive, Two-day Workshop
  • March 8-9, 2017 in Chicago
  • REGISTER ONLINE AT:

http://bit.ly/BoostTrainingTransfer