BMW Manufacturing Corp. Return on Investment Impact Study Gallery - - PowerPoint PPT Presentation

BMW Manufacturing Corp. Return on Investment Impact Study

Gallery Tour Review the posters with questions that will be addressed at each evaluation level Complete a Post-It note with additional questions you want addressed today

Founders of GAINS Profile

• Dr. Ivan Misner & Robert Davis

Purpose: To strengthen both new and established relationships Five Elements of the GAINS Profile

• 1. Goals
• 2. Accomplishments (Assets)
• 3. Interests (Passions)
• 4. Networks (IRS)
• 5. Skills

Results

Let’s Network using the GAINS Profile

Presenter: Ylonda Glover, MS Tri-Success Management Institute

GAINS Profile Focus on I for Interests

People are the Drivers Understanding People - Improves ROI What are your personal and professional interests? What are you most passionate about or what do you like doing?

Toni Hodges

TH & Company

• Independent consultant, providing consulting services

and workshops to private and public organizations

• Formerly managed Measurement and Evaluation for

Verizon Communications and Bell Atlantic

• Bell Atlantic selected APQC Best Practice for “Measuring

the Impact of Learning”

• Selected “Training’s New Guard”
• ROI Network Practitioner of the Year
• Author “Linking Learning and Performance”

Michelle M. Wentz

Assistant Manager BMW Manufacturing

• Assistant Manager of Training and Associate Development at BMW

Manufacturing 5000-associate company manufacturing the Z4 Roadster and X5 Sport Activity Vehicle Main responsibilities: Learning Technologies, e-Learning, Computer Applications Training, Training Measurement

• 12 years of broad-based training and development experience

working in the manufacturing and financial services industries

• Master of Science degree in Adult Education from the University of

Wisconsin

• Bachelor of Business Administration degree in Management, also

from the University of Wisconsin

Overview

• 1. ROI Methodology
• 2. Program Background
• 3. BMW Process
• 4. Results
• 5. Conclusions
• 6. Recommendations
• 7. Lessons Learned

Collect Data After Solution Implementation Collect Data During Solution Implementation Develop Evaluation Plans and Baseline Data Develop Objectives of Solution DATA COLLECTION EVALUATION PLANNING

• 1. Satisfaction /

Reaction

• 3. Application/

Implementation

• 2. Learning
• 4. Business Impact

ROI Process™ Methodology

Provides 6 types of data

Isolate the Effects of the Solution Convert Data to Monetary Values Calculate the Return on Investment Tabulate Costs Identify Intangible Measures

Generate Impact Study

REPORTING DATA ANALYSIS

• 5. ROI
• 6. Intangible Benefits

Provides 6 types of data

ROI Process™ Methodology

Benefits / Project Benefits Costs = Ratio Project Costs Net Project Benefits ROI = Project Costs

ROI Process™ Calculating the BCR and ROI

Leadership Mastery Program Background

• To continue leaders’ professional development after the

BMW Life Program

• Emphasis on business competencies, communication,

DISC style adapting and conflict resolution skills

• 3 classroom days followed by 8 hours of individualized

coaching

• Target Associates: Managers, Supervisors, and Individual

Contributors with Project Responsibilities

1. Two focus groups convened to determine potential business

• bjectives (21 LMP participants)

2. Reaction data collected from BMW evaluation form (158 LMP participants) 3. Job Application data collected with 360 Degree Assessment (pre and post data collected from 16 out of 34 associates) 4. Business Impact data collected from Leadership Mastery Program Review survey (128 LMP participants responded) 5. Tangible Benefits collected from Leadership Mastery ROI Survey and actual program costs (24 out of 36 LMP participants responded) 6. All data compiled and analyzed by Return On Training (ROT) Analysis Team (3) 7. Logic Map Task Force convened for input into Conclusions/ Recommendations (2 coaches, 4 associates)

Methodology for BMW Process Steps

• Reaction data high –ratings between 3.8 to 5.0
• Participants saw improvement in 13 of 18 competencies,

4 significantly so: 1. Communicates with Impact 2. Fresh Thinking 3. Positive Disposition 4. Decision Making

• Training and Coaching perceived to have a positive impact on

meeting business and performance objectives

• Those that received more coaching perceived higher impact to

business/performance objectives (Impact increased more than threefold for those who had more than 5 coaching sessions)

Results Reaction and Job Application

• 1. Select the business benefits the

Leadership Mastery program most positively impacted.

• 2. What is the monetary value of

each unit of improvement?

• 3. What was the change in this unit
• ver a 30-day period?
• 4. To what extent do you credit the

Leadership Mastery program for this change?

• 5. How confident are you of this

estimate? Question

• 1. Fewer hours spent in meetings
• 2. $30 per hour
• 3. 4 hours per month
• 4. 70% percent of change

accredited to Leadership Mastery program

• 5. 80% confident

Example

Results Calculating Tangible Benefits

Results Tangible Business Benefits

• 14 out of 24 respondents were able to estimate tangible benefits

(used in calculation)

• “Reduced number of issues reported” received highest estimated

tangible benefit of $50,284

• “Fewer hours spent in meetings” was the benefit realized most
• ften (2nd highest tangible benefit)

Participant example from Assembly Quality Department Reduced number of issues reported (Quality Improved) 10 issues reduced per month at 3 hours each @$30/hr. Isolation @ 40% Confidence level @ 60% Annual monetary benefit = $259

Results BCR and ROI Calculation

Benefits (Perceived Annual Monetary Value from 24 Participants -

Adjusted for Isolation and Confidence)

Costs

(Actual Costs for 24 Participants)

Reduced number of issues reported $50,284 Reduced overtime $46,444 Fewer hours spent in meetings $27,446 Reduced time to full productivity $14,275 Reduced turnover $13,970 Fewer associate complaints $9,714 Improved customer satisfaction $3,200 Reduced minutes in rework $300 Improved vehicle quality measures $175 Benefit Attributable to Training: $124,008 Design and Development: ($190.80 X 24 associates ) $4,579 Leadership Mastery Delivery Costs: Facilities + Instructor + Materials ($1924 X 24 associates) $46,176 Associate ParticipationCosts: $26,910 $57.48/hr (FL IV) X 24 hrs X 4 associates + $53.01/hr (FL V) X 24 hrs X 9 associates + $34.02/hr (FL VI) X 24 hrs X 11 associates ) (fullyloaded hourly rates) Report: $150/report X 24 associates $3,600 Coaching: $250/hour X 75 coaching hours $18,750 $45.02/hour X 75 coaching hours (Participant time in coaching sessions ) $3,377 Total Training Costs: $103,392 ROI Calculation: $124,008 (benefit attributable to training) - $103,392 (training costs )= .199 x 100 =19.9% $103,392 (training costs) Benefit/Cost Ratio Calculation: $124,008 (benefit attributed to training) = 1.20 = 1.20:1 $103,392 (training costs )

• Improved teamwork – 83%
• Reduced conflict – 74%
• Improved clarity of roles and responsibilities – 61%

–Improved associate job satisfaction – 39% –Improved associate development plans – 35% –Increased cross functionality – 30% –Others noted: improved customer satisfaction,

• rganizing skills, and communication

Results Intangible Benefits

• 1. Participants believed they would be able to apply the skills

they learned on the job

• 2. Leadership competencies improved as a result of the

program, according to participants: Decision-making, Communicates with Impact, Positive Disposition, and Fresh Thinking.

• 3. Training and coaching had a positive impact on participants’

performance

• 4. Training and coaching may be improved with more realistic

and relevant work-related focus

• 5. A better understanding by participants’ management of LMP

skills acquired and value expected would be beneficial

• 6. The LMP is leading BMW employees toward leadership

with a financial return

Conclusions

Recommendations Next Steps

1. Continue Leadership Mastery program 2. Share ROI project results with key audiences 3. Prioritize and implement LMP recommendations 4. Implement the following measurement best practices:

• Change our training design process to include the

UPFRONT development of business objectives for future

• courses. Seek input from Senior Management so that

training programs can be strategically linked to BMW’s business/ organizational objectives.

• Focus measurement efforts on learning and on-the-job

application / performance impact

• Measure ROI and Business Impact for a few select

training programs

Lessons Learned Putting the Pieces Together…

• Training Design process must include link to business

need, initiative, or improvement area Learning objectives are not enough Objective mapping process is key Provides strategic link to business Ensures concrete measures

• Prepare participants for accountability

Especially when collecting data using participant estimates Train instructors, designers, and learners in documenting, and tracking tangible benefits Action plans, business objectives, course design play a key role

Lessons Learned Putting the Pieces Together…

• Partner with Senior Management and Financial

departments Communicate ROI process (for training) Get financial requirements Look for ways to customize to

• rganization in order to establish

an ‘approved’ methodology

Lessons Learned Putting the Pieces Together…

• Keep participation high

Utilize ‘field’ perspective whenever possible through focus groups, and task forces Involve course participants, trainers, designers, managers, etc. Provides credibility, buy-in and more reliable data

Lessons Learned Putting the Pieces Together…

• Be careful using 360º feedback for performance data (level 3)

In this ROI study, used only as supporting data 360º Surveying is labor intensive Difficult to draw conclusions, since

• ver time, feedback givers will change

Lessons Learned Putting the Pieces Together…

• Anticipate personal bias

Expect parties with vested interest to try to influence study results Evaluator needs to remain neutral and ensure objectivity Explain ROI process (what’s okay and not okay to do) Push back if you need to

Lessons Learned Putting the Pieces Together…

Executive Response

CFO reaction to “ROI” VP reaction to “intangibles” CEO reaction to the importance of training Overall reaction to the training organization’s link to the business

Lessons Learned Executive Response

Executive Response

ISO Matrix Certification Auditor asked about training measurements, we offered the ROI impact study, and auditor asked for a copy of the study Received an “Honorable Mention” at Auditor’s Closing meeting as a best practice taking place within the

• rganization!

Lessons Learned Honorable Mention

Correlation Pyramid

• Reaction, Learning, Performance, ROE and ROI Data

Reaction, Learning, Performance and ROE Data Reaction, Learning and Performance Data Reaction and Learning Data Reaction Data

Why Are Levels of Evaluation Important?

Thank you for your attention. Let’s take a break!

Michelle M. Wentz

BMW Manufacturing Assistant Manager, Training and Development

Michelle.Wentz@bmwmc.com

Toni Hodges

TH & Company 410-956-0475 Tonihodges@mindspring.com www.tonihodges.com

Rotating Table Discussions

Process Everyone chooses a table 3 Rotations: 15 minute table discussion 5 minute for rotation to next table Be prepared to share insights from each table discussion

Debrief

Insights from table discussions Additional Questions Specific challenges going forward Form Networks for support and idea sharing

Michelle M. Wentz

BMW Manufacturing Assistant Manager, Training and Development

Michelle.Wentz@bmwmc.com

Toni Hodges

TH & Company 410-956-0475 Tonihodges@mindspring.com www.tonihodges.com

Concluding Comments from the ASTD Board

Patrice McGuire-President Elect Please complete the Level 1 Program Evaluation on your table!