Leveraging the Power of Surveys Lillian Thomas Analytics Manager - - PowerPoint PPT Presentation

▶
leveraging the power of surveys
SMART_READER_LITE
LIVE PREVIEW

Leveraging the Power of Surveys Lillian Thomas Analytics Manager - - PowerPoint PPT Presentation

Sep 15, 2023 206 likes •470 views

Leveraging the Power of Surveys Lillian Thomas Analytics Manager (Section Chief) HR Systems Analytics & Information Division (HR SAID) Office of Human Resources (OHR) National Institutes of Health (NIH) Why Use Surveys? Get more

slide-1
SLIDE 1

Leveraging the Power of Surveys

Lillian Thomas

Analytics Manager (Section Chief) HR Systems Analytics & Information Division (HR SAID) Office of Human Resources (OHR) National Institutes of Health (NIH)

slide-2
SLIDE 2

Why Use Surveys?

 Get more context  Fill the ‘gaps’ of quantitative data  Provide insight into employee opinions  Understand issues and develop solutions  Assess the effectiveness of programs  Improve processes  Track trends

4/3/2018 Leveraging the Power of Surveys 2

slide-3
SLIDE 3

Kick-Off Design

Develop

Implement Analyze/ Report Take Action

Survey Life Cycle

4/3/2018 Leveraging the Power of Surveys 3

Kick-Off Design Survey Develop Online Implement/Administer Analyze Data/Report Findings Take Action

slide-4
SLIDE 4

4/3/2018 Leveraging the Power of Surveys 4

Developing the OHR Pulse Survey

Get Leadership Buy-In Promote Survey

  • Leadership announcing survey

coming

  • Sharing previous survey

results/impact if applicable

Launch Survey

  • Monitor survey response rates
  • Continue to promote survey &

encourage participation

Determine Topics

  • f Interests

Develop Survey

  • Questions development
  • Provide draft survey for

leadership review

  • Test survey

CASE EXAMPLE

slide-5
SLIDE 5

4/3/2018 Leveraging the Power of Surveys 5

Reporting for the OHR Pulse Survey

Analyze Results Action Planning Implement Changes Debrief Leadership Communicate Findings

CASE EXAMPLE

slide-6
SLIDE 6

Survey Life Cycle Best Practices

4/3/2018 Leveraging the Power of Surveys 6

Kick-Off

Discuss survey

  • bjectives up

front and make sure they align with questions and the analyses.

Design

Evaluate survey questions and ensure they are free from errors and pitfalls.

Develop

Reduce survey- taker burden. Ensure confidentiality.

Implement

Sponsor to advertise surveys.

Analyze/Report

Ensure data is free from errors. Conduct analyses appropriate to

  • bjectives.

As reports mature, find ways to automate the report (or segments) for quicker dissemination.

Take Action

Communicate survey results. Take appropriate action. Communicate steps taken as a result of prior surveys.

slide-7
SLIDE 7

What analyses can we do with engagement surveys?

 Basic summary report  Significance testing  Cross-tabulation  Open-text analysis  Trend analysis  Cross analysis with other survey and HR data

4/3/2018 Leveraging the Power of Surveys 7

slide-8
SLIDE 8

Examples of Analysis

4/3/2018 Leveraging the Power of Surveys 8

Significance Testing Thematic/Open-Text Analysis Trend Analysis What is the difference in supervisor satisfaction between employees likely to leave in a year and employees intending to stay? Is that difference statistically significant or is that due to chance? What are people saying in their open text responses? What can we extract from the numerous lines of texts? How is the organization doing compared to last year? Are there any increases or decreases worth noting?

slide-9
SLIDE 9

Survey Data & Analytics

4/3/2018 Leveraging the Power of Surveys 9

SURVEY RESULTS

  • Pulse Survey
  • NIH Exit

Survey

  • FEVS

DATA/ANALYTICS

  • Workforce

Demographics

  • Separation

Data OUTCOMES

Workforce Planning Retention Strategies Action Planning Employee Engagement Improved Organizational Process

slide-10
SLIDE 10

Data Driven Action Planning

Initiatives/Programs Resulting from Action Planning Based on OHR Pulse Survey

4/3/2018 Leveraging the Power of Surveys 10

OHR Director’s Awards Ceremony OHR Training Program HR Opportunity Program ConnectMore Events

CASE EXAMPLE

slide-11
SLIDE 11

Key Takeaways

4/3/2018 Leveraging the Power of Surveys 11

Survey Participation

Advertise

  • Have someone important sponsor the

survey

  • Send reminder emails

Provide Confidentiality

  • Preserve confidentiality
  • Increase participation and valid

responses

Share Results

  • Share findings with participants

Take Action/Show Commitment

  • Show participant input is useful and

actions will be taken based on survey results

  • Increase customer buy-in

Survey Validity

Keep It Simple

  • Use succinct questions
  • Reduce instrument bias to get more

valid data

Survey Utility

Incorporate Multiple Sources

  • Consider incorporating other sources
  • f data to gain further insights
slide-12
SLIDE 12

Contact Information Lillian Thomas

Analytics Section Chief NIH/OHR/HR SAID thomaslm@mail.nih.gov 301.594.0924

4/3/2018 Leveraging the Power of Surveys 12

slide-13
SLIDE 13

Additional Resources: Types of Survey Questions

4/3/2018 Leveraging the Power of Surveys 13

Demographic Questions Select Best & Multi-Select Questions Likert Scale Questions Baseline/Overall Rating Questions Open-Ended Questions

slide-14
SLIDE 14

Additional Resources: Types of Survey Questions

4/3/2018 Leveraging the Power of Surveys 14

Demographic Questions

  • Questions that asks respondents their demographic

information

  • IC, Series, Grade, Pay Plan, Tenure
  • Include demographic questions if they are needed for analyses

Please select the employee type that best describes you.

  • Intern/ Fellow
  • NIH Employee
  • Contractor
slide-15
SLIDE 15

Additional Resources: Types of Survey Questions

4/3/2018 Leveraging the Power of Surveys 15

Select Best & Multi-Select Questions

  • Select Best: Respondents forced to select only one answer

Please select your preferred training delivery method.

  • Classroom
  • Online
  • Multi Select: Respondents can select multiple answers

What training areas are you interested in? Select all that apply. Project Management Information Systems Human Resources Management

slide-16
SLIDE 16

Additional Resources: Types of Survey Questions

4/3/2018 Leveraging the Power of Surveys 16

Likert Scale Questions

  • Rating questions
  • Degree of agreement, satisfaction, likelihood, & etc.

Please rate your agreement with the following statement: The instructor communicated the objectives effectively.

  • Strongly Disagree
  • Disagree
  • Neutral
  • Agree
  • Strongly Agree
slide-17
SLIDE 17

Additional Resources: Types of Survey Questions

4/3/2018 Leveraging the Power of Surveys 17

Baseline/Overall Rating Questions

  • Most commonly are likert scale questions
  • Ask the “overall experience

What is your overall satisfaction with the training session.

  • Very Dissatisfied
  • Dissatisfied
  • Neutral
  • Satisfied
  • Very Satisfied
slide-18
SLIDE 18

Additional Resources: Types of Survey Questions

4/3/2018 Leveraging the Power of Surveys 18

Open-Ended Questions

  • Provides respondents an opportunity to provide some general

comments. Please provide any additional comments you have regarding the training session.

  • Can be used to ask why respondents selected a specific answer.

You have indicated dissatisfaction with the statement above, please elaborate on your experience.

slide-19
SLIDE 19

Additional Resources: Common Survey Question Issues/Pitfalls

4/3/2018 Leveraging the Power of Surveys 19

Double/Multi-Barreled Questions Biased/Leading Questions Categories not Mutually Exclusive Categories are not Exhaustive Double Negative Questions Open-Ended Question that can be answered with yes/no

slide-20
SLIDE 20

Additional Resources: Common Survey Question Issues/Pitfalls

4/3/2018 Leveraging the Power of Surveys 20

Double/Multi-Barreled Questions

Bad example: Please rate how satisfied you were with the training content, instructor and materials. Good example: Please rate your satisfaction with the following:

  • Training content
  • The instructor(s)
  • Training materials provided

Tips on how to avoid error

  • Look for ‘and’, ‘or’ in the question
  • Break down the question and test whether you could answer each piece separately
slide-21
SLIDE 21

Additional Resources: Common Survey Question Issues/Pitfalls

4/3/2018 Leveraging the Power of Surveys 21

Biased/Leading Questions

Bad example: On average, a hands-on training format is rated the most preferred. What type of training format do you prefer?

  • Hands-On
  • Lecture
  • Online

Good example: What type of training format do you prefer the most?

  • Hands-On
  • Lecture
  • Online

Tips on how to avoid error

  • Avoid including opinions and supporting information
  • Keep it simple; Only include the question at hand
slide-22
SLIDE 22

Additional Resources: Common Survey Question Issues/Pitfalls

4/3/2018 Leveraging the Power of Surveys 22

Categories not Mutually Exclusive

Bad Example: Approximately how many in classroom trainings do you attend per year?

  • 0-1
  • 1-3
  • 3-5
  • 5 or more

Good Example: Approximately how many in classroom trainings do you attend per year?

  • 0-1
  • 2-3
  • 4-5
  • 6 or more

Tips on how to avoid error

  • Look for response options which overlap or repeat
  • Identify questions where a single response could fall into more than one response option/category; separate options into

independent categories

slide-23
SLIDE 23

Additional Resources: Common Survey Question Issues/Pitfalls

4/3/2018 Leveraging the Power of Surveys 23

Categories are not Exhaustive

Bad Example: How many years have you worked at NIH?

  • 1-4
  • 6-8
  • 10-12

Good example: How many years have you worked at NIH?

  • Less than 1 year
  • 1 year – Less than 5 years
  • 5 years – Less than 8 years
  • 8 years – Less than 12 years
  • 12 years or more

Tips on how to avoid error

  • Look for gaps between options
  • Look for limits which may exist in the range of response
  • Look for a potential to include an “Other” option
  • When applicable, add a text box (“Please Specify”) if interested in gathering more detailed information
slide-24
SLIDE 24

Additional Resources: Common Survey Question Issues/Pitfalls

4/3/2018 Leveraging the Power of Surveys 24

Double Negative Questions

Bad example: Please rate the following statement: The instructor was not effective in communicating the objectives.

  • Strongly Disagree  Strongly Agree

Good example: Please rate the following statement: The instructor was effective in communicating the objectives.

  • Strongly Disagree  Strongly Agree

Tips on how to avoid error

  • Look for ‘not’ in the question
  • Look for a contradiction; combination of ‘not’ and a negative response option
slide-25
SLIDE 25

Additional Resources: Common Survey Question Issues/Pitfalls

4/3/2018 Leveraging the Power of Surveys 25

Open-Ended Question that can be answered with yes/no

Bad Example: Do you have any other comments on your training experience? Good example: Please provide any additional comments you have regarding your training experience. Tips on how to avoid error

  • Test to see if “yes” or “no” will satisfy the question
  • Consider starting the question with “What,” “Please describe,” or “Please provide…” to

make it open-ended