Asking the right questions for IMPACT Jonathan Ferris Purdue - - PowerPoint PPT Presentation

Asking the “right” questions for IMPACT

Jonathan Ferris Purdue Extension – Fayette County

Purdue University is an Equal Opportunity/Equal Access institution.

The Goal

“Data collection is worthless unless people are able to use the findings and use them well. The methodological area that presents the most challenge …. is that of helping the people who should use information from program studies to interpret it such a way that it actually is used in a positive manner benefiting people and communities.” Sara Steele, Univ. Wisconsin Extension

Purdue University is an Equal Opportunity/Equal Access institution.

Or,

“How to keep this whole IRB/Human Subjects evaluation thing as simple as possible so as to not pull your hair

• ut and hopefully end up with a

simple yet effective impact statement that will blow away your district director and program leader as well as wow the folks in DC.”

Purdue University is an Equal Opportunity/Equal Access institution.

Where I’m coming from…

• In the past few years, I’ve reviewed

500+ IRB applications

• Many survey instruments are overly

complicated and detailed

• We all could use help in crafting

questions that get the types of answers we want. (None of us are trained in this!)

Even with IRB stuff, our jobs aren’t that bad…

Purdue University is an Equal Opportunity/Equal Access institution.

So how do we start thinking about this? (My humble opinion)

• 1. (Bear with me as I get philosophical for a

minute…) Think of a program you really enjoy doing in your county/area.

• Why do you do it?? What drives you?
• What do you really want your participants to

take away from it?

Purdue University is an Equal Opportunity/Equal Access institution.

More…

• 2. What would you REALLY like for the

impact statement to say?

• Imagine writing it now…what would it say?
• Don’t forget – you need to be realistic about what

you can accomplish in (for example) a 3 hour program.

• Be concise (this isn’t thesis research). Your

efforts will probably be summed up into a couple

• f sentences anyway!

Purdue University is an Equal Opportunity/Equal Access institution.

My personal pet peeve…

• 3. Only ask questions you NEED to ask!
• In other words, make sure there is a reason

for asking the question. Will the data it generates be part of your impact statement?

Example: What is your age? (Check one) __ Under 18 __ 18-24 __24-32 __32-40 __Over 40

Purdue University is an Equal Opportunity/Equal Access institution.

Some types of questions

• Program Improvement

Did this program… a) Meet your expectations b) Exceed your expectations c) Not meet your expectations Program improvement questions should be kept to a

• minimum. Maybe even just poll a few folks

afterwards? What if they had low expectations?

Purdue University is an Equal Opportunity/Equal Access institution.

“Scale questions”

The information on weed control was: a) Not beneficial at all b) Slightly beneficial c) Moderately beneficial d) Very beneficial As a result of this workshop, I am more knowledgeable about (XYZ) (Circle one) 4) Strongly agree 3) Agree 2) Disagree 1) Strongly Disagree

Purdue University is an Equal Opportunity/Equal Access institution.

More Scale questions…

To what extent was this meeting worth your time? (Circle

• ne)

Not at all A great deal 1 2 3 4 5 6 Please rate your knowledge of (XYZ) before and after the program by placing a (B) or (A) on the scale below Least Most

1 2 3 4 5 6 7 8 9 10

Purdue University is an Equal Opportunity/Equal Access institution.

Strongly Agree/Disagree

The resources provided were useful (check one)  Strongly Agree  Agree  Undecided  Disagree  Strongly Disagree

• The problem with this

sort of question (in my

• pinion), is that when

you try to report the findings, it becomes “wordy”. Also, in this specific case, the question could have been worded better. (“Will you use the resources provided?”)

Purdue University is an Equal Opportunity/Equal Access institution.

Irrelevant?

Gender: Male__ Female__ How old are you? What size farming

• peration do you have?
• Some of these sorts
• f questions may

have merit, if you plan to break the data out differently depending on their

• answers. If not, do

you need to know?

Purdue University is an Equal Opportunity/Equal Access institution.

Enough negative…let’s practice making questions better!

Did this program… a) Meet your expectations b) Exceed your expectations c) Not meet your expectations

• How about…

“Would you like to see similar workshops

• ffered in the

future?” (Yes/No)

Purdue University is an Equal Opportunity/Equal Access institution.

The information on weed control was: a) Not beneficial at all b) Slightly beneficial c) Moderately beneficial d) Very beneficial

• How about…

“Do you plan to change your weed management practices as a result

• f what you learned

today?” (Yes/No)

Purdue University is an Equal Opportunity/Equal Access institution.

As a result of this workshop, I am more knowledgeable about healthy snacks (Circle

• ne)

4) Strongly agree 3) Agree 2) Disagree 1) Strongly Disagree

• How about…

“Do you plan to offer more healthy snacks to your children as a result of today’s program?” (Yes/No)

Purdue University is an Equal Opportunity/Equal Access institution.

To what extent was this meeting worth your time? (Circle one) Not at all A great deal 1 2 3 4 5 6

• How about… “Would you recommend this

program to your friends?” (Yes/No)

Purdue University is an Equal Opportunity/Equal Access institution.

Please rate your knowledge of pond weed management before and after the program by placing a (B) or (A) on the scale below Least Most

1 2 3 4 5 6 7 8 9 10

• How about… “Did you learn something about

managing your pond weeds that will help you enjoy your pond more this year?” (Yes/No)

Purdue University is an Equal Opportunity/Equal Access institution.

Surveys – how can we “ramp it up” and make them more transformational?

• Think about audiences you work with who

attend your programs every year.

• Why not simply add a few questions for

returning attendees to your survey instruments?

• What could you ask them to see if

practices/behaviors have changed?

Purdue University is an Equal Opportunity/Equal Access institution.

If you have attended DPAC Field Days in the past, please take a moment to tell us about how the information you received at the field days has helped you in your farming operations.

• 1. How many years would you estimate you have attended the DPAC field day?

_______________

• 2. What keeps you coming back?
• 3. Please tell us about one or two specific things that you are doing differently in your
• peration as a result of something you learned at a DPAC Field Day.
• 4. Would you say that the information you’ve received at DPAC field days in the past

has helped you be more profitable? (Circle one) YES NO

• 5. Can you think of one thing you wish we’d change about this program? If so, what

is it?

(An Example of how this might work…)

Purdue University is an Equal Opportunity/Equal Access institution.

What results did we get?

• 3. Please tell us about one or two specific things that you are doing

differently in your operation as a result of something you learned at a DPAC Field Day. Herbicide use, corn stalk management, better herbicide selection (9x), weed management (8x), scout more before making a decision, conservation farming, GPS signal on front of tractor, changing weight of tractor, research more before investing time and $, drainage & pesticide safety, GPS, tillage, plant populations, insect resistance management, drift control, more pre- emergent herbicide usage, scouting

• 4. Would you say the information you received at DPAC field days in

the past has helped you be more profitable?

– Yes: 95% – No: 5%

Purdue University is an Equal Opportunity/Equal Access institution.

Another Example: (From Wild Game Workshops)

• 4. As a result of attending our workshops, would you say that you tend to keep or

bring home more fish & game? (Circle one) YES NO

• 5. As a result of attending our workshops, would you say that you hunt/fish more

now that you are better prepared to utilize the game you harvest? (Circle one) YES NO

(Keep in mind – sometimes you craft questions with ulterior motives in mind, like future sponsors/partners…)

Purdue University is an Equal Opportunity/Equal Access institution.

Results…

89% of returning respondents said they actually hunt or fish more (as a result of the workshops), now that they can better utilize the fish/game they harvest. 95% said they keep or bring home more fish/game as a result of attending our programs. Do you think the DNR might be interested in hearing about this? Future sponsor/partner?

Purdue University is an Equal Opportunity/Equal Access institution.

To reinforce the “End in mind” principle…

• Let’s practice coming up with a few basic

questions for a couple of common Purdue programs.

• Remember – we first need to think of what we

want the impact statement to say, then craft questions that will get us the information we want.

• Pick one program from the samples at your

table and come up with 3 questions to use.

Purdue University is an Equal Opportunity/Equal Access institution.

Purdue University is an Equal Opportunity/Equal Access institution.