Evaluation: Hot tips from The Ian Potter Foundation Dr Squirrel - - PowerPoint PPT Presentation

Evaluation: Hot tips from The Ian Potter Foundation

Dr Squirrel Main, Research and Evaluation Manager

Before we begin…

 Please note that IPF staff may be taking photos/videos

during this workshop for social media purposes. If you do not wish for your image to be included in the publicity, please let any staff member know.

Presentation Overview

 Standards of evidence  Level 1: Grantee tips and KPIs  Level 2: Endgame and sustainability  Level 3: Resources and ideas  Dissemination: Infographics

Evaluation vs. chasing unicorns…

Standards of Evidence

Level 1: Top tips

• 1. Train the volunteers conducting surveys
• 2. Ensure redundancy in the key knowledge holders (bikes-

cancer-hearts)

• 3. Hold immediate staff-wide dissemination-training as

soon as possible

• 4. Start early: consultations with stakeholders take a long

time

• 5. Capture baseline data
• 6. Use pre-existing, validated surveys (North Carolina

Family Assessment Scales)

• 7. Ensure all council permits, etc. are in order

Level 1: SMART KPIs

Take a moment to review your goals in pairs. Listen to your partner’s goals and provide feedback -

 Are they SMART?  Specific  Measurable  Achievable  Rewarding/Relevant  Time-bound

How can your partner improve their measurements? Have time? Take a moment to review your long-term outcomes in pairs….

 How will your partner begin to measure/collect data

for their longer-term outcomes?

 Challenges? Solutions?

Level 2: Evaluation key steps

Determine end-game Involve stakeholders Circulate tender Finalise proposal Review first draft Disseminate

Level 2: Decide your endgame



Source: Alice Guglev and Andrew Stern. What’s Your End-game? Global Development Incubator. 30 January 2014.

Then tailor your evaluation…

Inputs Activities Outputs Short- term

• utcomes

Medium- term

• utcomes

Long- term

• utcomes
• How does the program work?
• What did the program

accomplish?

• What has influence how the

program works?

• What outcomes resulted from the

program?

• When and to whom?
• What is the program’s

effectiveness? (Findings, not Rec’s)

• How do changes compare to other

programs/sectors?

Level 2: Who would you invite?

Level 3: Evaluation resources

• TIPFEP
• Social impact hub

Level 3: Data rehearsal

Level 3: Data sources

Hot tip: Simplicity

Pre & Post Test n= Pre-test Mean Post test-Mean Difference Significance Level Question 1 59 2.64 3.98 1.33

P< 0.001

Question 2 59 3.64 4.17 0.52

P< 0.001

Question 3 59 2.76 4,20 1.44

P< 0.001

Question 4 59 3.37 4.17 0.79

P< 0.001

Question 5 59 3.19 4.20 1.01

P< 0.001

Table 2 shows the mean differences between pre workshop and post workshop responses for each question from 59 participants. Data was subject to a paired t-test to determine the p value for each question. This indicated the

• utcome to be highly significant at p<0.01 as

the difference in mean is significant and not due to chance, which indicates attitudinal change and knowledge transfer. Table 2: Statistical Analysis of Pre-Post Workshop Questionnaire - Paired t-test Table 2 outlines the statistical analysis of the data presented in Tables 3 to 7.

Or this?

2.6 3.4 3.2 4.0 4.2 4.2 1 2 3 4 5 It's a waste of time to think about managing money In order to avoid debts and financial stress it’s important to understand how loans, credit cards, interest and mobile phone plans work The only way to pay bills and saving is to spend less than you earn

Growth in financial knowledge: Responses from 59 workshop participants

Pre-test Mean Post test-Mean

Read it Pull out one big quote and one big stat for people to remember Find other ‘pull’ quotes and numbers Think about the best way to represent each item White space!

Secret sauce to communicating outcomes: creativity?

https://thenounproject.com/ http://piktochart.com/ https://infogr.am/ Or is the secret sauce a good webpage?