Assessing the Effectiveness of Youth Engagement Initiatives - - PowerPoint PPT Presentation

assessing the effectiveness of youth engagement
SMART_READER_LITE
LIVE PREVIEW

Assessing the Effectiveness of Youth Engagement Initiatives - - PowerPoint PPT Presentation

Assessing the Effectiveness of Youth Engagement Initiatives Elections Canada 2013 BC Provincial General Election Conference University of British Columbia Centre for the Study of Democratic Institutions January 31st February 1st, 2014


slide-1
SLIDE 1

1

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Assessing the Effectiveness of Youth Engagement Initiatives

Elections Canada

2013 BC Provincial General Election Conference University of British Columbia – Centre for the Study of Democratic Institutions January 31st – February 1st, 2014

slide-2
SLIDE 2

2

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Why do evaluations?

  • Accountability
  • Learning

– What’s the theory/logic of the project/program? – Was the project/program delivered as intended? – What went well? What could be improved?

  • Capacity-building

– To do / to use evaluations (think evaluatively)

  • Knowledge-building

– Contribute to broader learning

slide-3
SLIDE 3

3

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Logic Model

Inputs Activities Outputs Immediate Outcomes Intermediate Outcomes Ultimate Outcomes Resources:

  • Staff
  • money

Products developed Promotions Events # orders # students reached # forms distributed / completed Learning Knowledge Interest Discussion Registering Teaching methods Voting Sustained interest Participation Voting External Context

slide-4
SLIDE 4

4

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Student Vote Evaluation

slide-5
SLIDE 5

5

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Student Vote Federal Participation 2004-2011

*While targeted to Grades 5 and higher, all grades may participate, and students as young as kindergarten do. Teachers in middle school (intermediate) select the curriculum appropriate for their class. Since 2006, over 60% of registered schools have been elementary or middle schools.

slide-6
SLIDE 6

6

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Evaluation objectives

  • Assess the overall impact of the SVP in

the context of the 41st federal general election

  • Provide findings that will enable Student

Vote to improve the SVP for future parallel elections

slide-7
SLIDE 7

7

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Evaluation Design

  • Treatment: Students, teachers, parents who had participated in the 2011 SVP
  • Control: Students, teachers who had not participated in 2011 SVP, but had

taken/taught a class related to civics during that school year

Students Teachers Parents

Pre- Post- Pre- Post- Pre- Post- Treatment 14,754 3,780 1,341 341 Control 1,071 106

slide-8
SLIDE 8

8

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Teachers

  • Knowledge

– Number of years teaching civics is significant (+) – Elementary teachers feel less informed than other teachers (secondary or middle)

  • Confidence

– 85% participants say confidence increased – Only 65% of control group report feeling confident teaching civics

  • Intent to use SVP in future

– 95% state they are very likely – High praise for materials and support received

slide-9
SLIDE 9

9

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Parents

  • Knowledge:

– 61% of parents felt that their child’s participation in the SVP increased their own knowledge of politics and elections – 84% felt that it provided their family with more opportunities to learn about politics

  • Engagement/Discussion:

– 65% of parents felt that it increased their own interest – 86% believe the SVP motivated their child to discuss politics

  • Intention to vote:

– Of the 73% who reported voting, 20% said that their child’s participation in SVP positively influenced their decision to vote

slide-10
SLIDE 10

10

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Students

  • Knowledge and understanding

– Number of correct responses (pre- vs. post)

  • Elementary: + 15 percentage points
  • Secondary: + 11 percentage points

– Number of correct responses (participants vs. control)

  • Elementary: +11 percentage points
  • Secondary: no significant difference
slide-11
SLIDE 11

11

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Students

  • Engagement in political discussion

– discussion with parents increased significantly as a result of SVP participation – especially in secondary students

  • Appreciation (It is “a civic duty to vote”)
  • Increase in secondary students
  • Political interest
  • Experiential learning had a positive effect
slide-12
SLIDE 12

12

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Intention to vote (frequency results)

82% 90% 84% 18% 10% 16%

Non-Participant (Control) Post-Program Participant Pre-Program Participant Yes No

Question: If you had been eligible to vote in the federal general election that was held on May 2nd, would you have voted?

slide-13
SLIDE 13

13

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Intention to vote

  • Multiple regression yielded mixed results
  • Methodological issues:

– Confusing question wording (“eligible”) – Many responded “not old enough” to open-ended follow up question – “Real” versus “parallel”

  • Do we need better methods to evaluate impact on

future voting behaviour?

  • Is it appropriate to try to measure this?

? ? ?

slide-14
SLIDE 14

14

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Mixed results

  • Confidence expressing views

– Pre- vs. post: no significant difference – Confidence much higher among control group (over-confident?) – But: Parallel election had positive impact

  • Critical thinking

– Frequencies (pre/post/control): no difference – Regression: mixed results

  • Methodological issues?

? ?

slide-15
SLIDE 15

15

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Methodological issues

  • Question specificity
  • Surveys too long and some questions too

complex for younger students

  • Very strong control group
  • Quantitative surveys may not be best

instrument for assessing all outcomes (need mixed methods approach)

slide-16
SLIDE 16

16

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Key recommendations

  • To strengthen the program:

– Expand efforts to reach civics and social studies teachers – Engage official parties

  • To strengthen civics in general:

– Experiential civics between official election periods

  • To strengthen the evaluation:

– Review survey instruments – Consider research to determine impact on voting through a ‘longitudinal’ study

slide-17
SLIDE 17

17

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Apathy is Boring youth registration pilot project

slide-18
SLIDE 18

18

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Project Overview - 1

  • Elections BC - Apathy is Boring: pilot project
  • Goal: to increase the number of youth

accurately registered to vote in advance of the 2013 BC general election

  • Two Components:

1. Education for Civic and Community Organizations 2. Youth Voter Registration Campaign

slide-19
SLIDE 19

19

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Project Overview - 2

  • Elections Canada – Apathy is Boring:

evaluation/assessment

  • Goal: evaluate the effectiveness of key tactics

using quasi-experimental methods and qualitative interviewing

  • Two components:

1. Group Registration/Turnout Comparison 2. Interviews with local organizations

slide-20
SLIDE 20

20

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Evaluation Project Components

  • 1. Implement a quasi-experiment to test the

effectiveness of registration drives conducted by Street Teams at concerts and events in March and April 2013

  • 2. Conduct qualitative interviews with
  • rganizations seeking to engage or mobilize

youth during 2013 BC election

– to get a deeper sense of their experiences in youth electoral engagement

slide-21
SLIDE 21

21

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.
  • 1. Registration Drives

quasi-experiment

  • Treatment group: youth who register as a

result of Apathy is Boring Street Teams activities

  • Control: youth who were not reached by this

initiative

  • Analysis: registration and turnout data

provided by Elections BC

slide-22
SLIDE 22

23

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Preliminary Results: Registration Drives

  • March 8 – April 19, 2013
  • Key Outputs:

– 51 events attended (target: 40) – 10,511 interactions (target: 14,000) – 390 “Register to Vote” packages given out – 511 “BC Has More Ways to Vote” packages given

  • ut

– 572 completed 200A forms (target: 8000)

slide-23
SLIDE 23

24

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Preliminary Results: Registration Drives

  • Analysis of 200A registration transactions
  • 508 forms completed by individuals 18-39

Create Confirm Update Total 18-24 164 40 102 306 25-29 36 18 91 145 30-34 8 13 22 43 35-39 3 6 5 14 Total 211 77 220 508

slide-24
SLIDE 24

25

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Voter turnout by age and transaction type

  • 60% turnout among those who completed form

Create Confirm Update Total 18-24

Y : 90 N : 74 Y : 28 N : 12 Y : 68 N : 34 Y : 186 N : 120

25-29

Y : 13 N : 23 Y : 13 N : 5 Y : 50 N : 41 Y : 76 N : 69

30-34

Y : 5 N : 3 Y : 9 N : 4 Y : 20 N : 2 Y : 34 N : 9

35-39

Y : 1 N : 2 Y : 6 N : 0 Y : 3 N : 2 Y : 10 N : 4

Total

Y : 109 N : 102 Y : 56 N : 21 Y : 141 N : 79 Y : 306 N : 202

slide-25
SLIDE 25

26

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Next steps

  • Obtain / analyse data for control group(s)
  • Analysis of qualitative interview data
  • Final report and presentation to EC (to be

shared with E-BC)

slide-26
SLIDE 26

27

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Lessons Learned

  • There is value in evaluating/assessing
  • Importance of clarity in program design
  • Build evaluation into program from the outset
  • Think about how evaluation will be used and by

whom – design accordingly

  • Be realistic about what you can achieve

(applies both to program and evaluation)

  • Use appropriate tools and methods to measure
  • Communicate findings clearly
slide-27
SLIDE 27

28

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

What’s next for EC?

  • Knowledge mobilization for youth engagement
  • Goals:

– Raise awareness and understanding of the issue among target audiences (youth-serving orgs) – Showcase/encourage target audiences to invest and take action on the issue

  • Tactics:

– New website: youth engagement research / tools – Regional workshop series (spring – fall 2014) – National conference (winter 2015)

slide-28
SLIDE 28

29

  • VOTE. SHAPE YOUR WORLD. VOTER, C’EST CHOISIR SON MONDE.

Thank You

Miriam Lapp Assistant Director Outreach and Research Elections Canada Miriam.Lapp@elections.ca