Promi mise ses s and Paradoxes in Un Underst standing Imp - - PowerPoint PPT Presentation

▶

Oct 27, 2022 244 likes •492 views

Promi mise ses s and Paradoxes in Un Underst standing Imp Impacts s of Ou Out of School Programs ms for Youth Annie Wright, Ph.D. Director of Evaluation Center on Research and Evaluation (CORE) SMU- Simmons Feb 10, 2017 Consultation

SLIDE 1

Promi mise ses s and Paradoxes in Un Underst standing Imp Impacts s of Ou Out of School Programs ms for Youth

Annie Wright, Ph.D. Director of Evaluation Center on Research and Evaluation (CORE) SMU- Simmons Feb 10, 2017

SLIDE 2

SLIDE 3

Center on Research and Evaluation

Evaluation

Consultation Developmental Formative Process Outcome - Summative

SLIDE 4

CORE

Areas

Out of School

Time

Early Childhood
Social &

Emotional Learning

Literacy
Parent

education Supports

Professional

development & teacher training

Leadership

training

Capacity

building

Strategic

partnerships Outcomes

Achievement

Gap

Readiness

Center on Research and Evaluation

SLIDE 5

CORE

Areas

Out of School

Time

Early Childhood
Social &

Emotional Learning

Literacy
Parent

education Supports

Professional

development & teacher training

Leadership

training

Capacity

building

Strategic

partnerships Outcomes

Achievement

Gap

Readiness

Center on Research and Evaluation

SLIDE 6

CORE

Areas

Out of School

Time

Early Childhood
Social &

Emotional Learning

Literacy
Parent

education Supports

Professional

development & teacher training

Leadership

training

Capacity

building

Strategic

partnerships Outcomes

Achievement
Achievement

gap

Readiness

Center on Research and Evaluation

SLIDE 7

Community Psychology

Center on Research and Evaluation

Understand individuals in context
Enhance the well-being of individuals and

communities through collaborative research and action

Prove and improve
Social science and community activism

Society for Community Research and Action (SCRA) – Division 27 of the American Psychological Association

SLIDE 8

Impact

Center on Research and Evaluation

Casual & Causal
Jingle & Jangle
Source & Audience

SLIDE 9

Impact – So What?

False Positive

Think that things work

when they really don’t False Negative

Think that things don’t work

when they really do

Center on Research and Evaluation

SLIDE 10

Impact

Did it work? How do you know it worked? Or didn’t? For whom? Why? How? Under what circumstances?

Center on Research and Evaluation

Sustain and Scale

SLIDE 11

Adults and Organizations Evidence based practices are necessary but not sufficient for achieving outcomes.

Wandersman

SLIDE 12

Adults and Organizations Evidence based practices are necessary but not sufficient for achieving outcomes.

Wandersman

Programs aren’t effective, only well- implemented programs are effective.

Durlak

SLIDE 13

Factors that support implementation

Features of the Setting Features of the Program Staff Characteristics Community Level Factors

See: Table 26.1 Examples of Factors That Influence Quality of Implementation Durlak, J.A. (2015). What everyone should know about implementation. In, Handbook on Social and Emotional Learning – Research and Practice. CASEL.

SLIDE 14

Adults and Organizations

Center on Research and Evaluation

Dallas Afterschool

Does coaching & training increase OST quality? Does OST quality increase

utcomes for

youth?

SLIDE 15

Adults and Organizations

Center on Research and Evaluation

Dallas Afterschool

Does coaching & training increase OST quality?

What elements of AQuA changed

within one year?

Does OST quality increase

utcomes for

youth?

SLIDE 16

Adults and Organizations

Center on Research and Evaluation

Dallas Afterschool

Does coaching & training increase OST quality?

Not all aspects of quality change at the

same rate (Programs & Activities)

Does OST quality increase

utcomes for

youth?

Depends on where they started

SLIDE 17

Organizational Readiness

R=MC2

Scaccia, J.P., Cook, B., Lamont, A., Wandersman, A. Castellow, J., Katz, J., & Beidas, R.S. (2015). A practical implementation science heuristic for

rganizational readiness: R = MC2. Journal of Community Psychology, 43, 4,

484-501.

Organizational readiness = motivation x general capacities x innovation specific capacities

SLIDE 18

Methods Matter

SLIDE 19

Low to High Inference

SLIDE 20

Program No Program Program No Program Program No Program

SLIDE 21

Methods Matter

Center on Research and Evaluation

Dallas Afterschool

Does coaching & training increase OST quality? Does OST quality increase

utcomes for

youth?

Does the AQuA score for the

OST site a student attends predict gains in achievement over one year?

SLIDE 22

Methods Matter

Center on Research and Evaluation

Dallas Afterschool

Does coaching & training increase OST quality? Does OST quality increase

utcomes for

youth?

Early elementary reading
Link to day & afterschool
DAS math > non DAS

SLIDE 23

Center on Research and Evaluation

Its amazing how far you can get when you don’t care who gets the credit.

SLIDE 24

Q&A

Center on Research and Evaluation