Welcome New Pennsylvania Commission on Crime and Delinquency - - PowerPoint PPT Presentation

welcome
SMART_READER_LITE
LIVE PREVIEW

Welcome New Pennsylvania Commission on Crime and Delinquency - - PowerPoint PPT Presentation

Welcome New Pennsylvania Commission on Crime and Delinquency Grantee Orientation The EPISCenter represents a collaborative partnership between the Pennsylvania Commission on Crime and Delinquency (PCCD), and the Prevention Research Center,


slide-1
SLIDE 1

The EPISCenter represents a collaborative partnership between the Pennsylvania Commission on Crime and Delinquency (PCCD), and the Prevention Research Center, College of Health and Human Development, Penn State University. The EPISCenter is funded by PCCD and the Department of Public Welfare. This resource was developed by the EPISCenter through PCCD grant VP-ST-24368.

Welcome

New Pennsylvania Commission on Crime and Delinquency Grantee Orientation

slide-2
SLIDE 2

Meet & Greet Activity

  • Help yourself to refreshments
  • Please join the table for your program
  • Introduce yourselves
  • Describe your organization
  • Describe why your organization/community selected the

program including:

  • Target Population
  • Targeted Risk and Protective Factors
  • Hopes for the future (after PCCD funding ends)
slide-3
SLIDE 3

The EPISCenter represents a collaborative partnership between the Pennsylvania Commission on Crime and Delinquency (PCCD), and the Prevention Research Center, College of Health and Human Development, Penn State University. The EPISCenter is funded by PCCD and the Department of Public Welfare. This resource was developed by the EPISCenter through PCCD grant VP-ST-24368.

PCCD’s Investment in Evidence- based Programs

slide-4
SLIDE 4

EPISCenter’s Three Key Functions

  • Build general prevention

capacity

  • Build program-specific

capacity

  • Facilitate interaction/

communication between systems

Transla'onal ¡ Research

EPISCenter ¡

(Preven'on ¡ Support ¡System)

Build ¡general ¡preven'on ¡capacity ¡ among ¡prac''oners ¡and ¡policy ¡makers ¡

Technical ¡ Assistance

Con'nuous ¡Quality ¡ Improvement

EBP Grantees & Community Coalitions (Prevention Delivery System) Penn State’s Prevention Research Center (Prevention Synthesis & Translation System) Resource Center Steering Committee (Policy Makers & Funders) Rhoades, Bumbarger & Moore (in press). The Role of a State-Level Prevention Support System in Promoting High-Quality Implementation and Sustainability of Evidence-based Programs. American Journal of Community Psychology.

slide-5
SLIDE 5

Pennsylvania’s EBP Dissemination in 1999…

slide-6
SLIDE 6

Pennsylvania’s EBP Dissemination in 2014…

slide-7
SLIDE 7

Resource ¡Center ¡ ¡ for ¡Evidence-­‑based ¡Preven'on ¡and ¡ ¡ Interven'on ¡Programs ¡and ¡Prac'ces ¡

Support to Community Prevention Coalitions Improve ¡Quality ¡of ¡ ¡ Juvenile ¡Jus'ce ¡Programs ¡ and ¡Prac'ces ¡ Support to Evidence- based Programs

Multi-Agency Steering Committee (Justice, Welfare, Education, Health)

A unique partnership between policymakers, researchers, and communities to bring science to bear on issues of public health and public safety.

The EPISCenter is a project of the Prevention Research Center, College of Health and Human Development, Penn State University, and is funded by the Pennsylvania Commission on Crime and Delinquency and the Pennsylvania Department of Public Welfare as a component of the Resource Center for Evidence-Based Prevention and Intervention Programs and Practices.

slide-8
SLIDE 8

The EPISCenter represents a collaborative partnership between the Pennsylvania Commission on Crime and Delinquency (PCCD), and the Prevention Research Center, College of Health and Human Development, Penn State University. The EPISCenter is funded by PCCD and the Department of Public Welfare. This resource was developed by the EPISCenter through PCCD grant VP-ST-24368.

PCCD Grantee Requirements

PCCD Representative

slide-9
SLIDE 9

Quality Reporting

  • Grantees are responsible for reporting in Egrants

quarterly.

  • Report can be initiated after previous quarter is approved.
  • Reports are due to be submitted by the 20th of the month after the

quarter is closed.

  • Data collection tools and other documents are submitted as

attachments

slide-10
SLIDE 10

Quality Assurance Process

  • Purpose: To assess site functioning, review your data

collection process, and ability to demonstrate and communicate impact.

Start Date QA Process Date July 1, 2014 July-September 2015

*Funds should have been budgeted in your grant for this process.

slide-11
SLIDE 11

Quality Assurance Process

  • The (insert name of County/Agency) site is implementing with sufficient
  • quality. Given the current implementation, it is reasonable to expect future

positive outcomes from the program. At this time, the areas identified as needing improvement are limited and as to be expected for the length of implementation time.

  • The (insert name of County/Agency) site needs to make significant changes

to their implementation and is currently not implementing with the level of quality that is expected for the amount of time it has been implementing. For programs in this category, it is PCCD's intention to work with the developer and grantee to articulate a plan of corrective action and timetable to bring the program into compliance with the developer's requirements.

  • The (insert name of County/Agency) site is not implementing the program as

designed and the developer does not believe that the site’s implementation quality is correctable within a reasonable amount of time.

slide-12
SLIDE 12

Outcomes Report

  • Purpose-
  • Summarize Data Collected to Measure
  • Program Reach
  • Implementation Quality
  • Impact

Start Date OR Due Date July 1, 2014 April 2016

slide-13
SLIDE 13

Outcomes Report

  • Quantitative Data
  • Performance measures and survey data
  • Qualitative Data
  • Feedback from participants, staff, and program partners
  • Changes to project plan, project modification requests
slide-14
SLIDE 14

Outcomes Report

  • Section 1: Description of Population Served
  • Section 2: Indicators of Program Impact
  • Section 3: Indicators of Implementation Quality
  • Section 4: Lessons Learned
  • Section 5: Program Sustainability
slide-15
SLIDE 15

Top Three Reasons to Call a PCCD Analyst

  • Project Modification Requests - Program and Fiscal

changes to your application

  • Quarterly Program/Fiscal Report questions
  • Any fiscal questions/concerns should be directed to the

fiscal contact listed in Egrants

slide-16
SLIDE 16

The EPISCenter represents a collaborative partnership between the Pennsylvania Commission on Crime and Delinquency (PCCD), and the Prevention Research Center, College of Health and Human Development, Penn State University. The EPISCenter is funded by PCCD and the Department of Public Welfare. This resource was developed by the EPISCenter through PCCD grant VP-ST-24368.

EPISCenter Technical Assistance

Roger Spaw EPISCenter Prevention Coordinator

slide-17
SLIDE 17

Evolution of Technical Assistance

  • 1998: PCCD invests in prevention programming.
  • Difficult to monitor implementations of a wide variety of programs.
  • Grantees experienced mixed outcomes; some programs worked

well, some failed.

  • It was hard to know about problems until “after the fact”.
  • No formal technical assistance was available.
slide-18
SLIDE 18

Evolution of Technical Assistance

  • 2001: Penn State’s Prevention Research Center begins

providing Technical Assistance to PCCD grantees.

  • Grantees get support from the beginning, but programs are varied

and not all of them have high levels of evidence of effectiveness.

  • Technical Assistance was mostly reactive.
  • No standard approach to quality assurance
  • Performance Measures varied greatly from grantee to grantee
  • Difficult to know if programs were having desired impact
slide-19
SLIDE 19

Evolution of Technical Assistance

  • 2008: EPISCenter was established to support programs

with a higher level of evidence of effectiveness.

  • “Blueprints” programs www.blueprintsprograms.com
  • EPISCenter now PROACTIVELY targets Technical

Assistance for 11 programs.

  • Goal: To advance high quality implementation, impact assessment,

and sustainability of programs in order to maximize the positive impact of prevention science on outcomes for Pennsylvania youth.

slide-20
SLIDE 20

Evolution of Technical Assistance

slide-21
SLIDE 21

How?: Explaining the Magic

“I think you should be a bit more explicit here in step 2.”

slide-22
SLIDE 22

Why Technical Assistance

  • Goal: To advance high quality implementation, impact

assessment, and sustainability of programs in order to maximize the positive impact of prevention science on

  • utcomes for Pennsylvania youth.
  • Objectives:
  • Assess and support program implementation
  • Promote program quality and fidelity
  • Support the data collection process
  • Maximize the ability to demonstrate and communicate impact
  • Increase local investment & sustainability
slide-23
SLIDE 23

How? Group Training and Networking

  • Networking Meetings
  • PCs organize regular networking opportunities to allow grantees

who are doing the same program to share lessons learned, tools, and resources.

  • Trainings
  • Often PCs will identify a common training need for several
  • grantees. They then identify experts who can meet this need and

facilitate in-person or webinar training events. These trainings can be geared towards all of the EBPs or just one program specifically.

slide-24
SLIDE 24

How? Personal Consultations

  • On-Site Consultations
  • PC visits you in person in your community to review the Web-based

resources and discuss any successes or barriers

  • Typically twice a year, but varies depending on program and need
  • Initial consultation always includes completion of an

Implementation Plan with action steps for PC and Grantee

  • Web-Based Consultations
  • Skype, Adobe Connect, and Go-To Meeting are frequently used to

connect with grantees

  • Email & Phone Calls
  • Both great ways to contact your PC with a question
slide-25
SLIDE 25

How? Web-based Resources

  • Web Based Resources: www.EPISCenter.psu.edu
  • Logic Models
  • Frequently Asked Questions
  • Readiness Guides
  • Implementation Manuals
  • Impact Measurement Tools & Guidance
  • Performance Measure Reporting Tools and Guidance
  • Links to research articles, Developer Websites, etc.
  • Templates for Outcomes Reporting to PCCD and community

stakeholders

  • Training videos
slide-26
SLIDE 26

Implementation Plan

slide-27
SLIDE 27

Demonstrating Program Impact

§ Solid data collection systems § Implementing with a commitment to quality and fidelity § Consistently analyzing process and outcomes data

slide-28
SLIDE 28

Who Provides Technical Assistance?

  • EPISCenter Prevention Coordinators or “PCs”
  • Prevention Coordinators:
  • Lee Ann Cook – IYS (MST, FFT, MTFC)
  • Kris Glunt – LST, TND, OBPP
  • Roger Spaw - ART, BBBS
  • Mary Ann Demi- SFP 10-14, PATHS
  • Elizabeth Campbell – MST, FFT, MTFC
slide-29
SLIDE 29

The EPISCenter represents a collaborative partnership between the Pennsylvania Commission on Crime and Delinquency (PCCD), and the Prevention Research Center, College of Health and Human Development, Penn State University. The EPISCenter is funded by PCCD and the Department of Public Welfare. This resource was developed by the EPISCenter through PCCD grant VP-ST-24368.

Planning & Start-up

Lee Ann Cook, MSS EPISCenter Prevention Coordinator

slide-30
SLIDE 30

Don’t Reinvent The Wheel!

  • Use EPIS Readiness Tools
slide-31
SLIDE 31
slide-32
SLIDE 32
slide-33
SLIDE 33
slide-34
SLIDE 34
slide-35
SLIDE 35
slide-36
SLIDE 36

Did you follow recommendations for:

  • Training Information & Coordination?
  • Local training is often available
  • Materials Purchasing Guidelines?
  • Do you really need that video or those stickers?
  • Real World Time Requirements
  • Staff Requirements?
  • How many are required?
  • What qualifications are required?
slide-37
SLIDE 37

Break Out

  • EPISCenter PC will review specific “start up” resources
  • Discuss: How is start up going for you?
  • Successes?
  • Barriers?
  • How can EPIS help?
slide-38
SLIDE 38

Lunch

  • On your own
  • Be back by 1:30 p.m. Please!
slide-39
SLIDE 39

The EPISCenter represents a collaborative partnership between the Pennsylvania Commission on Crime and Delinquency (PCCD), and the Prevention Research Center, College of Health and Human Development, Penn State University. The EPISCenter is funded by PCCD and the Department of Public Welfare. This resource was developed by the EPISCenter through PCCD grant VP-ST-24368.

Implementation Quality (IQ) – Ensuring the effectiveness of your program

Mary Ann Demi, M.S. Research Coordinator, EPISCenter

slide-40
SLIDE 40

Effective Program

  • What does it mean when a program has been determined

effective/evidence based?

  • The intervention is based on a theory of change that is documented

in a clear logic or conceptual model

  • Documentation exists that it has been effectively implemented in

the past with:

  • In a scientific manner
  • With high and rigorous standards
  • With credible and positive effects
  • The intervention is reviewed by a panel of informed prevention

researchers

slide-41
SLIDE 41

Basis for “Evidence”

  • Reduces risk due to a strong research design
  • Desired risk reduction (outcomes) is sustained over time
  • Desired outcomes are replicated in multiple sites
slide-42
SLIDE 42

Fidelity Leads to Effectiveness

  • What is fidelity?
  • Fidelity is:
  • Delivering a prevention or intervention program and adhering to the to

the original protocol or program model.

  • Implementing a program without removing parts of the program that are

essential to the program’s effectiveness—its core components.

  • Implementing a program without adding parts to the program that

are not part of the original protocol

FOLL FOLLOW THE RECIPE W THE RECIPE

slide-43
SLIDE 43

Logic Model

  • The “Logic Model’ is the recipe and is
  • Developed based on tested theory of change
  • Has key components (ingredients) and without those key

components recipe may fail

  • Often follow a logical sequence of cause and effect:

If this, than that ….. Program Activity Decreased Risk Behavioral Outcomes

slide-44
SLIDE 44

Logic Model Example:

Strengthening Families Program

Model created by Penn State EPISCenter

slide-45
SLIDE 45

Deviating from program design…

  • “When communities ‘tweak’ a program to suit their own

preferences or circumstances, they wind up with a different program whose effectiveness is unknown.”

Blueprints for Violence Prevention www.colorado.edu/cspv/blueprints

slide-46
SLIDE 46

Elements of Implementation Quality

  • Dosage
  • Adherence
  • Quality of Delivery
  • Program Reach
  • Participant Responsiveness
slide-47
SLIDE 47
  • Dosage
  • All lessons are taught
  • Lessons are taught the entire length of time prescribed
  • All materials correspond with lessons

Elements of Implementation Quality

slide-48
SLIDE 48
  • Fidelity/Model Adherence
  • Understand the theory behind the program
  • Comply with recommended staff qualifications
  • Contract with developers to train staff in the use of the model
  • Enroll intended participants (universal versus targeted)
  • Monitor fidelity using developed tools that identify key components

Elements of Implementation Quality

slide-49
SLIDE 49
  • Quality of Delivery
  • Program staff are trained by developer
  • Staff have characteristics that target population can relate to,
  • Staff share positive and enthusiastic attitudes
  • Facilities are appropriate for group size, safe, and welcoming
  • Materials and supplies are prepared in advance, sufficient for

number of participants, and are in good shape and of good quality

  • Efforts to keep participants engaged in the program (importance of

quality, trained and staff experience working with the target population)

Elements of Implementation Quality

slide-50
SLIDE 50
  • Program Reach (target audience)
  • Delivered to intended audience
  • Age range
  • Youth, Parent, or Family
  • Individual versus Group
  • Sufficient number of participants to run groups effectively

Elements of Implementation Quality

slide-51
SLIDE 51

Ways to Measure Implementation Quality

  • Fidelity Observation
  • Implementer Self-Report
  • Administrative Documents
  • Participant Self-Reports
slide-52
SLIDE 52

Why IQ or Fidelity Monitoring?

  • Measure IQ à Unlock black box
  • Ex: What is happening in the program?
  • Measure IQ à Better IQ
  • “What gets assessed gets addressed!”
  • Better IQ à Better Outcomes
  • Show funding agencies what you’ve done
slide-53
SLIDE 53
  • We’re here to support the best possible implementation in

your communities.

  • Contact your Prevention Coordinator for expert assistance
  • Consult our website for additional resources
  • www.episcenter.psu.edu

Need implementation Help?

slide-54
SLIDE 54

The EPISCenter represents a collaborative partnership between the Pennsylvania Commission on Crime and Delinquency (PCCD), and the Prevention Research Center, College of Health and Human Development, Penn State University. The EPISCenter is funded by PCCD and the Department of Public Welfare. This resource was developed by the EPISCenter through PCCD grant VP-ST-24368.

Data Collection and Reporting

Lee Ann Cook

slide-55
SLIDE 55

Process Data

  • How many?
  • How often?
  • How well?
slide-56
SLIDE 56

Outcomes Data

  • Did you achieve your intended impact?
  • Baseline Pre Test or Survey
  • Follow-up Post Test or Survey
  • Success Stories
slide-57
SLIDE 57

Data Analysis & Reporting

  • PCCD Excel Spreadsheet Data Analysis & Reporting

Tools

  • Generally each is designed to hold one year of data
  • Update and submit to PCCD via egrants on a quarterly basis
  • Provides summary data on each performance measure so that

providers can make adjustments to implementation as necessary

slide-58
SLIDE 58

Instructions

slide-59
SLIDE 59

Process Tab

slide-60
SLIDE 60

Outcomes Tab

slide-61
SLIDE 61

Raw Data Entry-Pre/Post

slide-62
SLIDE 62

Raw Data Entry-Fidelity

slide-63
SLIDE 63

Why SOOO much Data Collection?

slide-64
SLIDE 64

Strengthening Families 10-14 2012-2013 Reach

  • Active Grants
  • Facilitators Trained
  • Programs Completed
  • 478 Adults Enrolled
  • 437 Youth Enrolled
  • 74% of Participants Completed the Program
  • 77% of Participants Completed Pre/Post Measures
slide-65
SLIDE 65

Life Skills Training Fidelity 2012-2013

PA LST Model Fidelity Model Fidelity

The goal is to observe 20 percent of the lessons taught to assess model

  • fidelity. For LST “Minimum

Fidelity” is defined as having taught at least 75 percent of the lesson as designed by the developer.

84% ¡ 16% ¡

Lessons ¡Observed: ¡103 ¡

Mee'ng ¡Min. ¡ fidelity ¡ Needing ¡ improvement ¡

slide-66
SLIDE 66

Strengthening Families 10-14 2012-2013 Impacts

54% 54% 63% 45% 56%

0% 10% 20% 30% 40% 50% 60% 70%

Parents - Increased Substance Use Rules/ Expectations Parents - Improved Standard Setting Parents - Improved Family Problem Solving Youth - Improved Parent-Child Relationship Quality Youth - Improved Peer Pressure Refusal Skills

NUMBER OF PARTICIPANTS REPORTING POSITIVE CHANGE 349 parents/caregivers and 353 youth with completed pre/post tests 189 188 219 158 198

slide-67
SLIDE 67

Big Brothers Big Sisters Impacts 2010-2013

35% 43% 11% 26% 42%

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%

Family Relationship Peer Relationships Avoid ATOD Avoid Delinquency Academic Performance

Littles' Improvements

722 Annual Outcome Evaluations (POE and YOS)

slide-68
SLIDE 68

Data Reporting

  • Its never to early to start sharing data
  • Internally
  • Collaborative Board Reporting
  • School Board
  • How?
  • Community Reporting Templates
  • Outcome Report Templates
  • Comprehensive Outcome Report Required at the

end of Year Two.

slide-69
SLIDE 69

Project Towards No Drug Abuse Community Report Template

slide-70
SLIDE 70

Future Improvements: Logic Model Tabs

slide-71
SLIDE 71

Future Improvements: More Descriptive Analysis

slide-72
SLIDE 72

The EPISCenter represents a collaborative partnership between the Pennsylvania Commission on Crime and Delinquency (PCCD), and the Prevention Research Center, College of Health and Human Development, Penn State University. The EPISCenter is funded by PCCD and the Department of Public Welfare. This resource was developed by the EPISCenter through PCCD grant VP-ST-24368.

Kris Glunt

The 5 W’s of Sustainability

slide-73
SLIDE 73

Defining Sustainability

  • “…the continuation of community health or quality of life

benefits over time.”

  • A Blueprint for Sustainabiliy From Surviving to Thriving adapted from

Sanislaus County Children and Families

slide-74
SLIDE 74

WHEN should you begin planning?

  • From the beginning:
  • Consider efficiency from the start
  • Have data collection procedures in place
  • Utilize the resources provided through EPIS
  • Schedule regular meetings to discuss

YESTERDAY!!

slide-75
SLIDE 75

WHO should be involved in this process?

  • Administrators
  • Staff
  • Community Support
  • Identify Key Stakeholders
  • Strategic partnerships with similar interests/goals
  • Identify Key Champions
  • Serve as advocates for your program/agency
  • People within and outside of your organization
slide-76
SLIDE 76

WHAT tools can be used for this process?

  • Plan for Sustainability (A Blueprint for Sustainability From

Surviving to Thriving)

  • Sustainability: The Result of Building Relationships
  • Sustainability Planning Worksheet
  • Community/Outcomes Reports
  • Quality Assurance Letter Verification
  • http://episcenter.psu.edu/PSIBGF
slide-77
SLIDE 77

Sustainability Tools/Resources:

slide-78
SLIDE 78

Tools/Resources Continued:

slide-79
SLIDE 79

WHERE to turn for resources/funding?

  • In-kind Resources
  • Foundations
  • Government
  • Corporations
  • Businesses
  • United Way
  • Community-Service Organizations
  • Faith-based Organizations
  • Individuals
  • Schools/PTOs
slide-80
SLIDE 80

WHERE to turn…be strategic:

  • List selling points:
  • State the benefits to the participants
  • State the benefits to the community
  • Tie your product/program to the funder’s objectives and

terminology

  • Highlight how your outcomes match the potential funders’ goals
slide-81
SLIDE 81

WHY start planning so soon?

  • Planning for sustainability is a challenging process
  • 2 years will go fast
  • It’s important to start preparing from the beginning
  • Continue to make a positive impact in your community
  • Continue to allow good people to do good work
slide-82
SLIDE 82

HOW should we begin this process?

  • Consider efficiency from the start
  • Use the “Plan for Sustainability” as a guide to get started
  • Schedule regular meetings to develop a plan and make

changes as necessary

  • Have data collection procedures in place from the start
  • Discuss with your EPIS PC
  • Maintain strong ties with your Collaborative Board
  • Visit http://episcenter.psu.edu/PSIBGF for more resources
slide-83
SLIDE 83

For more information:

  • Planning for Sustainability: Impact Beyond Grant Funding
  • http://episcenter.psu.edu/PSIBGF