Common Measures: Planning & Implementing a Successful CYFAR - - PowerPoint PPT Presentation

Common Measures:

Planning & Implementing a Successful CYFAR Evaluation

Kyle R. Hawkey krhawkey@umn.edu

Planning for Successful CYFAR Evaluation

Evaluation: What’s in it for me?

• Concrete evidence about whether young

people are benefiting from your SCP project

• Project Improvement
• Project Resources
• Current/potential participant support
• Community relationships
• Funding/initiative requirements

CYFAR SCP Evaluation requirements

• 5 year logic model
• 5 year work plan
• Programs are REQUIRED to evaluate their

programs using the CYFAR Common Measures, including:

• PRE/POST evaluation
• Reporting outcomes
• Sharing outcomes

Required Common Measures

Core Competencies

This instrument measures behavior and attitudes related to five constructs that are common to the goals of all CYFAR programs: social conscience, personal values, caring, decision making, and critical thinking.

Youth & Adult Demographics

These two separate instruments collect program participant data including gender, age, grade level, ethnicity, race, military affiliation for both youth and adults

Youth & Adult Participation

These two separate instruments collect program participant dosage data including: frequency and length of participation in program, length of participation in 4-H, in-school activities, out-of-school activities, and

• ther activities.

Program Quality

This 22-item instrument measures different aspects of program quality, including physical/psychological safety, supportive relationships, positive social norms, support for efficacy/mattering, and opportunities for skill building.

Outcome-Specific Common Measures

Construct/Topic Version Type(s)

Leadership Development High School Nutrition Elementary, Middle School, & High School Parenting Adults Physical Activity Elementary, Middle School, & High School Science Elementary, Middle School, & High School Technology Elementary, Middle School, High School, & Adults Workforce Preparation Middle School & High School

Implementing a Successful CYFAR Evaluation

Resource Page

Survey Builder

Create a custom survey using vetted measures

1

Edit and save surveys to your account

2

Download your survey:

• Pre/Post
• PDF files

3

• Access the “Survey Builder”

from the top menu

• Click “Add New Survey”

Add a New Survey

Indicate whether you are:

• CYFAR Grantee
• YEAK Survey user

(4-H Science programs)

• Guest

Identify User Status

• Title your survey to locate

it in your account once saved

• Click “Add Survey” to

move to the next step

Title Your Survey

Select from:

• Middle School
• High School
• Elementary
• Adult

Select Target Audience

• Required measures

are pre-selected

• Select from Common

Measures approved for CYFAR

Select Your Measures

• Preview your survey
• Drag and drop to

reorder

• Scales stay in-tact

Preview & Reorder

How will you collect your data?

The CYFAR PDTA Team supports data collected using the on-line survey

• r printable paper/pencil

surveys.

File formats:

• Online survey link
• PDF
• Clicker survey

Download Survey Files

Elevated Account Privileges

• Your account must be elevated by the Evaluation

Team in order to:

• Download survey files
• Access application and software
• Upload data files
• Access data outputs and reports
• Send your username to the CYFAR PDTA Team

(cyfarpdta@umn.edu) to have your account elevated

IRB Requirements

• CYFAR grantees must consult with their local

institution to determine if the collection of CYFAR Common Measures will require IRB approval

• UMN’s IRB will not provide multi-institution level
• versight and Human Subjects approval for the

Common Measures data collection process

• Each site will safeguard their data by submitting only

de-identified data to the CYFERnetSEARCH.org website

IRB Requirements

UMN’s role in the CYFAR Common Measures data collection process does not constitute human subjects research because:

• CYFAR Common Measures data will be collected as part of routine

program evaluation and is not part of a specific intervention (program

• r activity) proposed by UMN
• CYFAR Common Measures data will be de-identified (stripped of

personal information and coded with a unique identifier provided by the site) when it is submitted by CYFAR SCP grantees to the UMN; therefore, UMN will not be able to trace data back to specific individuals and will not be able to identify individual persons based on their data

Common Measures Data Collection Process

Build Logic Model Build survey(s) Select short- term outcome indicators from approved CYFAR Common Measures Save and manage survey files (clicker, PDF, online) in your account

Phase One

Determine how survey will be administered Download survey files from CYFERnetSEARCH Review evaluation process including checking with your institution’s IRB

Phase Two

Common Measures Data Collection Process

Administer pre-surveys Upload data file(s) to CYFERnetSEARCH.org Download data set for grantee use Administer post-surveys Upload data file(s) to CYFERnetSEARCH.org

Phase Three

GRANTEES: Use CYFERnetSEARCH.org to create basic site-level reports of common measure data CYFAR PDTA Team: Use common measure data to provide aggregate-level analysis of CYFAR initiative

Phase Four

Questions?

cyfarpdta@umn.edu

Kyle R. Hawkey krhawkey@umn.edu