STARTING WITH THE END IN MIND How evaluation can support you in - - PowerPoint PPT Presentation
STARTING WITH THE END IN MIND How evaluation can support you in achieving program goals October 23, 2017 Sophia Mansori Sheila Rodriguez Gabriela Garcia Todays agenda Introduce our evaluation team 1 Review evaluation activities and
Sophia Mansori Sheila Rodriguez Gabriela Garcia
2
21st CCLC Fall Program Director’s Meeting | edc.org
1 2 3
3
21st CCLC Fall Program Director’s Meeting | edc.org
4
21st CCLC Fall Program Director’s Meeting | edc.org
5
21st CCLC Fall Program Director’s Meeting | edc.org
v Administer and support data collection from grantees (local evaluation report, annual survey) v Conduct site visits for evaluation studies v Provide annual reports on grantee activities and progress v Provide technical assistance on evaluation topics ü Submit an annual local evaluation report for each grant (Due Dec 1, 2017) ü Submit an annual survey for each grant (Spring 2018) ü Submit APR data and student roster information to the Benchmarking Tool ü Participate in site visits EDC Activities Grantee Expectations
21st CCLC Fall Program Director’s Meeting | edc.org
6
21st CCLC Fall Program Director’s Meeting | edc.org 7
8
21st CCLC Fall Program Director’s Meeting | edc.org
9
21st CCLC Fall Program Director’s Meeting | edc.org
10
21st CCLC Fall Program Director’s Meeting | edc.org
INPUTS ACTIVITIES OUTPUTS SHORT TERM OUTCOMES LONG TERM OUTCOMES
ISBE Federal Funding 21st CCLC program guidelines Statewide objectives Technical assistance provider Professional development and training in State Standards and NCLB Staff who are knowledgeable about the 21st CCLC program and issues in afterschool Provide technical assistance Offer professional development Funding Reviewing grants and approving program plans Provide opportunities for grantees and ISBE to interact (e.g., site visits, spring conferences, regular communication) Reporting and feedback to Department of Education, State of Illinois, and CCLC programs Monitoring State-wide evaluation # Trained professionals in afterschool programming # Networked professionals in afterschool programing Guidelines and examples of 21st CCLC programs in the State of Illinois # Programs serving students with the greatest need in all parts of state Data on grantee activities State-wide program evaluation report 21st CCLC programs develop plans for sustainability 10% or more increase in academic achievement for students who participate in 21st CCLC programs 21st CCLC program staff increase understanding of key areas of the statewide objectives 21st CCLC program staff have increased professional development opportunities Increase in staff trained to meet the needs of 21st CCLC programs, staff, and students Increase in program opportunities for students who are in the greatest of need ISBE increases understanding of program activities, successes and challenges Long term sustainability of CCLC programs via funding and resources Strong partnerships for families, schools, and communities Program staff, families, and communities increase awareness and knowledge of 21st CCLC and afterschool programs, statewide
Students who participate in 21st CCLC’s graduate from high school Students who participate in 21st CCLC’s apply to and enter college Students who participate in 21st CCLC’s are prepared for post secondary and/or career readiness Students who participate in 21st CCLC’s and their families demonstrate a strong knowledge base in the common core state standards and Illinois learning standards Students who participate in 21st CCLC’s demonstrate positive behavior and strong socio-emotional skills Grantees Funding Educator expertise Individual program guidelines and goals Localized resources Professional development and training Professional development Activities that strengthen the academic and social skills of students Activities that develop college and career readiness Activities that address behavior and socio-emotional skills Activities that encourage family and community involvement Local Evaluation # of staff who receive professional development # and dosage of activities that strengthen academics, college and career readiness, socio-emotional skills # Students who participate in activities that strengthen academics, college and career readiness, socio- emotional skills # of activities offered by programs that engage families and community # of families and community members involved in activities Reported increases in school attendance rates for students Students improve attitude towards school Students engage in and demonstrate positive behavior Students improve academic achievement Students develop college and career readiness skills Families and community members increase engagement in program activities and develop relevant skills and knowledge
EXTERNAL CONTEXTUAL FACTORS
11
Objective Evaluation Questions Spring Survey Local evaluation Site Visits APR data Objective #1: Participants will demonstrate an increased involvement in school activities and participation in other subject areas such as technology, arts, music, theater, sports and other activities. Do 21st CCLC programs provide opportunities for participants to increase participation in activities and subjects such as technology, the theatre and arts, and extracurricular activities such as sports and clubs? In what ways? For whom? X X X To what extent do program participants increase participation in activities and subjects such as technology, the theatre and arts, and extracurricular activities such as sports and clubs? X X X X Objective #2: Participants in the program will demonstrate increased academic achievement. To what extent do program goals and activities address and support increased academic achievement for program participants? X X X Have 21st CCLC program activities and services positively influenced student achievement outcomes (i.e., increased student test scores, grade promotion rates)? X X
21st CCLC Fall Program Director’s Meeting | edc.org
12
What Why When Who How What are the kinds of information and data that will help you understand this goal? Why are you collecting these data? What will they tell you? When is the best
appropriate time to collect this information? When do you need data in
decisions? Who needs to be included in the process of both collecting and interpreting data? How will you use these data? What processes are there to incorporate findings and make program changes?
21st CCLC Fall Program Director’s Meeting | edc.org
Evaluation use is what happens when you don’t just put a report in a file cabinet...
Inform parents
benefit from
Let funders know the extent of your services. Share student
school day teachers. Discuss results with staff, including challenges they face and what worked well. Communicate with Partners about program activities and connections.
21st CCLC Fall Program Director’s Meeting | edc.org
13
14
21st CCLC Fall Program Director’s Meeting | edc.org
Issue Data Response Program wanted to understand effectiveness and participant
§ 59% of participants who attended 60+ days made gains in at least one subject area. § 30% of participants attended 60 days or more Ø Recommendation in evaluation: Develop a retention plan. Ø Communicate with parents about the benefits of regular participation. Ø Engage staff in developing retention strategies Ø Collect more data to understand barriers to regular attendance
15
21st CCLC Fall Program Director’s Meeting | edc.org
Identify a program area to improve or better understand outcomes (e.g. student retention, behavior/discipline, achievement in math, parent involvement). What data do you currently have about this activity or outcome? What additional data do you need? How can you use these data? What processes do you have to use your findings and results? Share the program area you would like to improve. Share the data you currently and need to collect. What data do others at your table collect related to this area? What strategies have others used to address this topic? How have they used evaluation data to inform their work in this area? With your colleague… With your table… 1 2 3 1 2 3
16
21st CCLC Fall Program Director’s Meeting | edc.org