Investing in the Future: Evaluating the Kentucky College Coaching - - PowerPoint PPT Presentation
Investing in the Future: Evaluating the Kentucky College Coaching Program 2012-2014 Dr. Nicolas Antony Valcik, Director Dr. Kimberly Scruton, Professor Office of Institutional Research of Business Management West Virginia University Dean
Dean Business Administration, Marketing, and Management Methodist University
Office of Institutional Research West Virginia University
AmeriCorps funds paid for this assessment and report.
AmeriCorps funds paid for this assessment and report.
The authors would like to extend our thanks for all of the support given to us by the personnel at the Kentucky Campus Compact, Berea College, the Kentucky Higher Education Assistance Authority, and the Kentucky Department of Education Office of Career and Technical Education in compiling data when we requested it and putting the information in a usable format. The authors would also like to thank the Kentucky College Coach program participants who were interviewed for this project, who took time and effort out of their schedules to provide the authors with valuable qualitative information. The authors were impressed by the professionalism of all of the administrators of the Kentucky College Coach program. Without the support of all of these individuals, performing an accurate assessment of the Kentucky College Coach program would not have been possible. AmeriCorps funded this external assessment.
AmeriCorps funds paid for this assessment and report.
This assessment is to determine the state of the KCC program as well as delve deeper into characteristics that make the program successful with the “ignored cohort” that the evaluators discussed in the last assessment report. To restate the purpose of this assessment, the program evaluators were contracted through the Kentucky Campus Compact (paid by AmeriCorps) to provide an evaluation of the KCC program and to assess whether the KCC program has been effective in achieving its three main program goals:
AmeriCorps funds paid for this assessment and report.
The evaluation of the first two goals uses Quantitative data from KCC and KHEAA. 1) First goal measured: Increasing high school graduation rates 2) Second goal measured: The rate of high school students going to postsecondary education
The evaluation of the third goal uses quantitative and qualitative data collected from a variety of sources. 3) Third goal measured: Building of a college-going culture in the Commonwealth of Kentucky. The data collected and provided has allowed the program evaluators to determine the effectiveness and impact of the KCC program.
AmeriCorps funds paid for this assessment and report.
Kentucky Campus Compact (KCC) Berea College (2011 – 2014) Office of Career and Technical Education (OCTE) (2011 – 2013) Kentucky Higher Education Assistance Authority (KHEAA)
AmeriCorps funds paid for this assessment and report.
1) Qualitative Methodology 2) Quantitative Methodology 3) Spatial Data Analysis – Added for the 2012 – 2014 Evaluation 4) Site Visits – Added for the 2012 – 2014 Evaluation
AmeriCorps funds paid for this assessment and report.
The empirical portion of this study uses a quasi-experimental design to estimate the causal impact of the KCC intervention
programs.
AmeriCorps funds paid for this assessment and report.
determine comparability of the program and control group.
the KCC program and comparison group during the follow-up school year.
and ATC’s — and compared the results, since the main goal of this study was to capture as much detail as possible about the program and its participants.
AmeriCorps funds paid for this assessment and report.
regression analysis was applied.
students graduating from targeted high schools during the 2012-2014 school years after controlling for confounding variables.
mentoring 63.7% were female and 36.3% were male.
Of those, 61.3% were female and 38.7% were male.
AmeriCorps funds paid for this assessment and report.
AmeriCorps funds paid for this assessment and report.
Ethnicity Breakdown of Total Population
Indian and Pacific Islander/Native Hawaiian (2.5%).
were black, while the remaining 0.8% were Asian, American Indian and Pacific Islander/Native Hawaiian.
AmeriCorps funds paid for this assessment and report.
FRPL (free- and reduced-price lunch program) 87.9% of the mentored students also participated in free-and reduced- price lunch program (FRPL), while 80.4% of non-mentored students participated in FRPL program.
AmeriCorps funds paid for this assessment and report.
Mother’s Education
AmeriCorps funds paid for this assessment and report.
Father’s Education
AmeriCorps funds paid for this assessment and report.
Social Living Arrangement for Students
AmeriCorps funds paid for this assessment and report.
Individualized Mentoring
41.7%.
comprise only 0.79% as compared to 4.54% previous year.
individual in 2014. Furthermore, only two individuals had responded to local disasters in 2013 as compared to none in 2014.
AmeriCorps funds paid for this assessment and report.
Individualized/Group Mentoring and Persistence to Enroll
dependent on the individualized mentoring.
than that of not individually mentored (12.9%).
the college and those who did not, t(605.506) = 4.084, p = 0.000.
(28.3%) than students who did not meet with a mentor in a group (24.1%).
AmeriCorps funds paid for this assessment and report.
Effect of Individualized Mentoring on Student’s Success
success.
college.
12.4% students who did not receive individualized mentoring.
significantly greater individualized mentoring time (M = 8.21, SD=20.96) than students with unsuccessful application (M = 3.77, SD=14.42), t(563.085) = 4.058, P = 0.00 < 0.001. These results suggest that students receiving more individualized mentoring are more likely to have successful college application.
AmeriCorps funds paid for this assessment and report.
Effect of Group Mentoring on Student’s Success
received a significantly greater group mentoring time (M = 3.21, SD=6.05) than students with unsuccessful college application (M = 2.13, SD=4.40), t(579.899) = 3.391, P = 0.00 < 0.001.
successful college application.
AmeriCorps funds paid for this assessment and report.
Postsecondary Enrollment Seniors 2013 N Fall 2013 Spring 2014 Summer 2014 Fall 2014 Spring 2015 Summer 2015 Coached (38 Schools) 866 582 529 71 456 407 54 100% 67.2% 61.1% 8.2% 52.7% 47% 6.2% Non- Coached 6,365 3,491 3,225 585 2,904 2,604 412 100% 54.8% 50.7% 9.2% 45.6% 40.9% 6.5% 12.4% 10.4%
7.0% 6.1%
AmeriCorps funds paid for this assessment and report.
Postsecondary Enrollment Seniors 2014 N Fall 2013 Spring 2014 Summer 2014 Fall 2014 Spring 2015 Summer 2015 Coached (38 Schools) 448
294 246 26 100%
65.6% 54.9% 5.8% Non- Coached 4,938
2,729 2,481 361 100%
55.3% 50.2% 7.3%
10.4% 4.7%
AmeriCorps funds paid for this assessment and report.
Geospatial Analysis
seniors from 2012 to 2014.
same zip code in order to see if there were any patterns within the data that could not be seen by a traditional data analysis methodology.
AmeriCorps funds paid for this assessment and report.
In this GIS Map, Non-College Coached students throughout the state are clustered in densely populated in zip codes of major metropolitan areas (e.g. Frankfort). In comparison, KCC students for 2012 are served in both rural and urban communities.
AmeriCorps funds paid for this assessment and report.
In this GIS map, KCC students for 2012 are served in both rural and urban communities.
AmeriCorps funds paid for this assessment and report.
AmeriCorps funds paid for this assessment and report.
there are a few zip code areas of KCC students that did not have many going to college after their senior year in high school.
zip codes were over 60% that ended up attending college.
the Commonwealth in particular appeared to benefit from the program since all but a couple of zip codes were over 20% with the majority of zip codes showing a 61% to 100% college attendance rate.
AmeriCorps funds paid for this assessment and report.
number of communities sending their children to college is higher if their children are participating in the KCC program.
the KCC program that in fact went successfully into a technical or vocational trade after high school, which is also a goal of the program.
2014 with the non-participation of the ATC schools for that year.
AmeriCorps funds paid for this assessment and report.
In the 2013 GIS Maps, the population of KCC program students have shifted completely away from the Western part
Eastern zip codes of the
throughout the Commonwealth had between 0% to 20% college attending rates from KCC program students, which would suggest that areas of the Commonwealth that received KCC programs benefited from the positive intervention.
AmeriCorps funds paid for this assessment and report.
Commonwealth had between 0% to 20% college attending rates from KCC program students, which would suggest that areas of the Commonwealth that received KCC programs benefited from the positive intervention.
AmeriCorps funds paid for this assessment and report.
AmeriCorps funds paid for this assessment and report.
AmeriCorps funds paid for this assessment and report.
AmeriCorps funds paid for this assessment and report.
AmeriCorps funds paid for this assessment and report.
students have an increased percentage chance of going to college over non- college coached students.
available to tie in house hold income by zip code to perform a more definitive analysis to see if potential affluence of a student could be correlated to whether or not a student was more likely to go to college.
be an area that should be explored further in future studies.
AmeriCorps funds paid for this assessment and report.
how the program geographically has shifted from away from the Southern and Eastern parts of the Commonwealth presumably due to the discontinuation of the OCTE’s with the KCC program.
Commonwealth barren of participation in the KCC program for the 2014 year.
AmeriCorps funds paid for this assessment and report.
program, the KCC program for 2014 does not have any activity in most of the Southern and Western parts of the Commonwealth.
the KCC program, this clearly leaves large parts of the Commonwealth barren of students being advantage by such a program.
AmeriCorps funds paid for this assessment and report.
AmeriCorps funds paid for this assessment and report.
most of the zip code areas that had a college coaching program where students participated had percentages that were over 40% for their college attendance rates.
AmeriCorps funds paid for this assessment and report.
coordinators should investigate why rates were under 60% in those areas of the Commonwealth that participated in the KCC program in 2012 to 2014.
by percentage favors the KCC students over the non-college coached students
AmeriCorps funds paid for this assessment and report.
Financial Breakdown of KCC Coach Program – 2012 -2013
2012 – 2013 Berea College (12 Schools) KHEAA (30 Schools – plus 6 OTCE Schools) KCC Program Oversight Average $ Per student cost for coaches (2012 - 2013 financial data and number of schools) $359.45 (527 students for 12 schools in 2012 – 2013) $196.63 (2,004 students for 36 schools in 2012 - 2013) Not Applicable $ Per school cost for coaches (Less administrative costs included coaches’ training) $17,221 $14,594.44 $130,830 Scale up costs (Approximate estimate per school per administrative agency) $0 $15,683.33 N/A Administrative costs (fixed costs) $0 $29,400 $175,070 $ Per sponsor (Less administrative costs includes coaches training) $189,432 $394,050 (Murray State University subsidized 1 KHEAA school) $116,428 Total costs per administrative agency $189,432 $423,450 $305,900
Table 50
Financial Analysis – Costs of Coaches per High School and Student
not have ATC’s participating) and $918,782 (FY2013) to operate the three entities, is consistent from the previous year’s evaluation report.
dollars earned since the program is still very new and cohorts have not had time to graduate from college, much less enter the workforce.
the college coaching program and entered the workforce should allow for a calculation of a return on investment.
it will be necessary to grow the program by including additional high schools, which at some point will require an increase in administrative costs to provide proper oversight of the program without
AmeriCorps funds paid for this assessment and report.
Financial Analysis – Costs of Coaches per High School and Student
(Continued)
Financial Breakdown of KCC Coach Program – 2013 - 2014
2013 – 2014 Berea College (5 Schools) KHEAA (30 Schools plus 2 OTCE schools) KCC Program Oversight Average $ Per student cost for coaches (2013 - 2014 financial data and number of schools) $147.11 (320 students for 5 schools in 2013 -2014) $76.93 (5,504 students for 32 schools in 2013 - 2014) Not Applicable $ Per school cost for coaches (Less administrative costs included coaches’ training) $15,692.66 $14,115 $61,102 Scale up costs (Approximate estimate per school per administrative agency for coaches only) $0 $0 N/A Administrative costs (fixed costs) $0 $0 $202,797 $ Per sponsor (Less administrative costs includes coaches training) $47,078 $423,450 $0 Total costs per administrative agency $47,078 $423,450 $263,899
Table 51
difference in cost per student for each of the administrative
$359.45 while KHEAA averaged $196.63 per student.
while KHEAA averaged $76.93 per student.
KHEAA involvement in the KCC program rather than any implication that the KHEAA program appears to be more economical.
AmeriCorps funds paid for this assessment and report.
Discussion Points on the Three Goals Set Forth for the KCC Program to Accomplish
between students that were in the program versus students who did not participate in the program.
higher rate and create a “college-going culture” in Kentucky, the program has actually accomplished more than the original goals intended.
undergraduate and graduate programs in Kentucky, which impacts Kentucky residents after postsecondary education.
group, providing them with assistance within the rules, which permit them to do so.
program be funded by the state should federal funding become unavailable? Based on the evidence that the program evaluators have collected and analyzed, the KCC program is scalable and should be successful if administered and implemented in a very controlled manner.
AmeriCorps funds paid for this assessment and report.
Conclusion
Commonwealth of Kentucky.
increase the number of schools hosting college coaches.
Commonwealth of Kentucky, which bodes well for the overall future of the program.
surrounding communities, which is a very positive sign. Parental statements such as, “Enjoy your senior year, it’s the best year of your life” can cause high school seniors to delay applying to college and entering in the following fall semester, which creates a gap of continuous attendance in a school environment. Changing the perception of parents that have not pursued an education after high school will take generations of college coaches to alter within the Commonwealth of Kentucky. Without programs such as the KCC college coaches program, the chance of altering the perception of those parents is very low.
AmeriCorps funds paid for this assessment and report.
students spent together in a relationship, either individually or in a group, was also related to an increase in students’ enrollment and successful college application.
individualized mentoring.
mentoring.
enrollment and successful college application.
variables are held at constant.
Conclusion
(Continued)
AmeriCorps funds paid for this assessment and report.
AmeriCorps funds paid for this assessment and report.