Theory of Action and Results. Mara Vernica Santelices 1 Ximena Cataln - - PowerPoint PPT Presentation

▶
theory of action and results
SMART_READER_LITE
LIVE PREVIEW

Theory of Action and Results. Mara Vernica Santelices 1 Ximena Cataln - - PowerPoint PPT Presentation

Oct 17, 2023 236 likes •750 views

Institutional Admissions Programs Searching for Equity in Chiles Higher Education: Theory of Action and Results. Mara Vernica Santelices 1 Ximena Cataln 1 Catherine Horn 2 1 Pontificia Universidad Catlica de Chile 2 University of Houston

slide-1
SLIDE 1

1

Institutional Admissions Programs Searching for Equity in Chile´s Higher Education: Theory of Action and Results.

María Verónica Santelices1 Ximena Catalán1 Catherine Horn2

1Pontificia Universidad Católica de Chile

2University of Houston

This research was funded in part by Chile´s Ministry of Education through research Project F811363 and by Conicyt through Project ANILLO SOC1107.

slide-2
SLIDE 2

2

Introduction

  • Admissions criteria to higher education institutions have been

traditionally defined by their ability to predict students’ future academic performance. This has been one of the justifications for the use of standardized admission tests (in Chile, the University Selection Test or PSU).

  • The documentation of the benefits of diversity in the student

body (e.g., Deo, 2012; Hurtado, 2003; Milem, 2003; Milem & Hakuta, 2000) and the important gaps observed in performance on standardized test have mobilized interest in a broader set of admission criteria both at the national and institutional level.

  • Goal of this study: Contribute to the understanding of

institutional college admission programs implemented in some of the most selective universities in Chile with the goal to increase equity in access to higher education, focusing on their expected effects and observed effects in their respective institutions.

slide-3
SLIDE 3

3

Background

  • Chile: 60 universities (25 traditional universities), 44

Professional Institutes and 58 Centers for Technical

  • Education. In 2014 the total enrollment in higher education

was 1,144,299: 56,4% in Universities, 30,7% in IPs and 12,9% in CFTs.

  • Students in higher education increased significantly in

Chile during the last 25 years: 2014 enrollment was 39,3% (18-24 years old); was 13% in 1990.

  • Increased Financial Aid at the National and Institutional

level

  • There are significant differences on access by SES: net

rate of higher education access (18-24 years old) in the highest income quintile education is 59%, while in the lowest income quintile is only 22% + most high income students attend universities (accredited), while lower income students attend technical institutions.

  • Admissions indicators at the system level and at the

institution level: shifts from processes based mainly on standardized test to broader set of considerations.

slide-4
SLIDE 4

4

Literature review

  • Although there are studies on the effectiveness of institutional

admission programs in Chile, these are fragmented by institution (and major).

  • Studies show mixed results in academic performance of student

beneficiaries (Koljatic & Silva, 2012; Talent and Inclusion, 2012;. Castro et al, 2012; Devés, Castro, Mora & Roco, 2012; Trevino, Scheele & Flores, 2014).

  • There are not comprehensive and cross-sectional studies

and none has considered the program theory of action

  • International research shows that in the US state initiatives

promoting the inclusion of disadvantaged groups (outreach, percent plans) may have contributed to increased rates of admission of minority groups in selective (and not selective) public institutions, but there are not conclusive results regarding subsequent academic performance.

slide-5
SLIDE 5

5

Research Questions

1) To investigate the expected effects of institutional programs through the programs' theory of action. 2) To assess the effectiveness of institutional admission programs through: 2.1) The study the evolution of the proportion of disadvantaged groups enrolled in the universities under study, 2004 to 2013 2.2) The comparison of the grade point average and the persistence (academic performance) of benefiting students with comparable peers at the same institution.

slide-6
SLIDE 6

6

Methodology: main features

  • Sample: Catholic University of Chile, University of Santiago and

University of Chile.

  • Population: all institutions that are implementing institutional

admission programs with equity focus, approx. 21.

  • Mixed methods

– Qualitative methodology explored the main program features, theory of action and implementation: document analysis and interviews with key stakeholders (n = 16). – Quantitative analysis explored the effects on the admission rates of disadvantaged students and their subsequent academic achievement using application, admission and academic performance information for the relevant period.

slide-7
SLIDE 7

7

Research Questions

1) To investigate the expected effects of institutional programs through the programs' theory of action. 2) To assess the effectiveness of institutional admission programs we: 2.1) Study the evolution of the proportion of disadvantaged groups enrolled in the under study, 2004 to 2013 2.2) To compare the average grade point average and the persistence of students benefiting with comparable peers by institution.

slide-8
SLIDE 8

8

Teory of action-Propaedeutic (Univ of Santiago)

Components Desired Institutional EffectsEfectos Admission

Stregthen the relationship between the institution and society, in the context of a public University

Activities

Selection based on:

  • PSU scores
  • 100% attendance to the Propedeutic activities.
  • Commitment Letter from parents and students.
  • Grades of 4.0 or higher in all Propedeutic classes

(scales goes from 1 to 7).

  • Rank in the top 5% of the class ordered by Liberal

Arts program weighted application score (60% grade point average from the first three years of high school, 30% senior year grade point average, 10% grade point average from Propedeutic program).

Desired System-level Consequences

Increase access of talented youth who usually are not represented higher education (coming from disadvantage d background)

Retention

Financial Support: a) fellowship covering paid by USCH (solo pagan matrícula semestral); b) complementary funding from municipalities. Contribute to increase equity in access to higher education in Chile

Recruitment

Preliminary contact with schools and recruitment through the web page, Propedeutic programs network webpage, twitter and facebook. Academic support (remedation, tutoring) Non-focalized psychological support Motivate the persistence of program participants Selection into Propedeutic Program based on: a) high school seniors; b) grade point average from first three years of high school in top 10% of their class; c) since freshmen year in the same school d) from vulnerable schools, defined a priori. Improve the educational experience of all university students.

Ensure the quality of education through diversity.

Ensure the educational quality

  • f the institution

through the inclusion of talented low income students Increase educational expectations of secondary students Contribute to improve secondary education quality

slide-9
SLIDE 9

9

Expected goals-Institution level

Level

Objectives

Talent & Inclusion Prope- déutico SIPEE

I N S T I T U T I O N

  • 1. ¡Increase ¡the ¡access ¡of ¡disadvantaged ¡talented ¡

students ¡ ¡

  • 2. ¡Improve ¡ins8tu8onal ¡academic ¡experience ¡

and ¡educa8on ¡quality ¡ ¡through ¡diversity ¡ ¡

  • 3. ¡Improve ¡ins8tu8onal ¡academic ¡excellence ¡

through ¡the ¡access ¡of ¡talented ¡students ¡ ¡

  • 4. ¡Ensure ¡the ¡persistence ¡of ¡ ¡benefited ¡students ¡ ¡
  • 5. ¡Ensure ¡job ¡prospects ¡of ¡benefited ¡students ¡ ¡
  • 6. ¡Strengthen ¡rela8onship ¡between ¡ins8tu8on ¡

and ¡society ¡

  • 7. ¡Mo8vate ¡an ¡ins8tu8onal ¡ ¡reform ¡(support ¡for ¡

all ¡needy ¡students) ¡ ¡

slide-10
SLIDE 10

10

Expected goals-System level

Level Objec8ves ¡

Talent & Inclusion Prope- déutico SIPEE

S Y S T E M

  • 1. ¡Increase ¡of ¡educa8onal ¡expecta8ons ¡in ¡

high ¡schools ¡ ¡

  • 2. ¡Contribute ¡to ¡improve ¡secondary ¡

educa8on ¡quality ¡

  • 3. ¡Contribute ¡to ¡a ¡more ¡equitable ¡access ¡

to ¡higher ¡educa8on ¡ ¡

  • 4. ¡Contribute ¡to ¡a ¡more ¡democra8c ¡

society ¡

  • 5. ¡Provide ¡society ¡with ¡more ¡and ¡beMer ¡

professionals ¡ ¡

slide-11
SLIDE 11

11

Program Components

Component

Activities Talent & Inclusion Prope- déutico SIPEE

Recruitment

Focaliza8on ¡ ¡ Previous ¡academic ¡college-­‑related ¡ ac8vi8es ¡with ¡schools ¡ ¡

Admissions

HS ¡achievement ¡criteria ¡ ¡ Standardized ¡admission ¡test ¡achievement ¡ ¡ School ¡socioeconomic ¡criteria ¡ ¡ Specific ¡criteria ¡by ¡academic ¡program ¡ ¡

Retention

Ins8tu8onal ¡efforts ¡to ¡provide ¡financial ¡ support ¡ ¡ Ins8tu8onal ¡efforts ¡to ¡provide ¡academic ¡ support ¡ ¡ Ins8tu8onal ¡efforts ¡to ¡provide ¡ psychological ¡support ¡ ¡

slide-12
SLIDE 12

12

Research Questions

1) To investigate the expected effects of institutional programs through the programs' theory of action. 2) To assess the effectiveness of institutional admission programs through: 2.1) The study the evolution of the proportion of disadvantaged groups enrolled in the universities under study, 2004 to 2013 2.2) To compare the average grade point average and the persistence of students benefiting with comparable peers by institution.

slide-13
SLIDE 13

13

Results – Proportion of Low SES Enrollment

Evolution of low-SES students enrolled in first year during the period 2004-2013 (Univ of Santiago) Evolution of low-SES students enrolled in first year during the period 2004-2013 (University of Chile) Evolution of low-SES students enrolled in first year during the period 2004-2013 (Catholic University of Chile)

Blue=monthly family income below US$1,200 Red= public subsidized schools Green=mother´s education less than 4-yr college completed.

slide-14
SLIDE 14

14

Research Questions

1) To know the expected effects of institutional programs through deepening its main characteristics and theories of action. 2) To evaluate the effectiveness of institutional admission programs : 2.1) To study the evolution over the period 2004 to 2013

  • f the proportion of disadvantaged groups enrolled in

the universities that are the focus of this research 2.2) The comparison of the grade point average and the persistence (academic performance) of benefiting students with comparable peers at the same institution.

slide-15
SLIDE 15

15

  • Std. GPA -Propaedeutico (Test Score Control Group)

cohort year 1 year 2 year 3 year 4 year 5 year 6 PU No PU PU No PU PU No PU PU No PU PU No PU PU No PU 2008 Prom PPAN Z

  • 0.919

0.167

  • 0.586
  • 0.001
  • 0.258

0.012

  • 0.379
  • 0.222
  • 0.331
  • 0.106
  • 0.352
  • 0.048

N students 24 27 23 25 20 101 21 239 20 353 18 384 N majors 1 1 1 1 6 6 12 12 12 12 12 12 Delta(NPU-PU) 1.086 0.585 0.270 0.157 0.224 0.304 p value (NPU-PU) 1.000 0.955 0.728 0.634 0.686 0.758 2009 Prom PPAN Z

  • 1.030

0.245

  • 0.616

0.229

  • 0.983

0.041

  • 0.330

0.084

  • 0.633
  • 0.050

N students 30 49 28 46 25 159 24 358 23 593 N majors 1 1 1 1 6 6 11 11 20 20 Delta(NPU-PU) 1.275 0.846 1.024 0.415 0.583 p value (NPU-PU) 1.000 0.999 0.899 0.927 0.976 2010 Prom PPAN Z

  • 1.034

0.250

  • 0.543

0.185 0.048 0.053

  • 0.073
  • 0.041

N students 32 84 32 80 22 369 16 543 N majors 1 1 1 1 7 7 10 10 Delta(NPU-PU) 1.2836 0.7282 0.0046 0.0327 p value (NPU-PU) 1.000 0.999 0.509 0.555 2011 Prom PPAN Z

  • 0.887

0.198

  • 0.071

0.228 0.272

  • 0.060

N students 38 29 37 22 32 330 N majors 1 1 1 1 12 12 Delta(NPU-PU) 1.0850 0.2988

  • 0.3314

p value (NPU-PU) 1.000 0.899 0.081 2012 Prom PPAN Z

  • 0.779

0.170

  • 0.212
  • 0.084

N students 48 58 40 43 N majors 1 1 1 1 Delta(NPU-PU) 0.950 0.128 p value (NPU-PU) 0.999 0.693 2013 Prom PPAN Z

  • 0.491

0.124 N students 58 144 N majors 1 1 Delta(NPU-PU) 0.615 p value (NPU-PU) 1.000 PP1 PP2 PP3 PP4 PP5 PP6 Cohorts

  • verview

Delta(NPU-PU) 1.049 0.5171

  • 0.0551

0.4623 0.4483 0.3043 p value (NPU-PU) 1.000 0.997 0.967 0.965 0.965 0.758

slide-16
SLIDE 16

16

  • 1.0
  • 0.8
  • 0.6
  • 0.4
  • 0.2

0.0 0.2 0.4 Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Propedeutico Control (PSU)

  • 1.0
  • 0.8
  • 0.6
  • 0.4
  • 0.2

0.0 0.2 0.4 Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Propedeutico Control (Mother education)

Standardized Grade Point Average- Propaedeutico (All-Cohort Average)

slide-17
SLIDE 17

17

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Propedeutico Control (PSU) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Propedeutico Control (Mother education)

Persistence - Propaedeutico (All-Cohort Average)

slide-18
SLIDE 18

18

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Year 1 Year 2 Year 3 Year 4 Propedéutico Control (PSU)

Persistence - Propaedeutico (majors of transfer)- All Cohort Average

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Year 1 Year 2 Year 3 Year 4 Propedéutico Control (Mother education)

slide-19
SLIDE 19

19

  • The performance of Talent & Inclusion

students tends to be similar (sometimes higher) than their comparison groups, while students of the Propaedeutic show similar performance and sometimes lower than the comparison group.

  • Students entering via SIPEE to the University
  • f Chile are in an intermediate situation,

reflecting high volatility of SIPEE students` academic performance. Results on Academic Performance

slide-20
SLIDE 20

20

Propensity Matched Simple Comparisons

slide-21
SLIDE 21

21

Event History Analysis

slide-22
SLIDE 22

22

Discussion

  • The research on theories of actions of institutional programs

shows three main institutional objectives implemented through diverse activities clustered in three program components (Research Question 1):

Level

Program Goals

I N S T I T U T I O N

  • 1. ¡Increase ¡the ¡access ¡of ¡disadvantaged ¡

talented ¡students ¡ ¡

  • 2. ¡Improve ¡ins8tu8onal ¡academic ¡

experience ¡and ¡educa8on ¡quality ¡ ¡through ¡ diversity ¡ ¡

  • 4. ¡Ensure ¡the ¡persistence ¡of ¡ ¡benefited ¡

students ¡ ¡

Component

Recruitment Admissions Retention

slide-23
SLIDE 23

23

Discussion

  • The research on program effectiveness (Research Question 2)

shows:

Level

Program Goals

I N S T I T U T I O N

  • 1. ¡Increase ¡the ¡access ¡of ¡disadvantaged ¡

talented ¡students ¡ ¡

  • 2. ¡Improve ¡ins8tu8onal ¡academic ¡

experience ¡and ¡educa8on ¡quality ¡ ¡through ¡ diversity ¡ ¡

  • 4. ¡Ensure ¡the ¡persistence ¡of ¡ ¡benefited ¡

students ¡ ¡

Access Goal: sociodemographic composition at institutions does not change substantially pre and post program implementation. Programs may have helped maintain groups presence though (Research Question 2.1). Academic Success: analysis of grades and persistence suggests that students admitted through these programs perform on par with similar peers (Research Question 2.2).

slide-24
SLIDE 24

24

Limitations and Future Research

  • Sample: only CRUCH (traditional, publicly funded)

institutions, three selective institutions of RM, results are not generalizable.

  • Future studies: more admission programs could give a

fuller picture.

  • Focus on academic intra-institution comparison

because of data issues and in order to maintain comparability of groups.

  • Cannot talk about causality as methods used do not

control for rival hypotheses, specially student body composition.

  • Future studies: include voice of students, teachers

and administrators exploring changes in practices, curriculum, pedagogy and institutional climate.

slide-25
SLIDE 25

25

Thank you!

  • María Verónica Santelices, Ximena Catalán, Catherine Horn

Faculty of Education, Pontificia Universidad Católica de Chile and University of Houston. vsanteli@uc.cl, xrcatala@uc.cl, chorn@central.uh.edu

  • Acknowledgments: Fondo de Investigación y Desarrollo en

Educación del Ministerio de Educación (FONIDE)+Conicyt Project Anillo SOC1107

  • Related studies

– Santelices, M. V., Catalán, X., Horn, C., Venegas, A. (2015) High School Ranking in University Admissions at a National-Level: Simulation of Predictive Validity Results. Paper presented at the Annual Meeting of American Educational Association, April, Chicago, USA. – Santelices M.V., Catalán, X, Kruger, D. & Horn, C. Determinants of Persistence and the Role of Financial Aid: Lessons from Chile. Higher Education (forthcoming). doi: 10.1007/s10734-015-9906-6. – Horn, C., Santelices M.V., Catalán, X. (2014). Modeling the Impacts of National and Institutional Financial Aid Opportunities on Persistence at an Elite Chilean

  • University. Higher Education, 68, 471-488.
slide-26
SLIDE 26

26

Program Subsidized schools < USD 1.200 Mother <college PSU N

  • P. Universidad

Católica de Chile 2012 Talento Ingeniería 100,00% 82,93% 75,61% 657,7 41 Talento Derecho 100,00% 90,00% 100,00% 613,5 10 Talento Ing. Comercial 100,00% 60,00% 100,00% 658,2 5 Talento Arq, Diseño y College CN 80,00% 60,00% 40,00% 626,7 5 UC 32,40% 29,98% 39,31% 681,7 5.040 Universidad de Santiago 2013 Propedeútico UNESCO 100,00% 100,00% 87,93% 516,5 58 USACH 78,04 % 70,97% 68,31% 613,8 4.285 Universidad de Chile 2013 SIPEE Ingeniería 100,00% 100,00% 674,0 20 SIPEE Bioquímica 100,00% 100,00% 634,5 5 SIPEE Enfermería 100,00% 100,00% 623,6 5 SIPEE Nutrición y dietética 100,00% 100,00% 632,0 5 SIPEE Terapia ocupacional 100,00% 100,00% 632,4 5 SIPEE Bachillerato 100,00% 100,00% 614,5 9 SIPEE Arquitectura 100,00% 100,00% 622,3 11 SIPEE Adm. Pública 100,00% 100,00% 624,7 10 SIPEE Periodismo 100,00% 100,00% 628,9 6 SIPEE Antropología 100,00% 100,00% 650,8 7 SIPEE Ing. Comercial 100,00% 100,00% 670,7 26 SIPEE Kinesiología 100,00% 100,00% 644,8 4 SIPEE Derecho 100,00% 100,00% 679,8 10 SIPEE Psicología 100,00% 100,00% 635,2 14 SIPEE Ing. en Alimentos 100,00% 100,00% 604,6 2 SIPEE Diseño 100,00% 100,00% 605,6 1 SIPEE Tecnología Médica 100,00% 100,00% 635,5 8 SIPEE Medicina 100,00% 100,00% 684,0 10 SIPEE Historia 100,00% 100,00% 623,4 6 SIPEE Sociología 100,00% 100,00% 626,9 6 SIPEE Obstetricia 100,00% 100,00% 617,7 5 SIPEE Leng. y Literatura 100,00% 100,00% 625,7 2 SIPEE Geografía 100,00% 100,00% 625,8 1 SIPEE Qca. y Farmacia 100,00% 100,00% 619,1 5 SIPEE Odontología 100,00% 100,00% 672,4 5 SIPEE Cine y Tv. 100,00% 100,00% 618,3 1 SIPEE Fonoaudiología 100,00% 100,00% 624,0 5 SIPEE Ing. en RRNN 100,00% 100,00% 611,4 2 SIPEE Ing. en Información 100,00% 100,00% 658,1 2 Universidad de Chile 60,13% 53,30% 692,7 5493

Students characteristics

slide-27
SLIDE 27

27

Theory of action- Talent + Inclusion (Catholic University)

P e r s i s t e n c e / Graduation / Job Insertion Database with school information+massive mailing, visits of ofrmer students, meeting with counselors, through precollege programs (preuniversitario), college fairs+posters and fliers.

Desired Systemic Consequences Desired Institutional Effects Components Activities

Admission (by increasing the number of spots) Recruiting Defining eligible students:

  • Grades in top 10% of his/her class
  • Attended public or partly subsidized schools
  • Be within the first four lower income quintiles.
  • Apply to the Ministry´s financial aid programs.

Encourage the persistence of participating students through a good academic experience. Bring in talented students not represented at the Catholic University (to ensure the access of meritorious students regardless of socioeconomic status) Increase the presence of youngsters not usually present at Catholic University who have shown to have an outstanding performance considering their context. Ensure Academic Excellence through the diversity that the inclusion of talented students from deprived socioeconomic contexts generates. Increase educational e x p e c t a t i o n s o f secondary students (and through them, the quality

  • f secondary education)

Application: Pre-registration/application(personal information through a special website) + Special tests and interviews. Selection of students based:

  • Achievement based on special tests /interviews
  • PSU test

Additional financial aid to complement Ministry´s fellowships for tuition payment (only available for students with Ministry´s fellowship). Differential Strategies by academic program, such as:

  • Remedial programs (summer camp).
  • Early Diagnostic test
  • One-in-one class and tutorial.
  • Special classes and excercise sessions.
  • Early psicological interviewes for diagnosis.
slide-28
SLIDE 28

28

Contribute to a more equitable access to higher education in Chile

Theory of action – SIPEE (Univ of Chile)

Permit the acces

  • f young

meritorious students who come from a deprived context Admission (by increasing the number of spots) Final student selection, according to hierarchical level: 1° Based on Vulnerability Index, from highest to lowest. 2° Based in income quintile, from lowest to highest. 3° Based on grade point average´s ranking position, giving preference to those in the top 10% Recruitment Massive mailings (64.009 students, 836 schools). College Fair (300 als.). School Visits (98 schools in Metropolitan Region) Preselection of eligible students:

  • Income Quintiles I, II y III
  • Secondary education in public schools
  • Academic excellence in school (NEM above 5,5) and

PSU (above 600 or 650 depending on career) Academic Support for students: personalized classes by tutors. Allow for the diversity experience of students and académics Increase the number of meritorious students coming from deprived contexts. Ensure the persistence and social integration

  • f SIPEE

students, and promote an appropriate job insertion Improve Quality of Institutional Education Motivate an Institutional Reform that

  • ffers support

to all students who meet the SIPEE´s profile Raise school quality through the increase of educational expectations in high schools where SIPEE students come from. Contribute to build a more democratic and fair society, based on equal

  • pportunities for all

Components Desired Institutional Effects Activities Desired System-level Consequences

slide-29
SLIDE 29

29 cohort year 1 year 2 year 3 year 4 year 5 year 6 PU No PU PU No PU PU No PU PU No PU PU No PU PU No PU 2008 Prom PPAN Z

  • 0.96

0.17

  • 0.63

0.07

  • 0.26

0.02

  • 0.04
  • 0.15
  • 0.33
  • 0.07
  • 0.11

0.02 N students 23 54 22 49 20 228 20 410 18 440 17 397 N majors 1 1 1 1 6 6 11 11 12 12 11 11 Delta(NPU-PU) 1.130 0.704 0.280

  • 0.110

0.262 0.125 p value (NPU-PU) 1.000 0.994 0.734 0.368 0.716 0.628 2009 Prom PPAN Z

  • 1.072

0.208

  • 0.678

0.152

  • 1.081

0.024

  • 0.416

0.127

  • 0.673

0.060 N students 29 74 27 71 24 180 23 360 22 653 N majors 1 1 1 1 5 5 10 10 19 19 Delta(NPU-PU) 1.280 0.830 1.105 0.543 0.7325 p value (NPU-PU) 1.000 0.999 0.870 0.962 0.990 2010 Prom PPAN Z

  • 0.998

0.194

  • 0.554

0.161 0.040 0.084

  • 0.073

0.032 N students 31 64 31 60 21 286 16 432 N majors 1 1 1 1 7 7 10 10 Delta(NPU-PU) 1.1929 0.7141 0.0439 0.1058 p value (NPU-PU) 1.000 0.998 0.583 0.671 2011 Prom PPAN Z

  • 0.908

0.278

  • 0.076

0.117 0.272 0.085 N students 37 59 36 45 32 478 N majors 1 1 1 1 12 12 Delta(NPU-PU) 1.1868 0.1926

  • 0.1867

p value (NPU-PU) 1.000 0.848 0.203 2012 Prom PPAN Z

  • 0.846

0.215

  • 0.082

0.084 N students 42 97 34 74 N majors 1 1 1 1 Delta(NPU-PU) 1.061 0.166 p value (NPU-PU) 1.000 0.525 2013 Prom PPAN Z

  • 0.044

0.110 N students 47 90 N majors 1 1 Delta(NPU-PU) 0.065 p value (NPU-PU) 0.525 PP1 PP2 PP3 PP4 PP5 PP6 Cohorts

  • verview

Delta(NPU-PU) 1.066 0.4882 0.0068 0.4209 0.5504 0.1252 p value (NPU-PU) 1.000 0.990 0.513 0.964 0.984 0.628

  • Std. GPA – Propedéutico (Mother Ed. Control Group)
slide-30
SLIDE 30

30

Cohort Year 1 year 2 year 3 year 4 year 5 year 6 PU NoPU PU NoPU PU NoPU PU NoPU PU NoPU PU NoPU 2008 70,6% 62,8% 67,6% 60,5% 55,9% 48,8% 55,9% 48,8% 52,9% 41,9% 44,1% 34,9% 24a 27a 23a 26a 19a 21a 19a 21a 18a 18a 15a 15a 2009 63,8% 81,7% 59,6% 78,3% 51,1% 78,3% 42,6% 70,0% 38,3% 60,0% 30a 49b 28a 47b 24a 47b 20a 42b 18a 36b 2010 68,1% 69,4% 68,1% 68,6% 44,7% 42,1% 36,2% 38,0% 32a 84a 32a 83a 21a 51a 17a 46a 2011 79,2% 87,9% 77,1% 75,8% 58,3% 69,7% 38a 29a 37a 25a 28a 23a 2012 87,3% 85,3% 72,7% 69,1% 48a 58a 40a 47a Cohort year 1 year 2 year 3 year 4 year 5 year 6 PU NoPU PU NoPU PU NoPU PU NoPU PU NoPU PU NoPU 2008 71,9% 78,3% 68,8% 73,9% 59,4% 60,9% 59,4% 59,4% 56,3% 50,7% 46,9% 40,6% 23a 54a 22a 51a 19a 42a 19a 41a 18a 35a 15a 28a 2009 67,4% 86,0% 62,8% 84,9% 53,5% 77,9% 44,2% 73,3% 39,5% 64,0% 29a 74b 27a 73b 23a 67b 19a 63b 17a 55b 2010 68,9% 82,1% 68,9% 80,8% 44,4% 43,6% 35,6% 39,7% 31a 64a 31a 63a 20a 34a 16a 31a 2011 84,1% 85,5% 81,8% 73,9% 63,6% 62,3% 37a 59a 36a 51a 28a 43a 2012 87,8% 90,7% 69,4% 78,5% 43a 97a 34a 84a

Persistence – Propedéutico (Bachillerato)

PSU comparison Mother education comparison

slide-31
SLIDE 31

31

Discussion

  • The research on theories of actions of institutional programs

shows two main institutional objectives:

– Greater access of young talented usually not included in these institutions and; – Permanence of these student in order to successfully complete their studies. – Access Objective: sociodemographic composition before and after program implementation of the programs indicates that it does not change substantially as a result of inclusion programs. Programs may have helped maintain groups presence. – Academic Success Objective: The analysis of academic performance (measured both by notes and persistence rates) suggests that students admitted through theses programs do not always have GPA or persistence rates similar to those of their peers with scores PSU or educational level mother / comparable income.

slide-32
SLIDE 32

32

Conclusion

  • Access is not enough to ensure a successful

experience in tertiary education. The results of this study show that students entering via these programs have GPAs and persistence rates, at best, similar to the comparison group cases.

  • The more dissimilar is the academic profile of

students admitted through the new programs, the worse seems to be the academic performance of the group benefited relative to their comparison group.

  • One way to try to match the academic profile of

students admitted by this new programs is to start the academic preparation before and do it with greater intensity.

slide-33
SLIDE 33

33

Institutional programs background

  • The three institutional programs differ in how different

these new groups are compared to their colleagues. – Propedéutico: important differences in academic preparation (- 100 PSU scores). University of Santiago, 2007. – Talent and Inclusion: greater differences observed in socioeconomic status. Catholic University of Chile, 2011. – SIPEE: the situation varies considerably by academic

  • program. University of Chile, 2010.
slide-34
SLIDE 34

34

Dicussion

  • When comparing the findings of this research with the

previous literature, consistencies are observed showing a lower performance in students enrolled in USACH through the Propedéutico. The discrepancies in relation to the magnitude of the differences probably could be explained by conceptual and methodological aspects .

  • Previous studies on the SIPEE are also consistent with this

research, generally showing slight differences in favor of the group of freshmen students entering through PSU test, showing variations by academic program, cohort and monitoring period.

  • National studies regarding Talent and Inclusion are

consistent with the findings of this investigation, particularly when investigations considered PSU scores as a way to control for other variables that might be affecting the academic performance of students benefited.

slide-35
SLIDE 35

35

  • Theories of action have a similar structure: components,

activities, intended systemic effects and institutional consequences.

  • The components refer to the areas where programs and

admission policy have influence, distinguishing the recruitment , admission and retention activities.

  • Institutional effects: program´s goals at the institutional

level

  • Systemic effects: effects that programs expect to achieve at

the society level.

Results: Theories of action

slide-36
SLIDE 36

36

Cohort year 1 year 2 year 3 year 4 SIPEE No SIPEE SIPEE No SIPEE SIPEE No SIPEE SIPEE No SIPEE 2010 Prom Z

  • 0,427
  • 0,427
  • 0,084
  • 0,463
  • 0,448
  • 0,399
  • 1,144

0,015 N students 20 6 19 6 19 6 18 5 N majors 2 2 2 2 2 2 2 2 Delta(NPU-PU) 0,045

  • 0,379

0,048 1,159 p value (NPU-PU) 0,633 0,350 0,515 0,919 2011 Prom Z

  • 0,337
  • 0,199
  • 0,445

0,003

  • 0,298

0,078 N students 35 26 32 23 31 24 N majors 3 3 3 3 3 3 Delta(NPU-PU) 0,138 0,448 0,376 p value (NPU-PU) 0,680 0,916 0,911 2012 Prom Z

  • 0,444
  • 0,233
  • 0,444
  • 0,186

N students 104 75 93 71 N majors 10 10 10 9 Delta(NPU-PU) 0,210 0,258 p value (NPU-PU) 0,715 0,761 2013 Prom Z

  • 0,446
  • 0,363

N students 198 154 N majors 29 29 Delta(NPU-PU) 0,084 p value (NPU-PU) 0,674 Cohorts

  • verview

Delta(NPU-PU) 0,094 0,207 0,245 1,159 p value (NPU-PU) 0,742 0,797 0,725 0,919

  • Std. GPA- SIPEE (PSU weighted score control )
slide-37
SLIDE 37

37

  • Std. GPA- SIPEE (Fam. Income control )

cohort year 1 year 2 year 3 year 4 SIPEE No SIPEE SIPEE No SIPEE SIPEE No SIPEE SIPEE No SIPEE 2010 Prom Z

  • 0,472
  • 0,651
  • 0,084
  • 0,132
  • 0,448
  • 0,501
  • 1,144
  • 0,407

N students 20 20 19 19 19 18 18 17 N majors 2 2 2 2 2 2 2 2 Delta(NPU-PU)

  • 0,179
  • 0,048
  • 0,054

0,736 p value (NPU-PU) 0,430 0,458 0,442 0,817 2011 Prom Z

  • 0,337

0,152

  • 0,445

0,479

  • 0,298
  • 0,025

N students 35 35 32 32 31 32 N majors 3 3 3 3 3 3 Delta(NPU-PU) 0,489 0,925 0,273 p value (NPU-PU) 0,948 0,957 0,801 2012 Prom Z

  • 0,444

0,304

  • 0,444

0,195 N students 104 104 93 96 N majors 10 10 10 9 Delta(NPU-PU) 0,748 0,639 p value (NPU-PU) 0,967 0,943 2013 Prom Z

  • 0,446
  • 0,048

N students 198 198 N majors 29 29 Delta(NPU-PU) 0,398 p value (NPU-PU) 0,995 Cohorts

  • verview

Delta(NPU-PU) 0,458 0,601 0,142 0,736 p value (NPU-PU) 0,999 0,985 0,730 0,817

slide-38
SLIDE 38

38

Cohort year 1 year 2 year 3 year 4 SIPEE No SIPEE SIPEE No SIPEE SIPEE No SIPEE SIPEE No SIPEE 2010 95,0% 100,0% 95,0% 100,0% 95,0% 100,0% 90,0% 83,3% 19a 6a 19a 6a 19a 6a 18a 5a 2011 94,3% 100,0% 88,6% 88,5% 85,7% 88,5% 33a 26a 31a 23a 30a 23a 2012 87,5% 84,0% 78,8% 80,0% 91a 63a 82a 60a 2013 96,5% 94,8% 191a 146a Cohort year 1 year 2 year 3 year 4 SIPEE No SIPEE SIPEE No SIPEE SIPEE No SIPEE SIPEE No SIPEE 2010 95,0% 90,0% 95,0% 85,0% 95,0% 80,0% 90,0% 75,0% 19a 18a 19a 17a 19a 16a 18a 15a 2011 94,3% 100,0% 91,4% 94,3% 88,6% 94,3% 33a 35a 32a 33a 31a 33a 2012 87,5% 87,5% 78,8% 80,8% 91a 91a 82a 84a 2013 96,5% 100,0% 191a 198a

Persistence-SIPEE

PSU weighted score comparison Family income comparision

slide-39
SLIDE 39

39

Expected Program Goals

Level Objectives Talent & Inclusion Propedéutico SIPEE Institution level Increase the access of disadvantage talented students Improve the educational experience

  • f all students through diversity

Improve academic excellence through the access of talented students Ensure the persistence Motivate an institutional change System level Increase of educational expectations in high schools Contribute to improve secondary education quality Contribute to a more equitable access to HE and a more democratic society Give the society more and better professionals

slide-40
SLIDE 40

40

Standardized GPA –Talent + Inclusion (PSU control )

1º sem 2º sem 3º sem 4º sem 5º sem 6º sem Carrera TI No-TI TI No-TI TI No-TI TI No-TI TI No-TI TI No-TI 2011 Ingeniería Promedio (PPA Z)

  • 0,19
  • 0,38
  • 0,285
  • 0,584
  • 0,228
  • 0,903
  • 0,275
  • 0,709
  • 0,225
  • 0,796
  • 0,281 -0,743

N 18 11 18 11 16 10 16 8 15 8 15 8 Delta (NOTI-TI)

  • 0,190
  • 0,299
  • 0,675
  • 0,434
  • 0,571
  • 0,462

p value delta (NOTI-TI) 0,175 0,101 0,024 0,102 0,055 0,096 2012 Ingeniería Promedio (PPA Z)

  • 0,622
  • 0,521
  • 0,377
  • 0,677
  • 0,477
  • 0,751
  • 0,435
  • 0,665

N 27 6 27 6 26 6 25 6 Delta (NOTI-TI) 0,101

  • 0,300
  • 0,274
  • 0,230

p value delta (NOTI-TI) 0,664 0,123 0,182 0,234 Derecho Promedio (PPA Z)

  • 0,431
  • 0,266
  • 0,627
  • 0,547
  • 0,675

0,099

  • 0,554
  • 0,161

N 7 3 7 3 6 2 6 2 Delta (NOTI-TI) 0,17 0,08 0,77 0,39 p value delta (NOTI-TI) 0,617 0,545 0,807 0,645 Ingeniería Promedio (PPA Z)

  • 1,062
  • 0,663
  • 0,886
  • 0,639

N 41 20 41 20 Delta (NOTI-TI) 0,399 0,247 p value delta (NOTI-TI) 0,877 0,821 Derecho Promedio (PPA Z)

  • 0,506
  • 0,133
  • 0,68
  • 0,147

N 10 4 10 4 Delta (NOTI-TI) 0,373 0,533 2013 p value delta (NOTI-TI) 0,872 0,934 Ingeniería Comercial Promedio (PPA Z)

  • 0,634
  • 2,492
  • 0,276
  • 1,334

N 5 4 4 3 Delta (NOTI-TI)

  • 1,858
  • 1,058

p value delta (NOTI-TI) 0,133 0,18 Arquitectura, Diseño y College CCNN y Matemáticas. Promedio (PPA Z)

  • 2,051
  • 0,26
  • 1,158

0,056 N 5 5 5 5 Delta (NOTI-TI) 1,791 1,214 p value delta (NOTI-TI) 0,687 0,875 1º sem 2º sem 3º sem 4º sem Overview Delta (NOTI-TI) 0,112 0,060 0,167

  • 0,090

p value delta (NOTI-TI) 0,610 0,610 0,704 0,337 N cohort*majors 7 7 3 3

slide-41
SLIDE 41

41 1º sem 2º sem 3º sem 4º sem 5º sem 6º sem Carrera TI No-TI TI No-TI TI No-TI TI No-TI TI No-TI TI No-TI 2011 Ingeniería Promedio (PPA Z)

  • 0,19 -0,111 -0,285 -0,226 -0,228 -0,212 -0,275 -0,052 -0,225 -0,082 -0,281 -0,173

N 18 18 18 18 16 17 16 16 15 16 15 16 Delta (NOTI-TI) 0,079 0,059 0,016 0,223 0,143 0,108 p value delta (NOTI-TI) 0,616 0,579 0,52 0,803 0,695 0,651 2012 Ingeniería Promedio (PPA Z)

  • 0,622 -0,152 -0,377 -0,251 -0,477 -0,382 -0,435 -0,499

N 27 27 27 27 26 27 25 27 Delta (NOTI-TI) 0,47 0,126 0,095

  • 0,064

p value delta (NOTI-TI) 0,993 0,731 0,641 0,413 Derecho Promedio (PPA Z)

  • 0,431 -0,372 -0,627 -0,531 -0,675 -0,759 -0,554 -0,807

N 7 7 7 7 6 7 6 7 Delta (NOTI-TI) 0,059 0,096

  • 0,084
  • 0,253

p value delta (NOTI-TI) 0,554 0,594 0,431 0,309 2013 Ingeniería Promedio (PPA Z)

  • 1,064 0,109 -0,888 0,034

N 41 41 41 41 Delta (NOTI-TI) 1,173 0,922 p value delta (NOTI-TI) 0,999 0,999 Derecho Promedio (PPA Z)

  • 0,516
  • 0,82 -0,689 -0,993

N 10 10 10 9 Delta (NOTI-TI)

  • 0,304
  • 0,304

p value delta (NOTI-TI) 0,203 0,213 Ingeniería Comercial Promedio (PPA Z)

  • 0,634 0,587 -0,276 0,436

N 5 4 4 4 Delta (NOTI-TI) 1,221 0,712 p value delta (NOTI-TI) 0,971 0,825 Arquitectura, Diseño y College CCNN y Matemáticas Promedio (PPA Z)

  • 2,051 0,414 -1,158 0,483

N 5 5 5 5 Delta (NOTI-TI) 2,465 1,641 p value delta (NOTI-TI) 0,817 0,799 delta 0,738 0,465 0,009

  • 0,031

Overview p value 0,986 0,962 0,507 0,452 N 7 7 3 3

  • Std. GPA Talent + Inclusion (Mother Ed. Control)
slide-42
SLIDE 42

42

Persistence –Talent + Inclusion

Cohort Sem1 Sem2 Sem3 Sem4 Sem5 Sem6 TI NoTI TI NoTI TI NoTI TI NoTI TI NoTI TI NoTI 2011 100.0% 100.0% 100.0% 100.0% 88.9% 90.9% 88.9% 72.7% 83.3% 72.7% 83.3% 72.7% 18a 11a 18a 11a 16a 10a 16a 8a 15a 8a 15a 8a 2012 100.0% 100.0% 100.0% 100.0% 94.1% 88.9% 91.2% 88.9% 34a 9a 34a 9a 32a 8a 31a 8a 2013 100.0% 100.0% 98.4% 97.0% 61a 33a 60a 32a Cohort Sem1 Sem2 Sem3 Sem4 Sem5 Sem6 TI NoTI TI NoTI TI NoTI TI NoTI TI NoTI TI NoTI 2011 100.0% 100.0% 100.0% 100.0% 88.9% 94.4% 88.9% 88.9% 83.3% 88.9% 83.3% 88.9% 18a 18a 18a 18a 16a 17a 16a 16a 15a 16a 15a 16a 2012 100.0% 100.0% 100.0% 100.0% 94.1% 100.0% 91.2% 100.0% 34a 34a 34a 34a 32a 34a 31a 34a 2013 100.0% 100.0% 98.4% 98.3% 61a 60a 60a 59a

PSU comparison Mother Education comparison

slide-43
SLIDE 43

43

  • 0.9
  • 0.8
  • 0.7
  • 0.6
  • 0.5
  • 0.4
  • 0.3
  • 0.2
  • 0.1

1 sem 2 sem 3 sem 4 sem 5 sem 6 sem Talento e Inclusión Control (Mother education)

  • 0.9
  • 0.8
  • 0.7
  • 0.6
  • 0.5
  • 0.4
  • 0.3
  • 0.2
  • 0.1

1 sem 2 sem 3 sem 4 sem 5 sem 6 sem Talento e Inclusión Control (PSU)

Notas estandarizadas- Talento e inclusión

slide-44
SLIDE 44

44

0% 20% 40% 60% 80% 100% 120% 1 sem 2 sem 3 sem 4 sem 5 sem 6 sem Talent & Inclusion Control (PSU) 0.00% 20.00% 40.00% 60.00% 80.00% 100.00% 120.00% 1 sem 2 sem 3 sem 4 sem 5 sem 6 sem Talent & Inclusion Control (Mother education)

Persistencia - Talento e inclusión

slide-45
SLIDE 45

45

  • 1.4
  • 1.2
  • 1
  • 0.8
  • 0.6
  • 0.4
  • 0.2

0.2 Year 1 Year 2 Year 3 Year 4 SIPEE Control (PSU)

  • 1.4
  • 1.2
  • 1
  • 0.8
  • 0.6
  • 0.4
  • 0.2

0.2 0.4 Year 1 Year 2 Year 3 Year 4 SIPEE Control (Mother education)

Notas estandarizadas - SIPEE

slide-46
SLIDE 46

46

Persistencia - SIPEE

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Year 1 Year 2 Year 3 Year 4 SIPEE Control (PSU) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Year 1 Year 2 Year 3 Year 4 SIPEE Control (Familiar income)

slide-47
SLIDE 47

47

Metodología Objetivo 1

  • Se estudiaron las principales características de los

programas institucionales y de la nueva política de Ranking del CRUCH a través de un análisis documental. La recolección y lectura de documentos se sistematizó a través de matrices que caracterizan a los programas en distintas

  • dimensiones. A partir de estas matrices se elaboraron

diagramas explicativos para cada uno de los programas.

  • Se profundizó en la Teoría de Acción e Implementación de

cada uno de los tres programas mencionados y de la nueva política del CRUCH por medio de entrevistas con personas que han jugado roles claves en su desarrollo e implementación (5 para SIPEE, 5 para Talento e Inclusión, 3 para Propedeútico USACH y 3 para la nueva política del CRUCH, con un total de 16 entrevistas).

slide-48
SLIDE 48

48

  • Se estudió la composición socioeconómica por dependencia

del establecimiento de egreso de enseñanza media, nivel de ingresos y nivel educativo de la madre de los estudiantes matriculados en primer año durante el período 2004-2013 en de cada una de las universidades que son el foco de esta investigación.

  • La USACH presenta los mayores porcentajes de

estudiantes matriculados en primer año de alumnos con madres sin educación universitaria, provenientes de colegios de dependencia municipal y particular subvencionada y de familias con ingresos menores a $834.000. Le siguen la Universidad de Chile y la Universidad Católica de Chile. No se observan cambios significativos en la composición del alumnado posterior a la implementación de los programas estudiados.

Resultados -Admisión por NSE

slide-49
SLIDE 49

49

Resultados rendimiento académico

  • El análisis de rendimiento académico (medido tanto

por notas como por tasas persistencia) parece indicar que los alumnos beneficiados por los programas, no siempre presentan notas ni tasas de persistencia similares a las de sus compañeros con puntajes PSU o nivel educacional de la madre/ingreso comparables.

  • El desempeño de los alumnos Talento e Inclusión tiende

a ser similar, y en algunos caso superior, que el de sus grupos de comparación, mientras que los alumnos del Propedéutico muestran desempeño similar, y a veces inferior al de su grupo de comparación. El SIPEE se encuentra en una situación intermedia, reflejando alta volatilidad en la comparación en el desempeño de los alumnos SIPEE y sus grupos de comparación (posiblemente relacionado con un mayor número de carreras participantes).

slide-50
SLIDE 50

50

Limitations and Future Research (previous version)

  • Sample: only CRUCH (traditional, publicly funded)

institutions, three selective institutions of RM, results are not generalizable.

  • Future studies: more admission programs could give a

fuller picture.

  • Small number of students limits the power of the t-

test used to evaluate the statistical significance of differences.

  • Focus on academic program and intra-institution

comparison in order to maintain comparability of groups.

  • Future studies: include voice of students, teachers

and administrators exploring changes in practices, curriculum, pedagogy and institutional climate.