[PDF] - Using the Student Readiness Inventory (SRI) as part of a PDF Document

SLIDE 1

1

Using the Student Readiness Inventory (SRI) as part of a Comprehensive Intervention and Retention System

Alex Casillas

ACT, Inc.

29th Annual Conference on The First-Year Experience Denver, CO February 14, 2010

2

Agenda

Brief introduction to the SRI
Case studies

– Examining risk, resource use, and success at a 4-yr institution – COMPASS, SRI, behavior ratings & course success at a 2-yr institution

Implications
Questions

SLIDE 2

2

3

Six-year graduation rates average 57% at 4-year institutions (range = 36 to 75%, Horn & Nevill, 2006) and 34% at 2-year institutions (range not available,

Swail, 2004)

Many new students are ill-prepared to meet the hurdles they face upon entry into college. What can institutions do to help students succeed?

The Need

4

Prediction of Success

Historically, prediction of first-year college success has centered on standardized achievement and high school GPA There is growing evidence that academic success behaviors should be modeled with additional relevant attributes, including psychosocial factors (Le et al., 2005; Peterson et al.,

2006)

SLIDE 3

3

5

Robbins et al. (2004) conducted a meta-analysis

f 109 studies (N = 152,985) to examine the effect
f 3 types of predictors on academic

performance and retention:

– Traditional (e.g., standardized test scores, HS GPA) – Demographic (e.g., SES, race/ethnicity, gender) – PSFs (psychosocial and study skill factors)

Introduction to SRI

6

Psychosocial and study skill factors can be organized into 3 broad domains: Motivation & Skills

– Personal characteristics that help students to succeed academically by focusing and maintaining energies on goal-directed activities.

Social engagement

– Interpersonal factors that influence students’ successful integration into their environment.

Self-regulation

– Cognitive and affective processes used to monitor, regulate, and control behavior related to learning.

Introduction to SRI (cont.)

Le et al., (2005)

SLIDE 4

4

7

SRI Scales

Academic Self-Confidence
Steadiness

Self-Regulation

Social Activity
Social Connection

Social Engagement

Academic Discipline
Commitment to College
General Determination
Goal Striving
Study Skills
Communication Skills

Motivation & Skills SRI Scale Domain

See www.act.org/sri/components.html for scale definitions and sample items

8

Measuring Risk

The SRI also includes predictive indices

(based on a combination of SRI scales + achievement information):

– Academic Success Index: likelihood of GPA 2.0 or higher – Retention Index: likelihood of returning second year

All SRI scores (scales and indices) are

presented as percentile scores (range 1 to 99)

SLIDE 5

5

9

Sample SRI Student Profiles

Class of 2007 college students completed the SRI at

the beginning of their 1st semester (Fall 2003)

Sample profiles selected to contrast academic

preparation and motivation

Low High Low Student A Student B High Student C Student D Motivation ACT score

10 LOW ACT

LOW MOTIVATION HIGH MOTIVATION

Scale Percentile Percentile Academic Discipline General Determination Goal Striving Commitment to College Study Skills Communication Skills Social Connection Social Activity Academic Self-Confidence Steadiness 10 20 30 40 50 60 70 80 90 100 6 17 13 9 47 34 76 9 2 6

Low Medium High

10 20 30 40 50 60 70 80 90 100 95 72 80 51 59 99 49 83 46 71

Low Medium High

Low ACT score (composite = 14)

SRI Indices Academic = 1 Academic = 20 Retention = 3 Retention = 17 Outcome First-year GPA = 1.5 First-year GPA = 2.22 Fall 07 status = dropped out Fall 07 status = still enrolled

Student A Student B

SLIDE 6

6

11

High ACT score (composite = 27)

SRI Indices Academic = 60 Academic = 96 Retention = 50 Retention = 88 Outcome First-year GPA = 3.7 First-year GPA = 4.0 Fall 07 status = dropped out Fall 07 status = graduated

Student C Student D

HIGH ACT

LOW MOTIVATION HIGH MOTIVATION

Scale Percentile Percentile Academic Discipline General Determination Goal Striving Commitment to College Study Skills Communication Skills Social Connection Social Activity Academic Self

Confidence

Steadines s

10 20 30 40 50 60 70 80 90 100

3 55 4 22 31 46 7 15 32 9 Low Medium High

10 20 30 40 50 60 70 80 90 100

92 50 26 22 38 12 45 57 92 53 Low Medium High

12

Brief Recap

The previous slides suggest that academic

preparation and psychosocial factors both contribute to success. For example:

– High motivation will help an academically weak student to do better. – High motivation will help an academically strong student to excel.

These findings are consistent with both the

education and workforce literature.

SLIDE 7

7

13

Case Study:

Examining Risk, Resource Use & Success at a 4-yr Institution

14

Public Southwestern University
Over 13,000 Undergraduate Students and

1,100 Faculty

31% Dropout Rate across Freshman Year
Implemented Card Swiping System to Monitor

Resource Use

Background

SLIDE 8

8

15

Intervention Model

Implemented early in the (first) semester
Included groups known to be at risk
Collaborative and not duplicative

– academic advising and first-year programs – summer bridge program and first-year mentoring – other key groups interfacing with new students

Systematic
Addressed whole student (using 1-on-1

meetings between student and advisor)

Grounded in research

16

Student-Advisor Meeting

Above all else, connect with student
Acknowledge student’s strengths and identify campus

activities to reinforce them

Match student’s needs to campus resources

– See SRI Tool Shop: act.org/sri/studentguide/toolshop.html

Gather some data on SRI experience

– How did you hear about your appointment – How seriously did you take the SRI when you completed it?

SLIDE 9

9

17

Evaluation: Academic Performance

Avg 1st Sem GPA by Distribution Group

0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 Met 3.02 2.77 2.64 2.48 2.45 2.36 2.15 2.55 Not Met 2.83 2.28 2.41 2.19 2.06 1.97 1.91 2.23 Grp 1 Grp 2 Grp 3 Grp 4 Grp 5 Grp 6 Grp 7 ALL Student Affairs

All students who met to receive SRI results fared better than those who did not meet with Student Affairs staff.

18

Evaluation: Academic Performance (cont.)

Percent on Probation by Distribution Group

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% Met 31% 26% 23% 16% 15% 15% 10% 18% Not Met 41% 42% 21% 11% 28% 24% 14% 26% Grp 1 Grp 2 Grp 3 Grp 4 Grp 5 Grp 6 Grp 7 ALL Student Affairs

Students who did NOT meet regarding their SRI were more likely to end up on academic probation.

SLIDE 10

10

19

Resource & Services Utilization

GPA Ret. GPA Ret. GPA Ret.

2.68 .57 2.83 .79 2.67 .84

Advisory / Career Sessions

2.53 .60 2.95 .88

Academic Referrals

2.63 .69 2.91 .85 2.83 .72

Social Resources

2.67 .71 2.76 .77 2.87 .77

Recreational Resources

2.63 .70 2.92 .79 2.75 .75

Academic Services

Low Moderate High Level of Utilization

Resource/ Service Category

Robbins et al. (2009)

20

Association of Risk Level & Academic Service Use on Retention & First-Year GPA

.85 .84 .77

Retention

Low .81 .77 .72

Retention

Moderate .62 .72 .61

Retention

High .08 .23 .24

Gains show interaction

n GPA

3.22 2.81 2.35 4+ Sessions 3.38 2.81 2.41 1 – 3 Sessions 3.14 2.58 2.11 0 Use

GPA GPA GPA

Risk Level

Service Use Robbins et al. (2009)

SLIDE 11

11

21

Recommendations from Case Study (4-yr)

Coordinate institutional efforts across service

areas

Identify at-risk students early
Provide students with resources designed to

increase their opportunity to succeed

– Be strategic in your use of resources – Don’t be afraid of intrusive advising

Track student progress

– Use your data/feedback and improve the process

22

Case Study:

Developing an Early Intervention System at a 2-yr Institution

SLIDE 12

12

23

Located in a major urban center
Over 8,700 Students and 350 Faculty with a range of

programs (e.g., general studies, career/vocational)

Among students completing the college's placement

exams in Fall 2009:

– 58% were placed in developmental/remedial English. – 58% were placed into developmental/remedial Reading. – 84% were placed into developmental/remedial Math.

Working to implement a comprehensive First-Year

Experience Program

Background

24

Objectives of Early Intervention

Lend focus to first-year success by identifying

at-risk students

Customize student support services according

to students’ individual, varied needs

Target existing resources toward at-risk

students

Seek to yield higher returns on efforts and

college resources

SLIDE 13

13

25

New student Assessment (COMPASS & SRI)

Effective Course Placement Specific Course Activities

See: College Developmental Course Best Practices*

Early Warning Risk for Academic Failure and/or Drop-out Course and General Intervention Strategies

See: Early Intervention System

Assessment and Intervention Strategy

*not featured in this presentation 26

Learning Styles Workshop Study Skills Workshop Tutoring Center Career Advisement Workshop / One-on-one Career Consultation Writing Center Physical Science Tutoring Academic Advising Disability Access Center (referral) Social Support Network (referral) Faculty Referral COMPASS + SRI Institutional Criteria

(e.g. Repeating the same course)

Student Clubs and Organizations

Overview of Early Intervention System

SLIDE 14

14

27

Crosswalk of SRI Services

SRI Scales Support Services

Academic Discipline Academic Self-Confidence General Determination Commitment to College Communication Skills Steadiness Study Skills Goal Striving Social Activity Social Connection Tutoring Center X X Writing Center X Career Advisement Workshop X X One-on-one career consultation X X Advisement X X Physical Science Tutoring (if students take Physical Science courses) Disability Access Center (referral) Learning Styles Workshop Social Support Network (referral) X X Study Skills Workshop X Student clubs and organizations X X 28

Success in 2-yr College Courses Depends

n Academic and Psychosocial Risk

Success Rates in English Composition, by Academic and Psychosocial (SRI) Risk Levels

78% 68% 53% Low 58% 47% 32% Medium 44% 40% 23% High Academic Risk Level Low Medium High SRI Risk Level

Note: 2-yr college data from field study featured in Robbins et al., 2006.

SLIDE 15

15

29

Success in 2-yr College Courses Depends

n Academic and Psychosocial Risk (cont.)

Success Rates in Elementary Algebra, by Academic and Psychosocial (SRI) Risk Levels

69% 67% 50% Low 52% 41% 32% Medium 27% 28% 15% High Academic Risk Level Low Medium High SRI Risk Level

Note: 2-yr college data from field study featured in Robbins et al., 2006.

30

Faculty Ratings Based on Classroom Behavior

Instructors rated students’ classroom behavior using a five-point rating scale: 1 (almost never) to 5 (always)

– Participation

Student is actively engaged during group work; helps other students

with assignments; does his/her fair share of the work

Student is alert and attentive during class; asks/answers questions

– Attendance

Student attends class; stays for whole period

– Homework

Student completes assignments thoroughly and completely;

turns assignments in on time

SLIDE 16

16

31

Effects of In-class Student Behavior on Success

Success Rates in Elementary (precredit) Math, by COMPASS Pretest and Behavior Rating Levels

6% 44% 69% Low 17% 62% 91% Medium 59% 77% 92% High COMPASS Pretest Scores Low Medium High Behavior Rating Level

32

Recommendations from Case Study (2yr)

The three pillars of success:

Appropriately place and identify strengths and weaknesses

(using COMPASS diagnostics and SRI profiles)

Address motivational skill and engagement behaviors within and
utside classroom (using institutional resources)
Connect instruction to the diagnostic and curriculum targets

essential to academic achievement domain (e.g., Math, English)

Psychosocial Psychosocial factors factors Instruction and Instruction and Institutional Resources Institutional Resources Academic Academic Readiness Readiness

SLIDE 17

17

33

Implications

The SRI is useful for identifying at-risk

students.

Early identification and intervention with

students can facilitate positive outcomes.

– increased GPA – lower rates of academic probation – increased rates of continued enrollment

34

Implications (cont.)

Institution-wide initiatives designed to impact student success include several key elements:

– Provide early identification of at-risk students – Use multiple sources of information to identify students (e.g., achievement and placement scores, SRI results, faculty referrals) – Refer identified students to resources and interventions – Use a “whole student” approach for service delivery – Track student progress and outcomes

SLIDE 18

18

35

References

Horn, L. & Nevill, S (2006). Profile of undergraduates in U.S. postsecondary education institutions: 2003-2004: With a special analysis of community college students (NCES 2006- 184). Washington, DC: U.S. Dept. of Education, National Center for Education Statistics. Le, H., Casillas, A., Robbins, S., & Langley, R. (2005). Motivational and skills, social, and self- management predictors of college outcomes: Constructing the Student Readiness Inventory. Educational and Psychological Measurement, 65, 482-508. Peterson, C. H., Casillas, A., & Robbins, S. B. (2006). The Student Readiness Inventory and the Big Five: Examining social desirability and college academic performance. Personality and Individual Difference, 41, 663-673. Robbins, S. B., Allen, J. Casillas, A., Akamigbo, A., Saltonstall, M., Campbell, R., Mahoney,

E. & Gore, P.A. (2009). Associations of Resource and Service Utilization, Risk Level, and

College Outcomes. Research in Higher Education, 50(1), 101-118. Robbins, S., Allen, J., Casillas, A., Peterson, C., & Le, H. (2006). Unraveling the differential effects of motivational and skills, social, and self-management measures from traditional predictors of college outcomes. Journal of Educational Psychology, 98, 598-616. Robbins, S. B., Lauver, K., Le, H., David, D., Langley, R., & Carlstrom, A. (2004). Do psychosocial and study skill factors predict college outcomes? A meta-analysis. Psychological Bulletin, 130, 261-288. Swail, W. S. (2004, January 23). Legislation to improve graduation rates could have the opposite

effect. The Chronicle of Higher Education, B16.

For more information, visit: www.act.org/sri

36