Moving Forward: Lessons Learned From The Last 10 Years of Risk - - PowerPoint PPT Presentation

Moving Forward:

Lessons Learned From The Last 10 Years of Risk Modeling

Presentation Agenda

1) Provide a framework for defining risk 2) Examine the role of theory and personal bias in the modeling process 3) Help outline possible partnerships needed for successful processes (data extraction, intervention design, student outreach) 4) Review a list of data sources, variables and tools needed for successful program implementation 5) Guide participants in identifying next steps in their own Risk Identification Programs 6) Examine the impact individualized reports and ad-hoc reports play in day to day lives of advising administrators

• Top-tier public research university
• Founded by Russell H. Conwell in 1884
• Located in Philadelphia, PA
• Average Combined SAT (1170)
• Percent Admitted (79.9%)
• Undergraduate Enrollment: 29,416

Retention Rate: 89.7% Graduation Rates: (2012 Cohort) 4-year: 48% 6-year: 70%

Framework: Defining Risk

• At risk for what?
• Defining Retention and Academic Success
• 20% Rule

Data Mining Definition

70 - 90% - Spent Cleaning and Understanding data

The Role of Theory and Personal Bias in the Modeling Process

• Dataset Selection
• Variable Inclusion & Exclusion
• Tinto, Astin, Chicoring + Others

It Takes a Village (campus)

Business Intelligence Institutional Research Banner Competency Center Colleges & Advising Offices Housing Office Disability Resource Center Registrar Office Office of Orientation Career Center

Tools

• Cognos
• Query Studio
• Tableau
• SPSS (Modeler)
• Good Old Friend Excel
• Canvas LMS

Past

Regression Model

Present - Existing Structure

4 Models updated each year for all newly admitted Freshman & Transfer students ○ Admission - Placement ○ Orientation - Add/Drop ○ Add-Drop - W date ○ End of Term

Present: Configural Analysis (Classification or Segmentation)

Databases Variables Used - Student Life Cycle

○ Frozen Cohort Files ○ Admissions - (High School Info, demographic) ○ College/Major (Change of either) ○ Financial Aid (EFC, Pell, Unmet Need) ○ New Student Questionnaire (Q1 - Q83) ○ Placement Scores and Levels (Eng, Math, Aleks Sub Scores, SATs) ○ Orientation Information ○ Housing + LLC ○ Term Balance (Start - #of Hrs Registered & End of Term), T1 - T8 ○ Mid Semester Grades/Alerts (Cumulative) ○ Hold Indicators, Multiple W Grades ○ Special Populations ○ Flyin4 Participation ○ Delta GPA Term 1 GPA - HS GPA * Most Predictive

Models and Key Variables

Model 1: Admissions - NSQ Placement (June 15th)

• Admit Type
• # of Days between Deposit and First Day of Class or September 1 (Calculated)
• # of Days between Orientation and First Day of Class or September 1 (Calculated)
• Race*
• HS GPA, Rank, Percentile, Size of HS
• HS Code (GPA of all students who went to it before)
• Admissions Ranking
• Unmet Need/Offer Amount
• Academic Major/Program*
• Parental Education
• Admissions Letter Code*
• SAT/ACT
• NSQ ~ 15 Variables
• Placement (Math, ENG, Aleks Score)

Models and Key Variables

Model 2: Orientation - Registration - Change of College/Major (September 12th)

• Census Date - Day after Add/Drop period ends
• Change in college since Orientation (Y/N) as of Census Date
• # of Credits Registered (FT/PT)
• Date of Orientation (Late Orientation for TR)
• LLC vs Non LLC
• Balance as of Add/Drop
• Housing assignment (Name of the Building)
• Unmet Need
• EFC
• Pell Amount

Models and Key Variables Model 3: First Semester Half Mark (6 Weeks)

• 1 week after the “W” date
• # of UG Mid semester Alerts
• # of W grades (1 week after 2 date)
• Balance as of W date
• EFC & Unmet Need
• Total Amount Borrowed
• Student’s College/Major
• Special Populations

Models and Key Variables

Model 4: End of Term 1 Marker

• January 2nd
• Hold Indicator
• Term 1 GPA (Academic Standing)
• GPA Drop HS - Term 1 (1.25 drop from HS, Honors = .85)

Balance as of Final Grades > $200

• Housing assignment On or Off
• #of Credits Passed <12
• Math & Eng Grades
• # of UG MidSemester Alerts

Risk Liaison Intervention Training

Summer Meetings – July 11th and August 9th July: The Ones that Got Away: Canvas Review, Risk List Review & Group Introductions August: Designing Fall Interventions and Strategic Communications for students at Risk, on Probation and on Warning Fall Meetings (Fourth Wednesday of the Month @3PM) September Working with Data & Final Freshman & Transfer Risk List Review (Bring your laptop and Excel) October CSRDE Comparative Reports Review & Retention Workshop November Failed to Launch: Mid-Semester Alerts, LOA, WE Students December Designing Spring Interventions and Strategic Communications for students at Risk and on Warning Spring Meetings- Fourth Wednesday @3PM February The Murky Middle 2.0 – 2.5 March Intervention Discussion (What Worked & What Didn’t) April No Meeting May Intervention Assessment

Interventions

Addition of mentoring and peer mentoring supports Specialized Workshops (Financial Aid, Housing, etc) Mid Semester Initiatives (Letters, phone outreach) Outreach is done by Financial Aid, Housing, DRS, other supporting units not just advising Jobs on campus and work study positions

○ (Need to identify students eligible earlier)

Interventions

Advisor ratios review Specialized advisors focused on riskiest of students Advisor intervention design and data training Shift from intrusive to developmental & appreciative advising Shift from group to individual advising for riskiest of students

Retention Canvas Site

Advising Director Feedback

• Results of Qualitative interviews with 12 Schools/Colleges
• Qs Asked

○ How have you used Risk List? ○ What challenges have you faced in working with at-Risk students? ○ What data/reports would make your life easier? ○ What were the most successful interventions done by your department?

Future - Individualized Risk Models

Additional Information to be added to the models

• Retention Defined by College (i.e. Retained in Engineering not TU)
• Special Populations Tracked and Monitored

a.

Off/On Campus, first gen, low income, adult 24+, LLC, Pre-Business, Pre-Health, Undeclared with 60+ credits)

• Examining impact of Grades/Mid Semester Alerts in specific Courses

○ (First Semester ENG & MATH, STAT, ENG, etc.)

• Focus on Freshman, Sophomore & Transfer
• Focus on Specific Majors (Biology, Undeclared (all colleges), Psychology, Biochemistry, Business

Management, Pre-Pharmacy, Communication Studies, Computer Science, Early Childhood Education, Engineering, Political Science)

• June, September, January review of models and at-risk student lists/dashboards
• Monthly Advisor/Risk Liaison Workshops

Examining Internal University Student Migration

Examining Program Level Retention and Migration

CSRDE - National Student Consortium for Reports Webinars and Advisor Training Comparative data

Future Partnerships

Questions

Alexandra Yanovski-Bowers Assistant Director for Undergraduate Strategic Initiatives Office of the Vice Provost for Undergraduate Studies Temple University 500 Conwell Hall 1801 N. Broad Street Philadelphia, PA 19122 ayanovski@temple.edu p: 215-204-7596 https://www.linkedin.com/in/alexyanovski