Background UCC Enrollment (2002/2003) The University College of the - PowerPoint PPT Presentation

Overview Entry Assessment Testing as a Background Information † Predictor of Student Success Introduction to the Study † Findings † presented by Conclusions Cindy James † Assessment Centre Coordinator University College of the Cariboo Kamloops, British Columbia, Canada cjames@cariboo.bc.ca Background UCC Enrollment (2002/2003) The University College of the Cariboo (UCC) is a public educational institute offering both college University Degrees 5575 and university programs. Diplomas 1387 The main campus is † Vocational 1462 located in Kamloops, British Columbia, Developmental 1657 Canada. Seven regional † Totals 10081 offices/campuses within the central and southern interior of B.C. 1

UCC Assessment Centre Accuplacer Purpose: To provide general educational assessments to † WEB-based computer testing system † facilitate appropriate placement in courses/programs. † Utilizes Adaptive Technology To administer entry assessments for admission to † † Customized Testing various UCC programs. † Computer scored To coordinate and/or invigilate examinations for † other educational institutions and outside agencies. † Built-in Database Administered/Coordinate ~ 3500 assessments/tests † per year Accuplacer Tests UCC College Preparation Purpose Reading Comprehension † Offers adult learners the opportunity to complete Sentence Skills † prerequisites for admission into career, vocational and Arithmetic academic programs and/or to complete an ABE diploma. † Courses (offered year round) Elementary Algebra † English, Math, General Science, Biology, Physics, College Level Mathematics † Chemistry, Student Success, Psychology, Computing and Social Science. WritePlacer Plus & WritePlacer ESL † Placement Language of English Proficiency tests † Self – select for lower level courses (040 & 050) † Pre-requisites for the higher level courses (060) † 2

The Study Study Timeline The Big Question Spring/Summer session 2003 To improve the placement procedure and hence the Accuplacer tests administered to 6 different classes † success rate for College Preparation courses and of English and Math students on the first day of programs, should entry assessment testing be classes. mandatory for College Preparation applicants? Classes met on a daily basis for 7 weeks † The Consequential Question (total of 75 hours of instruction). Can Accuplacer Tests Scores Predict Student Final Grades collected at the end of each course. † Success/Nonsuccess in College Preparation Data analysis conducted during the Fall 2003. † Courses ? Study Participants Study Cohorts Math English Statistics Total # of # of Cohort Cohort participants participants Sample size (n) 93 Math 040 11 English 040 10 Females 48 Math 050 15 English 050 7 Males 45 Math 060 34 English 060 16 Average Age 27 Total 60 Total 33 3

Success Rates Regression Analysis Utilized † Linear & Multiple Regression Analysis Statistics Total To evaluate the linear relationship between the predictor variables (Accuplacer test scores) and the Successful Students 84 outcome (final grade). (final grade ≥ 50%) † Logistic Regression Analysis NonSuccessful Students 9 To evaluate the relationship between the predictor (final grade < 50%, DNC, Withdrew) variables (Accuplacer test scores) and the Success Rate 90 % dichotomous outcome (success vs nonsuccess). Linear & Multiple Regression Analysis Linear Regression Analysis Final Math Grade as a Function of Arithmetic Test Scores Math Cohort Pearson’s Correlation Coefficient (r) 110 100 Final Grade in Math Course (%) Accuplacer Test Scores 90 80 Final Grade Arithmetic Algebra Both Tests 70 Math 040 0.460 0.303 0.469 Math 060 60 (n = 10) Rsq = 0.1528 Math 050 50 0.290 0.392 0.397 Math 050 Rsq = 0.0841 (n = 13) 40 Math 060 Math 040 0.391 0.617 0.619 30 Rsq = 0.2117 (n = 32) 20 40 60 80 100 120 Accuplacer Arithmetic Test Score 4

Linear & Multiple Regression Analysis Linear Regression Analysis English Cohort Final Math Grade as a Function of Algebra Test Scores Pearson’s Correlation Coefficient (r) 110 100 Accuplacer Test Scores 90 Final Grade in Math (%) Reading Sentence 80 Final Grade Both Tests Comprehension Skills 70 English 040 0.285 0.641 0.669 Math 060 60 (n = 10) Rsq = 0.3806 50 Math 050 English 050 Rsq = 0.1533 0.071 0.143 0.172 40 (n = 6) Math 040 30 Rsq = 0.0916 English 060 20 40 60 80 100 120 0.508 0.563 0.575 (n = 15) Accuplacer Elementary Algebra Test Scores Linear Regression Analysis Linear Regression Analysis Final English Grade as a Function of Sentence Skills Test Scores Final English Grade as a Function of Reading Comprehension Test Scores 100 100 Final Grade in English Course (%) 90 90 Final Grade in English (%) 80 80 70 70 English 060 English 060 Rsq = 0.2580 Rsq = 0.3169 English 050 English 050 60 60 Rsq = 0.0050 Rsq = 0.0206 English 040 English 040 50 Rsq = 0.0814 50 Rsq = 0.4111 20 40 60 80 100 120 20 40 60 80 100 120 Accuplacer Reading Comprehension Test Score Accuplacer Sentence Skills Test Score 5

Logistic Regression Logistic Regression Analysis Sample Classification Table - Math 050 cohort Predictor – Elementary Algebra Test score Creates a model that predicts a dichotomous outcome based on one or more predictor Predicted Outcome Percentage variables. NonSuccess Success Correct NonSuccess (4) 2 2 50.0% For this Study: Success (11) 1 10 90.9% Outcome = Success or NonSuccess Overall 80.0% Predictor variables = Accuplacer test scores P-value 0.033 Cutoff = 0.5 Logistic Regression Logistic Regression Models Sample Classification Table - English 060 cohort Math Cohort Predictor – Sentence Skills Test score Overall Percentage Correct Predicted Outcome Elementary Arithmetic Both Tests Percentage Algebra NonSuccess Success Correct Math 040 90.9% 90.9% 100 %* NonSuccess (1) 0 1 0% (n = 11) Math 050 Success (15) 0 15 100.0% 73.3% 80.0%* 73.3% (n = 15) Overall 93.8% Math 060 88.2%* 88.2%* 94.1%* P-value 0.030 (n = 34) *Level of significance (p-value) < 0.05 Cutoff = 0.5 6

Logistic Regression Models Answers to the Questions? English Cohort † Can Accuplacer Tests Scores Predict Student Overall Percentage Correct Success/Nonsuccess in College Preparation Reading Sentence Both Courses ? Comprehension Skills Tests English 040 Yes! n/a n/a n/a (n = 10) † Should entry Assessment Testing be English 050 71.4% 100%* 100% Mandatory for College Preparation (n = 7) Applicants? English 060 100%* 93.8%* 100%* Yes, but used as an advising tool only! (n = 16) *Level of significance (p-value) < 0.05 What Now? Contact Information Cindy James Gather more data to support or refute the findings. UCC Assessment Centre Coordinator 900 McGill Road Test the prediction Models † Box 3010 Expand the study Kamloops, BC Canada † Initiate new studies † Phone: 250 – 828 – 5471 Fax: 250 – 371 – 5690 Email: cjames@cariboo.bc.ca Website: www.cariboo.bc.ca/dsd/assmnt/ 7