Extreme Quizzing: Extreme Quizzing: Increasing Student Success and Increasing Student Success and Retention Retention Gordon Gordon Hodge Hodge Gordon Gordon Hodge Hodge Mexico & Mexico & University of University of New New Mexico University University of of New New Mexico National National Center Center for for Academic Academic Transformation Transformation National National Center Center for for Academic Academic Transformation Transformation NCAT Second Annual Redesign Alliance Conference NCAT Second Annual Redesign Alliance Conference Orlando • March 16–18, 2008 Orlando • March 16–18, 2008 NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu
Intro Psych 105 is the largest of UNM’s 20 “killer” courses for lower division undergraduate students. Before the redesign, the drop, withdrawal, failure rate was 42%. Support Pew Grant Program in Redesign National Center for Academic Transformation Proposal for Redesign of General Psychology DOE Institute for Education Science ( IES) Grant Test-Enhanced Learning . NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu
Goals: While maintaining or exceeding standards … • Improve grades (but without inflation) • Use merit system ( no curving) • Comprehensive coverage • Achieve consistency across sections • Increase time on task NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu
Solution: WebCT Quizzes Quizzes Required Quizzes Required 3,000 Question Pool 3,000 Question Pool 3 Quizzes per week 3 Quizzes per week 20 Questions per quiz 20 Questions per quiz Timed & Deadlines Timed & Deadlines Unlimited repeats Unlimited repeats Highest score counts Highest score counts 1 Point per question 1 Point per question 1 Exam per week 1 Exam per week NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu
Instructions to Students Quizzes are not just a way to review or to test yourself. Quizzes are an important way to learn the material in the first place. NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu
Do Quizzes Matter? Students who do well on exams typically: Students who do well on exams typically: • Take quizzes many times (10-20x) • Take quizzes many times (10-20x) • Get perfect or near perfect scores • Get perfect or near perfect scores • Complete later quizzes in under 5 min • Complete later quizzes in under 5 min NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu
Do Quizzes Matter? Fall Fall 2005 2005 : Correlations Fall Fall 2005 2005 : Correlations r = .93 Exam Points Figure 1. High quiz scores predicted high exam score (n = 847 students). NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu
Do Quizzes Matter? Fall Fall 2005 2005 : Pretest vs. Posttest Fall Fall 2005 2005 : Pretest vs. Posttest Average Score (80 Questions) Figure 2. Students took an 80-question Pretest during the first week of the semester and the same test as the Final Exam. Students who performed well in the course, performed better on the Posttest Final than students who performed less well. NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu
Do Quizzes Matter? Fall Fall 2005 2005 : Ethnicity Fall Fall 2005 2005 : Ethnicity Percentage of Students Figure 3. Students of different ethnic backgrounds performed comparably. NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu
Do Quizzes Matter? Fall 2000 Fall 2000 – – Fall Fall 2007 2007 : 14 Semesters Fall Fall 2000 2000 – – Fall Fall 2007 2007 : 14 Semesters Passing Grades for Psych 105 Fall 2000 – Fall 2007 81% 80% 78% 80% 77% 77% Percentage of Students 77% 75% 75% 74% 73% 71% 63% 63% 59% 60% 41% 40% 37% 37% 29% 27% 26% 25% 25% 23% 23% 23% 22% 20% 19% 20% C or Above C- or Below (& Ws) 0% F00 F01 S02 F02 S03 F03 S04 F04 S05 F05 S06 F06 S07 F07 Tradi- Pilot Semester tional Figure 4. Compared to traditional designs (i.e., lectures, readings, exams), students who were required to complete quizzes performed better in the course. NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu
Do Required Quizzes Matter? Spring Spring 2002 2002 : Two Sections/Same Instructor Spring Spring 2002 2002 : Two Sections/Same Instructor Percentage of Students Figure 5. To determine whether quizzes need to be required in order for students to perform well in the class, two sections were taught by the same instructor. In Sec 001, quizzes were required; in Sec 002, quizzes were available but optional (i.e., students were encouraged to take them, but they received no points for doing so). Students performed better when quizzes were required (Sec 001) than when quizzes were optional (Sec 002). NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu
Do Required Quizzes Matter? Spring 2002 Spring 2002 : Two Sections/Same Instructor Spring 2002 Spring 2002 : Two Sections/Same Instructor Quizzes Required Quizzes Optional Percentage of Students 20 20 High Score 16 16 High Score High Score 12 12 A 8 8 B 4 4 C+ and C C- and Below 0 0 Number of Attempts 10 10 Number of Attempts Number of Attempts 8 8 6 6 4 4 (Figure 5) 2 2 0 0 Mean Duration (min) 12 12 Average Duration (Min) 9 9 Figure 6. Patterns of Quiz Taking (left). 6 When quizzes were required, students 6 scored higher, took them more times, 3 3 and for longer durations than when 0 0 quizzes were optional. Weekly Mastery Quizzes Weekly Mastery Quizzes Weekly Mastery Quizzes Weekly Mastery Quizzes NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu
Do Required Quizzes Matter? Spring Spring 2005 2005 : Two Sections/Different Instructors Spring Spring 2005 2005 : Two Sections/Different Instructors Percentage of Students Course Grade Figure 7. In spring 2005, to determine whether quizzes need to be required in order for students to perform well in the class, two sections were taught by different instructors. In Sec 001, quizzes were required; in Sec 002, quizzes were available but optional (i.e., students were encouraged to take them, but they received no points for doing so). Students performed better when quizzes were required (Sec 001) than when quizzes were optional (Sec 002). NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu
Do Required Quizzes Matter? Spring 2007 Spring 2007 : Alternating Weeks—Rqd vs Opt Qzs Spring Spring 2007 2007 : Alternating Weeks—Rqd vs Opt Qzs Mean In-Class Exam Performance S07 Sec 001 vs Sec 002 68 * * 66 * * 64 Points (out of 80) 62 60 * * 58 56 * * * * * * 54 52 * Sec 001 Qzs Rqd on Odd Exams * * * Sec 002 Qzs Rqd on Even Exams 50 Exam01 Exam02 Exam03 Exam04 Exam05 Exam06 Exam07 Exam08 Exam09 Exam10 Exam11 Exam12 Meth Neuro Devel S&P Consc Learn ThkIntel MoEmo Stress Pers AbTher Soc * * p<0.04 p<0.00 p<0.61 p<0.00 p<0.19 p<0.01 p<0.01 p<0.01 p<0.01 p<0.01 p<0.26 p<0.25 Exam p value (Single Factor Btwn ANOVA: Sec 001 vs Sec 002) Figure 8. In spring 2007, to determine whether quizzes need to be required in order for students to perform well on exams, two sections were counterbalanced each week for whether quizzes were required or optional. For weeks when quizzes were required, students performed better across sections than when quizzes were optional. NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu
Do Quiz Questions Matter? Are Are students students just just memorizing memorizing the the Are Are students students just just memorizing memorizing the the questions? questions? questions? questions? NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu NCAT Redesign Alliance • March 16-18, 2008 • Gordon Hodge • ghodge@unm.edu
Recommend
More recommend
