ONBOARDING SPECIFICATIONS GRADING Mai Yin Tsoi Mary Beth - - PowerPoint PPT Presentation

ONBOARDING SPECIFICATIONS GRADING

Mai Yin Tsoi Mary Beth Anzovino Angi Lively Karen Perell-Gerson Omar Villanueva

Problems with Traditional Assessment Truth in Grading

• What is an “A”? What is a “C”?
• Effort vs. Knowledge/Skills
• Purpose of Grades
• Effects of Grading
• Partial credit – reward for poor work
• “Game”

2

What is Specifications Grading?

• Outcomes-based assessment
• Student choice and control
• Mastery vs. non-mastery
• Multiple opportunities for mastery

3

Nilson, L. Specifications Grading: Restoring Rigor, Motivating Students, and Saving Faculty Time; Stylus: VA, 2014.

Benefits of Specifications Grading

• Addresses weaknesses and misconceptions
• Grades signify “mastery” – increases course rigor
• Clear communication about expectations
• Student-driven
• Second chances – “gentle fail”
• Less “emotional” involvement in grading

4

Ten Professors, Eight Courses, Four STEM Areas

PROFESSOR COURSE Mary Beth Anzovino CHEM 2211 – Organic Chemistry I Mai Yin Tsoi CHEM 1211 – Principles of Chemistry I CHEM 2211 – Organic Chemistry I Omar Villanueva CHEM 1211 – Principles of Chemistry I Emily Henary CHEM 1211 – Principles of Chemistry I Cynthia M. Woodbridge PHYS 1112 – Introductory Physics II (Algebra-based) Angi Lively MATH 1111* – Access/College Algebra (Co-requisite model) Mary George Whitney MATH 0989 – Foundations for College Algebra AmyH Erickson MATH 2210 – Calculus II Hank G. Harmon MATH 1111 – College Algebra Karen Perell-Gerson BIOL 2451 – Anatomy & Physiology I

5

• Backwards Design = List of Essential Outcomes (EO) / General

Outcomes (GO) Dr. Joshua Ring, Lenoir-Rhyne University

• Set up Grading Scale
• Wrote Assessments

– Wrote multiple versions

• Chose dates for Exam Hours
• Developed Token System

Initial Methodology: CHEM 2211 (Organic Chemistry I)

Ring, J. In CCCE ConfChem; ACS Division of Chemical Education; 2016.

Initial Assessment Plan: CHEM 2211 (Organic Chemistry I) A Pass 6 EOs + 13-15 GOs B Pass 6 EOs + 11-12 GOs C Pass 6 EOs + 9-10 GOs D Pass 6 EOs + 7-8 GOs F [Pass less than 6 EOs] OR [Pass 6 EOs and less than 6 GOs]

7

Revised Assessment Plan: CHEM 2211 (Organic Chemistry I)

• First round: final exam not integrated - “to keep B, no final!”
• Second round: integrated final exam - “needed to keep B!”

–Worth ~18.5% of total outcomes –Combined with mastered quizzes  Overall Grade

8

How To Write Objectives

• Backwards design of course (Wiggins & McTighe, 2005)

– Start with what you want the students to be able to do – Break the “what” into assessable outcomes – Consider what evidence is required to support a claim that student has

mastered that outcome or objective

• Use active verbs:

– Draw Lewis structures & predict molecular polarity – Classify isomeric relationships

9

Wiggins, G. and McTighe, J. Understanding by Design, Expanded 2nd ed; Association for Supervision and Curriculum Development: Alexandria, 2005.

A “Good” Objective

• “Predicting relative acidity & basicity.” (General Outcome, CHEM

2211K)

– Use quantitative information to compare acids & determine which is

stronger

– Use qualitative information to compare acids & determine which is

stronger

– Draw the mechanism (the “how”) for reactions involving acids & bases – Identify Lewis acids & bases

• Can write an assessment item for each of these!

10

How to Grade Assessments and Set Grading Policy

• Each question is correct or not – no partial credit

– Grading is quicker – immediate feedback for students – No debate over how much partial credit an answer was worth

• Instructor sets the benchmark for pass/fail

11

MATH 1111 – Access Algebra (Co-requisite Model)

• Developed list of Critical Skills (CS) / General Skills (GS)
• Prerequisite Materials in addition to skills used repeatedly in

subsequent skills were deemed “Critical”

• Set up Grading Scale
• Created Multiple Versions of Assessments for Multiple Attempts
• Chose dates for Three Exam Hours
• Created Supplemental Open Lab time
• Developed Token System
• Counseled Students Individually for Success Strategies

12

MATH 1111 – Access Algebra (Co-requisite Model) Class format:

• CLASS I) Instruction with Gradual Release of

Responsibility/Online Homework Assigned

• CLASS II) Reinforcing In-Class Practice
• CLASS III) Reality Check (5 question quiz – must earn at

least 4/5)

13

MATH 1111 – Access Algebra (Co-requisite Model)

Typical Class Meeting: 15 min: Reality Check on CS3 30 min: Reinforcing In-Class Collaborative Practice on CS2; Reality Check graded while they work; we go over answers to ICP together 5-10 min: Hand Back Reality Check/Discuss Correct Solutions 5-10 min: Break 60 min: Instruction on New Material/HW Assigned

14

MATH 1111 – Access Algebra (Co-requisite Model)

Supplemental Open Lab

• Class is scheduled TR 10:15-12:15
• Class is invited to supplemental open lab environment TR 8-9:45
• Work on Homework
• Earn Tokens
• Retake Assessments
• Get (some) Individual Instruction/Tutoring/Mentoring

15

BIOL 2451 – Anatomy and Physiology I

• Unique course – no calculations so finite number of questions from

the material

• 3 attempts throughout the semester for general objectives
• 5 application objectives for lecture material (case study application
• f material)
• 20 general objectives for lecture material (primarily physiology)
• 10 general objectives for laboratory material (primarily anatomy)

16

GRADING SCHEME FOR OVERALL LECTURE GRADE IN BIOL 2451K

Lecture Lab Grade Requirement Requirement A Pass 5 AOs + 18-20 GOs Pass 3 LRs + 9-10 GOs B Pass 5 AOs + 16-17 GOs Pass 3 LRs + 8 GOs C Pass 4 AOs + 14-15 GOs Pass 3 LRs + 7 GOs D Pass 4 AOs + 12-13 GOs Pass 3 LRs + 6 GOs F [Pass less than 4 AOs] OR [Pass 4 AOs and fewer than 12 GOs] [Pass less than 3 LRs] OR [Pass 3 LRs and fewer than 6GOs]

17

BIOL 2451 – Anatomy and Physiology I (Assessment Plan)

General Observations by Faculty

• Less grading time – “fighting for partial credit”
• Expectation of student to PREPARE before lecture finally

realized

• Higher expectations of student mastery
• Less “guilt” with grading
• Harder, less “sensitive” to student excuses

18

General Observations by Faculty

• Need to “PR” the system – A LOT!
• “Gentle fail”* – sensitivity to student perception of “fail”
• Administration comments:

–More feedback –Cheerleading

*Nilson, L. Specifications Grading: Restoring Rigor, Motivating Students, and Saving Faculty Time; Stylus: VA, 2014.

• Increase in student engagement,

attendance, and participation

• Students do not beg for “partial”

credit or bonus points

• High levels of student excitement

about course!

• Although more frequent

assessments, grading effort is lower

• Lowest withdrawal (“W”) rate at

midterm ever seen (only 1 student withdrew out of 52)

20

Observations by Villanueva in Principles of Chemistry I

Midpoint Student Survey in Principles of Chemistry I

21

Data obtained from n=45 students

“I truly feel like I am mastering the content in this class through specs grading (even if it takes several tries to pass a quiz)” 91% 9% Yes No 89% 11% Agree Disagree “After I pass a quiz, even though after several tries, I feel very confident in what I have learned in the course” Do you prefer specifications grading instead of a "traditional" three in-class exams and a final? 89% 11% Agree Disagree

Midpoint Student Survey in Principles of Chemistry I

22

Data obtained from n=45 students

“Specs Grading has changed how I learn Chemistry” “I didn't think I could do well in chemistry or learn chemistry, but specs grading is changing that (so far)” “Specs Grading allows me to choose what grade I can get in the class at the end of the semester (I have control over my own grade if I want to)”

98% 2%

Agree Disagree

91% 9%

Agree Disagree

80% 20%

Agree Disagree

STUDENT #14 – ORGANIC 1

1

Repeating student

2

“Excited” about tokens

3

At Midterm

C

4

Retook EO quizzes at least 1 x

5

Retook 12 (out of 15) GO quizzes at least 1 x

6

FINAL GRADE

A

23

Student #14 – Organic Chemistry I

STUDENT #3 - RILEY

1

Athlete

2

Wary about “All

• r

Nothing”

3

At Midterm

C

4

Retook 4 EO quizzes at least 1 x

5

Retook 8 (out of 15) GO quizzes at least 1 x

6

FINAL GRADE

A

24

Student #3 – Organic Chemistry I

SPECIFICATIONS GRADING – REINFORCEMENT FOR RETENTION

• Reinforcement = “really good base knowledge for

Orgo 2”

• “…I had to make adjustments on the stuff I wasn’t

retaining knowledge on…”

25

• “It gives me the power back over my own grade.”
• “Lots of quizzes…rather than cramming at the last minute.”
• “…helps to assess how well you…know the material.”
• “…more likely to go back and try and relearn the material.

I feel less discouraged.”

26

Student Comments (so far..)

Student Comments (so far..)

• “You aren't stressed about failing the class.”
• “It forces me to keep up with the material”
• “It is stressful at first, until I got the hang of it..”

• “You making me take that darn quiz, like, 7 times, made me learn

arrows back and forth. Now I am teaching my friends how to draw arrows in Orgo 2.” (CHEM 2211)

• “Appreciate the opportunity to go back and fix mistakes on quizzes

(re-do quiz) without penalty” (CHEM1211, CHEM1212, PHYS1112)

• “You have to be super precise, which is annoying at first, but it makes

sure you really know it!” (CHEM 2211)

28

Student Feedback at the End of the Semester

Criticisms of Specs Grading from Students

• “…if you miss one tiny detail it makes the entire problem

wrong.”

• “It does require the student to master the material

quicker.”

• “I feel like there is always a quiz to study for.”
• “The (quizzes) are very challenging.”
• “I am sacrificing sleep to study the best I can.”

29

Perceived Downsides to Specifications Grading

• Students miss being able to earn partial credit
• “You have to pass every test!”
• Must keep up with course materials
• Assessments are frequent

30

REFLECTIONS

• Limit use of Tokens
• Force use of Tokens
• Alignment of EO/GO with Chapters
• Improve Assessments
• Retake “suffering”

–Instructor –Student - used to “skating by” with partial credit

31

Reflections

Grading Spreadsheet – One Way To Track Grades

32

Future Directions for the Group

• Faculty involved have started a learning community on

campus

• Analysis of student error types on quizzes
• Analysis of student performance on:

–subsequence courses –Interdisciplinary courses

• Teaching and learning workshop to help interested faculty

get on board

33

Specifications Grading – Why Onboard?

• Increase rigor – course centered on core

content/skills

• Improve student-teacher relations
• Emphasize “what counts”
• Pride in “art of teaching”

Overall focus: what is BEST for student

34

ACKNOWLEDGEMENTS

• Georgia Gwinnett College School of Science and

Technology

• Dr. Joshua Ring
• Drs. Tsoi, M., Anzovino, M., Erickson, A.H., Harmon, H.,

Henary, E. Lively, A., Perell-Gerson, K., Villanueva, O., Whitney, M.G., Woodbridge, C.

• Dr. Linda Nilson
• Our families and friends

CONTACT INFORMATION

Georgia Gwinnett College School of Science and Technology ATTN: Dr. Mai Yin Tsoi

mtsoi@ggc.edu