Best Practice for Designing Exams to Assess Student Learning Seema - - PowerPoint PPT Presentation
Best Practice for Designing Exams to Assess Student Learning Seema C. Shah-Fairbank, P.E., PhD Director of Assessment and Program Review, Division of Academic Affairs Associate Professor of Civil Engineering Office of Assessment and Program
Office of Assessment and Program Review, Academic Affairs
Seema C. Shah-Fairbank, P.E., PhD
Director of Assessment and Program Review, Division of Academic Affairs Associate Professor of Civil Engineering
Tiffany Frontino
Administrative Analyst, Assessment Program Review
During Workshop
▫ Binary, Matching, Multiple Choice, Short Answer, Open Ended
▫ Not a holistic test score, but evaluation of specific questions ▫ Go beyond content and think about what you want students to learn. ▫ Results will provide you with evidence of student learning, which you can act on.
Post Workshop
Types of Test Questions Advantages Disadvantages
Binary True/False Easy to correct Range of responses are limited Easy to compile results Test creator must possess content mastery Instruction easy to understand High odds of accidental correctness Matching Easy to correct Only test lower-level objectives Easy to create Medium odds for accidental correctness Allows for many items to get tested Very Applicable to paring items Multiple Choice Easy to correct Generally limited to fact-based questions Can include distractors Does not allow for explanation Permits testing of a large body of material rapidly Not that easy to create Short Answer Closed Question Easy to correct Limits richness of response Easy to create Takes longer to correct Easy to insert during instruction Can result in variability Open Ended Easy to create Requires strong subject matter knowledge Allows for freedom in response Correction is labor intensive Appropriate for "why" and "how" Allows highly diverse responses
True False
A B C
Where?
How? Who? What? When?
▫ Make inferences about the topic based on data provided ▫ Select the correct answer by solving a quantitative problem ▫ Possible effect based on a situation ▫ Make a conclusion based on a graph
Do
about a single situation/problem
Don’ts
crucial to student learning.
reasoning ability not knowledge of the
“never”
Do
more than 7 items.
Don’ts
matching.
a. Make some of the options and distractor negative.
c. Make all options and distractors similar in length.
STEM Distractors Correct Answer
Do’s
situation/problem
have plausible situation
to be on a specific item.
Don’ts
and options
pronoun eliminates options from being correct
discrimination
guessing
Do
answer in.
Don’ts
complex sentences.
Do
the beginning of a questions.
they understand the expectations.
prior to writing.
Don’ts
the problem statement.
an in-depth response.
▫ As many as you need - Capture student learning ▫ Think about the time (student need 3x as long)
▫ Group items by test type ▫ How to record answer ▫ Whether or not to show work ▫ Point values for each item ▫ Neatness
▫ Don’t let one early incorrect answer repeatedly penalize a student ▫ Alternative: tell students to assume specific answer to prior item ▫ Students capacity is a limited resources
Program Level – Student Learning Outcomes
specific and measureable behaviors students will display to verify learning has occurred at the program level.
means they are worded with active verbs stating observable behaviors) and, 3) measurability.
Course Learning Outcomes
exams that are covered in the present course.
for assessment.
identify the unique knowledge and skills expected to be gained from a given course.
▫ Set a criterion for success – 75% of students will get each item correct ▫ Determine percent-correct for each items
▫ Did the students understand the item ▫ Was the item written clearly ▫ Are changes need to be made within this course or a prior course ▫ Are there curricular changes needed at the program level ▫ Are there different pedogeological methods to teach the material.
Seema C. Shah-Fairbank, P.E., PhD shahfairbank@cpp.edu
▫ https://www.cpp.edu/~academic-programs/program-review
questions/
tips/developing-assignments/exams/questions-types-characteristics-suggestions
▫ Meaningful and Manageable Program Assessment ▫ The College Instructor’s Guide to Writing Test Items: Measuring Student Learning