SLIDE 1
Introducing the research method
- f simulations
Comparing the validity of cut-score methods in higher education tests
Iris Yocarini
Introducing the research method of simulations Comparing the - - PowerPoint PPT Presentation
Introducing the research method of simulations Comparing the - - PowerPoint PPT Presentation
Introducing the research method of simulations Comparing the validity of cut-score methods in higher education tests Iris Yocarini Studying tests in Higher Education How accurate is the binding What is the most accurate way study advice
SLIDE 2
SLIDE 3
Researching tests in Higher Education
How accurate is the binding study advice decision? What is the most accurate way to correct for guessing on multiple choice tests? How accurate are different cut-score methods that determine the pass/fail test score on a test?
SLIDE 4
Researching tests in Higher Education
How accurate is the binding study advice decision? What is the most
accurate way to correct
for guessing on multiple choice tests? How accurate are different cut-score methods that determine the pass/fail test score on a test?
SLIDE 5
How to quantify accuracy?
SLIDE 6
Comparing cut-score methods in higher education tests
- Different methods
- Absolute method
- E.g. classical cut-score method
- Relative method
- Compromise method
- E.g. Cohen method
SLIDE 7
Comparing cut-score methods in higher education tests
- Previous studies
- Comparing consequences of different cut-score methods
From: Cohen-Schotanus, J., & Vleuten, C. P. M. (2010). A standard setting method with the best performing students as point of reference: Practical and affordable. Medical Teacher, 32, 154-160
Cut-scores
SLIDE 8
Comparing cut-score methods in higher education tests
- Previous studies
- Comparing consequences of different cut-score methods
From: Cohen-Schotanus, J., & Vleuten, C. P. M. (2010). A standard setting method with the best performing students as point of reference: Practical and affordable. Medical Teacher, 32, 154-160
Failure rates
SLIDE 9
To assess the accuracy simulations are required
SLIDE 10
Simulation studies
Simulations used for
- Predictions
- Performance
- Training
- Entertainment
- Education
- Proof
- Discovery
SLIDE 11
Simulation studies
General steps in simulating data
1.
Define a data generating model
2.
Simulate different conditions, varying parameters
3.
Introduce error or bias
4.
Evaluate input to output
SLIDE 12
Simulation studies
Challenges:
- Make assumptions about the true data generating model
- Complex behavior difficult to model
- With many factors you can’t model them all
- You obtain many results/ datasets
SLIDE 13
Simulation studies
Benefits
- Allows us to study aspects that are difficulty or impossible to investigate
- therwise
- Simulated data: no involvement/ physical interaction with the system used
to investigate
- Explicit and transparent research
- Many scenarios can be evaluated
- Assess which factor has more or less influence
SLIDE 14
Question? Mail me: yocarini@fsw.eur.nl
SLIDE 15
Simulation studies
Axelrod, R. (1997). Advancing the art of simulation in the social sciences. Obtaining, analyzing, and sharing results of computer
- models. Complexity,
3, 16-22. Cohen-Schotanus, J., & Vleuten, C. P. M. (2010). A standard setting method with the best performing students as point of reference: Practical and
- affordable. Medical Teacher, 32, 154-160.