An Item Response Theory Analysis for the Test of Understanding of Vectors (TUV) Suttida Rakkapao Department of Physics, Faculty of Science Prince of Songkla University THAILAND International Conference on Teaching Physics Innovatively August 17-19, 2015, Budapest, Hungary
OUTLINE • Test of Understanding of Vectors (TUV) • Objective • Item Response Theory (IRT) • Results and Discussion • Conclusions 2
TUV is a 20-item 5-choice research-based conceptual test of vectors (Barniol and Zavala, 2014). 3
Ten vector concepts used in the introductory course of a university level Vector Concept 1. Direction of a vector 2. Magnitude of a vector 3. Component of a vector 4. Unit vector in the Cartesian plane 5. Graphic representation of a vector 6. Vector addition 7. Vector subtraction 8. Scalar multiplication of a vector 9. Dot product 10. Cross product 4
Examples of TUV 5
The TUV is a reliable assessment tool based-on the classical test methods. 6
Item difficulty index ( P ) and Item discriminatory index ( D ) for item 1-10 of the TUV Mexican students (N=423) One of the most important item shortcomings of Classical Test Theory (CTT) is the item 0.28 0.11 0.20 0.55 0.61 0.80 0.59 0.24 0.38 0.50 parameters depend on test-takers. Desired values [0.3, 0.9] 0.59 0.17 0.26 0.53 0.52 0.29 0.51 0.49 0.51 0.61 Desired values [ ≥ 0.3] Thai students (N=774) 7
OBJECTIVE To analyze the test of understanding of vectors (TUV) using a one- parameter logistic model (1PL) of an item response theory (IRT), called Rasch model. 8
Item Response Theory (IRT) c = 0 1 P ( ) c (1 c )1 a ( b ) e a = 1 where b thedifficultyparameter Item a thediscrimination parameter parameters c = the guessing parameter person = theabilitylevel parameter P ( )= the probability that an examineewith that ability willgive a correct answer to theitem 9
JMetrik free software for Rasch Model (http://www.jmetrik.com/index.php) 10
An example of the Item Characteristic Curve (ICC) (created by PARSCALE) P b ( ) (1 c ) / 2 P ( ) 0.5 Desired values [ -2, 2] 11
Results and Discussion (1/3) Item 1-10 Difficulty index by CTT Desired values [ 0.3, 0.9] 0.62 2.08 1.06 -0.88 -0.60 -2.30 -1.09 0.80 -0.01 -0.59 Desired values [ -2, 2] Difficulty index by IRT (1PL) 12
Results and Discussion (2/3) Item 11-20 Difficulty index by CTT Desired values [ 0.3, 0.9] 0.10 1.64 0.09 -0.91 0.15 0.29 0.80 0.42 0.42 -1.85 Desired values [ -2, 2] Difficulty index by IRT (1PL) 13
Results and Discussion (3/3) Item 6 of dot product concept of TUV seem to be an easy question, proved by the IRT (1PL). 14
CONCLUSIONS Overall, the TUV contained items with different levels of difficulty values for monitoring students in different abilities. The IRT can be used to develop multiple-choice tests and to better understand the results, due to the independent of test-takers. 15
REFERENCES [1] P. Barniol and G. Zavala, Test of understanding of vectors: A reliable multiple-choice vector concept test, Phys. Rev. ST Phys. Educ. Res. 10, 010121 (2014). [2] R. K. Hambleton, H. Swaminathan and H. Rogers, Fundamentals of Item Response Theory (Sage Publications, Inc., California, 1991). [3] http://www.jmetrik.com/index.php 16
ACKNOWLEDGEMENT This study was supported by the Research Support from Faculty of Science, Prince of Songkla University (budget year 2015). Department of Physics Faculty of Science Prince of Songkla University 17
Recommend
More recommend
