Suttida Rakkapao Department of Physics, Faculty of Science Prince - - PowerPoint PPT Presentation

International Conference on Teaching Physics Innovatively August 17-19, 2015, Budapest, Hungary

Suttida Rakkapao

Department of Physics, Faculty of Science Prince of Songkla University THAILAND

An Item Response Theory Analysis for the Test of Understanding of Vectors (TUV)

OUTLINE

• Test of Understanding of Vectors (TUV)
• Objective
• Item Response Theory (IRT)
• Results and Discussion
• Conclusions

2

3

TUV is a 20-item 5-choice research-based conceptual test of vectors (Barniol and Zavala, 2014).

4

Ten vector concepts used in the introductory course of a university level Vector Concept

• 1. Direction of a vector
• 2. Magnitude of a vector
• 3. Component of a vector
• 4. Unit vector in the Cartesian plane
• 5. Graphic representation of a vector
• 6. Vector addition
• 7. Vector subtraction
• 8. Scalar multiplication of a vector
• 9. Dot product
• 10. Cross product

5

Examples of TUV

6

The TUV is a reliable assessment tool based-on the classical test methods.

7

Item difficulty index (P) and Item discriminatory index (D) for item 1-10 of the TUV Mexican students (N=423) Thai students (N=774)

item 0.28 0.11 0.20 0.55 0.61 0.80 0.59 0.24 0.38 0.50 0.59 0.17 0.26 0.53 0.52 0.29 0.51 0.49 0.51 0.61 Desired values [0.3, 0.9] Desired values [ ≥ 0.3]

One of the most important shortcomings of Classical Test Theory (CTT) is the item parameters depend on test-takers.

OBJECTIVE

8

To analyze the test of understanding

• f

vectors (TUV) using a

• ne-

parameter logistic model (1PL) of an item response theory (IRT), called Rasch model.

9

Item Response Theory (IRT)

( )

1 ( ) (1 )1

a b

P c c e





 

   

where thedifficultyparameter thediscrimination parameter = the guessing parameter = theabilitylevel ( )= the probability that an examineewith that ability willgive a correct answer to theitem b a c P    

c = 0 a = 1

person parameter Item parameters

10

(http://www.jmetrik.com/index.php)

JMetrik free software for Rasch Model

11

An example of the Item Characteristic Curve (ICC) (created by PARSCALE)

( ) 0.5 P   ( ) (1 ) / 2 P b c  

Desired values [ -2, 2]

Difficulty index by CTT

12

Difficulty index by IRT (1PL)

Desired values [ 0.3, 0.9] Desired values [ -2, 2] 0.62 2.08 1.06

• 0.88 -0.60 -2.30 -1.09 0.80
• 0.01 -0.59

Item 1-10

Results and Discussion (1/3)

Difficulty index by CTT

13

Difficulty index by IRT (1PL)

Desired values [ 0.3, 0.9] Desired values [ -2, 2] 0.10 1.64 0.09

• 0.91 0.15

0.29 0.80 0.42 0.42

• 1.85

Item 11-20

Results and Discussion (2/3)

14

Results and Discussion (3/3)

Item 6 of dot product concept of TUV seem to be an easy question, proved by the IRT (1PL).

CONCLUSIONS

• Overall, the TUV contained items with different

levels of difficulty values for monitoring students in different abilities.

• The IRT can be used to develop multiple-choice

tests and to better understand the results, due to the independent of test-takers.

15

[1] P. Barniol and G. Zavala, Test of understanding of vectors: A reliable multiple-choice vector concept test, Phys. Rev. ST Phys.

• Educ. Res. 10, 010121 (2014).

[2] R. K. Hambleton, H. Swaminathan and H. Rogers, Fundamentals

• f Item Response Theory (Sage Publications, Inc., California, 1991).

[3] http://www.jmetrik.com/index.php

16

REFERENCES

ACKNOWLEDGEMENT

Department of Physics Faculty of Science Prince of Songkla University

This study was supported by the Research Support from Faculty of Science, Prince of Songkla University (budget year 2015).

17