Detecting Detecting Outline DIF/DSF with DIF/DSF with PCMtrees PCMtrees Detecting Differential Item and Testing for DIF in Testing for DIF in Testing for DIF in the Rasch model the RM the RM Differential Step Functioning with Standard model tests Standard tests Standard tests Model-based recursive Model-based recursive Partial Credit Trees Model-based recursive partitioning partitioning partitioning Extension to the Extension to the Extending the model-based recursive partitioning approach PCM PCM Basil Abou El-Komboz, Achim Zeileis and Carolin Strobl DIF/DSF in the PCM DIF/DSF in the PCM to the Partial Credit Model (PCM) (Un)ordered threshold (Un)ordered threshold parameters parameters Differential item and step functioning in the PCM Visualization Visualization Example: Verbal Example: Verbal (Un)ordered threshold parameters in the PCM Aggression data Aggression data Summary Visualization in Partial Credit trees Summary References Example: Verbal Aggression data References Summary Detecting Detecting Differential Item Functioning (DIF) Standard model tests DIF/DSF with DIF/DSF with PCMtrees PCMtrees Testing for DIF in ◮ tests for k given groups Testing for DIF in the RM the RM graphical test, Andersen’s Likelihood-Ratio Test, Standard tests Standard tests Model-based recursive Model-based recursive Wald Tests partitioning partitioning is present when one or more items of a test Extension to the Extension to the + straightforward interpretation PCM PCM DIF/DSF in the PCM − only detect DIF in specified groups DIF/DSF in the PCM ◮ are easier or harder to solve for certain subjects (Un)ordered threshold (Un)ordered threshold parameters parameters ◮ latent-class approach Visualization Visualization Example: Verbal Example: Verbal ◮ even though they have the same latent trait Rost’s “Mixed” (mixture) Rasch model Aggression data Aggression data Summary Summary + identifies previously unknown groups with DIF References References − groups are not directly interpretable ⇒ 2nd step: describe groups with covariates (e.g., Cohen and Bolt, 2005)
Detecting Detecting Standard model tests Standard model tests DIF/DSF with DIF/DSF with PCMtrees PCMtrees 3 Testing for DIF in ◮ tests for k given groups Testing for DIF in the RM the RM 2 graphical test, Andersen’s Likelihood-Ratio Test, Standard tests Standard tests Model-based recursive Model-based recursive 12 ● Wald Tests ● partitioning partitioning Geschlecht = Frau 1 1 7 ● ● ● ● Extension to the Extension to the + straightforward interpretation PCM PCM 5 ● ● 0 11 ● ● 89 DIF/DSF in the PCM DIF/DSF in the PCM ● − only detect DIF in specified groups 6 ● 3 2 ● ● ● ● ● ● ● ● 4 ● ● (Un)ordered threshold (Un)ordered threshold 10 ● ● parameters parameters −1 ◮ latent-class approach Visualization Visualization Example: Verbal Example: Verbal Rost’s “Mixed” (mixture) Rasch model Aggression data Aggression data −2 Summary Summary + identifies previously unknown groups with DIF −3 References References − groups are not directly interpretable −3 −2 −1 0 1 2 3 ⇒ 2nd step: describe groups with covariates Geschlecht = Mann (e.g., Cohen and Bolt, 2005) (Mair, Hatzinger, and Maier, 2010, package eRm ) Detecting Detecting New: Model-based recursive partitioning Approach used in psychotree takes care of... DIF/DSF with DIF/DSF with PCMtrees PCMtrees ◮ selecting splitting variables ⇔ parameter instability tests + identifies previously unknown groups with DIF Testing for DIF in Testing for DIF in 0.4 + straightforward interpretation the RM the RM 1 score contributions gender Standard tests 0.2 Standard tests p = 0.006 Model-based recursive Model-based recursive partitioning partitioning 0 male female Extension to the Extension to the 2 PCM PCM age −0.4 p < 0.001 DIF/DSF in the PCM DIF/DSF in the PCM (Un)ordered threshold (Un)ordered threshold ≤ 34 > 34 parameters parameters 25 30 35 40 45 Visualization Visualization age Node 3 (n = 35) Node 4 (n = 74) Node 5 (n = 91) 4.66 4.66 4.66 ◮ selecting optimal cutpoints Example: Verbal Example: Verbal ● Aggression data Aggression data ● ◮ other multiple testing issues ● ● ● Summary Summary ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ◮ between variables in each split ● ● References References ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ◮ over successive splits −2.68 −2.68 −2.68 1 20 1 20 1 20 (Zeileis and Hornik, 2007; Zeileis, Hothorn, and Hornik, 2008; function raschtree in package psychotree Strobl, Malley, and Tutz, 2009; Strobl, Kopf, and Zeileis, 2010a,b)
Detecting Detecting Extending the model-based partitioning approach Extending the model-based partitioning approach DIF/DSF with DIF/DSF with PCMtrees PCMtrees Rasch trees 1 Rasch trees 1 gender gender p = 0.006 p = 0.006 Testing for DIF in Testing for DIF in male female male female the RM the RM 2 2 age age p < 0.001 p < 0.001 Standard tests Standard tests ≤ 34 > 34 ≤ 34 > 34 Model-based recursive Model-based recursive partitioning partitioning Node 3 (n = 35) Node 4 (n = 74) Node 5 (n = 91) Node 3 (n = 35) Node 4 (n = 74) Node 5 (n = 91) 4.66 4.66 4.66 4.66 4.66 4.66 ● ● Extension to the Extension to the ● ● ● ● ● ● ● ● ● ● ● ● ● PCM ● PCM ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● DIF/DSF in the PCM ● ● ● ● ● DIF/DSF in the PCM ● ● ● ● ● ● ● ● ● ● ● ● ● ● −2.68 −2.68 −2.68 −2.68 −2.68 −2.68 (Un)ordered threshold (Un)ordered threshold 1 20 1 20 1 20 1 20 1 20 1 20 parameters parameters Visualization Visualization Partial Credit trees Example: Verbal Example: Verbal Aggression data Aggression data Summary Summary References References Detecting Detecting Extending the model-based partitioning approach (Un)ordered threshold parameters in the PCM DIF/DSF with DIF/DSF with PCMtrees PCMtrees Rasch model 1.0 δ j1 δ j2 δ j3 Testing for DIF in Testing for DIF in the RM the RM ◮ scores are 0 or 1 0.8 Standard tests Standard tests Model-based recursive Model-based recursive ◮ each item has one location parameter = difficulty P(u ij =c| θ i , δ j1 ,..., δ j3 ) partitioning partitioning 0.6 Extension to the Extension to the ◮ DIF means item is more/less difficult for certain group PCM PCM 0 1 2 3 0.4 DIF/DSF in the PCM DIF/DSF in the PCM (Un)ordered threshold (Un)ordered threshold Partial Credit model parameters parameters 0.2 Visualization Visualization ◮ scores are between 0 and m j Example: Verbal Example: Verbal Aggression data Aggression data 0.0 ◮ different parametrizations: e.g. m j thresholds Summary Summary −5 0 5 10 15 � c References k =0 ( θ i − δ jk ) References ◮ DIF means entire item is more/less difficult e P ( u ij = c | θ i , δ j 1 , . . . , δ jm j ) = � m j � l k =0 ( θ i − δ jk ) l =0 e ◮ DSF means some steps are more/less difficult with � 0 (may cancel out so there is no overall DIF) k =0 ( θ i − δ jk ) = 0
Recommend
More recommend
