Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
The paired comparison method for latent variables An Application of - - PowerPoint PPT Presentation
The paired comparison method for latent variables An Application of - - PowerPoint PPT Presentation
The paired comparison method for latent variables An Application of the Bradley Terry Model Almut Thomas Michaela Garei Regina Dittrich Reinhold Hatzinger nd Workshop on Psychometric Computing February th - February th
Introduction
Introduction ❍ Instrument ❍ Example ❍ Standard Procedure ❍ Way out Analysing FIT with Paired Comparisons Methods Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Instrument
Introduction ❍ Instrument ❍ Example ❍ Standard Procedure ❍ Way out Analysing FIT with Paired Comparisons Methods Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Freizeit-Interessen-Test (FIT), Stangl based on Holland’s () RIASEC-model . . . . Realistic: practical, physical
Instrument
Introduction ❍ Instrument ❍ Example ❍ Standard Procedure ❍ Way out Analysing FIT with Paired Comparisons Methods Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Freizeit-Interessen-Test (FIT), Stangl based on Holland’s () RIASEC-model . . . . Realistic: practical, physical Investigative: intellectual, scientific
Instrument
Introduction ❍ Instrument ❍ Example ❍ Standard Procedure ❍ Way out Analysing FIT with Paired Comparisons Methods Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Freizeit-Interessen-Test (FIT), Stangl based on Holland’s () RIASEC-model . . . . Realistic: practical, physical Investigative: intellectual, scientific Artistic: creative, independent
Instrument
Introduction ❍ Instrument ❍ Example ❍ Standard Procedure ❍ Way out Analysing FIT with Paired Comparisons Methods Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Freizeit-Interessen-Test (FIT), Stangl based on Holland’s () RIASEC-model . . . . Realistic: practical, physical Investigative: intellectual, scientific Artistic: creative, independent Social: supporting, helping
Instrument
Introduction ❍ Instrument ❍ Example ❍ Standard Procedure ❍ Way out Analysing FIT with Paired Comparisons Methods Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Freizeit-Interessen-Test (FIT), Stangl based on Holland’s () RIASEC-model . . . . Realistic: practical, physical Investigative: intellectual, scientific Artistic: creative, independent Social: supporting, helping Enterprising: competitive, persuading
Instrument
Introduction ❍ Instrument ❍ Example ❍ Standard Procedure ❍ Way out Analysing FIT with Paired Comparisons Methods Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Freizeit-Interessen-Test (FIT), Stangl based on Holland’s () RIASEC-model . . . . Realistic: practical, physical Investigative: intellectual, scientific Artistic: creative, independent Social: supporting, helping Enterprising: competitive, persuading Conventional: detail-oriented, organizing
Example
Introduction ❍ Instrument ❍ Example ❍ Standard Procedure ❍ Way out Analysing FIT with Paired Comparisons Methods Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Which of the two alternatives would you prefer?
Example
Introduction ❍ Instrument ❍ Example ❍ Standard Procedure ❍ Way out Analysing FIT with Paired Comparisons Methods Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Which of the two alternatives would you prefer?
Build a greenhouse (R)
- grow and maintain
rare plants (I) in your own garden.
Example
Introduction ❍ Instrument ❍ Example ❍ Standard Procedure ❍ Way out Analysing FIT with Paired Comparisons Methods Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Which of the two alternatives would you prefer?
Build a greenhouse (R)
- grow and maintain
rare plants (I) in your own garden. Play as a musician (A)
- be a conductor (E)
in a folk group.
Example
Introduction ❍ Instrument ❍ Example ❍ Standard Procedure ❍ Way out Analysing FIT with Paired Comparisons Methods Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Which of the two alternatives would you prefer?
Build a greenhouse (R)
- grow and maintain
rare plants (I) in your own garden. Play as a musician (A)
- be a conductor (E)
in a folk group. Produce (R)
- sale (E)
christmas decoration.
Standard Procedure
Introduction ❍ Instrument ❍ Example ❍ Standard Procedure ❍ Way out Analysing FIT with Paired Comparisons Methods Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Sum the selected items of each scale Compare the means of the sub-scales
Standard Procedure
Introduction ❍ Instrument ❍ Example ❍ Standard Procedure ❍ Way out Analysing FIT with Paired Comparisons Methods Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Sum the selected items of each scale Compare the means of the sub-scales Each item has a different attractivity
Standard Procedure
Introduction ❍ Instrument ❍ Example ❍ Standard Procedure ❍ Way out Analysing FIT with Paired Comparisons Methods Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Sum the selected items of each scale Compare the means of the sub-scales Each item has a different attractivity The selection of an item depends on the offered alternative Comparison of sub-scale means is not appropriate
Way out
Introduction ❍ Instrument ❍ Example ❍ Standard Procedure ❍ Way out Analysing FIT with Paired Comparisons Methods Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Use of methods for Paired Comparisons
Way out
Introduction ❍ Instrument ❍ Example ❍ Standard Procedure ❍ Way out Analysing FIT with Paired Comparisons Methods Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Use of methods for Paired Comparisons FIT: different items different items for each sub-scale comparisons e.g. R : I, I : A,
Analysing FIT with Paired Comparisons Methods
Introduction Analysing FIT with Paired Comparisons Methods ❍ Problem ❍ Object Covariate ❍ Categorical Object Covariates ❍ Reparameterization Matrix ❍ Including Subject Covariates ❍ Sample ❍ Solution Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Problem
Introduction Analysing FIT with Paired Comparisons Methods ❍ Problem ❍ Object Covariate ❍ Categorical Object Covariates ❍ Reparameterization Matrix ❍ Including Subject Covariates ❍ Sample ❍ Solution Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
[R] [I] [I] [A] [A] [S] ... [R]
-
... [I]
-
[I]
-
[A]
-
[A]
-
[S]
-
... ...
Linear dependencies in the design matrix
Object Covariate
Introduction Analysing FIT with Paired Comparisons Methods ❍ Problem ❍ Object Covariate ❍ Categorical Object Covariates ❍ Reparameterization Matrix ❍ Including Subject Covariates ❍ Sample ❍ Solution Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Each sub-scale is treated as an object: R I A S E C
Object Covariate
Introduction Analysing FIT with Paired Comparisons Methods ❍ Problem ❍ Object Covariate ❍ Categorical Object Covariates ❍ Reparameterization Matrix ❍ Including Subject Covariates ❍ Sample ❍ Solution Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Each sub-scale is treated as an object: R I A S E C Each item is assigned to a sub-scale
Object Covariate
Introduction Analysing FIT with Paired Comparisons Methods ❍ Problem ❍ Object Covariate ❍ Categorical Object Covariates ❍ Reparameterization Matrix ❍ Including Subject Covariates ❍ Sample ❍ Solution Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Each sub-scale is treated as an object: R I A S E C Each item is assigned to a sub-scale
Categorical Object Covariates
Introduction Analysing FIT with Paired Comparisons Methods ❍ Problem ❍ Object Covariate ❍ Categorical Object Covariates ❍ Reparameterization Matrix ❍ Including Subject Covariates ❍ Sample ❍ Solution Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
The fact that an item belongs to one scale is treated as categorical object covariate
Categorical Object Covariates
Introduction Analysing FIT with Paired Comparisons Methods ❍ Problem ❍ Object Covariate ❍ Categorical Object Covariates ❍ Reparameterization Matrix ❍ Including Subject Covariates ❍ Sample ❍ Solution Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
The fact that an item belongs to one scale is treated as categorical object covariate e.g. item R has the attribute Realistic ⇒ on the object covariate R
Categorical Object Covariates
Introduction Analysing FIT with Paired Comparisons Methods ❍ Problem ❍ Object Covariate ❍ Categorical Object Covariates ❍ Reparameterization Matrix ❍ Including Subject Covariates ❍ Sample ❍ Solution Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
The fact that an item belongs to one scale is treated as categorical object covariate e.g. item R has the attribute Realistic ⇒ on the object covariate R ln m(jk)j = µ(jk)j + λO
j − λO k
Categorical Object Covariates
Introduction Analysing FIT with Paired Comparisons Methods ❍ Problem ❍ Object Covariate ❍ Categorical Object Covariates ❍ Reparameterization Matrix ❍ Including Subject Covariates ❍ Sample ❍ Solution Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
The fact that an item belongs to one scale is treated as categorical object covariate e.g. item R has the attribute Realistic ⇒ on the object covariate R ln m(jk)j = µ(jk)j + λO
j − λO k
linear reparameterization λO
j = xj · R + xj · I + xj · A + xj · S + xj · E + xj · C
Categorical Object Covariates
Introduction Analysing FIT with Paired Comparisons Methods ❍ Problem ❍ Object Covariate ❍ Categorical Object Covariates ❍ Reparameterization Matrix ❍ Including Subject Covariates ❍ Sample ❍ Solution Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
The fact that an item belongs to one scale is treated as categorical object covariate e.g. item R has the attribute Realistic ⇒ on the object covariate R ln m(jk)j = µ(jk)j + λO
j − λO k
linear reparameterization λO
j = xj · R + xj · I + xj · A + xj · S + xj · E + xj · C
λO
R = · R
Reparameterization Matrix
Introduction Analysing FIT with Paired Comparisons Methods ❍ Problem ❍ Object Covariate ❍ Categorical Object Covariates ❍ Reparameterization Matrix ❍ Including Subject Covariates ❍ Sample ❍ Solution Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
R I A S E C R I I A A S . . .
Including Subject Covariates
Introduction Analysing FIT with Paired Comparisons Methods ❍ Problem ❍ Object Covariate ❍ Categorical Object Covariates ❍ Reparameterization Matrix ❍ Including Subject Covariates ❍ Sample ❍ Solution Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
account for subject covariates: sex (G)
Including Subject Covariates
Introduction Analysing FIT with Paired Comparisons Methods ❍ Problem ❍ Object Covariate ❍ Categorical Object Covariates ❍ Reparameterization Matrix ❍ Including Subject Covariates ❍ Sample ❍ Solution Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
account for subject covariates: sex (G) ln m(jk)j|g = µ(jk)jg + λO
j − λO k + λOS jg − λOS kg
Including Subject Covariates
Introduction Analysing FIT with Paired Comparisons Methods ❍ Problem ❍ Object Covariate ❍ Categorical Object Covariates ❍ Reparameterization Matrix ❍ Including Subject Covariates ❍ Sample ❍ Solution Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
account for subject covariates: sex (G) ln m(jk)j|g = µ(jk)jg + λO
j − λO k + λOS jg − λOS kg
Sample
Introduction Analysing FIT with Paired Comparisons Methods ❍ Problem ❍ Object Covariate ❍ Categorical Object Covariates ❍ Reparameterization Matrix ❍ Including Subject Covariates ❍ Sample ❍ Solution Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Students of the University of Klagenfurt F Cultural Technical G sciences sciences total female male
Solution
Introduction Analysing FIT with Paired Comparisons Methods ❍ Problem ❍ Object Covariate ❍ Categorical Object Covariates ❍ Reparameterization Matrix ❍ Including Subject Covariates ❍ Sample ❍ Solution Results Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Design matrix y µ R I A S E C G F R y
-
I y
-
I y
-
A y
-
A y
-
. . .
Results
Introduction Analysing FIT with Paired Comparisons Methods Results ❍ Model Selection ❍ Worthplot Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Model Selection
Introduction Analysing FIT with Paired Comparisons Methods Results ❍ Model Selection ❍ Worthplot Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
anova(G+F,G,F)
- Resid. Df Resid. Dev
Df Deviance G+F . G .
-
- .
F .
- .
Worthplot
Introduction Analysing FIT with Paired Comparisons Methods Results ❍ Model Selection ❍ Worthplot Possible Refinement Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
estimated worth
Preferred Interests
0.10 0.15 0.20 female male R C I S E A A E C I S R
Possible Refinement
Introduction Analysing FIT with Paired Comparisons Methods Results Possible Refinement ❍ Account for the Differences ❍ Account for the Differences ❍ Item Difficulties ❍ Reparameterization Matrix ❍ Design Matrix ❍ Account for Item Difficulties Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Account for the Differences
Introduction Analysing FIT with Paired Comparisons Methods Results Possible Refinement ❍ Account for the Differences ❍ Account for the Differences ❍ Item Difficulties ❍ Reparameterization Matrix ❍ Design Matrix ❍ Account for Item Difficulties Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Whom would you choose as your statistical consultant?
Account for the Differences
Introduction Analysing FIT with Paired Comparisons Methods Results Possible Refinement ❍ Account for the Differences ❍ Account for the Differences ❍ Item Difficulties ❍ Reparameterization Matrix ❍ Design Matrix ❍ Account for Item Difficulties Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
easy items difficult items low weights high weights
Item Difficulties
Introduction Analysing FIT with Paired Comparisons Methods Results Possible Refinement ❍ Account for the Differences ❍ Account for the Differences ❍ Item Difficulties ❍ Reparameterization Matrix ❍ Design Matrix ❍ Account for Item Difficulties Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Where do we get these weights from?
Item Difficulties
Introduction Analysing FIT with Paired Comparisons Methods Results Possible Refinement ❍ Account for the Differences ❍ Account for the Differences ❍ Item Difficulties ❍ Reparameterization Matrix ❍ Design Matrix ❍ Account for Item Difficulties Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Where do we get these weights from? item difficulties from Rasch Model
Reparameterization Matrix
Introduction Analysing FIT with Paired Comparisons Methods Results Possible Refinement ❍ Account for the Differences ❍ Account for the Differences ❍ Item Difficulties ❍ Reparameterization Matrix ❍ Design Matrix ❍ Account for Item Difficulties Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
item difficulties are continous object covariates choosing item R wants an certain amount of the attribute ’Realistic’
[R] [I] [A] [S] [E] [C] R . . . . . . I . . . . . . I . . . . . . A . . . . . . A . . . . . . S . . . . . .
Design Matrix
Introduction Analysing FIT with Paired Comparisons Methods Results Possible Refinement ❍ Account for the Differences ❍ Account for the Differences ❍ Item Difficulties ❍ Reparameterization Matrix ❍ Design Matrix ❍ Account for Item Difficulties Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
y µ R I A S E C G F R y .
- .
I y
- .
. I y .
- .
A y
- .
. A y .
- .
. . .
Account for Item Difficulties
Introduction Analysing FIT with Paired Comparisons Methods Results Possible Refinement ❍ Account for the Differences ❍ Account for the Differences ❍ Item Difficulties ❍ Reparameterization Matrix ❍ Design Matrix ❍ Account for Item Difficulties Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
consider different item difficulties
Account for Item Difficulties
Introduction Analysing FIT with Paired Comparisons Methods Results Possible Refinement ❍ Account for the Differences ❍ Account for the Differences ❍ Item Difficulties ❍ Reparameterization Matrix ❍ Design Matrix ❍ Account for Item Difficulties Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
consider different item difficulties consider subject covariates sex and faculty
Account for Item Difficulties
Introduction Analysing FIT with Paired Comparisons Methods Results Possible Refinement ❍ Account for the Differences ❍ Account for the Differences ❍ Item Difficulties ❍ Reparameterization Matrix ❍ Design Matrix ❍ Account for Item Difficulties Results Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
consider different item difficulties consider subject covariates sex and faculty ln m(jk)j|l = µ(jk)jl + λO
j − λO k + λOS jl − λOS kl
λO
j = xj · R + xj · I + xj · A + xj · S + xj · E + xj · C
Results
Introduction Analysing FIT with Paired Comparisons Methods Results Possible Refinement Results ❍ Model Selection ❍ Worthplot Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Model Selection
Introduction Analysing FIT with Paired Comparisons Methods Results Possible Refinement Results ❍ Model Selection ❍ Worthplot Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
anova(G+F,G,F)
- Resid. Df Resid. Dev
Df Deviance G+F . G .
-
- .
F .
- .
Worthplot
Introduction Analysing FIT with Paired Comparisons Methods Results Possible Refinement Results ❍ Model Selection ❍ Worthplot Comparison of Results
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
estimated worth
Preferred Interests
0.10 0.15 0.20 female male R C E S I A A C E I R S
Comparison of Results
Introduction Analysing FIT with Paired Comparisons Methods Results Possible Refinement Results Comparison of Results ❍ Sub-Scale Means vs. Categorical Object Covariate ❍ Sub-Scale Means vs. Continous Object Covariate ❍ Categorical Object Covariates vs. Continous Object Covariates ❍ Conclusio ❍ Thank you
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
Sub-Scale Means vs. Categorical Object Covariate
Introduction Analysing FIT with Paired Comparisons Methods Results Possible Refinement Results Comparison of Results ❍ Sub-Scale Means vs. Categorical Object Covariate ❍ Sub-Scale Means vs. Continous Object Covariate ❍ Categorical Object Covariates vs. Continous Object Covariates ❍ Conclusio ❍ Thank you
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
estimated worth
Scale Means
0.12 0.14 0.16 0.18 0.20 0.22 female male R A C E I S A C R E S I estimated worth
Preferred Interests
0.10 0.15 0.20 female male R C I S E A A E C I S R
Sub-Scale Means vs. Continous Object Covariate
Introduction Analysing FIT with Paired Comparisons Methods Results Possible Refinement Results Comparison of Results ❍ Sub-Scale Means vs. Categorical Object Covariate ❍ Sub-Scale Means vs. Continous Object Covariate ❍ Categorical Object Covariates vs. Continous Object Covariates ❍ Conclusio ❍ Thank you
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
estimated worth
Scale Means
0.12 0.14 0.16 0.18 0.20 0.22 female male R A C E I S A C R E S I estimated worth
Preferred Interests
0.10 0.15 0.20 female male R C E S I A A C E I R S
Categorical Object Covariates vs. Continous Object Covariates
Introduction Analysing FIT with Paired Comparisons Methods Results Possible Refinement Results Comparison of Results ❍ Sub-Scale Means vs. Categorical Object Covariate ❍ Sub-Scale Means vs. Continous Object Covariate ❍ Categorical Object Covariates vs. Continous Object Covariates ❍ Conclusio ❍ Thank you
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
estimated worth
Preferred Interests
0.10 0.15 0.20 female male R C I S E A A E C I S R estimated worth
Preferred Interests
0.10 0.15 0.20 female male R C E S I A A C E I R S
AIC: . AIC: .
Conclusio
Introduction Analysing FIT with Paired Comparisons Methods Results Possible Refinement Results Comparison of Results ❍ Sub-Scale Means vs. Categorical Object Covariate ❍ Sub-Scale Means vs. Continous Object Covariate ❍ Categorical Object Covariates vs. Continous Object Covariates ❍ Conclusio ❍ Thank you
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –
the method of analysis makes a difference further refinement is still needed
Thank you
Introduction Analysing FIT with Paired Comparisons Methods Results Possible Refinement Results Comparison of Results ❍ Sub-Scale Means vs. Categorical Object Covariate ❍ Sub-Scale Means vs. Continous Object Covariate ❍ Categorical Object Covariates vs. Continous Object Covariates ❍ Conclusio ❍ Thank you
Thomas Gareiß Dittrich & Hatzinger The BTL for latent variables –