Shrinkage estimation of the three-parameter logistic model Michela - - PowerPoint PPT Presentation

Shrinkage estimation of the three-parameter logistic model

Michela Battauz (joint with Ruggero Bellio)

Department of Economics and Statistics University of Udine (Italy)

Psychoco 2020

Michela Battauz Shrinkage estimation of the 3PL model Psychoco 2020 1 / 20

The three-parameter logistic (3PL) model

Used for modelling the responses of a proficiency test with binary response items, when the probability of guessing is not zero. Probability of a correct response pij = Pr(Xij = 1∣θi;aj,bj,cj) to item j pij = cj + (1 − cj) exp{aj(θi − bj)} 1 + exp{aj(θi − bj)},

θi ability of person i Item parameters:

aj discrimination parameter bj difficulty parameter cj guessing parameter

−6 −4 −2 2 4 6 0.0 0.2 0.4 0.6 0.8 1.0 θ Prob

Michela Battauz Shrinkage estimation of the 3PL model Psychoco 2020 2 / 20

Maximum likelihood estimation

A convenient parameterization of the model, suitable for estimation is pij = cj + (1 − cj) exp(β1j + β2jθi) 1 + exp(β1j + β2jθi), with cj = exp(β3j) 1 + exp(β3j) . Marginal Maximum Likelihood Estimation (MLE) (Bock and Aitkin, 1981) usally adopted, where θi ∼ N(0,1) and the log-likelihood function is ℓ(β) =

n

∑

i=1

log ∫R

J

∏

j=1

pxij

ij (1 − pij)1−xij φ(θi)dθi,

Michela Battauz Shrinkage estimation of the 3PL model Psychoco 2020 3 / 20

The 3PL model in practice

Broadly used in applications Included in all major IRT software, including R packages ltm, mirt and TAM. Guessing parameters weakly identifiable (Patz and Junker, 1999), MLE has convergence problems. The estimates of the guessing parameters tend to have a negative bias.

n = 200

β ^

3

−25 −15 −5 50 100 150 200

n = 500

β ^

3

−12 −8 −4 50 100 150

n = 1000

β ^

3

−8 −6 −4 −2 50 100 150

n = 5000

β ^

3

−4 −3 −2 −1 50 100 150 200

Michela Battauz Shrinkage estimation of the 3PL model Psychoco 2020 4 / 20

Our proposal: shrinkage estimation of the 3PL model

Two main approaches: Penalized maximum likelihood estimation Model-based shrinkage estimation

Michela Battauz Shrinkage estimation of the 3PL model Psychoco 2020 5 / 20

Penalty on the guessing parameters

Penalized log-likelihood: ℓp(β) = ℓ(β) + J(β3) Penalty proportional to the log p.d.f. of the normal distribution J(β3) = − 1 2σ2

J

∑

j=1

(β3j − µ)2 , (implemented also in the mirt package) Ridge-type penalty J(β3) = −λ ∑

j<k

(β3j − β3k)2 , The two penalties are equivalent when µ = J−1 ∑j β3j, but the ridge-type penalty has only one tuning parameter. Empirical results were very similar, so we chose the ridge-type penalty.

Michela Battauz Shrinkage estimation of the 3PL model Psychoco 2020 6 / 20

Model-based shrinkage estimation

Application of the bias-reduction method (BR) proposed by Firth (1993) for a general parametric model, with estimating equation for β S(β) = ∇ℓ(β) − I(β)−1b(β), where I(β) is the expected Fisher information, and b(β) is the leading term of the asymptotic bias of the MLE. The estimator defined by solving S(β) = 0 has reduced finite sample bias, though it is asymptotically equivalent to the MLE. In many models for discrete data, a useful side effect of bias reduction is shrinkage of parameter estimates (Kosmidis, 2014).

Michela Battauz Shrinkage estimation of the 3PL model Psychoco 2020 7 / 20

Implementation

Both approaches were implemented the R package S3PL github.com/micbtz/S3PL. Integral approximated using Gaussian quadrature. Penalized likelihood:

package Rcpp to speed up computational time; tuning parameter λ selected using cross-validation; cross-validation error: the negative log-likelihood

Bias reduction:

package TMB for automatic differentiation and C++ implementation; Monte Carlo evaluation of required expected values; Parallel computing;

Michela Battauz Shrinkage estimation of the 3PL model Psychoco 2020 8 / 20

An illustrative example

Achievement data collected on students attending the third year of high school in Italy, tested in Mathematics n = 3843 students, J = 14 items Estimated correlation matrix of ˆ βMLE

ß11 ß21 ß31 ß12 ß22 ß32 ß13 ß23 ß33 ß14 ß24 ß34 ß11 ß21 ß31 ß12 ß22 ß32 ß13 ß23 ß33 ß14 ß24 ß34

Michela Battauz Shrinkage estimation of the 3PL model Psychoco 2020 9 / 20

Ridge-type penalization

0.0 0.5 1.0 1.5 2.0 3315.8 3316.0 3316.2 3316.4 λ CV error 0.0 0.5 1.0 1.5 2.0 0.00 0.10 0.20 0.30 λ c ^j

Michela Battauz Shrinkage estimation of the 3PL model Psychoco 2020 10 / 20

Comparison of estimates of the guessing parameters

• 0.0

0.1 0.2 0.3 0.4 0.5 0.0 0.1 0.2 0.3 0.4 0.5 MLE ridge

• 0.0

0.1 0.2 0.3 0.4 0.5 0.0 0.1 0.2 0.3 0.4 0.5 MLE BR

Ridge-type penalization yields a larger shrinkage of the parameters than BR.

Michela Battauz Shrinkage estimation of the 3PL model Psychoco 2020 11 / 20

A simulation study

True item parameters of 30 items taken from TIMSS 2015, Fourth Grade, Mathematics True guessing parameters:

TIMSS (very variable) constant cj = 0.2 ∀j

Sample size n = 200,500,1000 500 replications for each setting

Michela Battauz Shrinkage estimation of the 3PL model Psychoco 2020 12 / 20

Results of the simulation study

n = 200 constant guessing parameters

RMSE(β1) B(β1) RMSE(β2) B(β2) RMSE(β3) B(β3) MLE ridge BR 1 2 3 4 5 6 7

Michela Battauz Shrinkage estimation of the 3PL model Psychoco 2020 13 / 20

Results of the simulation study

n = 500 constant guessing parameters

RMSE(β1) B(β1) RMSE(β2) B(β2) RMSE(β3) B(β3) MLE ridge BR 0.0 0.5 1.0 1.5 2.0

Michela Battauz Shrinkage estimation of the 3PL model Psychoco 2020 14 / 20

Results of the simulation study

n = 1000 constant guessing parameters

RMSE(β1) B(β1) RMSE(β2) B(β2) RMSE(β3) B(β3) MLE ridge BR 0.0 0.5 1.0 1.5

Michela Battauz Shrinkage estimation of the 3PL model Psychoco 2020 15 / 20

Results of the simulation study

n = 200 variable guessing parameters

RMSE(β1) B(β1) RMSE(β2) B(β2) RMSE(β3) B(β3) MLE ridge BR 1 2 3 4 5

Michela Battauz Shrinkage estimation of the 3PL model Psychoco 2020 16 / 20

Results of the simulation study

n = 500 variable guessing parameters

RMSE(β1) B(β1) RMSE(β2) B(β2) RMSE(β3) B(β3) MLE ridge BR 0.0 0.5 1.0 1.5 2.0

Michela Battauz Shrinkage estimation of the 3PL model Psychoco 2020 17 / 20

Results of the simulation study

n = 1000 variable guessing parameters

RMSE(β1) B(β1) RMSE(β2) B(β2) RMSE(β3) B(β3) MLE ridge BR 0.0 0.5 1.0 1.5

Michela Battauz Shrinkage estimation of the 3PL model Psychoco 2020 18 / 20

Conclusions

MLE seems inaccurate even for large sample sizes. The BR method performs well for small sample sizes. For larger samples, ridge-type penalty performs better, especially when the true guessing parameters are constant.

Michela Battauz Shrinkage estimation of the 3PL model Psychoco 2020 19 / 20

References

Bock, R. D., Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443–459. Firth, D. (1993). Bias reduction of maximum likelihood estimates. Biometrika, 80, 27-38. Kosmidis, I. (2014). Bias in parametric estimation: reduction and useful side-effects. Wires Comp. Statist., 6, 185-196. Patz, R. J., Junker, B. W. (1999). Applications and extensions of MCMC in IRT: Multiple item types, missing data, and rated

• responses. J. Educ. Behav. Stat., 24, 342–366.

Thank you for your attention!

Michela Battauz Shrinkage estimation of the 3PL model Psychoco 2020 20 / 20