Models for Replicated Discrimination Tests: A Synthesis of Latent - - PowerPoint PPT Presentation

Models for Replicated Discrimination Tests

Models for Replicated Discrimination Tests: A Synthesis of Latent Class Mixture Models and Generalized Linear Mixed Models

Rune Haubo Bojesen Christensen & Per Bruun Brockhoff

DTU Informatics Section for Statistics Technical University of Denmark rhbc@imm.dtu.dk

August 13 2008

Rune H B Christensen (DTU) Models for Replicated Discrimination Tests UseR! 2008, Dortmund 1 / 19

Models for Replicated Discrimination Tests Motivation

This Talk is About

A non-standard type of mixed models Applicable to a range of discrimination tests Focus: Insight into models—not computational methods Motivated by examples from sensometrics and psychometrics (but also applicable in signal detection, medical decision making etc.) The models extend existing models by

◮ Modelling the covariance structure ◮ Having a close connection to psychological theory of cognitive

decision making

◮ Providing inference for individuals via random effect estimates Rune H B Christensen (DTU) Models for Replicated Discrimination Tests UseR! 2008, Dortmund 2 / 19

Models for Replicated Discrimination Tests Outline

Outline

1

Background

2

Models for Independent Data

3

Models for Replicated Data

4

Examples

5

Summary

Rune H B Christensen (DTU) Models for Replicated Discrimination Tests UseR! 2008, Dortmund 3 / 19

Models for Replicated Discrimination Tests Background

A Replicated Discrimination Test

Example: Coke Inc. wants to substitute a sweetener in a diet coke, A with a cheaper alternative B. Coke Inc. employs 30 consumers in a discrimination (triangle) test Each consumer performs the test 10 times (replications) Can consumers distinguish between the two recipes? Why Replications? Advantages:

◮ Cheap and Easy: Substitute some assessors with replications ◮ Information on difference between assessors

Challenges:

◮ Observations are often correlated and not independent Rune H B Christensen (DTU) Models for Replicated Discrimination Tests UseR! 2008, Dortmund 4 / 19

Models for Replicated Discrimination Tests Background

The Triangle Test

Two regular products and one new product are presented to the consumer Two A- products (a1, a2) and

• ne B-product (b1) are

presented to the consumer Task: Identify the odd product δ = µB − µA: A measure of discriminal ability and difference between products.

0.0 0.1 0.2 0.3 0.4 δ A B a1 a2 b1 Sensory magnitude

Rune H B Christensen (DTU) Models for Replicated Discrimination Tests UseR! 2008, Dortmund 5 / 19

Models for Replicated Discrimination Tests Background

The Triangle Test

1 2 3 4 5 6 0.0 0.2 0.4 0.6 0.8 1.0 δ π

Triangle Psychometric Function A B a1 a2 b1 Triangle Model

Answers are binomial: A proportion of correct answers Yi ∼ Bin(πi; ni) π0 ≤ πi < 1 Guessing probability: π0 = 1/3 Relation between πi and δi:

πi = f(δi) = ∞

• Φ
• −z

√ 3 + δ

• 2/3
• +Φ
• −z

√ 3 − δ

• 2/3

φ(z)dz

Psychometric function: Relates the probability of a correct answer to the ability to discriminate

Rune H B Christensen (DTU) Models for Replicated Discrimination Tests UseR! 2008, Dortmund 6 / 19

Models for Replicated Discrimination Tests Models for Independent Data

The Basic (Naive) Model (GLM)

Yi ∼ Bin(πi, ni) πi = ftriangle(δi) δi = δ A Generalized Linear Model (GLM) with

◮ Binomial distribution ◮ Psychometric function as inverse link function ◮ Simple linear predictor

Assumes π and δ identical for all individuals Family-object; triangle in package sensR for use with glm

◮ Extends discrimination tests to allow for explanatory variables ◮ Prepares the way for mixed effect models for discrimination tests Rune H B Christensen (DTU) Models for Replicated Discrimination Tests UseR! 2008, Dortmund 7 / 19

Models for Replicated Discrimination Tests Models for Replicated Data

Models for Replicated Discrimination Tests

Ignore covariance structure in data

◮ Basic GLM

Marginal Models (adjust se’s for overdispersion)

◮ quasi-binomial GLM

Latent Class Mixture model Conditional Models (model covariance structure)

◮ Generalized Linear Mixed Model (GLMM) ◮ Synthesis of Mixture and GLMM Rune H B Christensen (DTU) Models for Replicated Discrimination Tests UseR! 2008, Dortmund 8 / 19

Models for Replicated Discrimination Tests Models for Replicated Data

Models for Replicated Discrimination Tests

Ignore covariance structure in data

◮ Basic GLM

Marginal Models (adjust se’s for overdispersion)

◮ quasi-binomial GLM

Latent Class Mixture model Conditional Models (model covariance structure)

◮ Generalized Linear Mixed Model (GLMM) ◮ Synthesis of Mixture and GLMM Rune H B Christensen (DTU) Models for Replicated Discrimination Tests UseR! 2008, Dortmund 8 / 19

Models for Replicated Discrimination Tests Models for Replicated Data

Latent Class Mixture Models

Two-class model for discriminal ability Pi ∼ Bernoulli(p) Yi|pi ∼ Bin(πi; ni) πi = ftriangle(δi) δi = if pi = 0 δ if pi = 1 No dispersion among subjects with positive δ Often an inappropriate assumption!

0.0 0.2 0.4 0.6 0.8 δi δ 1 − p = P(δi = 0) p = P(δi > 0)

Rune H B Christensen (DTU) Models for Replicated Discrimination Tests UseR! 2008, Dortmund 9 / 19

Models for Replicated Discrimination Tests Models for Replicated Data

Generalized Linear Mixed Model

bi ∼ N(0, σ2

δ)

Yi|bi ∼ Bin(πi; ni) πi = ftriangle(δi) δi = δ + bi Assumes a continuous distribution for subjects Allows for dispersion among subjects Subjects can have δ < 0! Impossible in the triangle test

−2 2 4 0.0 0.1 0.2 0.3 0.4 δi δ σδ

Rune H B Christensen (DTU) Models for Replicated Discrimination Tests UseR! 2008, Dortmund 10 / 19

Models for Replicated Discrimination Tests Models for Replicated Data

Generalized Linear Mixed Model

bi ∼ N(0, σ2

δ)

Yi|bi ∼ Bin(πi; ni) πi = ftriangle(δi) δi = δ + bi Assumes a continuous distribution for subjects Allows for dispersion among subjects Subjects can have δ < 0! Impossible in the triangle test

−2 2 4 0.0 0.1 0.2 0.3 0.4 δi δ σδ

Rune H B Christensen (DTU) Models for Replicated Discrimination Tests UseR! 2008, Dortmund 10 / 19

Models for Replicated Discrimination Tests An Appropriate Alternative

Latent Class Mixed Model

Yi|bi ∼ Bin(πi; ni) δi ∼ F(δ, σ2

δ)

πi = ftriangle(δi) δi = δ + bi f(δi) =

• 1 − p,

δi = 0

1 σδ φ

• δi−δ

σδ

• ,

δi > 0 p = 1 − Φ(−δ/σδ) One-dimensional random effect with two attributes:

◮ Class probabilities ˜

pi

◮ The magnitude of

discriminal ability ˜ δi −2 2 4 0.0 0.1 0.2 0.3 0.4 δi δ σδ

P(δi = 0)= Φ(−δ σ σδ

δ)

P(δi > > 0)= 1 − − Φ(− −δ σδ)

Rune H B Christensen (DTU) Models for Replicated Discrimination Tests UseR! 2008, Dortmund 11 / 19

Models for Replicated Discrimination Tests An Appropriate Alternative

Estimation in Latent Class Mixed Model

Likelihood function ∼ marginal density of y f(yi) = (1 − p)f1(yi) + pf2(yi) Likelihood at δi = 0: f1(yi) = ni

yi

• πyi

0 (1 − π0)(ni−yi)

Likelihood at δi > 0: f2(yi) = 1

p

∞ fπ(yi|δi)φ((δi − δ)/σδ) σδ dδi where fπ(yi|δi) = ni

yi

• πyi

i (1 − πi)(ni−yi)

Define likelihood function as R-function via integrate

• ptimize with optim

Structure motives an EM algorithm

Rune H B Christensen (DTU) Models for Replicated Discrimination Tests UseR! 2008, Dortmund 12 / 19

Models for Replicated Discrimination Tests An Appropriate Alternative

Attenuation Effect

Marginal link function:

πm = Eδi[E[yi|δi]] = π0(1 − p) + ∞ ftriangle(δi)φ((δi − δ)/σδ)/σδ dδi

Marginal estimates are closer to “stationary points” rather than closer to zero Marginal link function depends

• n σ2

δ

1 2 3 4 5 6 7 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 δ π

Original link function Marginal link function δ0 Stationary point

Rune H B Christensen (DTU) Models for Replicated Discrimination Tests UseR! 2008, Dortmund 13 / 19

Models for Replicated Discrimination Tests Examples

Example Triangle Data

Model δ se(δ) σδ p Basic (GLM) 1.67 0.186

• Overdisp. GLM

1.67 0.257 Proposed Model 1.62 0.234 1.08 93.3% Difference in estimate of δ (attenuation effect) Basic model gives too small se’s Clear variation between subjects Large proportion of discriminators

Rune H B Christensen (DTU) Models for Replicated Discrimination Tests UseR! 2008, Dortmund 14 / 19

Models for Replicated Discrimination Tests Examples

Example Triangle Data

Random effect estimates as Conditional expectations: ˜ δi = E[δi|yi] = ∞

−∞

δif(δi|yi)dδi = ∞

−∞ δif(yi|δi)f(δi) dδi

f(yi) One extra integral per individual

−2 2 4 0.0 0.1 0.2 0.3 0.4 δi δ ^ = 1.62 σ ^δ = 1.08

Rune H B Christensen (DTU) Models for Replicated Discrimination Tests UseR! 2008, Dortmund 15 / 19

Models for Replicated Discrimination Tests Examples

Example Triangle Data

Class probabilities as Conditional expectations: ˜ pi = E[pi|yi] = ∞

−∞

PidF(Pi|yi) =

• Pi∈(0,1)

Pif(yi|Pi)f(Pi)/f(yi) = pf2(yi) (1 − p)f1(yi) + pf2(yi) Random effects are almost normal Expect similar results from GLMM

−2 −1 1 2 0.5 1.0 1.5 2.0 2.5

0.99 0.8 1 1 0.94 0.89 1 1 0.99 0.96 0.91 0.76 0.94 0.94 0.88 0.83 0.83 1 0.99 0.96 0.96 0.96 0.96 0.91 0.85 0.85 0.98 0.98 0.93 0.99

Normal Quantiles Observed Quantiles (δi)

Rune H B Christensen (DTU) Models for Replicated Discrimination Tests UseR! 2008, Dortmund 16 / 19

Models for Replicated Discrimination Tests Examples

Example: 2-AFC Test

Random effect estimates: ˜ δi, ˜ pi Clear tail on the left Clear lower bound on δ Discrete nature of data more clear

−2 −1 1 2 0.0 0.5 1.0 1.5 2.0

1 0.92 1 0.98 0.92 0.98 0.98 0.18 1 1 0.92 0.22 0.61 0.79 0.61 0.79 0.61 0.35 0.46 0.46 0.27 0.27 0.98 0.98 0.46 0.46 0.92 0.92 0.98 0.98

Normal Quantiles Observed Quantiles (δi)

Rune H B Christensen (DTU) Models for Replicated Discrimination Tests UseR! 2008, Dortmund 17 / 19

Models for Replicated Discrimination Tests Summary

Summary and Challenges

Summary Close connection to psychological theory Model-type apply to a range of discrimination test protocols (eg. triangle, duo-trio, 2-AFC and 3-AFC) A synthesis of Latent Class Mixture Models and GLMMs One-dimensional random effects with two attributes

◮ Class probabilities ˜

pi

◮ The magnitude of discriminal ability ˜

δi

Challenges Extension to additional

◮ fixed effects (changes area of integration) ◮ random effects (multi-dimensional integrals)

Variance of random effect estimates Implementation with better computational methods

Rune H B Christensen (DTU) Models for Replicated Discrimination Tests UseR! 2008, Dortmund 18 / 19

Models for Replicated Discrimination Tests Summary

Acknowledgments

Thanks to the Program Committee Thanks to Professor Per Bruun Brockhoff Thank you for listening!

Rune H B Christensen (DTU) Models for Replicated Discrimination Tests UseR! 2008, Dortmund 19 / 19