Measurement Equivalence Using Structural Equation Modeling T - - PowerPoint PPT Presentation

Measurement Equivalence Using Structural Equation Modeling T echniques Joseph J. Sudano, PhD

Assistant Professor, Medicine and Population and Quantitative Health Sciences, Case Western Reserve University Center for Health Care Research and Policy at MetroHealth Medical Center

Academy Health Annual Research Meeting New Orleans, LA June 26th, 2017 Special Acknowledgments: Adam Perzynski, PhD;

Measurement Equivalence Using Structural Equation Modeling T echniques

No financial conflicts of interest…yet…although I will mention that I am the current CEO of software startup Global Health Metrics, LLC.

3

Conceptual Model of Traditional OLS Regression

Tired Happy Feel Blue Income Occupation Education Enjoy Life Depression D1 e4 e3 e2 e1 Latent construct or factor Observed or manifest variables

4

Causal Modeling and Confirmatory Factor Analysis

(A special instance of Structural Equation Modeling) Tired Happy Feel Blue Occupation Income Education Enjoy Life Depression

D3

e4 e3 e2 e1

D1 D2

Acknowledgement: This study was funded by Grant number R01-AG022459 from the NIH National Institute on Aging. (Med Care 2011;49: 480–488).

Measuring Disparities: Bias in Self-reported Health Among Spanish-speaking Patients

J.J. Sudano1,2, A.T. Perzynski1,2, T.E. Love2, S.A. Lewis1,B. Ruo3, D.W. Baker3

1The MetroHealth System, Cleveland, OH; 2Case Western Reserve

University School of Medicine, Cleveland, OH; 3Northwestern University Feinberg School of Medicine

Measurement Models using Multiple Indicators

 Single items are unreliable  Single cases prevent generalizability  Use multiple indicators and large samples to

estimate the values of the latent, unobserved variables or factors

 The SF-36v2 uses multiple indicators

describing multiple factors in order to measure health more reliably.

Measurement Model of the SF-36v2 Modified

Objective & Significance

 Do observed differences in self-reported

health reflect true differences in health?

• Cultural and language differences may

create measurement bias

• If outcomes aren’t measuring the same

thing in different groups, we have a problem

Measurement Equivalence & Factorial Invariance

 It is only possible to properly interpret group

differences after measurement equivalence has been established (Horn & McArdle, 1992; Steenkamp & Baumgartner, 1998).

 “It may be the case that the groups differ … but it

also may be the case that extraneous influences are giving rise to the observed difference.” Meredith & T eresi (2006 p. S69)

 The external validity of any conclusion regarding

group differences rests securely on whether the measurement equivalence of the scale has been established (Borsboom, 2006).

Cross-sectional Study

 N= 1281  Medical patients categorized into four

groups: White, Black, English-speaking Hispanic and Spanish-speaking Hispanic.

 Multigroup Confirmatory Factor Analysis

(MGCFA)

T wo Types of Invariance

 Metric (Weak) Invariance

• Are the item factor loadings equivalent across

groups?

• Is a one unit change in the item equal to a one

unit change in the factor score for all groups?

 Scalar (Strong) Invariance

• Are the item intercepts equivalent across groups?
• Unequal intercepts results in unequal scaling of

factor scores

T wo Types of Invariance

 Metric (Weak) Invariance

• Are the item factor loadings equivalent across

groups?

• Is a one unit change in the item equal to a one

unit change in the factor score for all groups?

 Scalar (Strong) Invariance

• Are the item intercepts equivalent across groups?
• Unequal intercepts results in unequal scaling of

factor scores

T wo Types of Invariance

 Metric (Weak) Invariance

• Are the item factor loadings equivalent across

groups?

• Is a one unit change in the item equal to a one

unit change in the factor score for all groups?

 Scalar (Strong) Invariance

• Are the item intercepts equivalent across groups?
• Unequal intercepts results in unequal scaling of

factor scores

Self-rated health health e4

What happens to the model fit when we constrain all of these paths (loadings) to be equal across groups?

Weak invariance

Table 1: Goodness of Fit for SF-36 Multigroup Factorial Invariance Testing (N = 1281)

Model Description RMSEA (95% CI) CFI B-S χ2* df Ref ∆RMSEA ∆CFI B-S ∆χ2 ∆df 1 Unconstrained Model 0.028(.017 - .030) 0.936 3001 2172 2 Metric Invariance (Factor Weights) 0.029(.028 - .030) 0.931 3110 2253 1 0.001

• 0.005

109 81 3 Scalar Invariance (Intercepts) 0.033(.032 - .034) 0.907 3215 2358 2 0.004

• 0.024

105 105 4 Partial Scalar Invariance (B=W=HS not HE) 0.033(.032 - .034) 0.909 3179 2323 2 0.004

• 0.022

69 70 5 Partial Scalar Invariance (B=W=HE not HS) 0.030(.029 - .032) 0.921 3180 2323 2 0.001

• 0.010

70 70 6 2nd Order Structural Invariance** 0.030(.029 - .032) 0.921 3187 2333 2 0.001

• 0.010

77 80 7 2nd & 3rd Order Structural Invariance** 0.030(.029 - .032) 0.921 3196 2339 2 0.001

• 0.010

86 86 * The bootstrapped Bollen - Stine χ2 value is reported because of significant (p<.01) multivariate non-normality. ** Structural factor weights are constrained equal for Blacks, Whites and Hispanic English (Hispanic Spanish are unconstrained).

Table 1: Goodness of Fit for SF36 Multigroup Factorial Invariance Testing (N = 1281)

Model Description RMSEA (95% CI) CFI B-S χ2* df Ref ∆RMSEA ∆CFI B-S ∆χ2 ∆df 1 Unconstrained Model 0.028(.017 - .030) 0.936 3001 2172 2 Metric Invariance (Factor Weights) 0.029(.028 - .030) 0.931 3110 2253 1 0.001

• 0.005

109 81 3 Scalar Invariance (Intercepts) 0.033(.032 - .034) 0.907 3215 2358 2 0.004

• 0.024

105 105 4 Partial Scalar Invariance (B=W=HS not HE) 0.033(.032 - .034) 0.909 3179 2323 2 0.004

• 0.022

69 70 5 Partial Scalar Invariance (B=W=HE not HS) 0.030(.029 - .032) 0.921 3180 2323 2 0.001

• 0.010

70 70 6 2nd Order Structural Invariance** 0.030(.029 - .032) 0.921 3187 2333 2 0.001

• 0.010

77 80 7 2nd & 3rd Order Structural Invariance** 0.030(.029 - .032) 0.921 3196 2339 2 0.001

• 0.010

86 86 * The bootstrapped Bollen - Stine χ2 value is reported because of significant (p<.01) multivariate non-normality. ** Structural factor weights are constrained equal for Blacks, Whites and Hispanic English (Hispanic Spanish are unconstrained).

The Unconstrained Model Fits the Data Well

Table 1: Goodness of Fit for SF-36v2 Multigroup Factorial Invariance Testing (N = 1281)

Model Description RMSEA (95% CI) CFI B-S χ2* df Ref ∆RMSEA ∆CFI B-S ∆χ2 ∆df 1 Unconstrained Model 0.028(.017 - .030) 0.936 3001 2172 2 Metric Invariance (Factor Weights) 0.029(.028 - .030) 0.931 3110 2253 1 0.001

• 0.005

109 81 3 Scalar Invariance (Intercepts) 0.033(.032 - .034) 0.907 3215 2358 2 0.004

• 0.024

105 105 4 Partial Scalar Invariance (B=W=HS not HE) 0.033(.032 - .034) 0.909 3179 2323 2 0.004

• 0.022

69 70 5 Partial Scalar Invariance (B=W=HE not HS) 0.030(.029 - .032) 0.921 3180 2323 2 0.001

• 0.010

70 70 6 2nd Order Structural Invariance** 0.030(.029 - .032) 0.921 3187 2333 2 0.001

• 0.010

77 80 7 2nd & 3rd Order Structural Invariance** 0.030(.029 - .032) 0.921 3196 2339 2 0.001

• 0.010

86 86 * The bootstrapped Bollen - Stine χ2 value is reported because of significant (p<.01) multivariate non-normality. ** Structural factor weights are constrained equal for Blacks, Whites and Hispanic English (Hispanic Spanish are unconstrained).

The Unconstrained Model fits the data well The model with factor loadings constrained still fits the data well.

Metric (Weak) Invariance was Confirmed

I forget what an intercept is

 Scalar (Strong) Invariance

• Are the item intercepts equivalent across

groups?

 Intercept: the intercept in a multiple

regression model is the mean for the response when all of the explanatory variables take on the value 0.

 Could be called the “starting point”

Table 1: Goodness of Fit for SF-36v2 Multigroup Factorial Invariance Testing (N = 1281)

Model Description RMSEA (95% CI) CFI B-S χ2* df Ref ∆RMSEA ∆CFI B-S ∆χ2 ∆df 1 Unconstrained Model 0.028(.017 - .030) 0.936 3001 2172 2 Metric Invariance (Factor Weights) 0.029(.028 - .030) 0.931 3110 2253 1 0.001

• 0.005

109 81 3 Scalar Invariance (Intercepts) 0.033(.032 - .034) 0.907 3215 2358 2 0.004

• 0.024

105 105 4 Partial Scalar Invariance (B=W=HS not HE) 0.033(.032 - .034) 0.909 3179 2323 2 0.004

• 0.022

69 70 5 Partial Scalar Invariance (B=W=HE not HS) 0.030(.029 - .032) 0.921 3180 2323 2 0.001

• 0.010

70 70 6 2nd Order Structural Invariance** 0.030(.029 - .032) 0.921 3187 2333 2 0.001

• 0.010

77 80 7 2nd & 3rd Order Structural Invariance** 0.030(.029 - .032) 0.921 3196 2339 2 0.001

• 0.010

86 86 * The bootstrapped Bollen - Stine χ2 value is reported because of significant (p<.01) multivariate non-normality. ** Structural factor weights are constrained equal for Blacks, Whites and Hispanic English (Hispanic Spanish are unconstrained).

The Unconstrained Model fits the data well The model with factor loadings constrained still fits the data well. Constraining the intercepts results in a worsening of model fit

Table 1: Goodness of Fit for SF-36v2 Multigroup Factorial Invariance Testing (N = 1281)

Model Description RMSEA (95% CI) CFI B-S χ2* df Ref ∆RMSEA ∆CFI B-S ∆χ2 ∆df 1 Unconstrained Model 0.028(.017 - .030) 0.936 3001 2172 2 Metric Invariance (Factor Weights) 0.029(.028 - .030) 0.931 3110 2253 1 0.001

• 0.005

109 81 3 Scalar Invariance (Intercepts) 0.033(.032 - .034) 0.907 3215 2358 2 0.004

• 0.024

105 105 4 Partial Scalar Invariance (B=W=HS not HE) 0.033(.032 - .034) 0.909 3179 2323 2 0.004

• 0.022

69 70 5 Partial Scalar Invariance (B=W=HE not HS) 0.030(.029 - .032) 0.921 3180 2323 2 0.001

• 0.010

70 70 6 2nd Order Structural Invariance** 0.030(.029 - .032) 0.921 3187 2333 2 0.001

• 0.010

77 80 7 2nd & 3rd Order Structural Invariance** 0.030(.029 - .032) 0.921 3196 2339 2 0.001

• 0.010

86 86 * The bootstrapped Bollen - Stine χ2 value is reported because of significant (p<.01) multivariate non-normality. ** Structural factor weights are constrained equal for Blacks, Whites and Hispanic English (Hispanic Spanish are unconstrained).

The model with factor loadings constrained still fits the data well. Constraining the intercepts results in a worsening of model fit The fit is still poor if you allow intercepts for English-speaking Hispanics to vary

Table 1: Goodness of Fit for SF-36v2 Multigroup Factorial Invariance Testing (N = 1281)

Model Description RMSEA (95% CI) CFI B-S χ2* df Ref ∆RMSEA ∆CFI B-S ∆χ2 ∆df 1 Unconstrained Model 0.028(.017 - .030) 0.936 3001 2172 2 Metric Invariance (Factor Weights) 0.029(.028 - .030) 0.931 3110 2253 1 0.001

• 0.005

109 81 3 Scalar Invariance (Intercepts) 0.033(.032 - .034) 0.907 3215 2358 2 0.004

• 0.024

105 105 4 Partial Scalar Invariance (B=W=HS not HE) 0.033(.032 - .034) 0.909 3179 2323 2 0.004

• 0.022

69 70 5 Partial Scalar Invariance (B=W=HE not HS) 0.030(.029 - .032) 0.921 3180 2323 2 0.001

• 0.010

70 70 6 2nd Order Structural Invariance** 0.030(.029 - .032) 0.921 3187 2333 2 0.001

• 0.010

77 80 7 2nd & 3rd Order Structural Invariance** 0.030(.029 - .032) 0.921 3196 2339 2 0.001

• 0.010

86 86 * The bootstrapped Bollen - Stine χ2 value is reported because of significant (p<.01) multivariate non-normality. ** Structural factor weights are constrained equal for Blacks, Whites and Hispanic English (Hispanic Spanish are unconstrained).

The model with factor loadings constrained still fits the data well. The fit is acceptable if you allow intercepts for Spanish speaking Hispanics to vary

Scalar (Strong) Invariance is NOT

• Confirmed. Intercepts are lower

for Spanish-speaking Hispanics on nearly all items

• Total measurement equivalence of the SF-36v2

does not exist for Spanish-speaking Hispanics

• Caution should be used when comparing mean

scores on the SF-36v2 across these language groups

• Okay to use coefficients from OLS regressions

Thank you very much for attending!! And as a gentle reminder: Have some fun with your science!!!

31