Missing data due to a checklist-effect Dr. Thorsten Meyer 1) , Dr. - - PowerPoint PPT Presentation

▶
missing data due to a checklist effect
SMART_READER_LITE
LIVE PREVIEW

Missing data due to a checklist-effect Dr. Thorsten Meyer 1) , Dr. - - PowerPoint PPT Presentation

Sep 19, 2023 191 likes •361 views

Institut fr Sozialmedizin Campus Lbeck Missing data due to a checklist-effect Dr. Thorsten Meyer 1) , Dr. Ines Schfer 1) , Dr. Christine Matthis 1) Prof. Dr. Thomas Kohlmann 2) ; Dr. Oskar Mittag 1) 1) Institute for Social Medicine,

slide-1
SLIDE 1

Institut für Sozialmedizin Campus Lübeck

Folie 1 / 15

Missing data due to a ‘checklist-effect’

  • Dr. Thorsten Meyer1), Dr. Ines Schäfer1), Dr. Christine Matthis1)
  • Prof. Dr. Thomas Kohlmann2); Dr. Oskar Mittag1)

1) Institute for Social Medicine, University Clinics Schleswig-Holstein, Campus Luebeck 2) Institute for Community Medicine, Ernst-Moritz-Arndt-University, Greifswald

acknowledgement ‘Die Abschätzung von Rehabedarf bei aktiven Mitgliedern der Gesetzlichen Rentenversicherung: der Lübecker Algorithmus und seine Validierung’, A1-project, principal investigator: Prof. Dr. Dr. H. Raspe, Norddeutscher Verbundes für Rehabilitationsforschung, supported by BMBF and VDR (FKZ: 02 1 06)

slide-2
SLIDE 2

Institut für Sozialmedizin Campus Lübeck

Folie 2 / 15

Problem…

slide-3
SLIDE 3

Institut für Sozialmedizin Campus Lübeck

Folie 3 / 15

…and possible (simple-minded) solutions:

Som = 1.60 imputation by individual mean: Som = 1.15 imputation by group mean: Som = Missing according to manual: Som = 0.67 assumption of a „checklist effect“:

slide-4
SLIDE 4

Institut für Sozialmedizin Campus Lübeck

Folie 4 / 15

assumption: checklist-effect

In these respondents, all items with missing data in the respective scale are interpreted as „not at all“-responses.

definition by the following response pattern: (1) at least one missing value, and (2) at least one valid item response, and (3) no ‘not at all’-responses.

slide-5
SLIDE 5

Institut für Sozialmedizin Campus Lübeck

Folie 5 / 15

5 11 3 10 1 2 9 1 1 2 8 3 1 7 3 1 6 6 2 5 4 2 3 4 6 1 1 1 3 4 2 4 1 number of valid positive responses without “not-at-all”- responses 12 11 10 9 8 7 6 5 4 3 2 1 number of missing item responses

Number of missing item responses in relation to the number of valid positive responses (‘checklist effect’ highlighted in the diagonal) somatisation subscale of SCL-90-R

slide-6
SLIDE 6

Institut für Sozialmedizin Campus Lübeck

Folie 6 / 15

85,8 % (194) 12,4 % (28) work status blue-collar / manual worker white-collar / clerical worker 68,9 % (153) job full-time 78,1 % (178) 11,4 % (26) education secondary school (Hauptschule) secondary school (Realschule) 11,0 % (25) 75,4 % (172) 11,4 % (26) 2,2 % (5) family status single married divorced / separated widowed 73,2 % (167) male sex 50,1 (SD=6,5) age

  • n=228
  • primarily blue collar workers who previously

had filed applications for medical rehabilitation benefits (esp. due to back pain)

  • postal survey

Sample

slide-7
SLIDE 7

Institut für Sozialmedizin Campus Lübeck

Folie 7 / 15

no missing data checklist-effect

  • ther types of missing data

no valid item responses at all

75% 16% 9% 0%

somatisation subscale SCL-90-R

  • 1. prevalence of the checklist-effect?
  • 1. prevalence of the checklist-effect?
  • 2. stable phenomenon within the questionnaire?
slide-8
SLIDE 8

Institut für Sozialmedizin Campus Lübeck

Folie 8 / 15

succeeding items: questions on pain loci

slide-9
SLIDE 9

Institut für Sozialmedizin Campus Lübeck

Folie 9 / 15

depression (CES-D)

slide-10
SLIDE 10

Institut für Sozialmedizin Campus Lübeck

Folie 10 / 15

no missing data checklist-effect

  • ther types of missing data

no valid item responses at all

75% 16% 9% 0%

somatisation subscale SCL-90-R

  • 1. prevalence of the checklist-effect?
  • 2. stable phenomenon within the questionnaire?

83% 11% 6% 0% 87% 2% 10% 1%

pain items depressiveness CES-D

  • f those subjects with a checklist-effect in

the somatisation subscale, 7 out of 10 had a checklist effect in the pain items

  • all subjects with a checklist-effect in the pain

items had a checklist-effect in the somatisation subscale, too

slide-11
SLIDE 11

Institut für Sozialmedizin Campus Lübeck

Folie 11 / 15

  • 3. Do subjects with the postulated checklist-effect differ

from the other subjects with regard to social characteristics and health status?

No.

F=1.44; dfbetw=2; dfwithin=218 ;p=.239 η2=0.013 M=17.1 SD=8.8 M=16.3 SD=7.7 M=19.9 SD=10.6 depression (CES-D, 0-60) F=0.27; dfbetw=2; dfwithin=212; p=.76 η2=0.002 M=39.0 SD=17.9 M=37.8 SD=21.2 M=36.6 SD=19.9 vitality (SF-36, 0-100) χ2=3.395; df=6; p=.758; VC=.09 2 (1.2 %) 30 (17.6 %) 110 (64.7 %) 28 (16.5 %) 1 (5.0 %) 4 (20.0 %) 11 (55.0 %) 4 (20.0 %) 7 (20.6 %) 20 (58.8 %) 7 (20.6 %) health status good satisfactory not good bad χ2=3.7; df=2 p=.154; VC=.13 148b) (86.5 %) 23 (13.5 %) 15 (75.0 %) 5 (25.0 %) 28 (75.7 %) 9 (24.3 %) education lowera) medium or higherc) F=0.27; dfbetw=2; dfwithin=220; p=.36 η2=0.009 M=52,0 SD=7,6 M=50,3 SD=6,7 M=49,8 SD=6,3 age χ2=2.9; df=2 p=.229, VC=.11 125 (73.1 %) 12 (60.0 %) 30 (81.1 %) Sex male statistic, degrees of freedom, level of significance, effect size no missing data (n=171)

  • ther

missing data (n=20) „checklist

  • effect“

(n=37) test on difference between groups type of missing data

slide-12
SLIDE 12

Institut für Sozialmedizin Campus Lübeck

Folie 12 / 15

  • 4. results of different imputation procedures

0.61 0.58 0.62 0.67

sd

0.98 0.96 1.04 1.04

mean

1 28

N missing

227 228 228 200

N valid checklist- effect ML- estimation

(EM-algorythm)

manual- based + group mean manual- based

(max. 4 MD)

imputation M=.65 SD=.37 N=27

slide-13
SLIDE 13

Institut für Sozialmedizin Campus Lübeck

Folie 13 / 15

  • .304

n=221

  • .370

n=221

  • .317

n=222

  • .335

n=195 vitality (SF-36)

  • .391

n=225

  • .413

n=226

  • .373

n=226

  • .402

n=200 functional capacity (FFbH-R) .446 n=215 .436 n=216 .425 n=216 .437 n=193 rumination (PRSS) .496 n=215 .536 n=215 .504 n=215 .531 n=190 depression (CES-D) checklist- effect ML- estimation (EM- algorythm) manual- based + group mean manual- based (max. 4 MD) imputation

  • 5. different covariance structures?

Pearson correlation coefficient (all r p<.001)

slide-14
SLIDE 14

Institut für Sozialmedizin Campus Lübeck

Folie 14 / 15

Angaben in Prozent 1,4 16,2 30,6 8,7 8,8 13,9 A4-Survey Rehaantragsteller Kurgäste . 5 10 15 20 25 30 35 Checkliste Sonstige MD

checklist-effect in other surveys

slide-15
SLIDE 15

Institut für Sozialmedizin Campus Lübeck

Folie 15 / 15

Discussion and conclusions

Suggestion: Identification of possible checklist-effects in questionnaires + if present: additional coding of missing values as non-affirmative responses

  • 1. assumption of checklist effect accomplished an almost

complete imputation of missing values based on theory

  • 2. reduction of bias towards inclusion of less ill subjects
  • 3. the missingness does not seem to be conditional on some
  • ther variable(s) observed in the data (MAR)
  • 4. ML-estimation yielded similar mean and sd, but different

correlations identification of checklist-effect in other samples analyzing validity in methodological studies, e.g. cognitive survey