Structural Validity of the Tonic Immobility Scale in a Population - - PowerPoint PPT Presentation

Structural Validity of the Tonic Immobility Scale in a Population Exposed to Trauma: Evidence from Two Large Brazilian Samples

Michael Reichenheim Instituto de Medicina Social State University of Rio de Janeiro

Berkeley, 28 November 2017

departamento de epidemiologia

Rede Aferição

• Overall context

– Rede Aferição and PIEVF

• Procedural context

– Perspective  Cross-cultural adaptation – Road-map

• The paper (IMRaD)
• Beyond the paper / next steps

– New analyses – Way forward (towards a ‘Peri-Traumatic Scale’)

• Open debate and feedback

Presentation outline

Research Network for the Development of Instruments Applied to Measurement Models in Epidemiology (Rede Aferição) and Epidemiological Research Program on Family Violence (PIEVF)

Overall context

Research Network for the Development of Instruments Applied to Measurement Models in Epidemiology (Rede Aferição) and Epidemiological Research Program on Family Violence (PIEVF)

Overall context

Overall context

Research Network for the Development of Epidemiological Measurement Tools and Modeling

Rede de Pesquisa para o Desenvolvimento de Instrumentos de Aferição e Aplicação Modelos de Medida em Epidemiologia

Rede Aferição

ELANA (IMS / IN-UFRJ) SANDUC / EBIA (IMS / IN-UFRJ) Pesq Bolsa Família (IMS / IN-UFRJ) Projeto Violência no Namoro (IMS / FM-UERJ) Projeto TEPT (IMS / IFF) Projeto MUSA (UFBA) Projeto HADS (UEFS) Projeto PHQ (UEFS) Projeto EDE (UFSC) Projeto FCQ (UFTM) Rede FIBRA (FM-UERJ) Projeto MedRisk (ENSP) Rede Trauma (LADI-UFMG) Projeto OIDP (UFRGS)

Rosana Salles-Costa Claudia L Moraes Gabriela Interlenghi Claudia L Moraes Gloria V. Veiga Rosely Sichieri Claudia L Moraes Gustavo Lobato

Projeto Milênio (IMS / ENSP / UFRJ / UNIFESP)

Estela Aquino Greice Menezes Maria C Almeida Katia Freitas Tânia Araujo Katia Freitas Kionna Santos Alina Gomide Eduardo P Lima Elisabeth do Nascimento Mauro Mendlowicz João Luiz D Bastos Sybelle S Castro Sara Heitor Mônica Martins Roger K Celeste Roberto Lourenço Evandro Coutinho Claudia L Moraes Stella Taquette Rosana Salles-Costa Claudia L Moraes Rosely Sichieri Rubens Mattos

Cursos e material didático (IMS / INI / ENSP)

Yara Hökerberg Michael Reichenheim

Projeto ‘ELSA-Rio’ (ENSP)

Rosane Griep

Validação curva GPG (Intergrowth) (IN-UFRJ)

Gilberto Kac

Projeto ‘ERICA’ (UFRJ)

Kátia Bloch

Research Network for the Development of Research Instruments Applied to Measurement Models in Epidemiology (Rede Aferição) and Epidemiological Research Program on Family Violence (PIEVF)

Overall context

• Epidemiological Research Program on Family Violence

(PIEVF)

• Distributions, consequences, health services
• Covers (inter alia)

– Violence

• Family (psychological, physical, sexual)
• Youth / adolescence
• Community
• Elderly

– Substance abuse (alcohol and illicit drugs) – Nutritional and food ‘disorders’ (food insecurity) – Mental health

• Common mental disorders / depression / anxiety
• Post traumatic stress disorders (PTSD)
• Peri-traumatic disorders  tonic immobility, dissociative experience, physical

reactions

Overall context

• Key interest

– Brazilian epidemiology – Cross-culturally valid instruments

• Research
• Applied: prediction, risk assessment, awareness-raising (health services)

– Use / adapt available ‘international’ instruments

• Whenever possible
• To allow external comparisons

– Important  Cross-cultural adaptation processes (procedural models)

Overall context

• Cross-cultural adaptation model (Herdman et al; Guillemin et al., Beaton et

al.)

Procedural context

Conceptual Equivalence Evaluate whether construct and dimensions covered by the original instrument are relevant the new context for which it is being adapted. Item Equivalence Assess whether items 'mapping' the original instrument are relevant and ‘in tune’ with the new context and whether the 'intensity' of the items are being met (in principle). Semantic Equivalence Assess whether the wording/phrasing used in the adapted version converges equivalent meanings and ideas embedded in the items construed in the

• riginal version, thus leading to

congruent responses in both cultures regarding content and ‘intensity’. Measurement Equivalence Evaluate whether measurement properties of the original instrument and the adapted version are ‘invariant’. Operacional Equivalence Evaluate the compatibility between the characteristics of use in the source and target populations, so that there is 'efficacy' in measurement, even if the modus operandi ---layout, mode and application scenario-- differs.

• Cross cultural adaptation model  2 phases

Procedural context

Assessing conceptual equivalence Assessing

• perational

equivalence Pre-testing

1st phase (prototypic)

t

2nd phase (psychometric)

Reliability Analyses Assessing item equivalence Assessing semantic equivalence Assessing construct dimensional validity: adequacy of the internal structures Latent variable analyses (including IRT) Assessing content construct validity based

• n hypothesized relations

with other 'external' variables (incl. population invariance) + Other validities

COSMIN - COnsensus-based Standards for the selectionof health MeasurementINstruments

• A road-map to assessing ‘internal’ construct dimensional validity (item

and scale properties)

– Establishing configural structure equivalence (or its violation) – Establishing metric structure equivalence (or its violation)

• Evaluating item reliability / measurement error
• Examining 'method effect’  error/residual correlations (local independence) 

item content redundancy

• Assessing factor-based convergent validity
• Assessing factor-based discriminant validity

– Establishing scalar structure equivalence (or its violation)

• Examining item discrimination  positioning vis-à-vis the latent trait  item

coverage redundancy

• Assessing information coverage (latent trait)
• Evaluating scalability (including cutoff identification for id homogeneous groups)

– Establishing configural, metric and scalar ‘universality’ (‘generic diff’ across population subgroups)

Procedural context

Michael Reichenheim (IMS/UERJ) Evandro Silva Freire Coutinho (ENSP/FIOCRUZ) Wanderson Souza (DP/UFFRJ) Ivan Figueira (IPUB/UFRJ) Maria Inês Quintana (DP/UNIFESP) Marcelo Feijó de Mello (DP/UNIFESP) Rodrigo Affonseca Bressan (DP/UNIFESP) Jair de Jesus Mari (DP/UNIFESP) Sergio Baxter Andreoli (DP/UNIFESP)

On to the paper …

• Tonic Immobility (TI)

– Temporary catatonic-like state marked by a reversible motor inhibition, muscle hypertonicity, analgesia and relative unresponsiveness to external stimuli – Some authors regard it an evolutionary adaptive component working as the terminal defensive reaction when other resources are unavailable – Usually called as a ‘playing dead’ response in animals  TI is a consequence of a predatory attack in the wild when resistance is not successful – This response to threat seems to be evolutionarily beneficial, as it has been linked to higher survival rate to predatory attack in different species

Introduction

• Along with other peritraumatic stress reactions (dissociative

experiences, physical reactions, etc.), TI has been reported as a risk factor for Post Traumatic Stress Disorder – PTSD

• Among victims of violence (traumas), TI seems to predict

severity of PTSD and poor response to treatment

• Finding still contradictory, thus studies are still relevant …
• Measurement tools more so !
• To the best of our knowledge, there are only two

measurement tools available to evaluate TI in humans

– Tonic Immobility Questionnaire (TIQ) – Tonic Immobility Scale (TIS) proposed by Forsyth et al. in 2000

Introduction

• However,

– Still few studies on the structural (dimensional) validity of both instruments – In relation to the TIS  an important gap since it was the first measurement tool to be proposed and has been the most used so far

Introduction

• Forsyth et al. (2000) analyzing women with a history of

sexual abuse, satisfactorily evaluates 10 component items via ‘face validity’ (a priori content validity)

• Fusé et al. (2007)

– Exploratory Factor Analysis (n=88) and a single Confirmatory Factor Analysis (n=191) in sexually abused women suggested two distinct latent factors

• Tonic Immobility

froze/felt paralyzed unable to move though not restrained unable to call out or scream felt numb/no pain felt cold feared for life felt detached from self

Introduction

• Anxiety/fear

trembling/shaking fear/panic feelings of detachment from surroundings (environment)

• Although welcome as a start to the TIS’ scrutiny, this

psychometric history is rather incipient and surprising when considering time elapsed since its conception and given its continuous use over the years

– Narrow domain (sexual abuse) – Small samples – ‘Superficial’ assessment of configural and metric psychometric properties

• “Aiming to address these limitations and extend the scope of

use of TIS, the objective of this study was to reevaluate its dimensional structure through the sequential application in large representative samples of two Brazilian megacities, São Paulo and Rio de Janeiro”

Introduction

• Household-based representative samples from both cities:

São Paulo (20m) and Rio de Janeiro (12m)

• June 2007 to February 2008
• Stratified (according to homicide rates) and multi-stage

(census tract, household, subject) sampling with unequal probability of selection

• Prior screening of traumas and stressful events - list

– 11 items from WHO Mental Health Composite International Diagnostic Interview (CIDI) – 21 ad hoc items  e.g., exposure to assaultive violence or other shocking events (weapon, death threats, house broken in, IPV) or grief and suffering (sudden death or life-threatening illness of a close relative/person, car/motorcycle accident) – 86% reported at least 1 event

• Effective sample size  nSP = 2148 and nRJ = 1075

Methods: sampling and participants

Methods: measurement

• Multi-thematic questionnaire …
• The Tonic Immobility Scale (in full)

– 10 items – Item outcome space  7 scoring rubrics of increasing ‘intensity’

(e.g., item frozen from “able to move” to “completely paralyzed”)

frozen unable to move shaking no scream numb cold panic feared for life detached (self) detached (event)

• Process (modeling sequence)

– Step 1: assessing configural and metric structures in São Paulo

• Confirmatory Factor Analysis (CFA) of the ‘original’ 2-fator model,

also assessing factor-based discriminant validity

• Exploratory Structural Equation Model (ESEM) using geomin oblique

rotation (relaxing local independence …)

– Step 2: testing/corroborating structures in Rio de Janeiro

• CFA on the ‘best’ model suggested in São Paulo, also assessing

potential residual correlation (item content redundancies)

– Step 3: exploring tenability of reduced versions (aggregate sample)

• Restricted model fitting, removing previously found items forming

redundant pairs

• Preliminary assessment of latency mapping  scalability using

non-parametric IRT model

Methods: data analysis

Methods: data analyses

• Mplus (v.7 and v.8)
• Weighted Least Squares Mean and Variance Adjusted

estimator (WLSMV)

• Polychoric input matrices / ordinal probit models ( GRM)
• Modeling accounting for complex sampling procedure

(clustering and unequal sampling fractions)

• Model fit (indices)

– RMSEA (preferably 90%UCL)   0,06 – CFI   0.95 – TLI   0.95

Methods: data analysis

• Assessing residual correlations  ‘outlying’ Modification

Indices and/or Expected Parameter Changes r  0.20

• Assessing factor-based discriminant validity

– Contrasting the square root of the Average Variance Extracted of both factors with their factor correlation – Violation ‘accepted’ only if and p  0.05 (bs)

• Assessing scalability  Loevinger’s H coefficient / H  0.30

( ) ( 1 2) ve f f f

 

 



• Step 1

Results

  

SP

• Step 1

Results

  

SP



1 ( ) ( ) 1 * 2

.698 .97 .274 1 ( .302; .246)

f f e f v

 



      

2 ( ) ( ) 1 * 2

.766 .97 .205 1 ( .240; .169)

f f e f v

 



      

• Step 2

Results

  

SP

• Step 2 (cont’)

Results

  

RJ

• Step 2 (cont’)

Results

RJ SP

  

• Step 3

Results

  

• Step 2 (cont’)

Results

 

• Step 3 (cont’)

Results

shaking unable to move no scream numb cold feared for life detached (self) panic unable to move no scream numb cold detached (self)

Discussion

• Original bi-dimensional (configural) structure refuted 

factor-based discriminant validity violated

• Unidimensional structure admissible  yet ... local

independence assumption violated  item content redundancy

– feeling paralyzed (i1)  frozen (i2) – fear/panic (i7)  feared for life (i8) – detached from self (i9)  detached from environment (i10)

• Reduced structure seemingly also tenable

– Does not require complex modelling to deal with residual correlations – Latent score may be represented (operationalized) by the raw score – Operationally more efficient (timely)

• Same structure in both sample may suggest functional domain

invariance

Beyond the paper / next steps …

• Beyond configural / metric properties and the preliminary

scalar properties visited so far, further insight into the latter is required …

• New (tentative) analyses ...

Beyond the paper / next steps …

• Count (sw), Percent, Pt-biserial and IRT bi (10-item version)

(svy: irt grm ...)

Beyond the paper / next steps …

• Wright Map

Alternative B, 6-item version Original 10-item version

• Step 3 improved

Beyond the paper / next steps …

panic unable to move no scream numb cold detached (event) panic unable to move no scream numb cold detached (self)

Beyond the paper / next steps …

• Infit statistics

Original 10-item version Alternative B, 6-item version

Beyond the paper / next steps …

• Infit statistics

Original 10-item version Alternative C, 6-item version

Beyond the paper / next steps …

• Boundry Characteristic Curves (alt-B, 6-item version)

(svy: irt grm ...)

Beyond the paper / next steps …

• Item Information Functions and Test Information Function (alt-

B, 6-item version)

.5 1 1.5 2 Information

• 4
• 2

2 4 Theta

tis2 tis4 tis5 tis6 tis7 tis9

Item Information Functions

.2 .4 .6 .8 1 Standard Error 2 4 6 8 10 Information

• 4
• 2

2 4 Theta Test information Standard error

T est Information Function

(svy: irt grm ...)

Beyond the paper / next steps …

• Test Characteristic Curve (alt-B, 6-item version)

(svy: irt grm ...)

• Questions to be asked

– TIS ‘validated’ (broader sense)? – Adequately adapted? – Revisit theory  definitions of tonic immobility, dissociation, physical reactions, etc? – Construct underrepresentation  Reassess construct map? – Construct-irrelevance? – Assess other types of validity (consequential, fairness, etc.)?

Beyond the paper / next steps …

• Move towards a ‘Peri-traumatic Scale’ …

Beyond the paper / next steps …

Dissociation (PDEQ)

blanked out automatic action slow motion did not seem real spectator floating above felt disconnected did not notice things felt confused unsure of time or place numbness

Physical reaction (PRS)

shortness of breath dizziness or feeling faint accelerated heart tremors or shakes intense sweating nausea numbness / tingling (body) waves of heat or chills suffocation pain or discomfort in the chest difficulty control’ bladder/bowel

Tonic immobility (TIS)

froze/felt paralyzed unable to move unable to call out or scream felt numb/no pain felt cold feared for life felt detached (self) trembling/shaking fear/panic felt detached (environment)

Contact: michaelreichenheim@gmail.com Instituto de Medicina Social, UERJ

Thank you