Practice Settings Rochelle F. Hanson 1 , Benjamin E. Saunders 1 , - - PowerPoint PPT Presentation

Therapist-Reported Competence:

A Pragmatic Fidelity Measurement Strategy for Integration into Community Practice Settings

Rochelle F. Hanson1, Benjamin E. Saunders1, Jason Chapman2 Sonja Schoenwald1, Angela Moreland1

1Medical University of South Carolina 2 Oregon Social Learning Center

Presentation at the 9th Annual Conference

• n the Science of Dissemination & Implementation,

December 14, 2016, Washington DC

Acknowledgements

» Duke Endowment (No. 1582-SP) » NIMH (Grant No. R34 MH104470-02)

Colleagues

Elizabeth Ralston, PhD Elizabeth Hinson, MSW Rachael Garrett, MSW Carole Swiecicki, PhD Kim Reese, MSW

Dee Norton Lowcountry Children’s Center

Monica Fitzgerald, PhD

University of Colorado

Benjamin E. Saunders, PhD Angie Moreland, PhD Heidi Resnick, PhD Michael de Arellano, PhD Dan Smith, PhD Jan Koenig, MEd Faraday Davies Sara delMas Emily Fanguy

Medical University of South Carolina

Intro: Measuring Fidelity

» Observational coding methods » Expensive, time consuming, generally not feasible & sustainable in usual care/community-based settings » Potential alternative: therapist self-report

— Fidelity: Adherence vs. Competence

Focus of Presentation

» To examine feasibility of therapist self-report as measure of EBT use over course of training/implementation project » To examine relationships between therapist self-reported competence of an EBT (TF-CBT) and client outcomes (i.e., predictive validity)

Participants

Treatment Duration 8-20 Sessions Treatment Frequency 1 session/week

Targeted EBT

Psychoeducation Relaxation Affective

Modulation

Cognitive Coping Trauma Narrative

and Processing

In vivo Conjoint sessions Enhancing safety

Trauma auma Nar Narrativ ive e Phas hase e Int ntegration/ ion/ Cons

• nsolida
• lidation

ion Phas hase e Stabiliz bilization ion Phas hase e

Parenting Skills

Gradual Exposure Time

1/ 1/3 3 1/ 1/3 3 1/ 1/3 3 Fidelity: Prescribed Order of Components

CBLC Training Requirements for Clinical Participants

» Complete pre-work, including TF-CBTWeb, and pre-CBLC evaluation » Attend all learning sessions » Participate in 12/14 calls » Complete TF-CBT with a minimum of 2 cases (register 5) » Complete weekly metrics to assess use and competence in delivery of TF-CBT with training cases » Complete monthly metrics on delivery of TF-CBT with all clients on caseload » Complete post-CBLC evaluation

Participants

» Clinicians (n = 570) who attended LS1 of a Project BEST LC or CBLC (n = 11). — 516 identified at least one training case — 446 had at least one competence rating (total of 1,767 cases)

Training Cases

» 2,361 identified training cases » 1,614 (68.4%) had pre or post child-reported PTS » 816 (34.6%) had pre and post child-reported PTS (Jason)

u 60.2% girls u 55.3% White; 34.7% African American; 3.1% Hispanic; 7% Other u M child age = 12.34 (SD = 3.42; range = 4-23 years)

Fidelity Measurement

» Weekly online reports to assess:

— use of and perceived competence in delivering 11 TF-CBT components

» Treatment duration (specified as 16-20 sessions), » Inclusion of a caregiver, and » Prescribed order of components

Weekly Clinical Metrics

• Supervision minutes
• Registered cases seen

Each Case

• Parental involvement
• Which TF-CBT Components
• Perceived Competence in

delivery (less than adequate skill to expert skill)

• Barriers to adherence

Collected metrics up to 24 weeks

Results: Use of TF-CBT Components

» M weekly metrics completed per case = 11.26(SD = 4.77)

— 50.3% of clinicians completed at least 12 weeks of metrics — 33.5% of clinicians completed at least 14 weeks of metrics

» Treatment duration (# of weeks from pre to post assessment)

— M = 23.11 (SD = 9.3), Mode = 19.0, Range = 6-52 sessions

» Clinicians reported completing an average of 8.86 (of 11) TF- CBT components, and at least 10/11 with 50.8% of their cases

— All PRAC components were completed with 76% — TN completed with 79.1% — In Vivo completed with 44.3% — Enhancing safety completed with 78.4%

Perceived Competence

» 4 Competence Outcomes by Case

» Minimum, maximum, first and last competence ratings

» Variability in Competence Across Components

— Lowest ratings on exposure-based components; highest on psychoeducation, parenting skills and relaxation

» Change in Perceived Competence Over Time

— Mixed=Effects Regression Models [Clients (level-1) nested within Therapists (level-2)] — Change over time: # of months between first Competence rating for any client and the first Competence rating for each subsequent client

Growth Models: Change in Competence Over Time

TF-CBT Training Cases: PTSD

with pre/post PTSD assessments completed (n = 816)

Recent RCT Results: Cohen et al. (2011) pre-post child UCLA total: d = 0.64 Deblinger et al. (2011) mean pre-post for child outcomes: d = 0.94

UCLA (n = 297) CPSS (n =519) Pre Post Pre Post Mean 35.1 19.0 25.9 12.0 SD 13.4 11.9 10.3 9.4 d 1.3 1.4 t t(296)=19.8* t(518)=28.8*

*p<.001

TF-CBT Training Cases

Scoring Above Clinical Cut Score

20 40 60 80 100 Pre-Tx Post-Tx 71% 24%

[N = 816]

Percent

CPSS ≥ 15 UCLA ≥ 38 X2 (1) = 54.5, p < .001

Therapist Perceived Competence & Child Pre to Post Tx PTS Outcomes

» Analyses

— Three-level model with pre-post measurements (level-1) nested within clients (level-2) nested within therapists (level-3). — Therapist competence scores entered as predictors at client level. — To test for change, a dummy coded indicator was included at level-1 to differentiate the post-assessments from the pre- assessments — The model made use of all available data

Therapist Perceived Competence and Child Pre to Post Tx Outcome (PTS)

Those who received services from therapists with higher self-rated competence had greater improvements in PTS symptoms pre-to-post treatment

Discussion

» Variability in self-reported competence – lowest on exposure-based components, highest on PRAC » Significant improvements on 4 of 11 components (P, R, A, Conjoint) » Trend for TN development and Enhancing Safety » Children who received tx from therapists with higher self-rated competence had greater improvements in PTS sx pre to post treatment. » Potential for low cost, low burden, feasible measure of competence

Discussion: Study Limitations

» No control condition (selection bias; limited generalizability;

regression to the mean…)

» No objective fidelity coding for comparison with therapist self-report (i.e., is TF-CBT cause of reduction in symptoms??)

» Therapist rating style

Next Steps

» Ongoing data cleaning, cleaning and more cleaning » Therapist variables related to fidelity (e.g., therapist rating style) » Changes in adherence as related to implementation stages » Convergent validity » So many questions……

Questions?

Contact Information Rochelle Hanson hansonrf@musc.edu