Clinical Trials for Adaptive Intervention Designs: Workshop on the - - PowerPoint PPT Presentation

SCT Pre-conference Workshop May 18, 2014, 8am-12noon Organizer: Daniel Almirall, University of Michigan (List of titles and speakers on the next slide)

Clinical Trials for Adaptive Intervention Designs: Workshop on the Design and Conduct of Sequential Multiple Assignment Randomized Trials

• 1. Introduction to adaptive interventions and SMART

(Inbal “Billie” Nahum-Shani, Univ of Michigan)

• 2. Adaptive studies for Substance Use Disorders

(James R. McKay, Univ of Pennsylvania)

• 3. Building an adaptive implementation strategy

using a cluster-randomized SMART (Amy M. Kilbourne, Univ of Michigan)

• 4. Informing a full scale SMART to address obesity

(Sylvie Naar-King, Wayne State)

• 5. Adaptive interventions for children with autism

(Daniel Almirall, Univ of Michigan)

• 6. Discussion by Peter Thall, MD Anderson Cancer Ctr

SCT Pre-Conference Workshop on SMART

INTRODUCTION TO

ADAPTIVE INTERVENTIONS

AND

SMART DESIGNS

Inbal (Billie) Nahum-Shani

Nahum-Shani, I. May 2014: SCT

Adaptive Interventions (AIs)

• Treatment design
• Seeking to accommodate heterogeneity in response to treatment
• Intervention options are adapted to the specific and changing

needs of individuals

• Example (Marlowe et al., 2008; 2009; 2012)
• Adaptive drug court program for drug abusing offenders
• The goal: Minimize recidivism and drug use
• Operationalized by graduating from the drug court program

2

Nahum-Shani, I. May 2014: SCT

Adaptive Drug Court Program

As-needed court hearings + standard counseling Bi-weekly court hearings + standard counseling

Low risk High risk

As-needed court hearing + ICM Bi-weekly court hearing + ICM

Non-responsive Non-responsive

Jeopardy contract: “zero tolerance”

Non-compliant Non-compliant Non-compliant Non-compliant

3

Nahum-Shani, I. May 2014: SCT

First Stage Decision Rule

At point of entry into the program If risk = low Then, fist-stage intervention= {As-needed + SC} Else if risk=high Then, first-stage intervention = {Bi-weekly + SC}

• 1. Decision Point:

A time in which treatment options should be considered based on patient information (Yoshino et al., 2009)

• 2. Tailoring Variable:

Patient information used to make treatment decisions

• 3. Intervention options:

Type/Dose

• 5. Outcomes:

Distal àLong-term goal of intervention: Program graduation (14 consecutive weekly negative drug urine specimens) Proximalà Short-term goal of the adaptation: Compliance and response in the course of intervention (mediator)

• 4. Decision rule

4

Nahum-Shani, I. May 2014: SCT

Adaptive Intervention: 5 Elements

• 1. Decision Points
• 2. Tailoring Variable
• 3. Decision rule
• 4. Intervention Options
• 5. Proximal + Distal Outcomes

5

Trigger

• Monitoring
• Individualizing
• Delivering

Guide

Nahum-Shani, I. May 2014: SCT

Adaptation process

SMART Designs

• To aid in constructing empirically-based AIs
• SMART:
• Randomized Trials
• Multiple stages of randomization
• Each stage corresponds to a decision point

Nahum-Shani, I. May 2014: SCT

6

Example

• Adaptive Interventions for Children with ADHD (Pelham)
• Motivation:
• Debate about frontline treatment: MED or BMOD?
• About 30% do not respond well to either MED or BMOD
• What is the best “rescue” tactic:
• Enhance intensity or Augment with the other type of treatment?

ADHD SMART Study

Nahum-Shani, I. May 2014: SCT

8

R Medication BMOD Response Non-Response R Enhance (SG2) Augment (SG3) Response Non-Response R Enhance (SG5) Augment (SG6) Continue Med (SG1) Continue BMOD (SG4)

Questions We Can Address with SMART

• First-stage intervention options:
• Is it better to start with BMOD or MED?
• (SG1+SG2+SG3) vs. (SG4+SG5+SG6)
• Medication vs. BMOD
• Controlling for subsequent treatment

Nahum-Shani, I. May 2014: SCT

9

MED BMOD Response Non-Response Augment (SG3) Enhance (SG2) Response Non-Response Augment (SG6) Enhance (SG5) Continue (SG1) Continue (SG4)

Questions We Can Address with SMART

• Second-stage intervention options:
• Is it better to Enhance or Augment for non-

responders?

• (SG2+SG5) vs. (SG3+SG6)
• Enhance vs. Augment

Nahum-Shani, I. May 2014: SCT

10

MED BMOD Response Non-Response Augment (SG3) Enhance (SG2) Response Non-Response Augment (SG6) Enhance (SG5) Continue (SG1) Continue (SG4)

Questions We Can Address with SMART

• Embedded adaptive interventions

Nahum-Shani, I. May 2014: SCT

11

MED BMOD Response Non-Response Augment (SG3) Enhance (SG2) Response Non-Response Augment (SG6) Enhance (SG5) Continue (SG1) Continue (SG4)

At the beginning of school year Stage 1 = {MED}, Then, every month IF response = {NO} THEN stage 2 = {AUGMENT} ELSE IF response = {YES} THEN continue stage 1 At the beginning of school year Stage 1 = {BMOD}, Then, every month IF response = {NO} THEN stage 2 = {AUGMENT} ELSE IF response = {YES} THEN continue stage 1

VS.

At the beginning of school year Stage 1 = {MED}, Then, every month IF response = {NO} THEN stage 2 = {ENHANCE} ELSE IF response = {YES} THEN continue stage 1 At the beginning of school year Stage 1 = {BMOD}, Then, every month IF response = {NO} THEN stage 2 = {ENHANCE} ELSE IF response = {YES} THEN continue stage 1

Results:

Adaptive intervention ¡ Responders ¡ Non-Responders ¡ Estimated weighted mean ¡ Robust SE ¡ ¡ Stage 1 ¡ Stage 2 ¡ Sample size ¡ Sample mean ¡ Sample size ¡ Sample mean ¡ (1, -1) ¡ BMOD ¡AUGMENT ¡ 22 ¡ 4.64 ¡ 24 ¡ 4.08 ¡ 4.36 ¡ 0.15 ¡ (-1, -1) ¡ MED ¡ AUGMENT ¡ 36 ¡ 4.39 ¡ 17 ¡ 3.47 ¡ 4.00 ¡ 0.15 ¡ (1, 1) ¡ BMOD ¡INTENSIFY ¡ 22 ¡ 4.64 ¡ 22 ¡ 3.96 ¡ 4.17 ¡ 0.22 ¡ (-1, 1) ¡ MED ¡ INTENSIFY ¡ 36 ¡ 4.39 ¡ 18 ¡ 4.22 ¡ 4.27 ¡ 0.13 ¡

Primary outcome:

• Children’s school performance at month 8
• Based on the Impairment Rating Scale (IRS, Fabiano).
• Ranges from 1 to 5, with higher values= better school performance.

Primary Research Questions

• 1. Compare first stage intervention options
• 2. Compare second-stage intervention option
• 3. Compare embedded adaptive intervention

Nahum-Shani, I., Qian, M., Almirall, D., Pelham, W. E., Gnagy, B., Fabiano, G. A., ... & Murphy, S. A. (2012). Experimental design and primary data analysis methods for comparing adaptive interventions. Psychological methods, 17(4), 457. Oetting, A., Levy, J., Weiss, R., & Murphy, S. (2007). Statistical methodology for a SMART design in the development of adaptive treatment strategies. Causality and Psychopathology: Finding the Determinants of Disorders and their Cures. Arlington, VA: American Psychiatric Publishing, Inc.

Nahum-Shani, I. May 2014: SCT

13

Exploratory Research Questions

• More deeply tailored adaptive interventions.
• Using candidate tailoring variables not embedded in design.
• Tailoring Variables?
• Strongly moderate the effect of intervention options, in a way which

suggests that different intervention options should be offered depending on their values.

• Candidate tailoring variables:
• Medication prior to Stage 1
• Adherence to Stage 1

Nahum-Shani, I. May 2014: SCT

14

More Deeply Tailored Decision Rule:

At the beginning of school year

IF medication prior to stage 1={NO-MED} THEN stage 1 = {BMOD}. ELSE stage 1 = {MED} or {BMOD}. Then, every month IF response to stage 1 = {NO} THEN IF adherence to stage 1 = {LOW}, THEN stage 2 = {AUGMENT}. ELSE stage 2 = {Augment} or {ENHANCE}. ELSE continue stage 1.

Nahum-Shani, I. May 2014: SCT

15

At the beginning of school year Stage 1 = {MED}, Then, every month IF response = {NO} THEN stage 2 = {AUGMENT} ELSE IF response = {YES} THEN continue stage 1

Q-learning (Watkins, 1989; Murphy, 2005)

• Popular method from computer science.
• Extension of moderated regression to sequential decision

making

• You use Q-learning by conducting a sequence of regressions.
• One regression for each stage.
• At each stage you:
• Assess whether the candidate tailoring variable moderates the effect of

the intervention options at this stage

• In a way that is useful for making treatment decisions

Nahum-Shani, I. May 2014: SCT

16

Nahum-Shani et al., (2012). Q-Learning: A Secondary Data Analysis Method for Developing Adaptive Interventions. Psychological Methods, 17(4), 478-494

Stage 2:

Best tactic for non-responders, given adherence

Regress Y on O1, A1, O2, A2, O2*A2

Learn More

• Websites with resources:
• Methodology Center: http://methodology.psu.edu/ra/adap-inter/projects
• Key ideas:
• Murphy, S. A., (2005). An experimental design for the development of adaptive

treatment strategies. Statistics in Medicine, 24(10), 1455–1481.

• Murphy, S. A. (2003). Optimal dynamic treatment regimes. Journal of the Royal

Statistical Society, Series B, 65(2), 331-366.

• Overview of completed or ongoing SMART studies:
• Lei, H., Nahum-Shani, I., Lynch, K., Oslin, D., & Murphy, S. A. (2012). A

"SMART" design for building individualized treatment sequences. Annual Review of Clinical Psychology, 8, 14.1 - 14.28.

• Introduction to SMART
• Almirall D., Nahum-Shani, I., Sherwood, N.E., Murphy S.A. (accepted 2014, in

press). Introduction to SMART Designs for the Development of Adaptive Interventions: With Application to Weight Loss Research. Translational Behavioral Medicine.

Nahum-Shani, I. May 2014: SCT

Thank you!

• Email me questions/comments: Inbal@umich.edu
• Collaborators:
• U of Michigan: Statistical Reinforcement Learning Lab
• Susan Murphy: http://dept.stat.lsa.umich.edu/~samurphy/
• Danny Almirall: http://www-personal.umich.edu/~dalmiral/
• Linda Collins: http://methodology.psu.edu/people/lcollins

Nahum-Shani, I. May 2014: SCT

19

Sequential multiple assignment randomized trial (SMART) adaptive studies for SUD

James R. McKay, Ph.D.

University of Pennsylvania Philadelphia VAMC

Problems in SUD treatment

•  High dropout rate
•  PTs’ mixed reactions to “standard

care” in the treatment system:

n Behavioral interventions n Group counseling n 12-step model (i.e., AA approach)

•  Currently, treatment seekers with

substance use disorders (SUD) really do not have many TX options

Adaptive Treatment Study

•  Research Questions

n Does offering patients who do not engage in treatment a choice of other interventions improve outcomes? n Does offering patients who engage but then drop out a choice of other interventions improve outcomes? n Does a second attempt to offer TX choice to non-engagers improve outcomes?

Tailoring Variable

•  We are tailoring on IOP attendance

(rather than substance use)

•  Definition of “disengaged” was derived

through an expert consensus process

n At 2 weeks: failure to attend any treatment in the second week following intake n During weeks 3-7: failure to attend any IOP sessions for two consecutive weeks n At 8 weeks: failure to attend any IOP sessions in prior two weeks

Treatment Sites and Patients

•  Participants recruited from IOPs in

publicly funded and VA programs

•  Participants enrolled at intake
•  Two studies:

n Cocaine dependent (N=300), 80% with alcohol dependence n Alcohol dependent (N=200), 40% with cocaine dependence

•  Typical participant: African-American

male, around 40yo

Adaptive Protocol With Patient Choice

Intake to Specialty Care (IOP) Engaged Patients Non-Engaged Patients

Monitor for

Telephone MI For IOP Engagement Telephone MI With Choice

• f TX Option

IOP

Stepped Care Two weeks Medical Management

Self-Selection Randomization Still Non-Engaged Now Engaged Second Randomization

TEL MI W/Choice No Further MI Calls

Week 2 Week 8

CBT

Monthly Outcome Measures

•  Alcohol Use (for alcohol dependent Pts)

n Any use and any heavy use n Frequency of any and heavy use

•  Cocaine Use (for cocaine dependent Pts)

n Any use n Frequency of use n Urine toxicology

Study Participation

•  Engaged/Disengaged at Week 2:

n Study 1– 188 (63%) / 112 (37%) of 300 n Study 2– 123 (62%) / 77 (38%) of 200

•  Disengaged Weeks 3-7:

n Study 1—43 (23%) of 188 engaged at W2 n Study 2—24 (20%) of 123 engaged at W2

•  Still disengaged at Week 8:

n Study 1—66 (59%) of 112 disengaged W2 n Study 2—43 (56%) of 77 disengaged W2

What non-engaged MI-PC PTs select in weeks 2-7:

What non-engaged MI-PC PTs select at week 8: (at re-randomization)

Main Effects Analyses Alcohol Use in Patients Disengaged at 2 weeks

Any Alcohol Use in Month

Study 1 Study 2

p= .012 p= .028

Days of Alcohol Use per Week

Study 1 Study 2

p= .02 p= .015

Alcohol outcomes in combined sample (161 of 428 alc dep)

•  Any drinking:

n OR= 0.40, p= .0007

•  Any heavy drinking

n OR= 0.33, p= .001

•  Frequency of drinking

n B= -1.08, p= .009

•  Frequnecy of heavy drinking

n B= -1.09, p= .003

MI-PC= 0, MI-IOP= 1

Main Effects Analyses Alcohol Use in Patients Disengaged between weeks 3 and 7

Disengaged in weeks 3-7 in combined sample (N=73)

•  Any alcohol use

n OR= 0.54, p= .16

•  Any heavy alcohol use

n OR= 0.67, p= .36

•  Frequency of use

n B= -0.84, p= .23

•  Frequency of heavy use

n B=-1.03, p= .10

MI-PC= 0, MI-IOP= 1

Main Effects Analyses Alcohol Use in Patients Disengaged at both 2 and 8 weeks

Disengaged at weeks 2 and 8 in combined sample (N=86)

•  Any alcohol use

n OR= 1.12, p= .79

•  Any heavy alcohol use

n OR= 1.43, p= .45

•  Frequency of use

n B= -0.34, p= .58

•  Frequency of heavy use

n B= 0.02, p= .97

MI-PC= 1, no further outreach=0

Main Effects Analyses Cocaine Use Outcomes

Cocaine use (N= 409)

•  PTs disengaged at w2 (N=159):

n NS (P values .13 to .86)

•  PTs disengaged in w3-7 (N=69):

n NS (p values .16 to .74) (results in direction of IOP better than PC)

•  PTs disengaged w2 and w8 (N=84):

n NS (p values .14 to .42) (results in direction of NFO better than PC)

Conclusions

•  Providing substance dependent

patients who fail to engage in IOP a choice of other treatment options does not improve alcohol or cocaine use outcomes

•  In fact, outreach without a choice of
• ther treatments leads to better

alcohol use outcomes in those who do not engage in IOP initially

Conclusions

•  Also, no advantage to providing a choice
• f interventions to patients who engage

initially but then drop out

•  Finally, providing further outreach with a

choice of interventions to those not engaged at 2 and 8 weeks did not improve substance use outcomes compared to no further outreach

n Possible exception: patients with past rather than current dependence at intake

Encouraging results

•  It is possible to successfully implement a

SMART project in SUD patients

•  Use of telephone MI made it possible to at least

reach most patients after 1st and 2nd randomization, even though they were not engaged in treatment.

Challenges in Adaptive Treatment for Substance Dependence

•  PTs who are doing badly are hard to reach and

are often unwilling to participate further in treatment of any sort

•  Mechanisms of action in behavioral treatment
• ptions may not be sufficiently different that PT

doing poorly in one will respond to another

Funding

•  Support for this study provided by

NIH grants:

n P60 DA05186 (O’Brien, PI) n P01 AA016821 (McKay, PI) n K02 DA00361 (McKay, PI) n K24 DA029062 (McKay, PI) n RC1 AA019092 (Lynch, PI) n RC1 DA028262 (McKay, PI)

Collaborators

•  Penn

n Dave Oslin n Kevin Lynch n Tom Ten Have n Debbie Van Horn n Michelle Drapkin

•  Consultants

n Susan Murphy, U Michigan n Linda Collins, Penn State

Acknowledgments

Our Research Team

• 

Oubah Abdalla

• 

John Cacciola

• 

Rachel Chandler

• 

Dominic DePhilippis

• 

Michelle Drapkin

• 

Ayesha Ferguson

• 

Ellen Fritch

• 

Jessica Goodman

• 

Angela Hackman

• 

Dan Herd

• 

Laurie Hurson

• 

Ray Incmikoski

• 

Laura Harmon

• 

Megan Long

• 

Jen Miles

• 

Jessica Olli

• 

Zakkiyya Posey

• 

Alex Secora

• 

Tyrone Thomas

• 

Debbie Van Horn

• 

Sarah Weiss

• 

Tara Zimmerman

Improving Mental Health Outcomes:

Building an Adaptive Implementation Strategy Using a Cluster-randomized SMART

Amy M. Kilbourne, PhD, MPH

Acting Director, VA Quality Enhancement Research Initiative (QUERI) VA Ann Arbor Center for Clinical Management Research Professor of Psychiatry, University of Michigan

Acknowledgements

University of Michigan, VA (SMI Re-Engage), & Community (ROCC):

Daniel Eisenberg, PhD Danny Almirall, PhD Steve Chermack, PhD Edward Post, MD, PhD Michele Heisler, MD Michelle Barbaresso, MPH Sonia Duffy, PhD, RN Marcia Valenstein, MD Nicholas Bowersox, PhD Kristen Abraham, PhD Kristina Nord, MSW Hyungin Myra Kim, ScD Julia Kyle, MSW David Goodrich, EdD Celeste Vanpoppelen, MSW Zongshan Lai, MPH Peggy Bramlet, MEd Karen Schumacher, RN

University of Colorado:

Marshall Thomas, MD Jeanette Waxmonsky, PhD

• Univ. of Pittsburgh:

Harvard/VA Boston:

David Kolko, PhD Mark Bauer, MD Ronald Stall, PhD Carol Van Deusen Lukas, PhD

Columbia University: CDC:

Harold Pincus, MD Mary Neumann, PhD Funding: Royalties: NIMH R01 MH79994, R01 MH99898 New Harbinger Publications (~$200/year) VA HSR&D SDR 11-232, IIR 10-340

Outline

w Overview of implementation strategies w 2-arm adaptive implementation strategy design w SMART design - implementation strategies w Implications

Implementation and the 3T’s Road Map

Modified from Dougherty and Conway, JAMA 2008;299:2319-2321

Basic Biomedical Science Clinical Efficacy Knowledge Clinical Effectiveness Knowledge Effectiveness Studies Who benefits T2 Efficacy Studies What works T1

Implementation How

T3 Improved Population Health

5

Why Implementation Research?

Gaps in Treatment Quality

Condition Percentage of Recommended Care Received Breast Cancer 75.5% Hypertension 64.7% Depression 57.7% Diabetes 45.4% Alcohol Dependence 10.5%

McGlynn et al: N Engl J Med 2003;348:2635-2645

Delays in Research Adoption

1871 First recorded medical use 1949

First publication showing efficacy

1970

FDA approval Lithium for mania

The Need for Implementation Research

w New treatments take too long to get adopted w Providers lack tools to implement effective treatments w Large-scale treatment initiatives rolled out without adequate planning to sustain effects

Implementation- General Definition

“A deliberately initiated process, in which agents intend to bring into operation new or modified practices that are institutionally sanctioned, and are performed by themselves and other agents”

Key terms: Process Agents Institutionally sanctioned practices

May C. Towards a general theory of implementation. Imp. Sci. 2013

General Theory of Implementation

10

Implementation Strategies

Highly-specified, systematic processes used to implement treatments/practices into usual care settings w Guideline dissemination insufficient w Need buy-in from providers, healthcare leaders w Understanding barriers, facilitators to adoption

Implementation Strategies

Some Examples

w Evidence-based Quality Improvement (EBQI) w Promoting Action on Research Implementation in Health Services (PARiHS) w Getting to Outcomes (GTO) w Replicating Effective Programs (REP)

Replicating Effective Programs

Implementation Intervention Strategy

Pre-implementation

Identify need & program Identify settings Adapt & develop package- community working group input

Implementation

Disseminate package Training Technical assistance (brief) Evaluation

Dissemination

Outcomes Further diffusion, spread

REP was developed by the Centers for Disease Control to rapidly translate HIV prevention programs to community-based settings Based on Social Learning Theory, Rogers’ Diffusion model Emphasis on treatment fidelity and roll-out

Kilbourne AM, et al, Imp Sci 2007; Sogolow ED, AIDS Educ Prev. 2000

REP and Uptake of HIV Prevention Interventions in AIDS Service Organizations

10 20 30 40 50 60 70 80 90 100 Baseline 6 Month 12 Month Manual only Manual+training Manual+training+TA Kelly J, et al. AJPH 2000

Is REP Sufficient for Complex Programs?

w Collaboration across multiple providers w Start-up logistics w Leadership buy-in w Need for sustainability plan (after study is completed) REP can be augmented using other implementation strategies

Study #1: Enhanced vs. std. REP

(ROCC Study; R01 MH79994)

w Clustered RCT comparing Enhanced versus standard REP to promote provider use of a collaborative care model for bipolar disorder w Enhanced REP àprovider coaching (“Facilitation”) w 384 patients w/bipolar disorder, 7 outpatient clinics w Primary outcomes: Fidelity (# collaborative care sessions), mood disorder remission, quality of life

Kilbourne et al. Imp Sci 2007; Kilbourne et al. Psy Serv 2012

Enhanced REP Implementation Strategy

Kilbourne AM et al. 2012; Waxmonsky J et al. 2013

Pre- Implementation Identify need & program Identify settings Adapt & develop package- community working group input REP Implementation Disseminate package Training Evaluation Monitor response Facilitation (external) Barriers assessment Provider coaching and problem- solving- weekly calls Promote success Evaluation Outcomes Further diffusion, spread Process Evaluation Build business case: sustainability

Study Patient Characteristics

Overall N=384 Enhanced REP (n=221) Standard REP (n=163) Mean(SD) Mean(SD) Mean(SD) F (p) Age, years 42.0 (11.3) 42.2 (11.4) 41.8 (11.3) .36 (.72) N (%) N (%) N (%) Chi-sq (p) Female 256 (66.7) 146 (66.1) 110(67.5) .09 (.77) Non-White 108 (29.3) 54 (25.2) 54(34.8) 4.01 (.04) College Education 71 (18.8) 59 (27.1) 12(7.5) 23.2 (<.001) Unemployed 279 (72.7) 149 (67.4) 130(79.8) 7.2 (.007) Alcohol misuse 40 (10.7) 24 (11.2) 16 (10.0) .13 (.71) Illicit drug use 123 (32.0) 70 (31.7) 53 (32.5) .03 (.86)

REP and Patient-level Fidelity

Treatment Fidelity Measure REP package, training, TA REP package, training only % completing self- management sessions 64% 22% Total # contacts (self- management, care management) 8.1 (3.0) 5.5 (2.1)

12-Month Patient Outcomes

REP package, training, TA REP package, training only Mood disorder remission (PHQ-9 <5) 30.6% 17.7% Mental health quality of Life (SF-12) score 33.9 34.0

Secondary analyses adjusting for patient differences across sites revealed null findings comparing Enhanced, standard REP Small number of sites precluded sufficient power to detect differences in Enhanced versus standard REP

Is Enhanced REP Enough?

Need for an Adaptive Implementation Study

♦ REP may not be sufficient for improving patient

• utcomes across sites

♦ Facilitation is time-consuming and costs more ♦ Can sites solve barriers to treatment uptake on their own?

Study #2: Enhanced REP Adaptive Implementation Strategy

♦ Compare effectiveness of 2 adaptive implementation strategies enhance program uptake: Enhanced REP (+External Facilitation) for non-responsive sites immediately or later ♦ Two-arm cluster randomized trial taking advantage

• f a natural experiment of national program rollout

♦ REP initially used to implement program in 158 sites ♦ 88 non-responding sites randomized to receive added External Facilitation or continue standard REP

BMC CCT ISRCTN21059161;Davis et al AJPH 2012; Kilbourne t al. 2013

Primary Outcomes

Core Components of Outreach Program

• 1. Site-level updated documentation of patient

clinical status using electronic registry

• 2. Attempted contact by phone or mail
• 3. Patient scheduled appointment

Non-response defined as site with <80% of patients with updated clinical status documentation within 6 months (#1)

Re-Engage Adaptive Implementation Trial

National Implementation

Non- response

(N=88) Standard REP 158 Sites

Phase I

6 months Enhanced REP (N=39) Standard REP (N=49)

R Phase 2

6 months Enhanced REP 35 Sites

March 2012 August 2012

Standard REP (N=53) Low Response (N=35) Response (N=14)

Follow-up

12 months Standard REP All Sites

September 2012 February 2013 September 2013

Re-Engage 12 Month Results

Preliminary: Updated documentation (N=88 sites)

Re-Engage 12 Month Results

Preliminary: Attempted patient contact (N=88 sites)

Is External Facilitation Enough?

Building an Adaptive Implementation Strategy- SMART

w <50% patients with attempted contact w One “dose” of 6-month Facilitation took on average 7.5 hours per site w Site time commitment: 1-6 hours w Leadership buy-in: Need additional internal agent to address local barriers to treatment adoption? (Kirchner, et al. 2011)

Study #3: Designing SMART Trial on Facilitation

w External Facilitator (EF): coaching in technical aspects of clinical treatment or intervention w Internal Facilitator (IF): on-site clinical manager

w Direct reporting line to leadership w Some protected time w Address unobservable organizational barriers w Develop sustainability plan with leadership

Enhanced REP

Adding Facilitation based on PARiHS Framework

External facilitator (EF): off-site, research team, technical assistance Internal facilitator (IF): on-site provider with direct reporting line to leadership, protected time to build relationships, address unobservable

• rganizational barriers, develop sustainability plan

Kilbourne AM et al. 2013; Goodrich et al. 2012

Pre- Implementation Identify need & program Identify settings Adapt & develop package- community working group input REP Implementation Disseminate package Training Evaluation Monitor response Facilitation (Aim 1: Adaptive Implementation) External Facilitation Technical assistance Internal Facilitation Relationship- building/rapport Evaluation Outcomes Further diffusion, spread EF/IF Process Evaluation Build business case: sustainability

SMART REP Primary Aims

Among sites not initially responding to REP to implement collaborative care program, sites receiving External and Internal Facilitator (REP+EF/IF) vs External Facilitator alone (REP+EF):

• 1. Improved 12-month patient outcomes (QOL, sx)
• 2. Improved uptake (# collaborative care visits)

SMART REP (cont.)

w 80 community clinics (1600 patients) from Michigan, Arkansas, and Colorado w Sequential Multiple Assignment Randomized Trial (SMART) design w Non-response, within 6 months:

w <50% patients enrolled by provider in collaborative care program AND w Enrolled patients completing <75% collaborative care sessions

SMART REP Secondary Aims

w Effect of continuing REP+EF versus adding IF w Effect of continuing with REP+ EF/IF for a longer period of time

Figure ¡3: ¡SMART ¡Trial ¡Design ¡of ¡REP ¡Combined ¡with ¡External ¡(EF; ¡REP+EF) ¡and ¡Internal ¡Facilitation ¡(IF, ¡REP+EF/IF) ¡

¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡

REP ¡ k=100 ¡ sites ¡

Month ¡6 ¡ Assessment ¡ Month ¡12 ¡ Assessment ¡ As ¡

Month ¡18 ¡ Assessment ¡ Run-­‑In ¡Phase: ¡ ¡ All ¡sites ¡offered ¡REP ¡ to ¡implement ¡LG; ¡ Patients ¡start ¡LG ¡by ¡ Month ¡3 ¡ R ¡

Responders ¡ ¡ Non-­‑responders ¡ ¡ Responders ¡ ¡ Non-­‑responders ¡ ¡ R ¡ Start ¡of ¡ Study ¡ Month ¡24 ¡ Assessment ¡

• Cont. ¡follow-­‑up ¡

• Cont. ¡follow-­‑up ¡

SMART REP Design

SMART REP Implications

w Internal Facilitators (IFs) are costly for sites since they require additional time to recruit and administrative effort w Can off-site External Facilitation (EF) alone improve patient outcomes? w Delayed effect of adding IF or EF/IF among non-responsive sites, especially in smaller practices

Key Lessons

w Natural experiments

w Operational partner buy-in re: study design w National data sources (patient, provider) key

w Testing implementation intervention strategies

w Evidence base vs. time-sensitive opportunity w Cost and value of implementation interventions

Pilot ¡Studies ¡Informing ¡a ¡Full ¡ Scale ¡SMART ¡to ¡Address ¡ Obesity ¡Related ¡Health ¡ Dispari@es ¡

Sylvie Naar-King, Ph.D. Wayne State University Pediatrics

Key Co-Investigators: Deborah Ellis, Ph.D. WSU and Phillippe Cunningham MUSC

Funded by National Institutes of Health (NHLBI) and the American Diabetes Association

NIH ¡GOAL: ¡MORE ¡POTENT ¡OBESITY ¡ INTERVENTIONS ¡

• Context: ¡Why ¡obesity, ¡why ¡minori@es, ¡why ¡

adolescents ¡

• What ¡has ¡and ¡has ¡not ¡worked ¡so ¡far? ¡
• Small ¡pilots ¡to ¡overcome ¡these ¡barriers ¡
• Develop ¡an ¡adap@ve ¡behavioral ¡treatment ¡for ¡

adolescent ¡obesity ¡with ¡a ¡full ¡scale ¡SMART ¡ informed ¡by ¡pilot ¡studies ¡

Context ¡-­‑ ¡Evolu@on? ¡

Context ¡ ¡-­‑ ¡Evolu@on? ¡

Obesity ¡and ¡Disease ¡

• Physical ¡Ac@vity ¡and ¡Overea@ng ¡are ¡in ¡the ¡top ¡5 ¡causes ¡of ¡early ¡

mortality ¡ ¡

• Obesity ¡is ¡associated ¡with ¡a ¡host ¡of ¡cardiovascular ¡and ¡metabolic ¡

diseases ¡

• According ¡to ¡the ¡Na@onal ¡Cancer ¡Ins@tute, ¡obesity ¡is ¡associated ¡

with ¡increased ¡risk ¡for ¡the ¡following ¡cancers ¡

– Esophagus ¡ – Pancreas ¡ – Colon ¡and ¡rectum ¡ – Breast ¡(aTer ¡menopause) ¡ – Endometrium ¡ ¡ – Kidney ¡ – Thyroid ¡ – Gallbladder ¡

• A ¡BMI ¡reduc@on ¡of ¡only ¡1% ¡would ¡lead ¡to ¡avoidance ¡of ¡100,000 ¡

cancer ¡cases ¡by ¡2030 ¡ ¡

Obesity ¡

• Na@onal ¡Health ¡and ¡Nutri@onal ¡Examina@on ¡

Survey ¡(NHANES ¡data ¡through ¡2010) ¡

• 42% ¡of ¡US ¡adults ¡over ¡20 ¡are ¡obese ¡and ¡33% ¡

are ¡overweight ¡

• Obesity ¡in ¡adolescents ¡(12-­‑19 ¡yrs) ¡– ¡18% ¡
• Obesity ¡in ¡children ¡(6-­‑11 ¡yrs) ¡– ¡18% ¡
• Obesity ¡in ¡children ¡(2-­‑5 ¡yrs) ¡– ¡12% ¡(stabilizing ¡and ¡

may ¡be ¡reducing ¡but ¡only ¡in ¡White ¡children) ¡

Health ¡Dispari@es ¡in ¡Obesity: ¡ Adult ¡Women ¡

Health ¡Dispari@es ¡and ¡Obesity: ¡ Adolescent ¡Females ¡

Health ¡Dispari@es ¡and ¡Obesity: ¡Adult ¡ Men ¡

Health ¡Dispari@es ¡and ¡Obesity: ¡ Adolescent ¡Males ¡

Reducing ¡Health ¡Dispari@es ¡by ¡ Targe@ng ¡Adolescents ¡

• Obese ¡adolescents ¡become ¡obese ¡adults ¡

– Obese ¡adolescents ¡16 ¡@mes ¡more ¡likely ¡to ¡become ¡ severely ¡obese ¡adults ¡(The ¡et ¡al., ¡2010; ¡JAMA) ¡

• Adolescents ¡are ¡showing ¡comorbidi@es ¡normally ¡

associated ¡with ¡adult ¡obesity ¡(e.g., ¡over ¡200 ¡ youth ¡with ¡Type ¡2 ¡Diabetes ¡seen ¡at ¡CHM ¡in ¡one ¡ year) ¡

• Excess ¡medical ¡costs ¡associated ¡with ¡adolescent ¡
• verweight ¡are ¡es@mated ¡at ¡14 ¡billion ¡per ¡year ¡

• Affects ¡country’s ¡ability ¡to ¡protect ¡itself ¡ ¡-­‑ ¡25% ¡

not ¡eligible ¡for ¡the ¡military ¡due ¡to ¡obesity ¡

What ¡Works? ¡

• Eat ¡Less ¡and ¡Exercise ¡More!!! ¡

Skills ¡(Transdiagnos@c?) ¡

• Self-­‑monitoring ¡
• Environmental ¡Control ¡
• Managing ¡Cravings ¡(Impulses) ¡
• Ac@vity ¡Scheduling ¡ ¡
• Coping ¡with ¡Triggers ¡
• Refusal ¡Skills ¡
• Organiza@on ¡and ¡Planning ¡
• Emo@onal ¡Ea@ng ¡– ¡Managing ¡thoughts ¡and ¡

moods ¡ ¡

Why ¡isn’t ¡it ¡working? ¡

• Barriers ¡to ¡access ¡
• Poor ¡adherence ¡to ¡treatment ¡par@cularly ¡

among ¡minority ¡families ¡

– Ajendance ¡(eg. ¡high ¡dropout) ¡ – Following ¡treatment ¡recommenda@ons ¡ – Skills ¡prac@ce ¡(e.g., ¡homework) ¡

• Mo@va@on ¡for ¡skills ¡prac@ce ¡
• Mo@va@on ¡to ¡overcome ¡systemic ¡challenges ¡
• Addressing ¡these ¡issues ¡in ¡caregiver ¡and ¡teen ¡

First ¡we ¡try ¡home-­‑based ¡treatment! ¡

• 48 ¡African ¡American ¡adolescents ¡with ¡obesity ¡
• Tradi@onal ¡pilot ¡RCT: ¡6 ¡months ¡of ¡home-­‑based

¡ cogni@ve-­‑behavioral ¡skills ¡treatment ¡versus ¡ Shapedown ¡(office-­‑based ¡group) ¡

• 77% ¡female; ¡55% ¡single ¡parent ¡family; ¡mean ¡

income ¡28k; ¡average ¡baseline ¡BMI=38 ¡

• Medium ¡effect ¡sizes ¡.3-­‑.4 ¡(tx ¡group: ¡95% ¡
• verweight ¡to ¡88% ¡overweight, ¡3% ¡drop ¡body ¡fat, ¡

BMI ¡to ¡37) ¡

Funded by the American Diabetes Association Naar-King, Ellis et al. ( 2009) Journal of Adolescent Health

But ¡what ¡about ¡session ¡ ajendance? ¡

Quartile # of Sessions Weight Change 1 0 to 12 +11.5 2 13 to 21 +2.4 3 22 to 41

• 3.5

4 42 or more

• 10.3

What ¡about ¡mo@va@on? ¡

• Quan@ta@ve ¡data ¡(MacDonell ¡et ¡al., ¡2010) ¡

– Youth ¡mo@va@on ¡– ¡readiness ¡to ¡change ¡– ¡related ¡ to ¡dose ¡received ¡

• Qualita@ve ¡study ¡comparing ¡families ¡with ¡

successful ¡weight ¡loss ¡and ¡drop-­‑outs ¡(Carcone ¡et ¡al., ¡

2011) ¡

– Caregiver ¡mo@va@on ¡to ¡engage ¡in ¡weight ¡loss ¡ treatment ¡despite ¡significant ¡stressors ¡ ¡

Study ¡1 ¡Conclusions: ¡MST ¡for ¡Obesity ¡ in ¡Minority ¡Adolescents ¡

• Home-­‑based ¡skills ¡treatment ¡shows ¡some ¡

promise ¡to ¡improve ¡weight ¡in ¡African ¡ American ¡youth ¡but ¡modest ¡effects ¡for ¡ rela@vely ¡high ¡cost ¡

• Effects ¡are ¡strongest ¡among ¡those ¡who ¡

receive ¡a ¡full ¡dose ¡of ¡treatment ¡

• Interven@ons ¡to ¡directly ¡address ¡mo@va@on ¡

may ¡be ¡added ¡to ¡MST ¡to ¡increase ¡dose ¡of ¡ treatment ¡received ¡ ¡

What ¡are ¡best ¡prac@ces ¡to ¡ enhance ¡intrinsic ¡mo@va@on? ¡

• A specific method of communication
• Many RCTs and meta-analyses in adults,

particularly minority adults

• May be most effective as a brief

intervention or precursor to more intensive treatment

• First meta-analyses published in 2012 on

adolescent substance use; in press for pediatric health behaviors

How ¡do ¡we ¡adapt ¡MI ¡for ¡ adolescent ¡obesity? ¡

• 44 African American adolescents (12-17)

seen in CHM Primary Care

• Traditional Pilot RCT: 4 office sessions of

dietitian MI or dietary education over 3 months

• MI with adolescent to increase motivation

for food and activity changes

• MI with caregiver to increase motivation to

make behavioral changes to support youth

Funded by Children’s Research Center of Michigan; MacDonell et al., 2012

Study ¡2: ¡Intrinsic ¡Mo@va@on ¡-­‑ ¡MI ¡

Baseline (Mean, SD) 3-month (Mean, SD) Effect Size: Cohen’s d Paired samples t-test (Within subjects) Intrinsic Motivation for Exercise (1-7 scale for 11 items) MI 45.14 (11.41) 52.93 (13.26) 0.51 p = .06 CO 46.35 (15.91) 47.88 (12.06) p = .59 Intrinsic Motivation for Nutrition (1-7 scale for 11 items) MI 49.93 (14.17) 53.21 (12.28) 0.03 p = .30 CO 46.82 (14.82) 49.82 (11.99) p = .27 Fast food use per week (times eaten in past week) MI 2.14 (1.51) 1.07 (1.00) 0.88 p = .02 CO 1.41 (1.50) 1.71 (1.31) p = .49 Soft drink frequency (1-6 scale for single item) MI 3.00 (1.81) 2.25 (1.42) 0.20 p = .04 CO 3.07 (2.12) 2.67 (1.72) p = .51

Communica@on ¡Behaviors ¡

• Coded ¡transcripts ¡of ¡MI ¡sessions ¡
• Analyzed ¡rela@onships ¡between ¡provider ¡

communica@on ¡behaviors ¡and ¡adolescent ¡ mo@va@onal ¡statements ¡

• Using ¡sequen@al ¡analysis(GESQ) ¡iden@fied ¡key ¡

skills ¡within ¡an ¡MI ¡skill ¡set ¡that ¡are ¡par@cularly ¡ relevant ¡for ¡young ¡people ¡with ¡obesity ¡ (probability ¡that ¡a ¡provider ¡behavior ¡predicts ¡ adolescent ¡communica@on) ¡

Study ¡2 ¡Conclusions: ¡Intrinsic ¡ Mo@va@on ¡-­‑ ¡MI ¡

• Brief ¡MI ¡interven@on ¡(2 ¡sessions) ¡shows ¡

promise ¡for ¡increasing ¡mo@va@on ¡and ¡ producing ¡small ¡behavioral ¡changes ¡

• Consider ¡adding ¡MI ¡to ¡home-­‑based ¡

treatments ¡– ¡communica@on ¡founda@on ¡

• Important ¡lessons ¡learned ¡ ¡regarding ¡training, ¡

quality ¡assurance, ¡and ¡session ¡structure ¡ ¡

What ¡are ¡best ¡prac@ces ¡to ¡increase ¡ extrinsic ¡mo@va@on? ¡

• Small ¡pilot ¡mul@ple ¡point ¡baseline ¡(within ¡

subjects)design ¡(twice ¡weekly ¡weights ¡for ¡24 ¡ weeks); ¡N=6 ¡

• Skills ¡only ¡for ¡4-­‑12 ¡weeks ¡(varied ¡based ¡on ¡

ajendance) ¡then ¡

• Weekly ¡incen@ve ¡for ¡teen ¡for ¡weight ¡loss ¡and ¡

for ¡parent ¡for ¡teen ¡weight ¡loss ¡over ¡12 ¡weeks ¡

• Delivered ¡by ¡community ¡health ¡worker ¡

(paraprofessional) ¡ ¡

Study ¡3: ¡Extrinsic ¡Mo@va@on ¡-­‑ ¡CM ¡

• Linear ¡Mixed ¡Model ¡for ¡Repeated ¡Measures ¡

accounts ¡for ¡varied ¡baseline ¡period) ¡

• First ¡Model: ¡two ¡linear ¡piecewise ¡contrasts; ¡
• ne ¡to ¡establish ¡the ¡baseline ¡weight-­‑loss ¡trend

¡ and ¡the ¡other ¡to ¡test ¡for ¡change ¡in ¡trend ¡aTer ¡ the ¡introduc@on ¡of ¡CM ¡– ¡Difference ¡in ¡Linear ¡ Trend ¡NOT ¡SIGNIFICANT ¡

• Second ¡Model ¡taking ¡into ¡account ¡caregiver ¡

ajendance ¡(second ¡contrast) ¡

• ­‑5 ¡
• ­‑4 ¡
• ­‑3 ¡
• ­‑2 ¡
• ­‑1 ¡

• ver ¡baseline ¡interval, ¡b1. ¡

Study ¡3: ¡Extrinsic ¡Mo@va@on ¡-­‑ ¡CM ¡

Per protocol analysis

Study ¡3: ¡Extrinsic ¡Mo@va@on ¡-­‑ ¡CM ¡

U01: ¡Obesity ¡Related ¡ Behavioral ¡Interven@on ¡ Trials ¡– ¡6 ¡sites ¡

Orbit

Sponsored ¡by: ¡ ¡ NHLBI, ¡NIDDK, ¡OBSSR ¡

PUTTING IT ALL TOGETHER: FULL SCALE SMART

IDENTIFIED ¡COMPONENTS ¡FROM ¡ PILOT ¡STUDIES ¡

• FAMILY ¡WEIGHT-­‑LOSS ¡SKILLS: ¡From ¡STUDY ¡1 ¡
• INTRINSIC ¡MOTIVATION: ¡MOTIVATIONAL ¡

INTERVIEWING ¡ ¡(MI) ¡from ¡STUDY ¡2 ¡

• EXTRINSIC ¡MOTIVATION: ¡CONTINGENCY ¡

MANAGEMENT ¡(CM) ¡from ¡STUDY ¡3 ¡

WHICH ¡COMPONENTS ¡ARE ¡BEST ¡AND ¡ FOR ¡WHOM? ¡

Fit ¡Families ¡Key ¡Ques@ons ¡

PRIMARY ¡AIM ¡1: ¡What ¡is ¡the ¡effect ¡of ¡the ¡ini@al ¡ treatment ¡(i.e., ¡office-­‑ ¡vs. ¡home-­‑based ¡ treatment) ¡on ¡outcome ¡(Δ% ¡overweight, ¡% ¡ bodyfat, ¡metabolic ¡outcomes)? ¡ PRIMARY ¡AIM ¡2: ¡Among ¡the ¡non-­‑responders, ¡ what ¡is ¡the ¡effect ¡of ¡the ¡secondary ¡treatment ¡ (i.e., ¡skills ¡vs. ¡con@ngency ¡management) ¡on ¡

• utcome? ¡

¡

• ¡ ¡

Fit ¡Families ¡Key ¡Ques@ons ¡

• SECONDARY ¡AIM: ¡Which ¡adap@ve ¡

interven@on ¡(phase ¡1 ¡to ¡phase ¡2 ¡sequence) ¡ led ¡to ¡the ¡best ¡outcome? ¡

• What ¡caregiver ¡and ¡youth ¡characteris@cs ¡

moderate ¡these ¡primary ¡and ¡secondary ¡aims? ¡

– Baseline ¡Caregiver ¡and ¡Youth ¡Mo@va@on ¡ – Baseline ¡Youth ¡Execu@ve ¡func@oning ¡ – Single ¡Parent ¡Status ¡ – Barriers ¡to ¡Treatment ¡Par@cipa@on ¡ – Rela@ve ¡Reinforcing ¡Value ¡of ¡Food ¡

Fit ¡Families ¡Sample ¡

• Our ¡Sample ¡(N=181) ¡

– 67% ¡female, ¡Mean ¡age=13.75 ¡years ¡(12 ¡to ¡16) ¡ – Baseline ¡weight ¡ranged ¡from ¡133.00 ¡to ¡451.00 ¡ pounds ¡(M=229.97, ¡SD=51.13), ¡ – Percent ¡overweight ¡ranged ¡from ¡35.38% ¡to ¡218.47% ¡ (M=96.81, ¡SD=37.59). ¡ ¡ – Median ¡income ¡range=$12,000-­‑$15,999 ¡ – Caregivers’ ¡(primarily ¡mothers) ¡weight ¡ranged ¡from ¡ 133.00 ¡to ¡625.00 ¡pounds ¡(M=245.29, ¡SD=67.29), ¡ 88.4% ¡obese ¡(BMI≥30.0). ¡ ¡

Preliminary ¡Findings: ¡Within ¡Group ¡ Differences ¡in ¡Percent ¡Overweight ¡

• Overall ¡significant ¡weight ¡loss ¡in ¡phase ¡one ¡ ¡
• At ¡3 ¡months ¡[t(317.89) ¡= ¡2.11, ¡p ¡= ¡.035] ¡and ¡at ¡

7 ¡months ¡ ¡[t(317.95) ¡= ¡4.22, ¡p ¡< ¡.001] ¡

• From ¡baseline ¡to ¡the ¡end ¡of ¡Phase ¡2, ¡

par@cipants ¡across ¡condi@ons ¡reduced ¡ percent ¡overweight ¡by ¡2.96% ¡ ¡(95% ¡CI: ¡1.64%, ¡ 4.27%). ¡ ¡Clinically ¡significant? ¡

Between ¡Group ¡Differences ¡in ¡ Percent ¡Overweight ¡

HB-MIS also “primed the pump” for intervention dose, as those in HB-MIS attended significantly more sessions in Phase 2 regardless of tx condition

Preliminary ¡Conclusions ¡Regarding ¡ Primary ¡Aims ¡

• Small ¡amounts ¡of ¡weight ¡loss ¡achieved ¡across ¡condi@ons ¡
• Small ¡differences ¡between ¡office-­‑based ¡and ¡home-­‑based ¡ ¡
• Interven@on ¡dose ¡is ¡markedly ¡different ¡between ¡

treatment ¡components. ¡ ¡More ¡potent ¡interven@ons ¡ needed ¡to ¡make ¡increased ¡interven@on ¡dose ¡worthwhile! ¡

Future ¡Direc@ons ¡

• SMART ¡design ¡provides ¡many ¡opportuni@es ¡to ¡

guide ¡interven@on ¡development ¡ ¡Percent ¡body ¡fat ¡and ¡other ¡biomarkers ¡ ¡Tes@ng ¡the ¡full ¡adap@ve ¡interven@ons ¡ ¡Moderator ¡analysis ¡ ¡

• Small ¡Pilots ¡to ¡improve ¡potency ¡now ¡that ¡we ¡

now ¡how ¡to ¡more ¡effec@vely ¡deliver ¡treatments ¡

Developing Adaptive Interventions for Children with Autism who are Minimally Verbal: Two SMART Case Studies

• Univ. of Michigan, ∗Univ. of California Los Angeles, ∗∗Vanderbilt Univ.

Outline

Sequential, Individualized Treatment is Often Needed

• ver time in response to the specific needs and evolving status of the

Definition of an Adaptive Intervention

Definition of an Adaptive Intervention

Example of an Adaptive Intervention in Autism (Some Background First...)

Example of an Adaptive Intervention in Autism (Some Background First...)

Example of an Adaptive Intervention in Autism (Some Background First...)

Example of an Adaptive Intervention in Autism (Some Background First...)

• lder, minimally-verbal children with autism.

Example of an Adaptive Intervention in Autism

Example of an Adaptive Intervention in Autism

Continue: JASP Responders JASP Augment: JASP + AAC Slow Responders

How was response/slow-response defined?

Many Unanswered Questions when Building an Adaptive Intervention.

What is a Sequential Multiple Assignment Randomized Trial (SMART)?

What is a Sequential Multiple Assignment Randomized Trial (SMART)?

Example of a SMART in Autism Research

Example of a SMART in Autism Research

Example of a SMART in Autism Research

SMARTs permit scientists to answer many interesting questions useful for building a high-quality adaptive intervention.

Example of a SMART in Autism Research (N = 61)

Recall: The 3 AIs Embedded in the Example Autism SMART

Challenges in the Conduct of this Initial Autism SMART

Recall: The 3 AIs Embedded in the Example Autism SMART

Results from an Analysis of the Autism SMART

• 0.96

• 11.00

• 10.5

• 3.2

Analysis of Longitudinal Outcomes in the Autism SMART

Interventions for Minimally Verbal Children with Autism

Primary and Secondary Aims

What the original study did not aim to examine?

Interventions for Minimally Verbal Children with Autism

Conclusions and Some Final Remarks

Thank you!

Discussion Peter Thall Department of Biostatistics M.D. Anderson Cancer Center

Workshop P6: Clinical Trials for Adaptive Intervention Designs: Design and Conduct of Sequential Multiple Assignment Randomized Trials Society for Clinical Trials, Annual Meeting Philadelphia, PA Sunday, May 18, 2014

Disclaimer

No cute puppies were harmed in preparing this discussion.

OPINIONS The two main reasons to do SMARTs

• 1. They reflect actual medical practice: a multi-stage outcome-

adaptive process, or “dynamic treatment regime” (DTR).

• 2. If you want data for unbiased comparisons, randomize.

Together, these imply the scientific ideal is to be SMART à account for multiple stages of therapy à randomize at each stage, if it is ethical/practical Important Caveats

• 1. Actual implementation is hard!
• 2. Only viable DTRs should be included, not mathematical

fantasies that a physician would never use. Randy Millikan: “The future is combinations and sequences.”

Actual Goals of SMARTs

• 1. Estimate overall outcome means (expected survival or DFS

in oncology, probability of, or mean time to, disease worsening in a study to treat alcoholism, schizophrenia, etc.) for all DTRs in an unbiased fashion. Then rank and select.

• 2. Use the estimates and ranks to refine the DTRs, or possibly

modify physician/professional behavior

• 3. Don’t try to test hypotheses. There are usually far too many

DTRs, and hypothesis testing is silly anyway. p-values Data Analysis: Things almost never go as designed (dropouts, missed appointments, deviation from study design, etc.) à Methods for analyzing observational data typically are needed - IPTW, G-estimation, matching, imputation, etc.

Identifying the DTRs (blue à for randomization) Response A1 A1 No Response B1 Response A2 A2 No Response B2 The Regimes: (A1, A1, B1) = Give A1, repeat it if you get a response, switch to B1 if you don’t (A2, A2, B2) = Give A2, repeat it if you get a response, switch to B2 if you don’t There is no re-randomization - - but they still are adaptive.

Identifying the DTRs Response A1 No Response B1 Response A2 No Response B2 The Regimes: (A1, A1, B1) (A2, A2, B2) (A1, A2, B1) : Even if A1 gets a response, try A2 (A2, A1, B2) : Even if A2 gets a response, try A1 Note: One could easily get 4 more DTRs by switching B1 and B2. A1 A2 A1 A2

The SPARC SMART in Metastatic Renal Cancer

DFS = approx 9 months à Try 3 targeted agents : b = bevacizumab, s = sunitinib, t = temsirolimus Stage1 of Therapy At entry, randomize the patient among { b, s, t } Stage 2 of Therapy If the 1st failure is disease progression (not discontinuation) at time of progression re-randomize the patient between the two treatments not received initially “Try something. If it fails, try something else”

1st Line

b s t

2nd Line Strategy

s t b t b s (b, s) (b, t) (s, b) (s, t) (t, b) (t, s)

1st failure

• Induction chemo is given to de-bulk the disease
• Y1 = Chemo-Rad Outcome, Y2 = disease-free survival time
• CRT: 3 chemo types x 3 RT types x 2 RT fields = 18 CRTs
• [2 Ind Chemo = Y/N] x [18 CRTs] x [4 surg. cats.] à 144 DTRs

But only 81 with n> 0 in our dataset . . .

Data Mining DTRs for Esophageal Cancer: No Randomization Baseline Covs Induction Chemo No Induction Chemo Chemo- Radiation Therapy Surgery (3 types) No Surgery Y1 Y2 Y2

Inbal (Billie) Nahum-Shani: Intro to Adaptive Interventions and SMARTs The ‘Adaptive Drug Court Program’ is an example of an adaptive intervention. It is not a SMART ( no randomization ). Q: (in the figure) What happens if they are compliant or responsive? The ADHD SMARTer MED,BMOD study (pay attention!!) the one second stage rule is “Repeat a treatment that works; if not, Enhance (give more of the same trt).” The other second stage rule Augments (add a 2nd other type of trt)” Q: Why not also include “Switch to something else” for non- responders, instead of just “augment”? Page 11: Nice “If-then-else” computer program!! This will help people think about adaptive interventions as well-

• perationalized treatment regimes which is what they are.

James McKay : 2-Stage SMARTs for SUDs Acronyms SUD = Substance Use Disorder, e.g. drinking too much beer because you like your SUDs (or cocaine: SNORTs) TX = treatment (not the state where I live) IOP = Intensive Outpatient Program Goal Prevent relapse for alcoholics (or snorters) who graduate from residential programs and get clean Questions: Can a 2nd (adaptive) try with a different TX improve outcomes for SUD patients who § do not want standard TX § go to TX but leave (“drop out”) § rejected a first offer of TX

Standard SUD TXs Behavioral intervention, grp counseling, 12-step program Translations § Drop Out : The attempt at TX failed § Tailoring : Base actions on IOP attendance § Participant : Middle-aged African-American man § SMART : Repeatedly phone people who drop out until they come back to IOP, randomize if ‘disengaged’ § Outcome : How much alcohol or coke they say they are using, or an actual urine sample Conclusions: 1) If you leave it up to the subject, they are more likely to keep drinking / snorting - - - Relapse is a problem in SUD! 2) “Compliance” is the actual outcome that treatment should target.

Amy Kilbourne: Mental Health Implementation Science Focus on the “Research-to-Practice” gap by getting sites to adopt evidence based practices - - I could not agree more!!! Major Caveat - - Make sure your conclusions are very likely to be true before you make a recommendation. Q: What is the intervention you are trying to get sites to adopt in your SMART (Study #3)? Is there strong evidence for it? Excellent use of acronyms: EBQI, PARiHS, REP, GTO (also a great muscle car from the 60’s), TA, ROCC, QOL, SMI, VHA, VA NPCD, CSI REP was developed by the CDC to improve adoption of HIV prevention and treatment interventions. Q: Is this goal of REP the same as enhancing treatment fidelity, buy in, or compliance?

SMI Re-Engage Findings (Study#2) Comparison of Enhanced REP (n=39) vs Standard REP (n=49) based on % of veterans with updated documentation Q: Is the effect around March-April clinically significant or just statistically significant? Q: The adaptive intervention (ie, Standard REP) appears to catch-up. What are the policy implications of this?

SMART REP Trial on Facilitation

• EF+IF versus usual dissemination strategy
• 60+ community care clinics across 3 US states
• Applied to build cost-effective adaptive implementation

strategy SMART REP Trial (60+ clinics, 1200+ patients)

• Goals: Among sites not initially responding to REP, estimate

effect of adaptive implementation interventions in sites receiving REP+EF/IF vs REP+EF on 12-month outcomes

• SMART : Very complex, with run-in (all sites start with REP),

re-randomization among continued non-responders, etc. Q: Results likely to be very sensitive to definition of “response” at the site level. Is the definition based on data? Rationale? Q: The study is complex. What challenges do you anticipate?

Sylvie Naar-King: Pilot Studies of Obesity Nice pictures - - very Darwinian. Comment: Actual SMARTs are almost all “pilot” studies,

• f varying sizes, since N / [number of DTRs] is seldom

large (e.g., in cancer research). Motivation: We now live in Fat City. People are fat, and getting fatter, and being fat causes lots of terrible

• diseases. Getting less fat à better overall health à

This is Very Important!! Strategy: 1) Get ‘em while they’re young, i.e. target adolescents. 2) Eat less, exercise more. But behavior change is difficult to get people to do!

Pilot Studies of Obesity In home-based treatments, very little success, very dependent on adherence and motivation à Develop an adaptive intervention to enhance motivation using a SMART!! Comment: In summary of “Intrinsic Motivation (IM vs. CO)” Study 2, there appears to be too much reliance on p-values. Could try using posterior probability summaries. Q: Are the effect sizes clinically significant, actually meaningful? Next Strategy: Use extrinsic motivation (CM)

Pilot Studies of Obesity The “Home vs Office” SMART Q1: Why is 3% weight loss in 3 months a “response” ? Specifically, what is the rationale for this definition? Empirically-based? Q2: What happened to changing diet and exercise ? Is changing diet and increasing exercise part of the skills training components in stage 1 of treatment? Q3: Your first question focuses on delivering the intervention at HOME vs in the OFFICE/CLINIC? Why is this such an important scientific question to address? Q4: Your second question focuses on effect of CM? Can you clarify what is CM? Is CM a feasible intervention in the real-world?

Dan (“The Man”) Almirall: Adaptive Interventions for Childhood Autism: Two SMARTs Motivation: Traditional interventions to develop language skills in autistic children are lousy à Need for improvement Possibilities: JASP or AAC (acronyms are essential) The DTR: If JASP works continue it; o/w augment w. AAC, in 12-week stages. Outcome is 7-variate Y Comment: The way that Y is reduced to 1-dim response is critical à M advice: Use additive elicited utility weight function U(Y) = w(Y1) + … + w(Y7) The Kasari UCLA study rule becomes U(Y) > u* , or possibly with AR proportional to posterior mean E{U(Y)|θ}

Comparisons of Competing DTRs in the Kasari UCLA Autism Study Comment: The data seem most useful to a) Rank (ACC, ACC+) > (JASP, ACC) > (JASP, JASP+) b) Weed out inferior DTRs (JASP, JASP+), (JASP, ACC) With n=61, as usual, the results are only suggestive. 2nd Study On Social Play Same design, different twist - - - human behavior is complex!

Thanks for listening. Now it’s time for lunch. And eat a salad instead of french fries!!