Designing for Prevention: Putting Evidence-Based Prevention - - PowerPoint PPT Presentation

Designing for Prevention: Putting Evidence-Based Prevention Strategies into Practice in Diverse Communities

Bruce D. Rapkin, PhD Professor of Epidemiology and Population Health Division of Community Collaboration and Implementation Science Albert Einstein College of Medicine

Conclusions

• Participatory models of intervention research are

superior to top down models.

• Scientific rigor does not equal the randomized

controlled trial.

• Communities of shared interest must form around

Learning Systems - with successive studies leading to refinement of key distinctions among interventions, types populations and settings

• Comprehensive dynamic trials are intended to support

the learning system, by inventing and evolving interventions in place, drawing upon multiple sources of information gained during the conduct of an intervention.

Why do we need alternatives to the Randomized Clinical Trial model?

• The community argument
• The business practices argument
• The statistical argument
• The scientific argument
• The psychological argument

Community-Academic Relationships Imposed by the Medical Model

• Funders resist changing interventions promoted as

national standards (despite absence of external validity)

• Communities must figure out how to fit themselves to the

program – the program dictates the terms

• What communities know about prevention or engaging

clients is only relevant if it pertains to the manual

• A tightly scripted protocol does not respond to

collaborators’ circumstances

• Danger that lessons learned will be framed as “what the

community did wrong to make the program fail”

• Unwillingness to consider limits of research theories and

methods, local problems will remain unsolved

Is this any way to run a business?

• Businesses including clinics examine their

practices continually to seek improvements

• Research protocols are designed to resist or

restrict change over the course of a study, to ensure “standardization”

• Lessons learned must be “ignored” until the next

study

• Valuing fidelity over quality impedes progress to
• ptimal intervention approaches

What is a “Treatment Effect”?

• The RCT is designed to determine an

estimate of a population “treatment effect”

• Is the “treatment effect” a useful

construct?

– How is the effect determined by the initial composition of the sample? – Is information beyond aggregate change error variance or meaningful trajectories? – How does the control condition determine the effect’? Is this ignorable?

• 10
• 8
• 6
• 4
• 2

2 4 6 8 10 TIME 1 TIME 2 TIME 3

• 10
• 8
• 6
• 4
• 2

2 4 6 8 10 TIME 1 TIME 2 TIME 3

Does a Successful RCT Mean that Faithful Replication of an Intervention will Ensure Outcomes?

• Not necessarily because…

– Original RCT findings do not generalize to a “universe” – too dependent on context – Mechanics of interventions have different implications, depending on setting norms – Even the meaning & impact of core elements may be transformed by local ecology

• We don’t know because of the lack of

attention to external validity!

Desirable Features for Study Designs

• Must take into account diversity inherent in the

determinants of health and risk behavior

• Must recognize that different people can respond to the

same intervention in different ways, or in the same way for different reasons

• Must accommodate diversity and personal preferences
• Must avoid ethical dilemmas associated with

substandard treatment of some participants

• Must be responsive to evolving understanding of how to

best administer an intervention, and to local innovations and ideas

• Must contribute to community capacity building and

empowerment at every step of the research process

The Research Paradigm We Need…

• A Learning System
• A Community Science = A “WIKI”
• Who has input

– True integration of multiple methods and perspectives

• Who makes decisions?

– The peer review process – The community review process

• Progress toward adequate intervention theory

and practice can be quantified

We have (some of) the building blocks

But bridges are always built Somewhere -

Comprehensive Dynamic Trials Designs

• Comprehensive => use complete

information from multiple sources to understand what is happening in a trial

• Dynamic => built-in mechanisms for

feedback to respond to different needs and changing circumstances

• Trials => Systematic, replicable activities

that yield high quality information useful for testing causal hypotheses

Three CDT Designs

• Community Empowerment to enable

communities to create new interventions

• Quality Improvement to adapt existing

manuals and procedures to new contexts

• Titration-Mastery to optimize algorithms for

delivering a continuum of services

Rapkin & Trickett(2005)

CDT Community Empowerment Design

• Closest to the “orthodox” model of CBPR

– No pre-conceived “intervention” – No need for externally-imposed explanation of the problem or theory of change

• Common process of planning
• Common criteria for evaluating

implementation across multiple settings and/or multiple “epochs”

CDT Quality Improvement Design

• Starting point

– An evidence-based intervention – A established standard of practice – An innovation ready for diffusion

• Alternative to the traditional “top-down”

model of intervention dissemination

• Begin with a baseline intervention, then

systematically evolve and optimize it

CDT Titration to Mastery Design

• Suited to practice settings committed to

the client/patient/participant

• Does NOT ask about intervention effects?
• Rather, asks what combination of

interventions will get closest to 100% positive outcome most efficiently?

• Begins with a tailoring algorithm to

systematically apply a tool kit, which is then evolved and optimized

Contrasting Comprehensive Dynamic Trials with the Prevailing Paradigm

View of: CDT Paradigm RCT Paradigm Outcomes Replicability of processes Direct generalizability of outcomes Time Processes unfold over time Time dimension collapsed Dynamics Continuous improvement Frozen intervention protocol Knowledge Require community input Limited or no input Ecology Examine research partnerships No inquiry about researchers’ roles Rigor Problem solve using all evidence Closed to outside findings Precision Focus on particular responses Focus on aggregate response Causality Reciprocal causality Linear causality Validity Findings framed by context Findings presumed universal Synthesis Systems modeling is necessary Use of modeling limited

Ingredients of a Comprehensive Dynamic Trial

Just add community and stir…

How does the CDT Feedback Loop Work?

What Types of Data Are Needed?

• Outcome Indicators
• Fidelity
• Mechanistic Measures
• Intervention Processes
• Structural Impediments
• Adverse Events
• Propitious Events
• Context Measures

The Deliberation Process

• Key stakeholders should be involved in

deliberation

• Research systematically provides data to

stakeholders to make decisions about how to modify and optimize interventions

• Timing is based upon the study design
• The nature and extent of changes should be

measurable, and expressed in terms of intervention components and procedures

• Deliberation process should be bounded by

theory

Ethical Principles –

Lounsbury et al.

• Transparency
• Shared Authority
• Specific Relevance
• Rights of Research Participants

– Self-Determination – Third-Party Rights – Employees’ Rights

• Privacy
• Sound Business Practice
• Shared Ownership

A Model for Maximizing Partnership Success: Key Considerations for Planning, Development, and Self-Assessment – Weiss et al.

Environmental Factors Composition, Structure and Functions Characteristics of the Group Process Intermediate Indicators of Partnership Effectiveness

Development & Implementation of Programs & Activities

Outcome Indicators of Partnership Effectiveness

A CDT-QI Model to Disseminate an Evidenced- Based Approach to Promote Breast Cancer Screening

The Bronx ACCESS Project

The Albert Einstein Cancer Center Program Project Application Under Development

Theoretical Underpinnings

• Empowerment via Mediating Structures
• Diffusion of Innovation
• Community-Based Participatory Research
• Intervention Tailoring/Lay Health Advisors
• Amalgam of our team’s prior intervention

research

– Weiss – effective partnerships – Lounsbury – collaborative capacity – Thompson – trust and adherence – Goodman – leadership – Rapkin – problem-solving

Bronx ACCESS Conceptual Model of Dissemination

• Building Agency

Capacity & Readiness Optimizing Health Promotion Continuum Engaging Community in Adaptation & Tailoring Population Adherence Agency Routinization

+ +

+ +

Emergence

• f

Competing Priorities Barriers to Access Arise

• r Worsen

Erosion of Trust Exposure to Misleading Information Loss of Resources or Coverage

Dissemination involves dynamic, mutually-reinforcing processes that unfold over time

• Multi-level

interventions are intrinsic to the dissemination process, interacting to counter interference and promote desired

• utcomes over

time We hypothesize that this model of dissemination will lead to ever greater improvements in in adherence to breast screening guidelines, associated with increased agency institutionalizatio n of a continuum

• f evidence-

based tailored strategies

At any time, factors at individual, community,

• rganizational, or

systems levels can interfere with the dissemination process, and must be countered.

Initial Conditions

• Social, cultural and

economic population characteristics

• Baseline adherence to

guidelines

• Agency resources
• Agency reach
• Community

resources & involvement

• Local service ecology
• Baseline levels of

sources of interference

2 3 1

Intervention at Three Levels

• Individual (Continuum via Lay Health Advisors)

– Women from three Bronx neighborhoods, out of adherence with, or uncertain about, screening guidelines

• Organizational (Process Consultation)

– Community social service agencies able to reach large numbers of medically-underserved women

• Community (Adaptation via Participatory

Research)

– Representatives of different sectors with relevant knowledge to help guide disseminations

• PLUS – Modeling processes to inform

policy

Building Agency Capacity & Readiness Optimizing Health Promotion Continuum Engaging Community in Adaptation & Tailoring Population Adherence Agency Routinization

+ +

+ +

Changing the Rules to Conduct Research in the Real World

• How to incorporate local input in an evidence

based paradigm?

– Solution: Fidelity gets a vote, but not a veto

• How to deal with cultural and risk specificity of

mammography screening interventions?

– Solution: disseminate a “suite” of theoretically equivalent strategies as a tool kit

• How to address agencies’ many priorities?

– Solution: Encompass these as “community targeted strategies” for outreach & retention

How Do You Get Science Out of All That Data?

In Any One CDT …

• Analyses are intrinsic to intervention
• The program should get better as it goes along
• Experimental effects may be examined in context
• Particularly interested in accounting for diverse

trajectories and patterns of responses

• Ability to steer toward optimal intervention components
• Case study of community problem solving
• Able to examine setting impacts, sustainability

The Real Payoff – Science as a Community Process

The Epistemology of CDT

• A Community Science = A “WIKI”
• A learning system
• Who has input

– True integration of multiple methods

• Who makes decisions?

– the peer review process

• Progress toward theory development can be

quantified

• Theory can be (provisionally) completed

The Comparative Effectiveness Matrix

What outcomes distinct are associated with different intervention approaches? How do characteristics of target population affect

• utcomes?

How are outcomes affected by history, resources, and contexts?

Oipc|t

The conditional probability

• f an outcome,

for this type of intervention with this population in this context, given what is known at the present time.

Wiring Up the Comparative Effectiveness Matrix:

Systems Dynamics? Neural Networks? Genetic Algorithms?

1) An intervention 3) in different contexts 1 2 3 4 5 6 7 8 2) experienced by different people 4) may lead to different

• utcomes.

ARROWS indicate probabilistic pathways Oipc|t at time T

Tx

Scientific Enterprise Needed to Support this paradigm

• Evaluation for funding will consider soundness of

researchers’ relationships with communities

• Multiple sources of data will gain importance
• Emphasis on practice-based evidence
• Case studies of planning, decision making and

community involvement will be highly important

• Awareness that results depend on context, so a single

trial will not receive undue weight

• Investigators will work in tandem to create service

suites and knowledge bases

• Meta-analysis will grow more important, as a way of

integrating multiple types of studies

Where’s the Science?

• In community process.
• In understanding how researchers’ roles

and activities impact CBPR.

• In the evolution and refinement of

intervention implementation strategies, through dynamic exchange and reflection.

• It emerges out of the synthesis of CBPR

findings

Conclusions

• Participatory models of intervention research are

superior to top down models.

• Scientific rigor does not equal the randomized

controlled trial.

• Communities of shared interest must form around

Learning Systems - with successive studies leading to refinement of key distinctions among interventions, types populations and settings

• Comprehensive dynamic trials are intended to support

the learning system, by inventing and evolving interventions in place, drawing upon multiple sources of information gained during the conduct of an intervention.