MDICx webinar series From Stories to Evidence: Quantitative - - PowerPoint PPT Presentation

▶
mdicx webinar series from stories to evidence
SMART_READER_LITE
LIVE PREVIEW

MDICx webinar series From Stories to Evidence: Quantitative - - PowerPoint PPT Presentation

Feb 22, 2024 153 likes •685 views

MDICx webinar series From Stories to Evidence: Quantitative patient-preference information to inform product- development and regulatory reviews Shelby Reed Professor, Duke School of Medicine F. Reed Johnson Professor, Duke School of

slide-1
SLIDE 1

1

February 15, 2018

MDICx webinar series From Stories to Evidence: Quantitative patient-preference information to inform product- development and regulatory reviews

Shelby Reed

Professor, Duke School of Medicine

  • F. Reed Johnson

Professor, Duke School of Medicine

Juan Marcos Gonzalez

Assistant Professor, Duke School of Medicine

slide-2
SLIDE 2

From Stories to Evidence: Quantitative patient-preference information to inform product-development and regulatory reviews

MDIC Webinar February 15, 2018

Shelby Reed

Professor, Duke School of Medicine

  • F. Reed Johnson

Professor, Duke School of Medicine

Juan Marcos Gonzalez

Assistant Professor, Duke School of Medicine

This is presentatio tion refle lects ts th the v vie iews o

  • f t

the a auth thors a and should ld n not t be c construed to to r represent th the p polic licie ies o

  • f t

the U U.S. FDA.

Preference Evaluation Research Group (PrefER)

slide-3
SLIDE 3

Patient-centered healthcare movement

3

slide-4
SLIDE 4

Who is qualified to make relative-importance value judgments?

  • Clinicians
  • Politicians
  • Patients

– Informed consent – Individualistic ethic: Individuals are the best judge of their own welfare

“Patient preferences are critical in determining when a product’s benefits outweigh its risks… .”

  • - Robert M. Califf (JAMA 2017)

4

slide-5
SLIDE 5

Increasing Support from FDA

4

FDA’s guidance on benefit-risk determinations for device approvals describes patient tolerance for risk and perspective on benefit as an explicit factor the agency may consider in approval decisions.

slide-6
SLIDE 6
  • Dr. Rob Califf, FDA Deputy, Former FDA Commissioner

6

You don’t know people’s preferences unless you ask them. How do people look at these differences? And I fell in love with the discrete- choice experiments, which I had heard about from the Business School, but had not seen in action and I think that provides major

  • advantages. So this is a great day for me. It’s you know been a long
  • time. Who would have thought it would come from the device world?

… I think it’s a major triumph for the device world that we’re here today, not just talking about it, but with the FDA very involved. To the extent that FDA takes preferences seriously, I think it’s a great day.

Release Event for the MDIC Framework for Integrating Patient Perspective into Medical Device Benefit-Risk Assessments and the FDA Center for Devices and Radiological Health Draft Guidance, May 13, 2015

slide-7
SLIDE 7

History of CDRH interest in PPI

7

Based on Levitan, NIH HCS Collaboratory and PCORnet Grand Rounds, 2016

slide-8
SLIDE 8

Preference-Sensitive Regulatory Decisions

Risks Weight Loss

Patient Engagement Advisory Board (2015) Bennett Levitan Available at: https://www.noexperiencenecessarybook.com/wwzgN/powerpoint-presentation.html

Diet & Exercise

?

8

Less weight loss Lower risks

slide-9
SLIDE 9

Regulatory Impact of the Study

  • EnteroMedics’s Maestro Rechargeable

System for weight loss

  • Device failed to meet its original trial endpoints
  • Device was approved in January 2015

– First new obesity device approved by FDA since 2007 – First approval to result from CDRH's patient preference initiative

http://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/ucm430223.htm

9

slide-10
SLIDE 10

“FDA understands that patients and care-partners who live with a disease or condition on a daily basis and utilize devices in their care may have developed their own insights into and perspectives on the benefits and risks of devices reviewed...”

  • -August 2016

Guidance

Center for Devices and Radiological Health

10

slide-11
SLIDE 11

What are “preferences”?

“Qualitative or quantitative statements of the relative desirability or acceptability of attributes that differ among alternative interventions.”

Medical Device Innovation Consortium (PCBR Framework Report 2015)

Stories from individuals Evidence representative

  • f a group

Often obtained from surveys Defined by what people are willing to give up

  • r
  • health states
  • care processes
  • convenience features
  • other

Characteristics

  • r features
slide-12
SLIDE 12

Approaches

  • Qualitative methods (focus groups, public meetings)

– Identify areas of concern – Provide context for product-development and regulatory decisions

  • Simple quantitative methods (ranking, threshold)

– Prioritization – Tradeoffs involving only two outcomes

  • More advanced quantitative methods (choice experiments, best-worst scaling)

– Tradeoffs involving more than two outcomes – Statistical preference measures (risk tolerance, minimum acceptable benefit, time equivalents) – Publishable regulatory-quality evidence

  • Today’s focus: discrete-choice experiments

12

slide-13
SLIDE 13

Discrete-Choice Experiments to Quantify Patient Preferences

  • Developed, tested, and validated over past 40 years in

– market research – transportation planning – environmental economics – health

  • Daniel McFadden received the Nobel Prize in Economics in

2000 for conceptual and statistical foundations

  • Increased interest and regulatory support because of

commitment to patient-centered healthcare

10

slide-14
SLIDE 14

Example Choice Task: Cystic Fibrosis

14

Mohamed, Johnson, Balp, Calado, The Patient 2016

slide-15
SLIDE 15

Choice-Experiment Features

  • Also known as choice-based conjoint analysis
  • Alternatives consist of combinations of features
  • Preferences among alternatives depend on the relative

importance of features

  • Respondents indicate choices among hypothetical

alternatives

  • Statistical analysis of pattern of choices indicates relative

importance of features

12

slide-16
SLIDE 16

Survey Development

Endpoints Choice Tasks Pretest Interviews Experimental Design

Prefer X

Prefer Y

Prefer X

Prefer Y

Prefer X Level B1 Level B2 Level R1 Level R2

Prefer Y Level B1 Level B2 Level R1 Level R2

Data Collection & Analysis

Preference Weights Definitions

Steps in a Choice-Experiment Study

16

slide-17
SLIDE 17

Applications over product life cycles

CLINICAL DEVELOPMENT REGULATORY REVIEW ACCESS

Weighted endpoints Benefit- risk Value frameworks Personalized medicine

USE

Evidence reviews

GUIDANCE

Study 1 2 3 Quality +

  • ++

N 100 50 300 EndptA

  • +

+ EndptB + +

  • R

Tx A Tx B

Outcome 1 Outcome 2 Outcome 3 SAE 1 SAE 2 SAE 3

Side effects Cost Convenience Alternatives Health outcomes Disease severity

17

slide-18
SLIDE 18

Applications over product life cycles

CLINICAL DEVELOPMENT REGULATORY REVIEW ACCESS

Weighted endpoints Benefit- risk Value frameworks Personalized medicine

USE

Evidence reviews

GUIDANCE

Weighted PRO endpoints

EXAMPLE

18

Study 1 2 3 Quality +

  • ++

N 100 50 300 EndptA

  • +

+ EndptB + +

  • R

Tx A Tx B

Outcome 1 Outcome 2 Outcome 3 SAE 1 SAE 2 SAE 3

Side effects Cost Convenience Alternatives Health outcomes Disease severity

slide-19
SLIDE 19

European Organisation for Research and Treatment of Cancer

Mohamed AF, Hauber AB, Johnson FR, Coon CD. Patient preferences and linear scoring rules for patient reported outcomes. Patient. 2010;3(4):217-27.

Patient-Reported Outcome Quality of Life Questionnaire Preference Weights

16

slide-20
SLIDE 20

Applications over product life cycles

CLINICAL DEVELOPMENT REGULATORY REVIEW ACCESS

Weighted endpoints Benefit- risk Value frameworks Personalized medicine

USE

Evidence reviews

GUIDANCE

Weight- Loss Devices

EXAMPLE

21

Study 1 2 3 Quality +

  • ++

N 100 50 300 EndptA

  • +

+ EndptB + +

  • R

Tx A Tx B

Outcome 1 Outcome 2 Outcome 3 SAE 1 SAE 2 SAE 3

Side effects Cost Convenience Alternatives Health outcomes Disease severity

slide-21
SLIDE 21

Preferences and Regulatory Decisions

Regulators’ Benefit- Risk Threshold

Benefit Risk

  • Disapprove

+ Approve

  • +

+ + + + +

Patients’ Benefit-Risk Threshold

+ + +

  • +

22

slide-22
SLIDE 22

Preference-Sensitive Regulatory Decisions

Risks Weight Loss

Patient Engagement Advisory Board (2015) Bennett Levitan Available at: https://www.noexperiencenecessarybook.com/wwzgN/powerpoint-presentation.html

Diet & Exercise

?

23

Less weight loss Lower risks

slide-23
SLIDE 23

FDA Obesity Study

24

Ho, Gonzalez, Lerner, et al., Surgical Endoscopy. 2015

slide-24
SLIDE 24

FDA Obesity Study

25

Ho, Gonzalez, Lerner, et al., Surgical Endoscopy. 2015

slide-25
SLIDE 25

FDA Obesity Study

Ho et al. Surgical Endoscopy (2015)

30% weight loss ≈ 1.2% mortality risk

26

Ho, Gonzalez, Lerner, et al., Surgical Endoscopy. 2015

slide-26
SLIDE 26

Weight-Loss Device Decision Tool

27

slide-27
SLIDE 27

Applications over product life cycles

CLINICAL DEVELOPMENT REGULATORY REVIEW ACCESS

Weighted endpoints Benefit- risk Value Personalized medicine

USE

Evidence reviews

GUIDANCE

Value Frameworks

EXAMPLE

28

Study 1 2 3 Quality +

  • ++

N 100 50 300 EndptA

  • +

+ EndptB + +

  • R

Tx A Tx B

Outcome 1 Outcome 2 Outcome 3 SAE 1 SAE 2 SAE 3

Side effects Cost Convenience Alternatives Health outcomes Disease severity

slide-28
SLIDE 28

Value Frameworks

2014 2015 2016 2017 29

slide-29
SLIDE 29

StdCare ExpTrx StdCare ExpTrx

QALYs QALYs C C ICER − − =

To compare cost-effectiveness for treatments across conditions, outcomes must be measured using the same units.

Traditional Value Assessment

slide-30
SLIDE 30

Other Elements of Value

Other Elements

  • f Value

Option value Value of knowing Equity Value of hope Impact

  • n
  • thers

Dosing regimen

Clinical benefits

Cost

Value

Adapted from Garrison L, 2016 and Neumann PJ, 2016.

31

slide-31
SLIDE 31

ASCO Value Framework Scoring Rubric

0 to 100 (OS>PFS>RR)

Clinical benefit

  • 20 to 20

Toxicity

0 to 10

Palliation*

0 to 20

Survival curve*

0 to 10

Trx-free interval*

0 to 20

Quality of life*

Clinical benefit Toxicity Bonus points* Net health Benefits Cost per month

DAC:____ Patient payment:___

slide-32
SLIDE 32

Drug Abacus from MSKCC

$12,000 to $300,000 10% to 30% 1.0 to 3.0 1.0 to 3.0 1.0 to 3.0 1.0 to 3.0

Monthly drug price

http://www.drugabacus.org/drug-abacus/tool/

slide-33
SLIDE 33

Applications over product life cycles

CLINICAL DEVELOPMENT REGULATORY REVIEW ACCESS

Weighted endpoints Benefit- risk Value frameworks Personalized medicine

USE

Evidence reviews

GUIDANCE

Crohn’s Disease

EXAMPL E

34

Study 1 2 3 Quality +

  • ++

N 100 50 300 EndptA

  • +

+ EndptB + +

  • R

Tx A Tx B

Outcome 1 Outcome 2 Outcome 3 SAE 1 SAE 2 SAE 3

Side effects Cost Convenience Alternatives Health outcomes Disease severity

slide-34
SLIDE 34

More time in remission Higher risk of cancer Lower risk of infections Less use of steroids

TNF-α inhibitors vs. Corticosteroids for Crohn’s disease

Evidence Reviews: Benefits and harms of Crohn’s disease therapies

More time in remission Higher risk of cancer Lower risk of infections Less use of steroids

35

slide-35
SLIDE 35

Choice Question Example: Crohn’s Disease

PCORI, NCT02316678; PI: James Lewis

36

slide-36
SLIDE 36

Patient-Centered Comparative Effectiveness Research

Patient Preferences Real-World Data DCE

Simulation Model

Time equivalents Treatment 1 Treatment 2

5.5 3.7

Attributes Add-on therapy Hospital- izations Survival Adverse event 1 Adverse event 2

Time equivalents Treatment 1 Treatment 2

Group 1

Group 2

Group 3

Outcomes Add-on therapy Hospital- izations Survival Adverse event 1 Adverse event 2

Time Equivalents Event rates: Treatment 1 Treatment 2

Time equivalents Treatment 1 Treatment 2

Group 1

Group 2

Group 3

slide-37
SLIDE 37

Comparative Effectiveness

Anti-TNFs Prolonged CS Difference

Remission-time equivalents, mean (SD) 5.3 (4.0) 4.5 (3.7) 0.8 (0.5 – 1.1)

Medicaid and Medicare claims analysis Markov model DCE

PCORI, NCT02316678; PI: James Lewis

38

slide-38
SLIDE 38

Potential Patient-Centered Applications

Effects Treatment 1 Treatment 2 Add-on therapy

  • +

Hospitalizations +

  • Survival
  • +

Adverse event 1 +

  • Adverse event 2
  • +

Traditional Comparative Effectiveness Research

Effects Treatment 1 Treatment 2 Add-on therapy

  • +

Hospitalizations +

  • Survival
  • +

Adverse event 1 +

  • Adverse event 2
  • +

Time equivalents 

Preference-based Comparative Effectiveness Research

slide-39
SLIDE 39

Latent-Class Choice-Model Estimates

Crohn’s Disease

  • 10
  • 9
  • 8
  • 7
  • 6
  • 5
  • 4
  • 3
  • 2
  • 1

0 4 8 12 0 4 8 12 0 4 8 12 0 2 8 12 0 5 1530 0 2 5 8 0 2 5 8 Severe Duration Moderate Duration Mild Duration Steroid Duration Infection Risk Cancer Risk Surgery Risk Efficacy Class Steroid Class Risk Class

PCORI, NCT02316678; PI: James Lewis

40

slide-40
SLIDE 40

Comparative Effectiveness

Remission-time equivalents Anti-TNFs Prolonged CS Difference (95% CI) Efficacy Class 1.3 (6.7) 0.1 (6.4) 1.3 (0.8, 1.7) Steroid Class 6.9 (2.9) 6.4 (2.7) 0.6 (0.4, 0.8) Risk Class 7.8 (2.6) 7.3 (2.5) 0.5 (0.3, 0.7)

PCORI, NCT02316678; PI: James Lewis

Medicaid and Medicare claims analysis Markov model DCE

41

slide-41
SLIDE 41

Applications over product life cycles

CLINICAL DEVELOPMENT REGULATORY REVIEW ACCESS

Weighted endpoints Benefit- risk Value frameworks Personalized medicine

USE

Evidence reviews

GUIDANCE

Shoulder dislocation

EXAMPLE

42

Study 1 2 3 Quality +

  • ++

N 100 50 300 EndptA

  • +

+ EndptB + +

  • R

Tx A Tx B

Outcome 1 Outcome 2 Outcome 3 SAE 1 SAE 2 SAE 3

Side effects Cost Convenience Alternatives Health outcomes Disease severity

slide-42
SLIDE 42

Preferences in Practice Guidelines

“Clinicians must communicate evidence-based

  • ptions for treatment, inclusive of their benefits

and risks, and patients must be allowed to express their goals and preferences.”

Gynecol Oncol. 2016;143(1):3-15.

slide-43
SLIDE 43

Potential Patient-Centered Applications

Patient characteristics

demographic and contextual variables

Choice questions

generate patient-level preference weights, or classify patients into preference groups

Decision model

predict expected outcomes with alternative treatments

Compute net benefits

combine expected outcomes with preference weights

Decision aids

slide-44
SLIDE 44

First-time shoulder dislocation

Personalized Medicine

45

slide-45
SLIDE 45

5 10 15 20 25 30 35 Limits on Shoulder Motion Avoid High-Risk Activities Duration of PT Chance of Recurrence Out-of-Pocket Cost

Relative Importance

Importance of Attributes in Shoulder Dislocation

N=374

Personalized Medicine

Streufert BD, Reed SD, Johnson FR, Huber JC, Orlando LA, Taylor DC, Mather III RC. Orthop J Sports Med. PMID 28377932

46

slide-46
SLIDE 46

Decision-analytic model

Personalized Medicine

Adaptive choice questions to generate patient-level preference weights Demographics and contextual variables

Patient Output:

47

Clinical literature

slide-47
SLIDE 47

Benefits of Personalized Medicine

Patients

  • More informed patients
  • Improved patient-provider

communication

  • Greater patient satisfaction
  • Improved adherence and

patient outcomes

Health System

  • Improved efficiency of health

care delivery

  • Documentation of patient-

centered care

  • Provide justification for

changes in practice patterns

slide-48
SLIDE 48

Opportunities and Challenges

  • Documenting patient concerns about new devices is useful for

regulatory reviews and at other points in product life cycles.

  • Stated-preference methods are relatively unfamiliar and there

is limited experience with health applications.

  • Methods are highly flexible and can be adapted for evaluating

almost any preference-sensitive decision.

  • FDA has provided guidance on developing patient-preference

evidence for devices.

49

slide-49
SLIDE 49

50

DISCUSSION…

Please submit your question via the chat box

slide-50
SLIDE 50

Next session

  • How do you define study objectives and scope?
  • What study-team skills are required?
  • How do you work effectively with the technical team?
  • What steps are required for conducting a regulatory-

quality study?

  • How long will it take and what will it cost?

51

Planning and implementing a choice-experiment study

slide-51
SLIDE 51

Contact Information

Shelby Reed

shelby.reed@duke.edu 919 668 8991

Reed Johnson

reed.johnson@duke.edu 919 668 1075

Juan Marcos Gonzalez

jm.gonzalez@duke.edu 919 668 5157

52

slide-52
SLIDE 52

53

Join us for the next 2 sessions

  • March 15 - Session 2: Example applications and lessons learned—

instrument development

  • April 19 - Session 3: Example applications and lessons learned—

analysis and reporting

  • Recordings will be available on http://mdic.org/mdicx