Benefit and risk considerations in medical decision making Douwe - - PowerPoint PPT Presentation

Introduction Graphical method SMAA for BR Extensions Summary

Benefit and risk considerations in medical decision making

Douwe Postmus1, Gert van Valkenhoef, Hans Hillege

Department of Epidemiology, University Medical Center Groningen, The Netherlands

1Corresponding author. Email: d.postmus@umcg.nl

Introduction Graphical method SMAA for BR Extensions Summary

Medical decision making

Health policy decision making

Given the evidence produced by phase II and phase III studies, should a new anti-depressant be allowed on the market? Which of the available anti-depressants should be eligible for reimbursement?

Clinical decision making

Which anti-depressant should be prescribed to a patient presenting with severe signs and symptoms?

Introduction Graphical method SMAA for BR Extensions Summary

Challenges in drug benefit-risk assessment

Dealing with multiple comparisons and trade-offs Measurement of benefit is closely defined whereas risk is generic

Decrease in body weight of 5kg versus 10% increase in the incidence of psychiatric disorders

Balancing short and long term effects Changing from probability statements about the data given the truth (Frequentist) to probability statements about the truth given the data (Bayesian)

Introduction Graphical method SMAA for BR Extensions Summary

Ad-hoc versus rational decision making

Source: Baltussen et al., Cost Effectiveness and Resource Allocation 2006, 4:14

Introduction Graphical method SMAA for BR Extensions Summary

Advantages of the use of MCDA

It helps to structure the problem It makes the need for subjective judgments explicit and the process by which they are taken into account transparent It provides a focus and language for discussion, leading to better considered, justifiable, and explainable decisions The analysis serves to complement and challenge intuition; it does not seek to replace intuitive judgment or experience

Introduction Graphical method SMAA for BR Extensions Summary

How to balance model complexity and usability?

Introduction Graphical method SMAA for BR Extensions Summary

A simple graphical method

The ‘Lynd & O’Brien’ model: Probabilistic simulation method Compares 2 alternatives On 2 criteria (benefit vs. risk) Sample (∆B, ∆R) values from a joint distribution Plot them on a plane Count how many points are below the threshold µ

Lynd LD and O’Brien BJ, Advances in risk-benefit evaluation using probabilistic simulation methods: an application to the prophylaxis of deep vein thrombosis, Journal of Clinical Epidemiology 57 (2004) 795–803.

Introduction Graphical method SMAA for BR Extensions Summary

Benefit-risk plane

+Benefit A +Benefit B +Risk A +Risk B

Trade-off Trade-off µ B better A better p =

a a+b

count b count a

Introduction Graphical method SMAA for BR Extensions Summary

Using the graphical method to assess the benefit-risk profile of two second-generation anti-depressants (ADDIS)

Introduction Graphical method SMAA for BR Extensions Summary

Using the graphical method to assess the benefit-risk profile of two second-generation anti-depressants (ADDIS)

Introduction Graphical method SMAA for BR Extensions Summary

Using the graphical method to assess the benefit-risk profile of two second-generation anti-depressants (ADDIS)

Introduction Graphical method SMAA for BR Extensions Summary

Using the graphical method to assess the benefit-risk profile of two second-generation anti-depressants (ADDIS)

0.0 0.2 0.4 0.6 0.8 1.0 2 4 6 8 HAM−D Responders Probability density Sertraline Fluoxetine 0.0 0.2 0.4 0.6 0.8 1.0 2 4 6 8 10 Dropouts Probability density Sertraline Fluoxetine

Introduction Graphical method SMAA for BR Extensions Summary

Using the graphical method to assess the benefit-risk profile of two second-generation anti-depressants (ADDIS)

Introduction Graphical method SMAA for BR Extensions Summary

Using the graphical method to assess the benefit-risk profile of two second-generation anti-depressants (ADDIS)

Introduction Graphical method SMAA for BR Extensions Summary

Limitations of the graphical method

The method by Lynd & O’Brien applies to two drugs that are evaluated on two criteria In most cases, more than two criteria need to be considered

Multiple safety criteria Various measures of therapeutic effect Costs

How can the multi-criteria assessment be extended to the general m × n problem without losing the possibility to consider

Uncertainty in the criteria measurements Imprecision in the decision maker’s preferences

Introduction Graphical method SMAA for BR Extensions Summary

Stochastic multi-criteria acceptability Analysis (SMAA)

SMAA is an MCDA method for ranking a set of m alternatives that are evaluated on a set of n criteria It is assumed that the decision makers’ preference structure can be represented by the additive value function V (a) =

n

• i=1

wivi(a) The partial value functions reflect the decision makers’ preferences for different levels of achievement on the individual criteria The weights indicate how much more important the swing from worst to best on one criterion is compared to the swing from worst to best on the other criteria

Introduction Graphical method SMAA for BR Extensions Summary

Uncertainty in the criteria measurements

uPoI distributions

Introduction Graphical method SMAA for BR Extensions Summary

Imprecision in the weights

Total lack of preference information is represented by a uniform distribution over the weight space If some preference information is available, the weight space can be restricted with linear constraints

Introduction Graphical method SMAA for BR Extensions Summary

SMAA descriptive indices

Rank acceptability index share of weights and measurements making an alternative have ranks 1, . . . , m (most preferred, second most, etc.) Central weight vector center of gravity of the favourable weight space: “Which preferences support an alternative to be the most preferred one?” Confidence factor probability for an alternative to be preferred when preferences equal its central weight vector: “Are the measurements sufficiently precise?”

Introduction Graphical method SMAA for BR Extensions Summary

Case study: second-generation anti-depressants

Placebo-controlled randomized clinical trial:

Fluoxetine Venlafaxine Placebo

Criteria (selected by expert):

Benefit: efficacy (treatment response) Risks: nausea, insomnia, anxiety

Tervonen T, Van Valkenhoef G, Buskens E, Hillege HL, Postmus D, A stochastic multicriteria model for evidence-based decision making in drug benefit-risk analysis, Statistics in Medicine 30 (2011) 1419–1428.

Introduction Graphical method SMAA for BR Extensions Summary

Criteria measurements

0.0 0.2 0.4 0.6 0.8 1.0 2 4 6 8 Efficacy Probability density Venlafaxine Fluoxetine Placebo 0.0 0.2 0.4 0.6 0.8 1.0 5 10 15 Nausea Probability density Venlafaxine Fluoxetine Placebo 0.0 0.2 0.4 0.6 0.8 1.0 2 4 6 8 10 12 Insomnia Probability density Venlafaxine Fluoxetine Placebo 0.0 0.2 0.4 0.6 0.8 1.0 10 20 30 40 Anxiety Probability density Venlafaxine Fluoxetine Placebo

Introduction Graphical method SMAA for BR Extensions Summary

Preference-free analysis

Figure: Rank acceptability indices

Introduction Graphical method SMAA for BR Extensions Summary

Ordinal ranking of the weights for mild depression

1 Nausea 2 Anxiety 3 Efficacy 4 Insomnia

Figure: Rank acceptability indices

Introduction Graphical method SMAA for BR Extensions Summary

Ordinal ranking of the weights for severe depression

1 Efficacy 2 Nausea 3 Anxiety 4 Insomnia

Figure: Rank acceptability indices

Introduction Graphical method SMAA for BR Extensions Summary

Discussion

The results of the preference-free analysis showed that there are clear trade-offs among the three drugs However, depending on the scenario considered, it was still difficult to make an informed decision

High uncertainty in the criteria measurements due to a relatively small sample size An ordinal ranking of the weights resulted in insufficient discrimination for the severe depression scenario and possibly misleading results for the mild depression scenario

Introduction Graphical method SMAA for BR Extensions Summary

Including evidence from multiple studies

Drug benefit-risk analysis is ideally based on evidence synthesized from multiple trials or possibly a complex network of trials

Figure: Evidence network (25 studies in total)

Introduction Graphical method SMAA for BR Extensions Summary

Pair-wise meta-analyses are ill suited

Relative effects have to be assessed against a common comparator, and not all evidence structures have a single treatment against which all others are compared

Selection bias: arbitrary exclusion of evidence Sensitivity analysis with different comparators

When a large number of treatments are available, most evidence will be indirect regardless of the chosen common comparator Solution: to apply mixed treatment comparison (MTC) for evidence synthesis in SMAA-based drug benefit-risk analysis

Van Valkenhoef G, Tervonen T, Zhao J, De Brock B, Hillege HL, Postmus D, Multicriteria benefit-risk assessment using network meta-analysis, Journal of Clinical Epidemiology 65 (2012) 394–403.

Introduction Graphical method SMAA for BR Extensions Summary

Measurement scales

Meta-analysis results in relative measurements

E.g. odds ratio, mean difference Statistically more robust Hard to interpret clinically

For decision making, we need absolute measurements

E.g. risk, change from baseline Choose a baseline treatment and estimate absolute effect Sample effects of other treatments conditional on that

Problem: how to estimate absolute effect?

No general answer, lots of options

Introduction Graphical method SMAA for BR Extensions Summary

MTC/SMAA for drug benefit-risk analysis

choose criteria run incons. model investigate run cons. model construct SMAA model k := 1 [cons.] [incons.] [inconsistency explained] [unexplained] estimate baseline identify or perform systematic review [all criteria done (k = n)] select criterion k k := k + 1 [k < n]

Introduction Graphical method SMAA for BR Extensions Summary

Incorporating more precise preference information

w[p] w[d] w[b]

• Tervonen T, Van Valkenhoef G, Basturk N, Postmus D, Hit-And-Run enables efficient weight generation for

simulation-based multiple criteria decision analysis, European Journal of Operational Research 224 (2013) 552-559.

Introduction Graphical method SMAA for BR Extensions Summary

Incorporating more precise preference information

w[p] w[d] w[b]

• Tervonen T, Van Valkenhoef G, Basturk N, Postmus D, Hit-And-Run enables efficient weight generation for

simulation-based multiple criteria decision analysis, European Journal of Operational Research 224 (2013) 552-559.

Introduction Graphical method SMAA for BR Extensions Summary

Drug benefit-risk assessment: current and future challenges

There has been an increasing interest in MCDA for drug benefit-risk analysis, but developing models that are both theoretically sound and clinically useful has proven to be far from straightforward The ultimate aim will be to arrive at methodologies that allow decision makers to simultaneously explore

Uncertainty in the model structure (i.e. number of alternatives and criteria, level of detail) Uncertainty in the preference statements (i.e. shape of the partial value functions, criteria weights) Uncertainty in the criteria measurements

We have started to develop a flexible set of tools to address all these aspects (www.drugis.org)

Introduction Graphical method SMAA for BR Extensions Summary

Thank you

Questions?