Drug benefit-risk assessment using multi-criteria decision analysis - - PowerPoint PPT Presentation

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Drug benefit-risk assessment using multi-criteria decision analysis

Douwe Postmus1, Gert van Valkenhoef, Hans Hillege

Department of Epidemiology, University Medical Center Groningen, The Netherlands

1Corresponding author. Email: d.postmus@umcg.nl

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Pharmaceutical decision making

Pharmaceutical decision making

Based on assessing benefits and risks of two or more drugs Ideally by considering all available clinical evidence

Given outcome of clinical trials, should a new anti-depressant be allowed on the market? Which anti-depressant is most suited for severely depressed patients?

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Challenges in drug benefit-risk assessment

Dealing with multiple comparisons and trade-offs Measurement of benefit is closely defined whereas risk is generic

Decrease in body weight of 5kg versus 10% increase in the incidence of psychiatric disorders

Balancing short and long term effects Changing from probability statements about the data given the truth (Frequentist) to probability statements about the truth given the data (Bayesian)

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Ad-hoc versus rational decision making

Source: Baltussen et al., Cost Effectiveness and Resource Allocation 2006, 4:14

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Advantages of the use of MCDA

It helps to structure the problem It makes the need for subjective judgments explicit and the process by which they are taken into account transparent It provides a focus and language for discussion, leading to better considered, justifiable, and explainable decisions The analysis serves to complement and challenge intuition; it does not seek to replace intuitive judgment or experience

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Learning objectives

To know the different phases in a multi-criteria analysis To be able to identify suitable objectives and to know how these objectives can be organized hierarchically into a value tree To be able to summarize the clinical input data in a format suitable for a multi-criteria analysis To have a basic understanding of multi-attribute value theory (MAVT) and how it can be used to assess the decision maker’s preference structure To know how the preference modeling part of the MCDA process is supported by the ADDIS software

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

The process of MCDA: PrOACT-URL framework

Source: EMA benefit-risk methodology project

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Case study: benefit-risk assessment of rimonabant in

• verweight or obese patients with type 2 diabetes

Rimonabant, a selective cannabinoid type 1 receptor blocker, has shown to reduce body weight and improve cardiovascular and metabolic risk factors in non-diabetic overweight or obese patients Can rimonabant, in combination with diet and exercise, produce a clinically meaningful reduction in bodyweight, blood glucose levels, and cardiovascular risk factors in

• verweight or obese patients with type 2 diabetes?

Do these favorable effects outweigh the side effects, such as depressed mood disorders, nausea, and dizziness?

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Alternatives

Placebo Rimonabant 5 mg/day Rimonabant 20 mg/day ...

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Identification of objectives

The use of MCDA calls for the identification of criteria against which the decision alternatives are to be evaluated These criteria are usually organized hierarchically into a value tree with higher-level constructs at the top of the tree and comprehensive and measurable attributes at the bottom In specifying the value tree, a balance must be found between completeness and conciseness The model should be usable with reasonable effort There should not be two or more criteria measuring the same concept (non-redundancy)

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Example value tree

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Assignments - 1

The following criteria emerged from the problem structuring process for the rimonabant case study

1

Change from baseline in body weight

2

Change from baseline in waist circumference

3

Change from baseline in fasting glucose

4

Change from baseline in HbA1c

5

Anxiety

6

Depressed mood disorders

7

Hypoglycaemia

8

Discontinuations due to adverse events

Organize the above criteria hierarchically into a value tree Is the resulting value tree suitable for the purpose of a multi-criteria analysis?

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Value tree for the rimonabant case study

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Clinical evidence

Drug benefit-risk assessment is generally based on data collected from randomized controlled trials (phase II and phase III studies) This includes outcome measures such as

Incidences (i.e., the fraction of the sample that develops a certain condition over a given period of time) Changes in the levels of a continuous response variable (e.g., blood pressure) Discontinuation rates

How to organize and present this data for the purpose of drug benefit-risk assessment?

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Effects table

Unit of measurement Placebo Rimonabant Rimonabant 5 mg/day 20 mg/day Change from baseline % of body weight

• 1.5

in body weight at baseline Change from baseline mmol/L 0.33 in fasting glucose Hypoglycaemia % of patients 2 Anxiety % of patients 3 Discontinuations due to % of patients 5 adverse events

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Assignments - 2

Complete the previously introduced effects table by using the data from the Lancet publication

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Effects table

Unit of measurement Placebo Rimonabant Rimonabant 5 mg/day 20 mg/day Change from baseline % of body weight

• 1.5
• 2.4
• 5.5

in body weight at baseline Change from baseline mmol/L 0.33 0.30

• 0.64

in fasting glucose Hypoglycaemia % of patients 2 1 5 Anxiety % of patients 3 1 5 Discontinuations due to % of patients 5 8 15 adverse events

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Role playing game

Consensus meeting to approve or reject market authorization of rimonabant as a complementary therapy in the treatment of type 2 diabetes

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

A simple choice between two drugs

Drug A Drug B Weight loss (% of body weight) 7% 4% Anxiety 10% 3% To model the decision maker’s preference structure for the above problem, consider the following value trade-off: Starting at a value of 4%, how large should the increase in weight loss be to just compensate for an increase in the incidence of anxiety from 3% to 10%?

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Indifference curves

Suppose that an increase in weight loss from 4% to x% is just sufficient to compensate for an increase in anxiety from 3% to 10% We then say that the decision maker is indifferent between the

• utcomes (4% WL, 3% Anx) and (x% WL, 10% Anx)

The line connecting all outcomes that are indifferent to (4% WL, 3% Anx) is called an indifference curve

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Indifference curves

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Dug A ≻ drug B

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Problem statement

Let A be the set of decision alternatives from which the decision maker has to make a simple choice or which the decision maker has to rank from best to worse Associated with each a ∈ A is a vector of criteria measurements (xa

1, . . . , xa n), where xa k denotes the performance

• f alternative a on criterion k

The objective in MAVT is to construct a value function v : Rn → R, such that for any two points x and y in the evaluation space X ⊆ Rn v(x) = v(y) ⇔ x ∼ y v(x) > v(y) ⇔ x ≻ y

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

The additive value function

A value function exists when

1

any two points x and y in X are comparable: x ∼ y, x ≻ y, or y ≻ x

2

the preference relation is transitive: x y and y z ⇒ x z

If, in addition, value trade-offs between any two criteria do not depend on the levels of the other criteria (preferential independence assumption), the decision makers preference structure can be represented by the additive function v(x, w) = w1v1(x1) + · · · + wnvn(xn)

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Assessment of the partial value functions

The partial value functions are constructed in such a way that equal increments in vk represent the same increase in value to the decision maker

The reduction in weight from 0% to 5% results in the same value increment as the reduction in weight from 5% to 20% This implies that if the decision maker is willing to pay x to increase weight loss from 0% to 5%, he or she should also be willing to pay x to increase weight loss from 5% to 20%

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Assessment of the weights

The weights indicate how much more important the swing from worst to best on one criterion is compared to the swing from worst to best on the other criteria An ordinal ranking of the weights can be obtained by rank

• rdering the swings from worst to best on all criteria

wk > wl implies that if the decision maker has to choose between improving either criterion k or criterion l from the worst to the best value, he or she would improve criterion k

Given an ordinal ranking of the weights, different techniques are available to assign exact values to them

As a percentage of the increase in overall value resulting from the swing on the most highly ranked criterion By manipulating the performance on a higher ranked criterion until indifference is reached between the manipulated scale swing and the swing from worst to best on a lower ranked criterion

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Demonstration ADDIS software (www.drugis.org)

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Uncertainty and imprecision

We might be able to say that the top speed of a car is between 200 and 250 km/h, but we are uncertain about the real value A decision maker may only be able to provide imprecise preference statements

Ordinal ranking of the weights Classification of differences into importance classes (e.g., weak, strong, or extreme)

How do uncertainty and imprecision affect our results?

Simple, one-way sensitivity analysis can be conducted to assess uncertainty/imprecision in a single parameter What if all parameters are uncertain or imprecise?

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Stochastic multi-criteria acceptability analysis (SMAA)

In SMAA, uncertainty in the criteria measurements and imprecision in the preferences are represented by means of probability distributions Probability distributions for the criteria measurements can be

• btained by fitting Bayesian statistical models to the data

Beta distribution for incidences Normal distribution for (log-transformed) continuous measurements

Weights are assumed to be uniformly distributed within the convex polytope defined through a set of linear constraints The partial value functions are generally assumed to be linear, but this assumption can be relaxed if considered inappropriate

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Uncertainty in the criteria measurements

uPoI distributions

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Imprecision in the weights

Total lack of preference information is represented by a uniform distribution over the weight space If some preference information is available, the weight space can be restricted with linear constraints

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

SMAA descriptive indices

Rank acceptability index share of weights and measurements making an alternative have ranks 1, . . . , m (most preferred, second most, etc.) Central weight vector center of gravity of the favourable weight space: “Which preferences support an alternative to be the most preferred one?” Confidence factor probability for an alternative to be preferred when preferences equal its central weight vector: “Are the measurements sufficiently precise?”

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Assignments - 3

Go to http://mcda.clinici.co and select the rimonabant case study Press next to set the scale ranges equal to its default values Press next to assume that all partial value functions are linear What are the probabilities that placebo, rimonabant 5 mg/day, and rimonabant 20 mg/day are ranked first when preference information on the weights is not available? How do these probabilities change when the decision maker ranks the criteria scale swings in the following order: change in weight ≻ change in fasting glucose ≻ discontinuations ≻ anxiety ≻ hypoglyceamia How do these probabilities change when the ranking in the criteria scale swings changes to: discontinuations ≻ anxiety ≻ hypoglyceamia ≻ change in weight ≻ change in fasting glucose?

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Role playing game cont’d

Use the ADDIS web service (http://mcda.clinici.co; select the rimonabant case study, and press next to set the scale ranges equal to its default value) to construct a value function that represents your preference structure for the rimonabant case study Which alternative is optimal if the resulting value function correctly reflects your preference structure? How does this relate to the choice you made previously without the use of a formal decision support method?

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Summary

MCDA provides a framework for the systematic and transparent analysis of complex decision problems involving value trade-offs There are three types of uncertainty involved with using MCDA

Uncertainty in the model structure (i.e. number of alternatives and criteria, level of detail) Uncertainty in the preference statements (i.e. shape of the partial value functions, criteria weights) Uncertainty in the criteria measurements

At www.drugis.org we provide a flexible set of tools to address all these aspects

Introduction Problem structuring Consequences Trade-offs Uncertainty Summary

Thank you

Questions?