Tuning Epidemiological Study Design Methods for Exploratory Data - - PowerPoint PPT Presentation

Tuning Epidemiological Study Design Methods for Exploratory Data Analysis in Real World Data

Andrew Bate Senior Director, Epidemiology Group Lead, Analytics

15th Annual Meeting of the International Society of Pharmacovigilance, Prague, 30 October 2015

Disclosures and Acknowledgement

• I am a full time employee of Pfizer and hold stocks and

stock options

• The work presented here represent a component of a

multi-year internal to Pfizer methodology programme to test and better understand Active Surveillance approaches thanking the following for their contributions to the production of these results

– Xiaofeng Zhou, Kathy Liu, Sundaresan Murugesan, Rongjun Shen, Richard Gong and Bing Cai

• Desvenlafaxine (Pristiq) used for this study is a Pfizer

product

2

Overview

• The importance of assessing signal detection capability in

longitudinal observational data

• Current knowledge about signal capability in such data
• Read out the results of quantitative signal detection

testing on method testing focussing on

– Adapting and assessing standard epidemiological approaches for signal detection – ‘Test set’ of more recently marketed products – Focussed detailed analysis of pre-defined small set of drug-

• utcome pairs

3

Evaluate Product Risks

Approval

Active Surveillance Monitor and detect signals in defined patient cohorts using innovative analytic methods Risk Minimization Evaluate the effectiveness of risk minimization measures (e.g., product label/education)

Standing Cohorts Characterize Patient Risk Profile

EMRs Claims Registries

Increasingly Important to Harness the Power of Real World Evidence for Safety

Post Approval Safety Studies Compare medication risks in the real world, as prescribed and taken during routine clinical practice

4

Good (but Imperfect) Method Performance Against Well Established Safety Reference Set

5

Statistical Method UK EMR THIN Median across OMOP lab of 10 RWA databases

Lower 95% CI >1 Sensitivity Specificity Sensitivity Specificity

PRR 0.67 0.68 0.44 0.73 USCCS 0.78 0.59 0.44 0.68 HDPS 0.50 0.76 0.56 0.82

Comparison to OMOP reference set of 53 well studied drug outcome pairs of established drugs showed no single best method Generalizability to NMEs?

• USCCS and HDPS are two very different analytic approaches extensively used

by the epidemiological community– but with clearly different designs might be more effective for different drug-event pairs

– Evaluated extensively for observational studies

• PRR is a transparent approach and a key signal detection method for

spontaneous reports - ideal reference method

Approach to Gain Insights Into Signal Detection Utility in Longitudinal Observational Data Resources for More Recently Marketed Products

6

Active Surveillance Algorithms

Inform understanding

• f signal

detection method performance for recently marketed products

UK EMR US Claims

Drugs of Focus 5 Outcomes of Interest Databases

More understanding on utility of different algorithms for different kinds of

• utcomes

More clarity on how signal detection on such data can add value compared to signals generated from other data sources externally

Focused analysis

Reference Health Outcomes of Interest Used for Method Testing

• The study focused on ability of methods to detect

labelled outcomes

• Selected outcomes

– For Pristiq

• Hypertension, Orthostatic hypotension, Proteinuria, Hyperlipidemia,

All Fractures

– For Humira

• Acute Myocardial Infarction (AMI), GI Perforation, Herpes Zoster,

Interstitial Lung Disease, Lymphoma, Pneumonia

• A priori had varying levels of confidence in how well

suited the databases will be for signal detection of these

• utcomes with different methods

– Examining the variability - an important study component

7

Epidemiologic Design and Statistical Methods Implementation for Primary Analysis

8

PRR USCCS HDPS

Choice of comparator All ¡other ¡drugs ¡ N/A ¡ Comparator ¡for ¡Humira: ¡Other ¡ biologics, ¡Comparator ¡for ¡ Pris<q ¡: ¡ ¡An<depressant, ¡ specifically ¡ ¡Cymbalta ¡ ¡(THIN): ¡ used ¡Lexapro ¡as ¡study ¡drug ¡ Case definition 1st ¡occurrence ¡of ¡ the ¡outcome ¡of ¡ interest ¡ ¡ 1st ¡occurrence ¡

• f ¡the ¡outcome ¡
• f ¡interest ¡ ¡

1st ¡occurrence ¡of ¡the ¡outcome ¡

• f ¡interest ¡ ¡

¡ Risk period Exposure ¡dura<on ¡ plus ¡30 ¡days ¡from ¡ the ¡end ¡of ¡the ¡ exposure ¡ ¡ Exposure ¡ dura<on ¡plus ¡30 ¡ days ¡from ¡the ¡ end ¡of ¡the ¡ exposure ¡ ¡ Ever ¡aLer ¡first ¡prescrip<on ¡of ¡ study ¡drug ¡or ¡comparison ¡drug ¡

Drug: defined by diagnoses codes NDC codes in Optum and multilexID in THIN. Indication not considered explicitly, but implicit in choice of comparator HDPS: 5 dimensions (prescription, inpatient procedures, outpatient procedure, inpatient diagnoses, outpatient diagnoses) for Optum database; 3 dimensions (prescription, procedure and diagnoses) for THIN database. Data until end of 2010

Details of Statistical Method Testing for USCCS

• USCCS model considers one drug and one AE which is

assumed arising from a non-homogeneous Poisson process

• It automatically controls for time-fixed covariates
• It only includes cases in the analysis which are then

used as their own controls.

– The USCCS conditional likelihood – Key assumption: conditionally independent events and events conditionally independent of exposure

9

Event ¡X ¡ ¡ ¡ ¡Drug ¡A ¡ ¡ ¡A ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡X ¡

Details of Statistical Method Testing for HDPS

10

Event X

PS Estimation

Event X

PS Estimation

Drug A Drug B

• Create cohorts of patients who use drug A or drug B
• Estimate Propensity Score (PS) for a given exposure for each individual
• based on multiple variables that might be potential confounders (e.g. age,

gender, other therapy use)

• Risk Estimation then adjusted for confounding using Propensity Scores

– PS-based adjustment done with different methods including logistic regression

• r SMR weighted method

Results for Method Testing on Humira Data – Several, but Not All, Pairs Highlighted

11

Humira in Optum

AMI GI Perforation Herpes Zoster Interstitial Lung Disease Lymphoma Pneumonia

PRR

1.01 ¡(0.85) ¡ 1.33 ¡(1.25) ¡ 1.97 ¡(1.80) ¡ 0.59 ¡(0.57) ¡ 0.89 ¡(0.70) ¡ 0.85 ¡(0.79) ¡

SCCS

1.12(0.87) ¡ 0.78(0.71) ¡ 1.41(1.23) ¡ 0.69(0.66) ¡ 1.43(1.02) ¡ 1.15(1.04) ¡

HDPS

0.97(0.83) ¡ 1.22(1.14) ¡ 1.12(1.02) ¡ 0.99(0.95) ¡ 0.82(0.66) ¡ 1.00(0.94) ¡ Note: Results shown are point estimate with lower bound of 95% confidence interval (LBCI) in the parentheses. Highlighted in red are those with LBCI greater than one.

Results for Method Testing on Antidepressant Data – Few Pairs Highlighted

12

Pristiq in Optum Lexapro in THIN

Fracture

Hyperlipidemia Hypertension

Orthostatic Hypotension Proteinuria Fracture

Hyperlipidemia Hypertension

Orthostatic Hypotension Proteinuria

PRR

0.50(0.47) 0.37(0.36) 0.37(0.35) 0.81(0.72) 0.46(0.40) 0.21(0.20) 0.31(0.30) 0.19(0.18) 0.49(0.42) 0.30(0.25)

USCCS

0.96(0.89) 0.40(0.39) 0.40(0.37) 0.97(0.83) 0.98(0.80) 1.22(1.14) 1.03(0.94) 0.60(0.56) 1.51(1.22) 1.32(1.06)

HDPS

0.70(0.67) 0.66(0.62) 0.62(0.61) 0.59(0.53) 0.63(0.54) 1.47(1.35) 1.87(1.67) 1.47(1.33) 1.37(1.11) 1.53(1.24)

Note: Point estimates shown with lower bound of 95% confidence interval (LBCI) in parentheses LBCI scores greater than 1 are highlighted in red

Secondary Analyses

• Additional analysis performed for SCCS and HDPS

– Varied risk period selections – Varied criteria of case selections (or inclusion criteria) – Examined variation across the two heterogeneous database by also conducting analysis of Lexapro in Optum data for direct comparison to Lexapro versus Cymbalta in THIN

• For HDPS

– Compared Propensity Score contribution by changing covariates in model, risk estimation method and also reviewed crude estimates – In depth consideration of approaches to PS matching and underlying PS distribution assessment (HDPS)

• For SCCS

– Examined capability of monitoring change in estimates over time

13

14

• All outcomes highlighted for Humira with

incident case

• Same trend across study pairs - fractures,

hypotension and proteinuria highlighted, hypertension and hyperlipideamia less so

• Very similar pattern for Lexapro in Optum

data (results not shown)

• By contrast varying time at risk period had

limited impact

Pristiq- Optum Lexapro

• THIN

Humira- Optum

Outcome Criterion change for SCCS Increases Estimates Consistently

‘New case’ is the subset of ‘all cases’ with at least 12 months enrollment on or before the 1st outcome

15

Pristiq- Optum Lexapro

• Optum

Lexapro

• THIN
• When varying adjustment approach, adjusting

for demographics and propensity score, performed best and regression approach highlighted more SDRs than SMR weighting

• New case after the first exposure clearly more

effective at highlighting SDRs (results not shown)

• Some indication that in Optum data that

restricted to outcomes within first 30 days of first exposure improved performance further, not observed in THIN data (result not shown)

HDPS Adjusted Odds Ratio for Antidepressants in Optum and THIN

16

HDPS Adjusted Odds Ratio for Humira in Optum

While adjustment for demographics led to higher scores than unadjusted estimates, when comparing Humira to all bDMARDs the benefit of SMR or regression based HDPS was less clear Possibly due to heterogeneity across a drug class as comparator (general lack of research on drug class as comparator when using HDPS)

17

• Distribution of propensity score of

the group exposed to Lexapro in the Optum database.

• Distribution of propensity score of

group unexposed to Lexapro (comparator group) in the Optum database.

• Some differences observed in Odds Ratio estimates with HDPS when varying

approach for using PS distributions

• Specifically logistic regression compared to SMR weighting method

Ø Such differences can be indicative of non-balanced PS distributions

Propensity Score Distributions Similar – But Not Identical

• Herpes zoster reported in clinical trials for Humira
• Analyses of real world data identified herpes zoster as an issue post-marketing

requiring further investigation

US Claims Data Also Potentially Useful for Identifying Signals Earlier: Optum Data

Results of Self Controlled Case Series (SCCS)

18

5 7 13 17 23 28 33 39 42 48 53 60 64 76 86 95 105 116 139 159 172 189 213 239 264 295 312 338 351 378 397

1 2 3 4 5 6 7

2003Q2 2003Q3 2003Q4 2004Q1 2004Q2 2004Q3 2004Q4 2005Q1 2005Q2 2005Q3 2005Q4 2006Q1 2006Q2 2006Q3 2006Q4 2007Q1 2007Q2 2007Q3 2007Q4 2008Q1 2008Q2 2008Q3 2008Q4 2009Q1 2009Q2 2009Q3 2009Q4 2010Q1 2010Q2 2010Q3 2010Q4

Relative Incidence with 95% LBCI

LBCI=Lower Bound Confidence Interval

Generally observed flat trends over time for most drug- adverse event pairs

Limitations

• In US Insurance claims and UK EMRs, solely defining outcome

by diagnosis code (ICD9 or Read Code) is not always accurate

• USCCS: Method can be volatile to selection of risk period

– Necessarily risk window selection and confounder adjustment approaches will be suboptimal in a signal detection framework, as generic parameter selection needed that are effective for large numbers of drug-outcome pairs (e.g. acute v chronic outcomes)

• HDPS: comparator group selection is challenging: methods

highlight SDRs if difference between drug of interest and comparison drug/group.

19

References

• Bate A, Evans SJW. Quantitative signal detection using spontaneous ADR reporting.

Pharmacoepidemiology and Drug Safety 2009 18(6): pp 427-436

• Cai B, Liu Q, Geier J, Bate A. An Algorithm To Predict Biologic DMARD Use in the THIN Database.

Pharmacoepidemiology and Drug Safety 22: S391

• Cai B, Murugesan S, Geier J, Bate A. 2014 Applying High Dimensional Propensity Score (HDPS) in a

Exploratory Data Analysis with a US Claims Database for Recent Medicinal Products. Pharmacoepidemiology and Drug Safety 23: S13

• Schneeweiss S, Rassen JA, Glynn RJ, Avorn J, Mogun H, & Brookhart MA. (2009). High-dimensional

propensity score adjustment in studies of treatment effects using health care claims data. Epidemiology 20(4), 512.

• Whitaker HJ, Hocine MN, & Farrington CP. (2008). The methodology of self-controlled case series
• studies. Statistical methods in medical research. 18(1) 7-26.
• Zhou X, Cai B, Murugesan S, et al. An Application of Univariate Self Case Control Series Method

Using THIN Database in the OMOP CDM for Active Drug Safety Surveillance. Pharmacoepidemiology and Drug Safety 21: S283

• Zhou X, Murugesan S, Bhullar H, Liu Q, Cai B, Wentworth C, Bate A. 2013 An Evaluation of the THIN

Database in OMOP Common Data Model for Active Drug Safety Surveillance Drug Safety. 36(2) pp: 119-134

• Zhou X, Shen R, Geier J, Bate A. 2014 Adapting and Evaluating Self-Controlled Case Series Method

(SCCS) for Signal Screening of a Recently Marketed Drug Using a US Claims Database Pharmacoepidemiology and Drug Safety 23: S181

20

Discussion

• For well established products and outcomes previous research shows

that signals can be detected, but with imperfect performance

• Our work suggests a similar position for more recently

marketed products and the potential value for signal detection in observational databases

– No single approach, nor set of design choices, performed uniformly better – Volatility and some unpredictable variation in observed scores emphasized challenges for routine signal detection in such data

• Use of a range of approaches is therefore likely to be important in

signal detection (e.g. for acute and chronic outcomes)

– Even more so than spontaneous reports, because of the complexity and heterogeneity of underlying data, and longitudinal nature of data sets

21