Model-Based Meta-Analysis with NONMEM Jae Eun Ahn, Ph.D. College - - PowerPoint PPT Presentation

▶
model based meta analysis
SMART_READER_LITE
LIVE PREVIEW

Model-Based Meta-Analysis with NONMEM Jae Eun Ahn, Ph.D. College - - PowerPoint PPT Presentation

Oct 13, 2022 328 likes •657 views

Longitudinal Aggregate Data Model-Based Meta-Analysis with NONMEM Jae Eun Ahn, Ph.D. College of Pharmacy, Kangwon National University 7 Sep 2012 WCoP, Seoul, Korea Acknowledgments Department of Pharmacometrics, Pfizer, Inc. Jonathan

slide-1
SLIDE 1

Longitudinal Aggregate Data Model-Based Meta-Analysis with NONMEM

Jae Eun Ahn, Ph.D. College of Pharmacy, Kangwon National University 7 Sep 2012 WCoP, Seoul, Korea

slide-2
SLIDE 2

Acknowledgments

  • Department of Pharmacometrics, Pfizer, Inc.
  • Jonathan French, Sc.D., Metrum Research Group

Ahn and French, Longitudinal aggregate data model-based meta-analysis with NONMEM: approaches to handling within treatment arm correlation, JPKPD (2010) 37: 147-201

2

slide-3
SLIDE 3

Negative study results from PoC study in acute schizophrenia

3

Better

PANSS Total (Change From Baseline) Ahn et al., ASCPT 2012 Visit (week) PANSS: Positive & Negative Syndrome Scale

slide-4
SLIDE 4

How is the placebo response in the literature?

Mean Change from BS in PANSS Total Score at 6 Weeks for Acute Subjects

RE Model for All Studies

  • 85
  • 75
  • 65
  • 55
  • 45
  • 35
  • 25
  • 15
  • 5

5 15 25 PANSS Total Change from BS at Week 6 Sunovion Study D1050006-39,2010 Sunovion Study D1050049-41,2010

  • C. M. Canuso-3,2010

Sunovion Study D1050229-43,2010 Sunovion Study D1050196-42,2010 Sunovion Study D1050231-45,2010 Vanda Study ILPB202-50,2009 Canuso CM-4,2009 Organon Study 041022-31,2009 Organon Study 041023-33,2009 Organon Study 041004-29,2009 Vanda Study ILP3000ST-48,2009 Organon Study 041021-30,2009 Potkin SG study 2-4,2008 Potkin SG study 3-5,2008 Marder SR-3,2007 McEvoy et al.,2007 Cutler et al.,2006 NDA 20-825 study 1,2001

  • 6.2
  • 12.3
  • 10.8
  • 14.7
  • 5.5
  • 15.2
  • 18.4
  • 25.5
  • 10.1
  • 10.8
  • 5.3
  • 4.1
  • 11.1
  • 3.5
  • 7.6
  • 7.6
  • 2.4
  • 5.3
  • 0.4

2.7 2.3 1.9 1.6 2.2 1.7 3.3 2.2 1.7 1.6 2.3 2.1 1.6 1.6 1.5 2.4 1.9 2.1 2.1 50 72 95 127 90 116 35 80 93 123 59 127 100 156 160 110 108 88 83

  • 6.20 [ -11.57 , -0.83 ]
  • 12.30 [ -16.85 , -7.75 ]
  • 10.80 [ -14.52 , -7.08 ]
  • 14.70 [ -17.84 , -11.56 ]
  • 5.50 [ -9.75 , -1.25 ]
  • 15.20 [ -18.53 , -11.87 ]
  • 18.40 [ -24.84 , -11.96 ]
  • 25.50 [ -29.76 , -21.24 ]
  • 10.10 [ -13.43 , -6.77 ]
  • 10.80 [ -13.94 , -7.66 ]
  • 5.30 [ -9.81 , -0.79 ]
  • 4.10 [ -8.29 , 0.09 ]
  • 11.10 [ -14.24 , -7.96 ]
  • 3.50 [ -6.55 , -0.45 ]
  • 7.60 [ -10.61 , -4.59 ]
  • 7.59 [ -12.20 , -2.98 ]
  • 2.38 [ -6.04 , 1.28 ]
  • 5.30 [ -9.36 , -1.24 ]
  • 0.40 [ -4.58 , 3.78 ]
  • 9.25 [ -11.96 , -6.55 ]

Delta PANSS Total SE N Study Author, Year Delta PANSS Total + 95%CI

PANSS Total change from baseline at week 6 in placebo group

Courtesy: Sima Ahadieh and Vikas Kumar, Pfizer, Inc.

slide-5
SLIDE 5

Comparison of the Placebo Response

1 2 3 4

  • 30
  • 20
  • 10

10

Placebo

Time (Weeks) Change from Baseline in PANSS Total Model Simulation (90% PI) with Overlaid Observed

Mean change from baseline PANSS Total in placebo group seems to be greater than model estimated mean from the literature meta-analysis but within 90% PI

Courtesy: Sima Ahadieh and Vikas Kumar Pfizer, Inc.

slide-6
SLIDE 6

Meta-Analysis

  • A quantitative review and synthesis of results from

related but independent studies (Normand, 1999) Meta-analysis estimate = weighted average

  • More precise inferences (pooling data)
  • Point of reference for in-licensing opportunities
  • Relationship b/w early (Phase 1, 2) and later (Phase 3)

endpoints

  • Quantitative predictions of the probability of a successful

phase 3 study

6

slide-7
SLIDE 7

Model-Based Meta-Analysis

  • Incorporates parametric model to describe dose-

response, time effects, etc.

7

Lalonde RL et al., Clin Pharmacol Ther 2007;82:21-32

Model-Based Drug Development

Drug & Disease Models Competitor Info. & Meta-Analysis Design & Trial Execution Models Data Analysis Model Quantitative Decision Criteria Trial Performance Metrics

slide-8
SLIDE 8

Benefits and Challenges

  • Address uncertainty and

heterogeneity of studies

  • Increase statistical power
  • Improve estimates of

treatment effect

  • Lead to new knowledge
  • Formulate new questions
  • Publication bias
  • Incomplete description of

trial design and methods

  • Combining aggregate

data (AD) and individual patient-level data (IPD)

  • Appropriately accounting

for the correlation between time points within each study

8

Lalonde RL et al., Clin Pharmacol Ther 2007;82:21-32

slide-9
SLIDE 9

Longitudinal Aggregate Data

9

Response variable = Mean outcome over time within an arm

  • Observations within a study

are correlated

– Because the patients come from a common population – Use the study as ID (the grouping factor for the first level random effects)

  • Mean observations over time

within a treatment arm are correlated

– Because they are based on the same set of patients

 Need to account for more than two-level random effects

slide-10
SLIDE 10

10

How to handle within-treatment arm correlation?

  • 1. IOV

Random Effects Population PK Analysis with IOV Meta-Analysis* 1st Level (ID) Individual (Inter-Individual Variability, IIV) 2nd Level Occasions (Inter-Occasion Variability, IOV) RUV Usually assumed to be independent

* Modified from Laporte-Simitsidis et al. Inter-Study Variability in Population Pharmacokinetic Meta-Analysis: When and Howe to Estimate It? J of Pharm Sci 89: 155-167 (2000)

Study (Inter-Study Variability, ISV) Treatment arm (Inter-Arm Variability, IAV) Can be correlated

slide-11
SLIDE 11

11

How to handle within-treatment arm correlation?

  • 2. L2
  • L1 (=ID): group together the data records of the same

realization of etas

  • L2: group together the data records of the same

realization of epsilons

– Within a L1 observation, the L2 random effects can be made to be correlated between L2 observations

  • A way to handle correlation of multivariate observations

within individual records

– PK and PD – Parent and metabolite – Replicates

  • S. L. Beal, L. B. Sheiner, and A. J. Boeckmann (Eds.). NONMEM Users Guides, ICON Development Solutions, Ellicott City, MD,

1989-2006

slide-12
SLIDE 12

What if ignoring the correlation?

  • Simulation Experiments

12

slide-13
SLIDE 13

Impact On

  • Estimating the drug effect
  • Predicting the outcome of interest in a

future study

– E.g., what is the probability that the observed mean difference from placebo is at least c2, for a given design? 

2 ik i

P Y Y c  

slide-14
SLIDE 14

Simulation Model and Study Design

Parameters Correlation Low Med High E0 30 EMAX 10 ED50 250 K0 0.1 OMGS (inter-study variability) 1 1 1 OMGP (inter-patient variability) 3 9 27 SIG (residual unexplained variability) 9 9 9 Correlation = OMGP/(OMGP+SIG) 0.25 0.5 0.75 TV (total variance = OMGS+(OMGP+SIG)/n) 1.12 1.18 1.36 100 mg Effect 2.86 500 mg Effect 6.67 14

w = 1/sqrt(n)

500 simulations per design; NONMEM VI (level 1.2)

w Dose ED Dose E time k E Y

p s MAX

           ) ( ) exp(

50

  

ID Phase Sample Size (per dose) Doses Time Points 1 2 50 0, 50, 100, 200, 400 0, 1, 2, 4 2 2 50 0, 100, 300, 500, 1000 0, 1, 2, 4 3 2 50 0, 100, 200, 300, 400 0, 2, 4, 8, 12 4 2 50 0, 300, 400, 500, 600 0, 2, 4, 8, 12 5 3 100 0, 200, 400 0, 4, 8, 12 6 3 100 0, 400, 600 0, 4, 8, 12 7 3 100 0, 200, 600 0, 4, 8, 12 8 3 250 0, 200, 400 0, 4, 8, 12 9 3 250 0, 400, 600 0, 4, 8, 12 10 3 250 0, 200, 600 0, 4, 8, 12

slide-15
SLIDE 15

15

Estimation Models

  • A1: Inter-study on E0, Inter-arm on E0

(=simulation model)

  • A2: Inter-study on E0 with correlated residual

errors (L2)

  • A3: Inter-study on with uncorrelated residual

errors (ID=Study)

  • A4: Inter-arm on E0 with uncorrelated residual

errors(ID=Arm)

slide-16
SLIDE 16

Results - Parameter Estimates

Hardly any bias in the fixed effect parameter estimates

(mean estimation error: -1 ~ 7%)

When the within arm correlation was ignored (A3), residual variability was inflated When the study effects were ignored (A4), inter- patient variability is under- estimated (close to the true first level random effects)

16

  • 100

50 100 150 200 1 2 3 4 ISV

  • 100

50 100 150 1 2 3 4 IAV 100 200 300 400 500 1 2 3 4 RUV

  • 50

50 100 150 1 2 3 4 TOTVAR

  • 100

50 100 150 200 1 2 3 4 ISV

  • 100

50 100 150 1 2 3 4 IAV 100 200 300 400 500 1 2 3 4 RUV

  • 50

50 100 150 1 2 3 4 TOTVAR

  • 100

50 100 150 200 1 2 3 4 ISV

  • 100

50 100 150 1 2 3 4 IAV 100 200 300 400 500 1 2 3 4 RUV

  • 50

50 100 150 1 2 3 4 TOTVAR

Inter-Study Inter-Arm Residual Total Var

L

  • w

M e d H i g h

A1 A2 A3 A4

slide-17
SLIDE 17

Standard Error*: 95% CI for High Dose Effects

Width of 95% CI ↑ for A1-A3 as correlation ↑

Information from longitudinal measurements ↓ , as correlation ↑

Not A4

17

Low

simulation # A1 100 200 300 400 500 2.5 3.0 3.5 4.0 simulation # A3 100 200 300 400 500 2.5 3.0 3.5 4.0 simulation # A4 100 200 300 400 500 2.5 3.0 3.5 4.0

Medium

simulation # 100 200 300 400 500 2.5 3.0 3.5 4.0 simulation # 100 200 300 400 500 2.5 3.0 3.5 4.0 simulation # 100 200 300 400 500 2.5 3.0 3.5 4.0

High

simulation # 100 200 300 400 500 2.5 3.0 3.5 4.0 simulation # 100 200 300 400 500 2.5 3.0 3.5 4.0 simulation # 100 200 300 400 500 2.5 3.0 3.5 4.0

Low Medium High

A1 A3 A4

*Delta method to approximate the SE using variances and covariance

  • f EMAX and ED50

White line = True value

slide-18
SLIDE 18

Distribution:

18

0.0 0.1 0.2 0.3 20 40 60 80 120 A1, 100 mg 0.6 0.7 0.8 0.9 1.0 20 60 100 140 A1, 500 mg 0.0 0.1 0.2 0.3 20 40 60 80 120

Medium Correlation; Delta=2.5

A3, 100 mg 0.6 0.7 0.8 0.9 1.0 20 60 100 140 A3, 500 mg 0.0 0.1 0.2 0.3 20 40 60 80 120 A4, 100 mg 0.6 0.7 0.8 0.9 1.0 20 60 100 140 A4, 500 mg

Gray line = True probability

 

2 50 2 2

1 1 1

EMAX ik ED ik P i ik

D c D n n                       

A1 A3 A4

Low Dose High Dose

A1 A3 A4

Δ) Y Y P(

plcb i, drug i,

 

slide-19
SLIDE 19
  • Not distinguish the treatment arms in one study

from the other

  • Arm-to-arm variability is overestimated

– Wider distribution of the observed difference

  • Tends to overestimate the probability for low dose,

and underestimate for high dose

Potential Problem with A4 (Ignoring study effects)

19

low dose (100 mg)

delta P 5 10 0.0 0.2 0.4

high dose (500 mg)

delta P 5 10 0.0 0.2 0.4

slide-20
SLIDE 20

VPC of the observed difference for each method*

20

B1

Dose (mg) Delta (treatment-placebo) 200 400 600 800 1000

  • 6
  • 5
  • 4
  • 3
  • 2
  • 1
  • bs delta

sim median sim quartile

B2

Dose (mg) Delta (treatment-placebo) 200 400 600 800 1000

  • 6
  • 5
  • 4
  • 3
  • 2
  • 1

B3

Dose (mg) Delta (treatment-placebo) 200 400 600 800 1000

  • 6
  • 5
  • 4
  • 3
  • 2
  • 1

B4

Dose (mg) Delta (treatment-placebo) 200 400 600 800 1000

  • 6
  • 5
  • 4
  • 3
  • 2
  • 1

25th-75th

*Simulation and estimation strategies (Part B in Ahn and French, JPKPD 2010) are different from the previous example (Part A) but presented to make a point

slide-21
SLIDE 21

Concluding Remark

  • Multi-level random effects models for

longitudinal MBMA with NONMEM

Should expect correlation between longitudinally observed mean values Inclusion of treatment arm-level random effects only is not recommended

21

slide-22
SLIDE 22

backup

22

slide-23
SLIDE 23

Example Data Set

23 REFID=ID ARM NTRT DAY absolute SE SD BASELINE TRT2 L2 FLG 100 91 25.3 0.304003 2.9 25.3 8 1 1 100 91 42 17.7 9999 9999 25.3 8 1 2 100 1 95 24.8 0.307794 3 24.8 9 2 1 100 1 95 42 14.2 9999 9999 24.8 9 2 2 100 2 92 24.9 0.312772 3 24.9 9 3 1 100 2 92 42 15.1 9999 9999 24.9 9 3 2 100 3 91 25.7 0.335451 3.2 25.7 9 4 1 100 3 91 42 16.5 9999 9999 25.7 9 4 2 107 75 20.42 0.475737 4.12 20.42 8 5 1 107 75 56 13.65 0.815219 7.06 20.42 8 5 2 107 1 82 19.9 0.463812 4.2 19.9 3 6 1 107 1 82 56 13.1 0.958546 8.68 19.9 3 6 2 107 2 37 21.43 0.598412 3.64 21.43 5 7 1 107 2 37 56 14.4 1.232992 7.5 21.43 5 7 2 108 90 17.79 0.498586 4.73 17.79 8 8 1 108 90 7 15.85 9999 9999 17.79 8 8 2 108 90 14 14.77 9999 9999 17.79 8 8 3 108 90 28 13.92 9999 9999 17.79 8 8 4 108 90 42 12.6 9999 9999 17.79 8 8 5 108 90 56 12.8 9999 9999 17.79 8 8 6 108 90 56 13.51 0.712567 6.76 17.79 8 8 7 108 1 91 17.47 0.545108 5.2 17.47 3 9 1 108 1 91 7 16.13 9999 9999 17.47 3 9 2 108 1 91 14 14.27 9999 9999 17.47 3 9 3 108 1 91 28 12.9 9999 9999 17.47 3 9 4 108 1 91 42 11.48 9999 9999 17.47 3 9 5 108 1 91 56 11.41 9999 9999 17.47 3 9 6 108 1 81 56 12.04 0.701111 6.31 17.47 3 9 7

slide-24
SLIDE 24

$INPUT ID ARM … L2 FLG $DATA $PRED FL1=0 FL2=0 FL3=0 FL4=0 FL5=0 IF (FLG.EQ.1) FL1=1 IF (FLG.EQ.2) FL2=1 IF (FLG.EQ.3) FL3=1 IF (FLG.EQ.4) FL4=1 IF (FLG.EQ.5) FL5=1 ERR1=FL1*ERR(2)+FL2*ERR(3)+FL3*ERR(4) ERR2=FL4*ERR(5)+FL5*ERR(6) ERRC=ERR1+ERR2 E0 = THETA(1) EMAX = THETA(2) LED50 = THETA(3) K0=THETA(4) W=1/SQRT(NTRT) MU = E0*EXP(-K0*TIME)-POST*EMAX*DOSE/(EXP(LED50)+DOSE) F = MU + ETA(1) Y = F + W*(ERR(1)+ERRC) $THETA (0, 30 ) ;E0 (0, 10) ;EMAX (5,) ;ED50 (0, 0.1) ;K0 $OMEGA 1 $SIGMA 3 $SIGMA BLOCK (1) 9 $SIGMA BLOCK (1) SAME $SIGMA BLOCK (1) SAME $SIGMA BLOCK (1) SAME $SIGMA BLOCK (1) SAME Allows for correlation between

  • bservations

within an arm Study-to-study variability

How to Set up

24

slide-25
SLIDE 25

Output Interpretation (compound symmetry correlation)

SIGMA - COV MATRIX FOR RANDOM EFFECTS - EPSILONS **** EPS1 EPS2 EPS3 EPS4 EPS5 EPS6 EPS1 + 3.64E+00 EPS2 + 0.00E+00 7.57E+00 EPS3 + 0.00E+00 0.00E+00 7.57E+00 EPS4 + 0.00E+00 0.00E+00 0.00E+00 7.57E+00 EPS5 + 0.00E+00 0.00E+00 0.00E+00 0.00E+00 7.57E+00 EPS6 + 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 7.57E+00

325 . 57 . 7 64 . 3 64 . 3 variation total component shared    ρ

25

slide-26
SLIDE 26

26

Compound Symmetry

2 2 2 1 2 1 2 2 2 1 2 2 2 1 2 1 2 1 2 1 2 2 2 1 2 1 2 1 2 1 2 2 2 1 2 1 2 1

1 ... ... ... ... ... ... 1 ... 1 ) ( ... ... ... ... ... ... ... I 1                                                                       where

slide-27
SLIDE 27

ARM= L2

Example: Replicates*

ID TIME AMT DV FLG L2 1 200 . . 1 1 2 . 17.9 1 2 . 17.3 1 1 4 . 8.53 1 4 . 8.99 1

$ERROR Y = F + ERR(1) $SIGMA 1

; common errors

+ ERR(2)*(1-FLG) + ERR(3)*FLG $SIGMA BLOCK (1) 1 $SIGMA BLOCK (1) SAME ; cor = 0.5

; replicate-specific errors

1 1 2 2

* Karlsson MO, Beal SL, Sheiner LB. Three new residual error models for population PK/PD analyses. J Pharmacokinet Biopharm 23: 651-672 (1995)

27

slide-28
SLIDE 28

28

Imagine data arise from this simple patient-level model …

2

3 1 4

( ) exp ( )

ij t study patient ij i ij ij ij

Dose Y t t Dose

     

      

Yij(t) is the response for patient j in the ith study at time t

2

~ (0, )

study i S

N  

2

~ (0, )

patient ij P

N  

2

( ) ~ (0, )

ij t

N  

The study-level variability represents different patient populations across studies. The patient-level variability represents variability across patients within a study.

slide-29
SLIDE 29

29

… but that we only observe the mean responses over time

{ }:

1 ( ) ( )

ij ik

ik ij ij Dose D ik

Y t Y t n

 

2

3 1 { } 4

1 ( ) exp ( )

t study patient ik ik i ij ij ij ik ik

D Y t t D n

     

      

( )

ik

Y t

is the mean response for dose Dk in the ith study at time t

( , , )

ik

f D t θ

slide-30
SLIDE 30

30

We can view this model in two ways

 

2

3 1 { } 4

1 ( ) exp ( )

t study patient ik ik i ij ij ij ik ik

D Y t t D n

     

      

 

*

1 ( ) ( , , )

study ik ik i ik ik

Y t f D t t n      θ 1 1 ( ) ( , , ) ( )

study arm ik ik i ik ik ik ik

Y t f D t t n n        θ

   

* 2 2 { }

1 ( ) ~ 0,

patient ik ij ij P ij ik

t N n        

where

 

2

( ), ( )

ik ik P

Cov t t     

 

2 2 2

( ), ( )

P ik ik P

Corr t t        

where

2

~ (0, )

arm ik P

N  

2

( ) ~ (0, )

ik t

N  

{ }

1

arm patient ik ij ij ik

n   

{ }

1 ( ) ( )

ik ij ij ik

t t n   

and

Doesn’t depend on time * * * *

slide-31
SLIDE 31

STDY=ID ARM=L2 DOSE TIME NTRT DV FLG (ARM) (ARM, FLG) 1 11 50 30.08 1 η1 ε1(11) ε2(11,1) 1 11 1 50 27.06 2 η1 ε1(11) ε2(11,2) 1 11 2 50 24.75 3 η1 ε1(11) ε2(11,3) 1 11 4 50 20.87 4 η1 ε1(11) ε2(11,4) 1 12 50 50 31.18 1 η1 ε1(12) ε2(12,1) 1 12 50 1 50 25.67 2 η1 ε1(12) ε2(12,2) 1 12 50 2 50 22.89 3 η1 ε1(12) ε2(12,3) 1 12 50 4 50 19.08 4 η1 ε1(12) ε2(12,4) 1 13 100 50 29.41 1 η1 ε1(13) ε2(13,1) 1 13 100 1 50 24.50 2 η1 ε1(13) ε2(13,2) 1 13 100 2 50 22.54 3 η1 ε1(13) ε2(13,3) 1 13 100 4 50 17.93 4 η1 ε1(13) ε2(13,4) 1 14 200 50 30.19 1 η1 ε1(14) ε2(14,1) 1 14 200 1 50 23.50 2 η1 ε1(14) ε2(14,2) 1 14 200 2 50 20.46 3 η1 ε1(14) ε2(14,3) 1 14 200 4 50 16.83 4 η1 ε1(14) ε2(14,4) 1 15 400 50 29.83 1 η1 ε1(15) ε2(15,1) 1 15 400 1 50 21.34 2 η1 ε1(15) ε2(15,2) 1 15 400 2 50 19.02 3 η1 ε1(15) ε2(15,3) 1 15 400 4 50 14.55 4 η1 ε1(15) ε2(15,4) 2 16 50 29.01 1 η2 ε1(16) ε2(16,1) 2 16 1 50 26.16 2 η2 ε1(16) ε2(16,2) 2 16 2 50 23.16 3 η2 ε1(16) ε2(16,3) 2 16 4 50 18.78 4 η2 ε1(16) ε2(16,4) 2 17 100 50 28.74 1 η2 ε1(17) ε2(17,1) 2 17 100 1 50 22.43 2 η2 ε1(17) ε2(17,2) 2 17 100 2 50 20.48 3 η2 ε1(17) ε2(17,3) 2 17 100 4 50 14.47 4 η2 ε1(17) ε2(17,4)