EMEA Performance Indicators Pre-Authorisation Bo Aronsson EMEA - - PowerPoint PPT Presentation

1

EMEA Performance Indicators Pre-Authorisation

Bo Aronsson EMEA EMEA-EFPIA Info Day 2009

2

Contents

• Analysis Period
• EMEA questionnaires
• Results
• Summary
• Work in progress (Predictors of outcome)
• Conclusions

3

Analysis Period

• EMEA Data Set: All applications with
• utcome between 1 January 2003 and 31

December 2008

– Period of questionnaires follows annual reporting to Management Board – Source:

• Questionnaires to (co-)rapporteurs
• Scientific Memory Database
• EFPIA Data Set: October 06-October 08

4

EMEA Questionnaires EMEA Questionnaires

• Two versions have been used in 2003-2008
• “Old” version, implemented in 2000

– 10-point Scale (0 dissatisfied to 10 satisfied)

• “New” version implemented in 2007

– Keep some of the same domains from “old” questionnaire – Includes new domains (e.g., Scientific Advice) – 5-point Likert Scale (1 agree to 5 disagree) – Note: validation ongoing

• Questionnaires administered after day 80
• Average scores between Rapporteur/Co-rapporteur, per product
• Exclusion of duplicates

5

EMEA Questionnaires Compared

5 Yes

• RMP/PVP

1 Yes

• Communication

4 Yes

• Scientific Advice
• 3

Yes SPC, PL, Labelling

• 3

Q NC C Study Reports

• 3

Q NC C Summary

• 2

NC C Overview 11 Q NC C 3 Q NC C Evidence-Data/Design 3 Q NC C 3 Q NC C Dossier Presentation No. V2007 No. V2000

Item

Q= quality, NC=non-clinical, C=clinical, CPh=clinical pharmacology; CE=clinical efficacy, CS=clinical safety, PVP=pharmacovigilance plan.

6

Data Set 2003-2008 (N=209)

• Allows to explore 2 domains and Parts of

Dossier

– Presentation of the Dossier for Q, NC and C – Evidence (Data/Studies) included in the dossier for Q, NC and C

209 44 36 43 33 32 21

• No. Questionnaires ("new" +

"old") 74 63 61 86 92 89 70 Compliance (%) 281 70 59 50 36 36 30

• No. Outcomes

31 30 1

• No. "new" quest.

178 14 35 43 33 32 21

• No. "old" quest.

Total 2008 2007 2006 2005 2004 2003 Year

7

Product Characteristics (N=209)

40.67 85 Scientific Advice 75.12 157 Positive Outcome 13.87 29 Other 7.18 15 V 11.00 23 N 23.92 50 L 17.70 37 J 7.18 15 C 5.26 11 B 13.88 29 A ATC 26.79 56 Orphan Status Percent Frequency

8

Questionnaire Results

• All scores converted to 10-point Scale (0 dissatisfied

to 10 satisfied)

9

Presentation of Dossier (N=209)

Quality Non-clinical Clinical 2 4 6 8 10 Score

10

Evidence by Module (N=209)

Quality Non-clinical Clinical 2 4 6 8 10 Score

11

Presentation By Time (N=209)

0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00 1-Jan-03 1-May-03 29-Aug-03 27-Dec-03 25-Apr-04 23-Aug-04 21-Dec-04 20-Apr-05 18-Aug-05 16-Dec-05 15-Apr-06 13-Aug-06 11-Dec-06 10-Apr-07 8-Aug-07 6-Dec-07 4-Apr-08 2-Aug-08 30-Nov-08

Outcome Score

Quality Non-clinical Clinical

• Poly. (Quality)
• Poly. (Non-clinical)
• Poly. (Clinical)

12

Evidence by Module by Time (N=209)

1 2 3 4 5 6 7 8 9 10 1-Jan-03 1-May-03 29-Aug-03 27-Dec-03 25-Apr-04 23-Aug-04 21-Dec-04 20-Apr-05 18-Aug-05 16-Dec-05 15-Apr-06 13-Aug-06 11-Dec-06 10-Apr-07 8-Aug-07 6-Dec-07 4-Apr-08 2-Aug-08 30-Nov-08

Outcome Score

Quality Non-clinical Clinical

• Poly. (Quality)
• Poly. (Non-clinical)
• Poly. (Clinical)

13

Evidence by Orphan (N=209)

No Orphan 2 4 6 8 10

Non-clinical

No Orphan 2 4 6 8 10

Quality

Score No Orphan 2 4 6 8 10

Clinical

14

Is the Score Associated with Outcome and Clock-stop?

15

Success by Outcome Year (N=209)

IsPos 1 PERCENT 10 20 30 40 50 60 70 80 90 100 YearFinal 2003 2004 2005 2006 2007 2008

16

Clinical Evidence versus Outcome (N=209)

Neg. Pos. 2 4 6 8 10

Data Clinical

Score

17

Average Score and Clock-stop (N=209)

0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00 200 400 600 800

Clock-stop (Days) Average Score

(all Presentation/Evidence for Q, N, C)

18

Summary

• Majority satisfaction

– No new time trends – Orphan status associated with lower satisfaction with Evidence for all modules (and Presentation, data not shown)

• Satisfaction with Clinical Evidence associated with
• utcome and clock-stop
• Need to improve compliance with questionnaire
• Future

– Further validate new questionnaire and explore new domains (work in progress) – Predictors of Outcome (work in progress)

19

Predictors for Outcome - SA

Work in progress

20

Proportion of Proportion of MAAs MAAs that received SA (by that received SA (by

• utcome year)
• utcome year)

38% 47% 56% 0% 10% 20% 30% 40% 50% 60% 2006 (n=50) 2007 (n=59) 2008 (n=70)

SA given

21

Distribution of Scientific Advice over eligibility Distribution of Scientific Advice over eligibility -

• 2008

2008

Eligibility with/without SA 2008

6 5 16 11 1 1 3 18 5 4 5 10 15 20 25 30 35

Biotech Mandatory Indication Orphan New Active Substance Significant Innovation/Patients interest WHO Generic

No SA SA

22

Did the Company follow SA?

• 31 “new” questionnaires in

study period

– 16 with SA given – 6/16 (35%) show poor compliance according to Rapporteurs (score <5)

• Is SA or compliance to SA

related to outcome?

Compliance Poor Good

23

Size of company, success rate and Compliance with SA

36% 29% 50% 56 151+*** 63% 34% 72% 53 21-150** 84% 46% 89% 83 Top 20 largest* Compliance with SA Proportion with SA Success rate Number applications

Pharma size

*Top 20 largest (n=83) defined as being among the 20 largest companies **21-150 (n=53) defined as being among the 21 – 150 largest companies ***151+ (n=56) defined as not being among the 150 largest companies based on Total revenues 2005 according to Scrips Pharmaceutical League Tables 2006.

Regnstroem et al., (in manuscript)

24

Predictors of Outcome

Data on ranking was only available for 148 applications (work in progress) Stepwise logistic regression. Compliance, retrospectively assigned in Regnstroem et al. (in manuscript)

78.311 1.175 9.593

SA & Compliant vs. (No SA

• r Not Compliant)

0.744 0.122 0.301

Major Objection on RCT

(No vs Yes)

1.901 1.16 1.485

Clinical Evidence (0-10)

3.36 1.079 1.904

Company Size (1: 151+; 2: 21-150; 3: Top 20 largest) 95% Wald Confidence Limits Point Estimate Effect Odds Ratio Estimates

25

Conclusions

• Most important factors associated with
• utcome

– Compliance with Scientific Advice – Company Size – Rapporteurs’ satisfaction with Clinical Evidence submitted – Major Objections on the Lack or Randomised Controlled Trials

26

Acknowledgments

• Analysis

– Francesco Pignatti – Franz Koenig – Jan Regnstroem

• Data Management

– Esther Cozar Calvente – Nadia Kresse – Monica Simeoni