Developing Bioanalytical Methods Balancing the Statistical Tightrope - - PowerPoint PPT Presentation

Developing Bioanalytical Methods Balancing the Statistical Tightrope

“Lee: can I use this number?”

Pr Process Develo lopment GSK, 19 1997

2

“it’s about 40” “about 40?” “probably...”

3

Enlightenment?

5

Unconscious Conscious Incompetent Competent

Co Conscio iousness

Blooms Taxonomy the 4 stages of f competence

6

Me

A Statistical God

Using Statistics

1. Potency assays are key in in mak aking medicines

• 2. Bioassays are

re very variable

• 3. Statistics will help

lp you understand your data

• 4. Understanding your data will re

reveal if control

exists

• 5. Your level of control allows you to judge RIS

ISK

• 6. Regulators globally re

require it

Why? Six Reasons

9

The Regulator & Assay Control

• 1. Pharmaceutical cGMPs for the 21st

Century

• 2. PAT
• 3. ICH Q2: Validation of Analytical

Procedures

• 4. ICH Q8: Pharmaceutical Development
• 5. ICH Q9: Quality Risk Management
• 6. ICH Q10: Quality Pharmaceutical

Systems

Regulators have been asking for this for years! QbD

10

Statistics The complete solution?

Or this? Your assay?

12

Or this?

• r your assay?

13

Statistics - an Amazing Transition

14

Bioassays will always be variable You can improve it

• by understanding it
• Focusing effort in right places
• This brings control
• You can manage expectations
• This is understood by regulators

15

Why assay variation matters?

product variation + assay variation + inaccuracy Many satisfactory OOS batches likely to fail (potentially costing £Ms) because of combination of assay method & process inaccuracy & variation

A few unsatisfactory batches may even pass specification due to a combination

• f assay method and

process variability

16

Our Control Strategy

What does the scientist need to achieve? i.e. selectivity, accuracy, precision linearity Measure Analyse Improve Control Define Identify & prioritise analytical CNX parameters eXperimental parameters

e.g., DoE Regression

Noise parameters

e.g., MSA, Precision

Control parameters

Fix & control

Method Robustness Method Ruggedness Method Control Strategy & reduce Risk prior to Validation → Routine Use & Continuous Improvement Input into

17

Generating Bio ioassay Data

18

• 1. Speak with your statistician before

generating data 2. 2.See Rule 1

The Rule les

19

Lot’s data ≠ Value

20

21

Statistics are a tool

22

Which Tools?

Design

QC

Precision Accuracy Linearity etc.

TIME

UCL LCL

Stage 1: Qualification Tool Fishbone, Minitab Stage 2: Development Tools

DX8, JMP, Minitab

Stage 3: Validation Tools

Nested, CELLULA

Stage 4 QC Tools

CELLULA, Shewhart chart, CUSUM

Technology Transfer YES NO

What’s Appropriate Knowledge?

• Learning takes time
• Will you use it often enough?
• It’s not an academic pursuit
• Activities must add value
• do what’s necessary

24

Scope & Design

26

Define & Scope

How is the assay performing?

Prec/TOL2-sided = 6 x 16.76 100 = 1.01

Parameters (e.g. 15)

pDNA NaCl pH Tube Length Time Seeding Density Ratio of Transfection Temperature Agitation and level Vector – type, conc Addition Order

• Q. How Many parameters?
• Q. Which parameters?
• Q. What ranges?
• A. Existing knowledge
• A. Common sense
• A. Practical limits

29

Define & Scope

Drill down - map out assay - build understanding & scope

Assay Flow

30

Define & Scope

Drill down & map out assay to build understanding & scope

Attention is focused toward key steps and the parameters involved in these steps

Cause & Effect Diagram (Fishbone) helps think your assay through

Identify & prioritise analytical CNX parameters

Scope & Screen

31

Scope ranges with simple experiments

Scoping Experiments

Explore mildest to most forcing conditions

32

Revealing the Big Hitters

Temptation

34

OFAT

pH pDNA NaCl

Provides estimates

• f effects at set

conditions of the

• ther factors and

no interaction effects.

Building Understanding

35

Building Understanding Factorial Design

Estimates effects at different conditions to estimate interactions Design of Experiments DOE

250 1300

900 300 500 1800 350 600 2400 2600

36

work towards a Robust Optimum

Optimise the parameters that survived the initial screening

Optimisation

37

The tools allow you to simulate scenarios using the data you’ve built up

Simulations

Visual simulation of expected performance relative to specification

Is the Model Correct?

38

39

Ideal Settings Control Space Design Space Method stretch…what if?

The evaluation of robustness should be considered during the development phase and should show the reliability of an analysis with respect to deliberate variations in method parameters

ICH Q2B, 1994

Validate & Verify

Assay Control: control the parameters inside boundaries

40

Even if you go outside the control boundaries, the assay will have enough flexibility to deal with it without an OOS

41

Working within the control boundaries will keep the assay under control

Summary - Data Driven Development

Scope

Explore mildest to most forcing conditions

Optimize

Estimate & utilize interactions to move towards optimum conditions

Verify

Rattle the cage to deliver a design space

QC/TT

Transfer to QC to validate on batches & bring into routine use Identify few potential key parameters Focus on vital few & narrow ranges

Screen

43

Precision

It may be considered at three levels:

1.

Repeatability

2.

Intermediate precision

3.

Reproducibility

ICH Q2A, 1994

Repeatability

1 analyst in 1 laboratory on 1 day injecting 6 times

Summary Statistics Number of Values Mean Standard Deviation Coefficient

• f Variation

Lower 95% CI for Mean Upper 95% CI for Mean t30 PS 6 223.27 6.43 2.88% 216.52 230.02 45

Intermediate Precision

As well as sample variation, this study still provides information on repeatability

46

• 1 analyst in 1 laboratory on
• 1 day
• injecting 6 samples
• each tested 6 times

So we compare the mean values for each sample (over replicate results per sample)

Intermediate Precision

Variance Components Factor df Variance % Total Sample 5 27.8535 21% Repeat 30 102.6361 79% 35 130.4896 100% Standard Mean Deviation RSD 216.24 11.4232 5.28%

47

and the others…..? Precision within a laboratory but with different analysts, on different days, with different equipment…reflects the real conditions within one laboratory

ICH Q2A 1995

48

49

Y 52000 52500 53000 53500 54000 54500 55000 55500 56000 5 10 15 20 25 Sample Peak Area

Data collect using several analysts using several instruments

• ver several days:

Intermediate Precision

50

Y 52000 52500 53000 53500 54000 54500 55000 55500 56000 5 10 15 20 25 Sample Peak Area

Potentially misleading: large analyst-to-analyst variation present:

Analyst 1 Analyst 2 Analyst 3

Intermediate Precision

51

better examined looking at multiple sources of variation within an assay

Intermediate Precision

want to understand major sources of variation such as sample, prep, analyst etc.

52

Intermediate Precision

Can also perform Unbalanced designs

Intermediate Precision

One operator performs multiple injections on single preparation; Two operators perform single injections on multiple preparations

53

54

…. sent to and analysed by other lab

C B



A C B A

Samples from laboratory:

multiple laboratories; typically run as an inter- laboratory cross-over study, with each participating lab sending samples to every other lab and analysing all samples (including own)

Reproducibility

       

Can use analysis of variance (ANOVA) to look for differences or biases between labs

Alternatively look for “analytical equivalence”

Reproducibility

Risk Management

The level of effort, formality and documentation.. ..should be commensurate with the level of risk

ICH Q9

Evaluation of the risk to quality should be based on scientific knowledge & ultimately link to the protection of the patient Is the bioassay fit for purpose and under control?

56

57

Before & After

How is the assay performing?

P/TOL2-sided = 6 x 16.76 100 = 1.01

58

Better

P/TOL2-sided = 6 x 6.99 100 = 0.42

Before & After

59

Ris isk Management

Method Understanding, Control and Capability (MUCC)

Understand impact of variation upon risk… Capability & Precision

Capable? Control?

Risk Management Loop

Understanding?

Statistical Process Control (SPC) Charts

60

Understanding?

Capability & Precision

Capable? Understanding?

100 = 1.01 P/TOL2-sided = 6 x 16.76

Capable? Control?

Ris isk Management

61

46 41 36 31 26 21 16 11 6 1 225 210 195 180 Observation t30 PS _ X=199.87 UCL=220.77 LCL=178.96 I Chart Investigate out-of-control points.

P/TOL2-sided = 6 x 6.99 100 = 0.42

Ris isk Management

1.Build a good basic understanding of stats but don’t need to become guru 2.Involve a statistician, at least at the beginning 3.Build understanding of your bioassay (QbD) – it’s a must 4.Get to grips with Bioassay Variability

Summary

62

“Lee: can I use this number?”

63

“Yes – it’s 42 ”

 0.05 wit ith 95% Confid idence for the st statisticians in in the audience

64

…

Acknowledgments

• Dr. Paul Nelson – Prism TC Ltd

Pictures from “The Cartoon Guide to Statistics” Larry Gonick & Woollcott Smith

65