Developing Bioanalytical Methods Balancing the Statistical Tightrope
“Lee: can I use this number?” Pr Process Develo lopment GSK, 19 1997 2
“it’s about 40” “about 40?” “probably...” 3
Enlightenment?
5
Blooms Taxonomy the 4 stages of f competence Incompetent Competent Conscious iousness Conscio Unconscious Co 6
A Statistical God Me
Using Statistics
Why? Six Reasons Potency assays are key in 1. in mak aking medicines 2. Bioassays are re very variable lp you understand your data 3. Statistics will help 4. Understanding your data will re reveal if control exists 5. Your level of control allows you to judge RIS ISK 6. Regulators globally re require it 9
The Regulator & Assay Control Regulators have been asking for this for years! QbD 1. Pharmaceutical cGMPs for the 21st Century 2. PAT 3. ICH Q2: Validation of Analytical Procedures 4. ICH Q8: Pharmaceutical Development 5. ICH Q9: Quality Risk Management 6. ICH Q10: Quality Pharmaceutical Systems 10
Statistics The complete solution?
Or this? Your assay? 12
Or this? or your assay? 13
Statistics - an Amazing Transition 14
Bioassays will always be variable You can improve it - by understanding it - Focusing effort in right places - This brings control - You can manage expectations - This is understood by regulators 15
Why assay variation matters? product variation + A few unsatisfactory assay variation + batches may even inaccuracy pass specification due to a combination of assay method and process variability Many satisfactory OOS batches likely to fail (potentially costing £Ms) because of combination of assay method & process inaccuracy & variation 16
Our Control Strategy What does the scientist need to achieve? i.e. selectivity, accuracy, precision linearity Define Identify & prioritise analytical CNX parameters Measure C ontrol N oise e X perimental parameters parameters parameters Analyse e.g., MSA, e.g., DoE Fix & control Input into Precision Regression Method Method Improve Ruggedness Robustness Method Control Strategy & reduce Risk prior to Control Validation → Routine Use & Continuous Improvement 17
Generating Bio ioassay Data 18
The Rule les 1. Speak with your statistician before generating data 2. 2.See Rule 1 19
Lot’s data ≠ Value 20
21
Statistics are a tool 22
Which Tools? QC UCL Stage 4 Technology QC Tools YES Transfer CELLULA, Shewhart chart, LCL CUSUM NO TIME Stage 1: Qualification Tool Stage 3: Fishbone, Minitab Validation Tools Nested, CELLULA Precision Stage 2: Accuracy Design Development Tools Linearity etc. DX8, JMP, Minitab
What’s Appropriate Knowledge? • Learning takes time • Will you use it often enough? • It’s not an academic pursuit • Activities must add value do what’s necessary 24
Scope & Design
Define & Scope How is the assay performing? Prec/TOL 2-sided = 6 x 16.76 100 = 1.01 26
Parameters (e.g. 15) pDNA NaCl pH Tube Length Time Seeding Density Ratio of Transfection Temperature Agitation and level Vector – type, conc Addition Order
Q. How Many parameters? Q. Which parameters? Q. What ranges? A. Existing knowledge A. Common sense A. Practical limits
Define & Scope Drill down - map out assay - build understanding & scope Assay Flow 29
Define & Scope Drill down & map out assay to build understanding & scope Attention is focused toward key steps and the parameters involved in these steps Cause & Effect Diagram (Fishbone) helps think your assay through Identify & prioritise analytical CNX parameters 30
Scope & Screen Scope ranges with simple experiments Scoping Experiments Explore mildest to most forcing conditions 31
Revealing the Big Hitters 32
Temptation
Building Understanding OFAT Provides estimates of effects at set conditions of the NaCl other factors and no interaction pDNA effects . pH 34
Building Understanding 2400 Factorial Design 2600 1300 900 1800 Estimates effects at different conditions to estimate interactions 350 600 250 300 500 Design of Experiments DOE 35
Optimisation Optimise the parameters that survived the initial screening work towards a Robust Optimum 36
Simulations The tools allow you to simulate scenarios using the data you’ve built up Visual simulation of expected performance relative to specification 37
Is the Model Correct? 38
Validate & Verify The evaluation of robustness should be considered during the development phase and should show the reliability of an analysis with respect to deliberate variations in method parameters ICH Q2B, 1994 Method stretch…what if? Ideal Settings Control Space Design Space 39
Assay Control: control the parameters inside boundaries 40
Working within the control boundaries will keep the assay under control Even if you go outside the control boundaries, the assay will have enough flexibility to deal with it without an OOS 41
Summary - Data Driven Development Scope Screen Optimize Verify QC/TT Transfer to QC to validate on batches & bring into routine use Identify few potential Explore mildest Estimate & utilize key parameters to most forcing interactions to move Rattle the cage to Focus on vital few & conditions towards optimum deliver a design narrow ranges conditions space
43
Precision It may be considered at three levels: Repeatability 1. Intermediate precision 2. Reproducibility 3. ICH Q2A, 1994
Repeatability 1 analyst in 1 laboratory on 1 day injecting 6 times Summary Statistics Number of Standard Coefficient Lower 95% CI Upper 95% Values Mean Deviation of Variation for Mean CI for Mean t30 PS 6 223.27 6.43 2.88% 216.52 230.02 45
Intermediate Precision 1 analyst in 1 laboratory on • 1 day • injecting 6 samples • each tested 6 times • As well as sample variation, this study still provides information on repeatability 46
Intermediate Precision So we compare the mean values for each sample (over replicate results per sample) Variance Components Factor df Variance % Total Sample 5 27.8535 21% Repeat 30 102.6361 79% 35 130.4896 100% Standard Mean Deviation RSD 47 216.24 11.4232 5.28%
and the others…..? Precision within a laboratory but with different analysts, on different days, with different equipment…reflects the real conditions within one laboratory ICH Q2A 1995 48
Intermediate Precision Data collect using several analysts using several instruments over several days: Y 56000 55500 55000 54500 Peak Area 54000 53500 53000 52500 52000 0 5 10 15 20 25 Sample 49
Intermediate Precision Potentially misleading: large analyst-to-analyst variation present: Y 56000 55500 55000 54500 Peak Area 54000 53500 53000 52500 52000 0 5 10 15 20 25 Sample Analyst 1 Analyst 2 Analyst 3 50
Intermediate Precision better examined looking at multiple sources of variation within an assay want to understand major sources of variation such as sample, prep, analyst etc. 51
Intermediate Precision 52
Intermediate Precision Can also perform Unbalanced designs One operator performs multiple injections on single preparation; Two operators perform single injections on multiple preparations 53
Reproducibility multiple laboratories; typically run as an inter- laboratory cross-over study, with each participating lab sending samples to every other lab and analysing all samples (including own) …. sent to and analysed by other lab A B C A Samples from laboratory: B C 54
Reproducibility Can use analysis of variance (ANOVA) to look for differences or biases between labs Alternatively look for “analytical equivalence”
Risk Management The level of effort, formality and documentation.. ..should be commensurate with the level of risk ICH Q9 Evaluation of the risk to quality should be based on scientific knowledge & ultimately link to the protection of the patient Is the bioassay fit for purpose and under control? 56
Before & After How is the assay performing? P/TOL 2-sided = 6 x 16.76 100 = 1.01 57
Before & After Better P/TOL 2-sided = 6 x 6.99 100 = 0.42 58
Ris isk Management Method Understanding, Control and Capability (MUCC) Understand impact of variation upon risk… Understanding? Risk Capable? Management Loop Statistical Capability Process Control & Precision (SPC) Charts Control? 59
Ris isk Management Understanding? Understanding? Capable? P/TOL 2-sided = 6 x 16.76 Capable? 100 Control? = 1.01 Capability & Precision 60
Ris isk Management P/TOL 2-sided = 6 x 6.99 100 = 0.42 I Chart Investigate out-of-control points. 225 UCL=220.77 210 t30 PS _ X=199.87 195 180 LCL=178.96 1 6 11 16 21 26 31 36 41 46 61 Observation
Recommend
More recommend
