New CLSI Document for the Validation of Methods Preformed by Flow - - PowerPoint PPT Presentation

new clsi document for the validation of methods preformed
SMART_READER_LITE
LIVE PREVIEW

New CLSI Document for the Validation of Methods Preformed by Flow - - PowerPoint PPT Presentation

New CLSI Document for the Validation of Methods Preformed by Flow Cytometry Sneak Peek and Update Virginia Litwin, Ph.D. Vice President, Immunology CLSI H62 Validation of Methods Performed by Flow Cytometry Sneak Peek Overview


slide-1
SLIDE 1

New CLSI Document for the Validation of Methods Preformed by Flow Cytometry – Sneak Peek and Update

Virginia Litwin, Ph.D. Vice President, Immunology

slide-2
SLIDE 2

2

CLSI H62 – Validation of Methods Performed by Flow Cytometry

Sneak Peek Overview

  • Document Writing Committee
  • Process and Timelines
  • Content Highlights
slide-3
SLIDE 3

H62 Document Writing Committee

Leadership

  • Virginia Litwin, Chair
  • Teri Oldaker, Vice Chair
  • Raul Louzao, Secretary
  • Dave Sterry, CLSI Standards Director

Voting Members David Barnett, Jacqueline, Cleary, Tom Denny, Cherie Green, Wolfgang Kern, Natalia Kokorina, Jennifer Stewart, Lili Wang Contributors and Reviewers Elena Afonina, Ahmad Al Samman, Tony Bakke, Fiona Craig, Bruce Davis, Lorella Di Donato, Steve Eck, Nancy Fine, Ben Hedley, Shuguang Huang, Jerry Hussong, Andrea Illingworth, Cassie Jiang, Mike Keeney, Natalia Kokorina, Sarah Maremont, Laura Marszalek, Kathy Muirhead, Andy Rawstron, John Schmitz, Alan Stall, Maryalice Stetler-Stevenson, Horacio Vall, Alessandra Vitaliti-Garami, Paul Wallace, Brent Wood, Yuanxin Xu

3

slide-4
SLIDE 4

Document Writing Committee Composition

Affiliations

  • Academia
  • Biopharmaceutical
  • CRO
  • Clinical Laboratories
  • Reagent/Instrument

Manufacturers

  • Government
  • FDA
  • NIST

4

Scientific Societies

  • AAPS
  • CAP
  • ESCCA
  • ICCS
  • ISAC

Provenance

  • Canada
  • Germany
  • Switzerland
  • UK
  • USA
slide-5
SLIDE 5

Special Reviewers

ICCS, Advocacy Committee AAPS, Flow Cytometry Action Program Committee

5

Ruth Barnard, Steve Eck, Catherine Fleener, Fiona Germaschewski, Christele Gonneau, Cherie Green, Chris Groves, Michael Hedrick, Shuguang Huang, Shibani Mitra- Kaushik, David Lanham, Virginia Litwin, Thomas McCloskey, Thomas McIntosh, Maxime Moulard, Sam Pine, Kruti Shah, Ulrike Sommer, Soren Sonder, Jennifer Stewart, Yongliang (Steve) Sun, Alessandra Vitaliti, Dave Williams, Sam Witherspoon, Yuanxin Xu, Chelsea Xue Thomas Denny, Pranav Dorwal, Jeannine Holden, Jerry Hussong, Wolfgang Kern, Virginia Litwin, Sara Monaghan, Teri Oldaker, Andy Rawstron, Stephanie Toney, Christopher Trindade, Paul Wallace

slide-6
SLIDE 6

Process and Timelines

Document publication (Word/InDesign), December 19 Final Draft, August 19 Final CLSI vote (20 days), September 19 Circulate right to appeal (30 days), July 19 Prepare responses and finalize comments , June 19 Circulate Draft / Open Comment (60 days), February 19 Final Draft Approved by Voting Members November 2018 Face-to-face Kick-off (Kansas City, MO) September 2017

6

slide-7
SLIDE 7

Impact

  • Extensive review process
  • American National Standards Institute (ANSI)

compliant

  • Alignment with International Organization for

Standardization (ISO)

  • CLSI serves as the ANSI-appointed Secretariat for the ISO

Technical Committee 212 (ISO/TC 212)

  • Regulatory agencies often recognize CLSI

guidelines

slide-8
SLIDE 8

Document Outline

Chapter 1 Scope Chapter 2 Quality System Essentials Teri Oldaker Chapter 3 Fit for Purpose / Iterative Approach Fiona Craig Chapter 4 Instrument Qualification, Setup, and Standardization Cherie Green Chapter 5 Assay Development and Optimization Ben Hedley Chapter 6 Assay Validation Steve Eck Chapter 7 Examination Phase/ Post-Examination Phase Raul Louzao

Pre-Examination Phase

slide-9
SLIDE 9

Chapter 1

Scope

slide-10
SLIDE 10

Scope

  • Recommendations and Practical Instructions
  • One-stop shopping
  • Current best practices
  • Summarize recent white papers and scientific advances
  • Target Audience
  • Basic research laboratories (non-regulated)
  • Clinical (regulated US and ex US)
  • Drug discovery, development, and manufacturing (regulated

and non-regulated)

  • Reagent, assay, and instrument manufacturers
  • Regulatory agencies
slide-11
SLIDE 11

Out of Scope

  • Out of Scope
  • Individual cell type-specific assay development
  • The validation of flow cytometric assays for soluble analytes
  • Third-party software and LIS interface validation
slide-12
SLIDE 12

Chapter 4

Instrument Qualification, Setup, and Standardization

slide-13
SLIDE 13

Chapter 4 Outline

4 Instrument Qualification, Setup, and Standardization

4.1 Installation Qualification and Operational Qualification (IQ, OQ) 4.2 Performance qualification (PQ)

4.2.1 Linearity and Dynamic Range 4.2.2 Electronic Noise 4.2.3 Resolution 4.2.4 Carryover

4.3 Cross-instrument, cross-site standardization

4.3.1 Examples of Cross-standardization

4.4 Compensation:

4.4.1 General factors to consider for calculating compensation: 4.4.2 Types of compensation controls 4.4.3 Compensation and Linearity

4.5 Longitudinal Performance 4.6 Qualification and verification of instrument for intended purpose

slide-14
SLIDE 14

Chapter 4--Take Home Message

  • Instrument qualification is often neglected
  • The foundation of good data
slide-15
SLIDE 15

Goals of Instrument & Software System Qualification

Establish and maintain a controlled environment that can produce reliable data over a long period of time Ensure integrity and reconstruction of data Support lifecycle of the system by establishing procedures from installation to decommission

slide-16
SLIDE 16

Installation Qualification

INSTALLATION

PARAMETER

PASS/FAIL CRITERIA DOCUMENTATION NOTES Environment Benchtop and associated lab space meet vendor specifications Checklist with vendor requirements, positive notation of Pass/Fail and initial and date Consider space requirements for instrument/computer footprint and additional clearance fo future maintenance Utilities Temperature and humidity

  • f lab space

meets vendor specification Checklist with vendor requirements, positive notation of Pass/Fail and initial and date Equipment used to perform verification should be documented in report appended to the checklist Electrical Electrical requirements meet vendor specifications Checklist with vendor requirements, positive notation of Pass/Fail and initial and date Equipment used to perform verification should be documented in report appended to the checklist Hardware Verify all components are installed Document instrument specifications (model, serial number, manufacturer date) Include all associated components, if any, including automated sample acquisition modules, uninterrupted power supplies, etc.

slide-17
SLIDE 17

Operation Qualification

OPERATIONAL

PARAMETER

PASS/FAIL CRITERIA DOCUMENTATION NOTES Software functionality Perform automated system functions (startup, QC) Screenshot and/or report with positive notation of Pass/Fail, initial and date Include automated maintenance procedures System alerts Stress the system to demonstrate that system detects problems and displays appropriate warnings Document warnings displayed with screenshots, initial and date Visual cues can also be used to prompt user to change fluids Example: Attempt to acquire data with low fluidics level or disconnected computer cable Example: Fluidics icons change color when low levels are detected. System should have warning and not allow further acquisition until fluidics issues are addressed Optical precision Run calibration beads to verify %CVs, detector sensitivity and laser power

  • utput meets vendor

specifications Checklist with vendor requirements, positive notation of Pass/Fail and initial and date Include any automated QC report, all testing reagents should be documented in a report attached to the checklist Automated sample acquisition Acquire triplicates of testing material (beads or cells) in randomly distributed locations in carousel

  • r plate

Checklist with positive notation of successful sample acquisition, Pass/Fail and initial and date There is some overlap in PQ; replicate samples could also be used to demonstrate

  • precision. OQ can be

performed using beads whereas PQ requires intended use biological samples.

slide-18
SLIDE 18

Performance Qualification Optical alignment Linearity and dynamic range Detection efficiency (Q) Electronic noise (SDen) Background signal (B) Overall resolution of the detection system, which is impacted by efficiency, background, electronic noise Acquisition carryover

slide-19
SLIDE 19

Look What’s New!

19

  • The National Institute for Standards and Technology (NIST)
  • Fluorescence calibration beads with traceable equivalent

number of reference fluorophores

  • Enable us to speak the same language

Traceable ERF Value Assignment to Commercial Microparticles nt to Commercial Microparticles

FC Bead Ex Laser (nm) SRM 1934 FITC 488 Fluorescein PE 488 Fluorescein BB515 488 Fluorescein PerCP 488 Nile Red PerCP-Cy5.5 488 Nile Red PE-Cy7 488 Nile Red APC 633 APC APC-R700 633 APC APC-H7 633 APC APC-Cy7 633 APC V450 405 Coumarin 30 BV421 405 Coumarin 30 V500-C 405 Coumarin 30 BV510 405 Coumarin 30 BV605 405 Coumarin 30 Six Peak Hard Dyed Micro- particles Ex Laser (nm) SRM 1934 Intensity 2-6 488 Fluorescein Intensity 2-6 488 Nile Red Intensity 2-6 633 APC Intensity 2-6 405 Coumarin 30

Cytometry Part A ●

  • 73A: 279-288, 2008; Flow Cytometry Protocols: Third Edition, p53-65, 2011

Current Protocols in Cytometry, 75:1.29.1-14, 2016; Flow Cytometry Protocols: Fourth Edition (in press)

Traceable ERF Value Assignment to Calibration Beads. Flow Cytometry Quantitation Consortium 81 Federal Register 136 (15 July 2016), pp. 46054-46055 ERF Value Assignment to Cytometer Calibration Beads Submitted by Consortium Members

slide-20
SLIDE 20

Two step process:

Aim: Provides evidence of linear range/proportionality and resolution, provides evidence of comparability within experiment and between experiments on single instrument Aim: Transforms fluorescence scale to ABC scale, provides reasonable instrument independent transferable scale

  • 1. Establish linear range in

fluorescence scale using beads assigned with “Equivalent number

  • f Reference Fluorophores (ERF)

values

  • 2. Anchor the fluorescence scale

(FS) to a benchmark cell material with a known protein expression in the unit of Antibodies Bound per Cell (ABC)

Benchmark scale

Lili Wang Building Measurement Assurance in Flow Cytometry CYTO Workshop 13

ERF for Benchmarking Instrument Scale

slide-21
SLIDE 21

Instrument Standardization

  • Why standardize?
  • Inter-instrument variation
  • Major source of variability
  • Within the same lab
  • Between experiments
  • Multicenter clinical trials
  • Goal of instrument standardization
  • Reproducibly set gains (PMT voltages) to achieve equivalent

fluorescence measurements (MFIs)

  • Experiment to experiment
  • Instrument to instrument
  • Lab to lab
  • Platform to platform
  • Accurately measure / assign fluorescence spillover values which

are used for fluorescence compensation

  • Maintain consistent longitudinal fluorescence measurements

21

slide-22
SLIDE 22

Instruments Compatible for Standardization

  • Similar excitation lasers and collection optics
  • Have stable fluidics
  • Be sufficiently sensitive to discriminate dim

fluorescence signals

  • Give reasonably low background (photon and/or

electronic noise)

  • Produce linear signal across dynamic range for

intended use

  • Produce data conforming to the current Flow

Cytometry Standard (FCS) data format

22

slide-23
SLIDE 23

Instrument Standardization Recent Advances

  • New instrumentation
  • Built-in, automated processes for setup and between

instrument standardization

  • Existing instruments
  • Processes for reducing between instrument/platform variability
  • Peer reviewed publications
  • Cytometry Part A 73:279, 2008
  • Cytometry A 81:567, 2012
  • Cytometry Part A 85:1037, 2014
  • Cytometry B 90:159, 2015
  • Vendor derived process
  • M. Ettinger. A New Method For QUANTITATIVE STANDARDIZATION of Flow

Cytometry Instruments. Contract Pharma. 2015

  • I. Athanasiadou and C. Gonneau. Challenges of flow cytometry for global

clinical trials. ESCCA, 2017

23

slide-24
SLIDE 24

Case Study Inter-instrument variability is reduced when instruments are standardized

Standardization Process Two Identical Instruments Fluorescence intensity readout Best Detector Worst Detector None Manufacturer’s setup process Daily Setup passed 17.9 %CV 37.4 %CV Standardized Instrument (hard dyed beads) 9.36 %CV 0.93 %CV Standardized Instrument (true fluorescence) <5%CV <5%CV

  • M. Ettinger. A New Method For QUANTITATIVE STANDARDIZATION of Flow Cytometry Instruments. Contract Pharma.

2015

  • I. Athanasiadou and C. Gonneau. Challenges of flow cytometry for global clinical trials. ESCCA, 2017
slide-25
SLIDE 25

Want More Details???

Validation the Key to Translatable Flow Cytometry a Three Part Series:

Instrument Qualification

29 October 2018 cytou.org

slide-26
SLIDE 26

Chapter 5

Assay Development and Optimization

slide-27
SLIDE 27

Chapter 5 Outline

Chapter 5: Pre-examination Assay Development and Optimization 5.2 Assay Evaluation 5.2.1 Steric hindrance 5.2.2 Tandem Fluorochrome interactions 5.2.3 Differences in affinity 5.2.4 IgG dependence 5.2.5 Fc Binding 5.3 Assay Optimization 5.3.1 Premade Cocktailed Antibody Combinations 5.3.2 Preliminary Stability: Reagent/Cocktail stability 5.3.3 Specimen Stability 5.3.4 Controls 5.3.5 Data Acquisition 5.3.6 Data Analysis 5.3.7 Documentation

5.1.1 Assay Development 5.1.2 Define the Assay 5.1.3 Considerations 5.1.4 Matrix 5.1.5 Viability 5.1.6 Antigen and Antibody Selection 5.1.7 Fluorophore Selection 5.1.8 Other Reagents 5.1.9 Sample Cell Concentration 5.1.10 Sample Lysis 5.1.11 Antibody titrations 5.1.12 Blocking 5.1.13 Fixatives 5.1.14 Gating Strategies 5.1.15 Incubation Temperature

slide-28
SLIDE 28

Process Map

Assay Objectives Assay Design Optimization Initial Characterization

  • Precision
  • Stability
slide-29
SLIDE 29

Initial Considerations

  • What do you want to measure?
  • How will you define your population of interest?
  • Positive selection antigens
  • Negative selection antigens
  • In what matrix?

Establish the Assay Objective

  • Number and type of lasers
  • Number of detectors
  • Number of markers
  • Number of tubes
  • Filter configuration
  • Fluorophore selection

What instrument do you have?

slide-30
SLIDE 30

Assay Types

  • Immuno-phenotyping assays
  • mAb
  • Multimers
  • Leukemia/Lymphoma diagnostic assays
  • Minimal residual disease (MRD) monitoring
  • Phos flow assays
  • Receptor occupancy (RO) assay
  • Functional assays
  • Intra-cellular cytokine detection
  • Pharmacokinetics (PK) assays
  • CAR-T
  • Other cell-based therapies
slide-31
SLIDE 31

Data Output

  • Relative percentage of a parent population
  • Cell concentration (cells/unit volume)
  • Fluorescence intensity
  • Percent bound, percent free for RO assays
  • Phenotypic description
slide-32
SLIDE 32

Matrix

  • Whole blood (anticoagulant choice)
  • Bone marrow aspirates/cores (anticoagulant choice)
  • Other body fluids (CFS, sputum)
  • PBMC/BMMC
  • Tissues
  • TILs
  • Cell lines
  • Other (marine, bacteria, ….)
slide-33
SLIDE 33

Blood Collection Materials

Anticoagulant Description Pros Cons EDTA

  • Sodium Heparin
  • ACD
  • Sodium Citrate
  • Stabilization Tubes
slide-34
SLIDE 34

Assay Design

  • Reagents
  • Antigen/Fluorophore pairing
  • mAb clone evaluation
  • Reagent titration
slide-35
SLIDE 35

Goals of Titration

  • Determine specificity and staining intensity of an antibody
  • Minimize compensation requirements
  • Determination of quantity of new antibody to be used
  • Signal should provide adequate room for evaluation of dim or

bright expression

  • Quality control of new antibody lots
  • Antibody performance should compared to known lot

1 mL 2 mL 3 mL 5 mL 8 mL 10 mL

slide-36
SLIDE 36

Optimal Titer

Volume MFI+ MFI- SDneg S/N SI % Gated 10uL 160.088 2.178 1.215 73.5 100.3 70 8uL 127.259 1.537 1.055 82.8 113.0 70 5uL 92.462 1.075 0.859 86.0 117.4 70 2.5uL 52.466 0.61 0.502 86.0 117.4 70 1uL 32.016 0.492 0.492 65.1 88.8 70 0.5uL 19.258 0.422 0.274 45.6 62.3 70

Staining Index = ( − ) 2 () = ! −

slide-37
SLIDE 37

Assay Design

  • Wash/lyse/fix sequence and buffer evaluation
  • Acquisition Templates
  • Number of events to acquire
  • Thresholds
  • Voltage Settings
  • Gating Strategy
  • The population of interest should be included in the gate
  • Other cell subsets/non-specific events should be excluded

Lymph PC Blasts Monos Grans Debris Lyse A 62% 0% 27% 0% 6% 3% Lyse A 60% 0% 30% 0% 3% 5% Lyse C 27% 0% 15% 0% 3% 54% Lyse D 11% 0% 6% 0% 1% 81% Lyse E 29% 0% 17% 0% 2% 51%

slide-38
SLIDE 38

Other Reagents

  • 7-amino-actinomycin D (7AAD)
  • 4’6-diamidino-2-phenylindole (DAPI)
  • Propidium Iodide (PI)
  • Detergents and Solvents
  • Lysis Reagents
  • Fixatives
  • Blocking Reagents
slide-39
SLIDE 39

Want More Details???

Step-by-Step Multi-parameter Panel Design

Jennifer Wilshire Tomas Baumgartner

15 October 2018 cytou.org

slide-40
SLIDE 40

Chapter 3 Outline

Chapter 3: Fit for Purpose Approach to Analytical Method Validation for Flow Cytometric Methods

3.1 Considerations 3.1.1 Bioanalytical Data Categories and Calibration Curves 3.1.2 Reference Standards for Flow Cytometry 3.2 Application of Standard Validation Parameters for Flow Cytometric Methods 3.2.3 Accuracy/Trueness 3.2.4 Linearity 3.2.5 Specificity and Selectivity 3.2.6 Sensitivity 3.2.7 Precision 3.2.8 Stability 3.2.9 Assay carryover 3.2.10 Reference Intervals 3.3 Fit-for-Purpose Approach

slide-41
SLIDE 41

Fit-for-Purpose Validation Concept

  • Fit
  • All data must be reliable
  • Purpose
  • Any purposes
  • Fit-for-Purpose
  • Analytical validation requirements
  • Specific to the current intended use of the data
  • Specific to the regulatory requirements, if any, associated

with that use

  • Practical, iterative approach

Lee et al. Pharmaceutical Research, 22:499, 2005

slide-42
SLIDE 42

Iterative Validation Approach

Assay Development / Optimization Initial Validation

  • Assess required

parameters for INITIAL intended use Assay Implementation Extended Validation

  • Assess additional

parameters appropriate to NEW intended use

42

slide-43
SLIDE 43

Challenges for Validation in Flow Cytometry

  • The complexity of cellular analytes/measurands
  • Increase complexity of cellular analytes in disease state samples
  • The technology
  • Highly complex
  • Highly flexibility
  • The reagents
  • Highly complex
  • mAb, fluorescent tags, tandem dyes
  • The rate of technological advances
  • Technology
  • New instruments
  • New software
  • Reagents
  • New fluorophores
  • The rapid rate of biological discoveries
  • New subsets identified
  • Phenotypic definitions of existing subsets change
  • The lack of TRUE reference material
  • The fact that data are not derived from a calibration curve

43

slide-44
SLIDE 44

Validation Parameters and Flow Cytometry

ALWAYS

  • Specificity
  • Precision/Robustness
  • Sensitivity
  • Limit of Detection
  • Limit of Quantitation
  • Stability
  • Reference Intervals

SOMETIMES

  • Interference (Matrix,

Drug) IT'S COMPLICATED

  • Accuracy
  • Linearity
  • Selectivity

“NEVER”

  • Range of

Quantification

  • Incurred Sample

Reanalysis

  • Normal Signal

Distribution

  • Prozone Effect

What are the Validation Parameters? Can they be evaluated in Flow Cytometry?

44

slide-45
SLIDE 45

Bioanalytical Data Categories

Definitive Quantitative Relative Quantitative Quasi-quantitative Qualitative

Lee et al. Pharmaceutical Research, 22:499, 2005

45

slide-46
SLIDE 46

Definitive Quantitative Data

  • Calibration curve
  • Reference Standards
  • Well defined
  • Fully representative of the

endogenous analyte

  • Numeric results are interpolated from

the calibration curve

  • Intended use of the data
  • Determine the absolute quantitative

values for unknown samples

  • Example
  • LC-MS assay for PK
  • Accuracy demonstrated by

spike/recovery with well defined standard

Talanta 70.4 (2006): 678-690

46

slide-47
SLIDE 47

Relative Quantitative Data

Image from proteintech

  • Calibration curve
  • Reference Standards
  • “Less” well defined
  • Not fully representative of the

endogenous biomarker

  • Numeric results are interpolated from the

calibration curve

  • Intended use of the data
  • Estimate the quantitative values for unknown samples
  • Emphasis on temporal changes in concentrations rather than absolute

concentrations

  • Examples
  • Cytokine enzyme immunoassays
  • Accuracy demonstrated by spike/recovery with standards

47

slide-48
SLIDE 48

Quasi-Quantitative Data

  • Intended use of the data
  • Estimate the quantitative values
  • Emphasis on temporal changes in

concentrations rather than absolute concentrations

  • Examples
  • Flow cytometric assays
  • Population frequency
  • MRD
  • Quasi
  • Having some resemblance to
  • Possession of certain attributes of
  • Does not use calibration curve
  • No reference standards
  • Numeric data are reported
  • Results are expressed in terms of a characteristic of the test sample

Image from Clinical Laboratory News, 12: 8, 2013

CD4 CD4 CD4

48

slide-49
SLIDE 49

Qualitative Data

  • No calibration standards
  • Non-numeric data are reported
  • Results are expressed in terms of a

characteristic of the test sample

  • Categorical data are reported
  • nominal (yes/no) format
  • ordinal (+, ++, +++) format (semi-

quantitative) (EP12)

  • Intended use of the data
  • Characterization of the samples
  • Examples
  • Leukemia/Lymphoma characterization for

diagnosis

  • Anti-nuclear Antibodies
  • Genetic marker/SNP

AML example from Paul Wallace

49

slide-50
SLIDE 50

Impact of Type of Data on Validation Design

Validation Parameter Definitive Quantitative Relative Quantitative Quasi-Quantitative Qualitative Accuracy √ √

  • Precision

√ √ √

  • Sensitivity

√ LLOQ √ LLOQ √ √ Specificity √ √ √ √ Dilutional Linearity √ √

  • Matrix Stability

√ √ √ √

Bioanalytical Data Category vs Validation Parameter

Lee et al. Pharmaceutical Research, 22:499, 2005

50

slide-51
SLIDE 51

Assay Risk Categories

Clinical risk Purpose / Intended use of assay Low  Basic research assay  Drug discovery assay  Clinical trial biomarker assay (exploratory end point) Moderate  Laboratory developed test used as an aid to diagnosis  Clinical trial biomarker assay (secondary endpoint) High  Clinical trial biomarker assay (primary endpoint)  Clinical trial biomarker assay (enrollment criteria endpoint)  Complementary diagnostic  Combination product / Companion diagnostic

slide-52
SLIDE 52

Want More Details???

Validation the Key to Translatable Flow Cytometry a Three Part Series: cytou.org

Method Validation Overview, Concepts 13 August 2018

slide-53
SLIDE 53

Chapter 6 Outline

Chapter 6: Analytical Method Validation

6.1 Validation Planning Phase (Say It!) 6.1.1 Validation Plan 6.1.1.1 Acceptance Criteria 6.1.2 Quantitative Data/ Methods 6.1.3 Qualitative Data/ Methods 6.2 Validation Execution (Do It!) 6.3 Validation Reports (Prove It!) 6.4 Fit-for-Purpose Validation Plans

slide-54
SLIDE 54

Quantitative Data/ Methods

  • Accuracy/Trueness
  • Specificity and Selectivity
  • Sensitivity
  • Sensitivity – Analytical (LOB/LOD)
  • Sensitivity - Functional (LLOQ)
  • Precision
  • Experimental Design
  • Precision acceptance criteria
  • Linearity
  • Linearity for Relative Quantitative Data
  • Linearity for Receptor Occupancy (RO) Assays
  • Linearity for Quasi-Quantitative Data
  • Stability
  • Specimen Stability
  • Processed Sample Stability
  • Assay carryover (instrument)
  • Reference Intervals
slide-55
SLIDE 55

Qualitative Data/ Methods

  • Accuracy/Trueness
  • Specificity
  • Sensitivity
  • Precision
  • Stability
  • Assay carryover (instrument)
  • Reference Intervals
slide-56
SLIDE 56

Validation Test Menus

*International Medical Device Regulatory Forum

Regulatory Setting Intended Use of Data Assay Type Validation Recommendation Non-regulated Basic research Novel assay Fit-for-Purpose Validation Level 1 Non-regulated Drug discovery Novel assay Fit-for-Purpose Validation Level 1 Non-regulated Exploratory endpoint in clinical development Novel assay Fit-for-Purpose Validation Level 1 Non-regulated Secondary endpoint in clinical development Novel assay Fit-for-Purpose Validation Level 2 Clinical Laboratory Patient care/treatment IVD/CE Approved Kit Verification per CLIA Clinical Laboratory Patient care/treatment clinical risk LDT CLIA/IMDRF* Validation GLP??? Primary endpoint in clinical development Novel assay Full Validation Level 1 Manufacturing (GMP) Regulatory submission for new diagnostic test Novel assay Full Validation Level 2 Manufacturing (GMP) CDx Novel assay Full Validation Level 2

slide-57
SLIDE 57

Test Menu Structure

Parameter Comments Samples Replicates Analytical Runs Accuracy/Trueness Specificity Selectivity Sensitivity LOD LLOQ Precision Repeatability (Intra-assay) Reproducibility (Inter-assay) Inter-operator Inter-instrument Linearity Stability Specimen Processed Sample Carryover Reference Intervals Documentation Validation Plan Validation Report QA Review ✓ ✓ ✓ Parameter Comments Samples Replicates Analytical Runs Accuracy/Trueness Specificity Selectivity Sensitivity LOD LLOQ Precision Repeatability (Intra-assay) Reproducibility (Inter-assay) Inter-operator Inter-instrument Linearity Stability Specimen Processed Sample Carryover Reference Intervals Documentation Validation Plan Validation Report QA Review ✓ ✓ ✓

slide-58
SLIDE 58

Want More Details???

Validation the Key to Translatable Flow Cytometry a Three Part Series: cytou.org

Method Validation Planning and Execution 10 September 2018

slide-59
SLIDE 59

Additional Resources Guidelines for the use of flow cytometry and cell sorting in immunological studies

Volume47, Issue10 Special Issue: Featuring the Guidelines for the use of flow cytometry and cell sorting in immunological studies October 2017 Pages 1584-1797

slide-60
SLIDE 60

Additional Resources

Special Issue: Validation of Cell‐ ‐ ‐ ‐Based Fluorescence Assays: Practice Guidelines from the International Council for Standardization of Haematology and the International Clinical Cytometry Society Volume 84, Issue 5 Pages: 279-357 September/October 2013

slide-61
SLIDE 61

Additional Resources

  • Recommendations for the Validation of Flow Cytometric Testing During Drug

Development: I Instruments. JIM, 363:104-119, 2011.

  • Recommendations for the Validation of Flow Cytometric Testing During Drug

Development: II Assays. JIM 363:120-134, 2011.

  • Validation of Cell-Based Fluorescence Assays: Practice Guidelines from the

International Council for Standardization of Haematology and International Clinical Cytometry Society. Cytometry Part B: Clinical Cytometry Special Issue volume 84B:2013

  • Recommendations for the Evaluation of Specimen Stability for Flow Cytometric

Testing During Drug Development. JIM, 418:1, 2015.

  • Recommendations for the development and validation of flow cytometry-based

receptor occupancy assays. Special Issue: Receptor Occupancy by Flow Cytometry, Cytometry Part B: Clinical Cytometry, 90B; 141, 2016.

  • Best practices in Performing Flow Cytometry in a regulated environment: feedback

from experience within the EBF. Bioanalysis 9:1253, 2017.

  • ISAC CYTO University Webinars (available for download at cytou.org)
  • Validation, the Key to Translatable Flow Cytometry
  • Part 1: Method Validation- Overview, Concepts, August 13
  • Part 2: Planning and Executing, September 10
  • Part 3: Instrument Qualification, October 29
slide-62
SLIDE 62

Summary

  • Flow cytometric methods provide high quality, reliable

data

  • Validation parameters appropriate for soluble analytes

are not appropriate

  • Validation guidelines….coming soon!

62

slide-63
SLIDE 63

Acknowledgment

  • CLSI H62 Document Writing Committee
  • Teri Oldaker, Co-chair
  • Slide Sharing
  • Cherie Green, Genentech
  • Jennifer Stewart, FCS Labs
  • Ben Hedley, LHSC

63

slide-64
SLIDE 64

64 Figure courtesy of Ira Schieren, Columbia University

Questions?

64

slide-65
SLIDE 65

Contact Information Virginia Litwin, Ph.D. Vice President, Immunology Caprion Biosciences Inc. Montreal, Quebec Canada VLitwin@caprion.com WWW.CAPRION.COM LinkedIn: https://www.linkedin.com/in/virginia-litwin-99869511/

65