Data Management and Integrity Laboratory Practices Matt Davis and - - PowerPoint PPT Presentation

Data Management and Integrity

Laboratory Practices

Matt Davis and Gaye Camm Senior Inspectors, Manufacturing Quality Branch Medical Devices and Product Quality Division, TGA

21 November 2019

What is data integrity?

• Data Integrity is the extent to which data is:

– Complete – Consistent – Accurate

• Throughout the Data lifecycle:

– Initial generation and recording – Processing – Use – Retention, archiving, retrieval and destruction

• (PIC/S Good Practices for Data Management and Integrity PI 041)

1

Creating the right environment

• Data management controls embedded in PQS

– System design to ensure good DI practices – QRM approach to data integrity – Ongoing risk review data criticality/risk – Self Inspection

• Clear understanding of importance of data integrity at all

levels of the organisation

• Internal reporting is encouraged
• Mature, open management approach to data integrity

Rationalisation

2

Risk management approach to data integrity

Data Criticality Data Risk

• Data Criticality

– CQA Batch release data > cleaning records – Data relating to product quality/safety

• Data Risk

– Vulnerability of data to alteration, deletion, recreation, loss or deliberate falsification

• Outcome - Effective control strategy to

manage identified risks

3

ALCOA+ Principles

Attributable

• Clearly

indicates who recorded the data or performed the activity

• Signed /

dated

• Who wrote

it / when

Legible

• It must be

possible to read or interpret the data after it is recorded

• Permanent
• No

unexplained hieroglyphic s

• Properly

corrected if necessary

Contemporaneous

• Data must

be recorded at the time it was generated

• Close

proximity to

• ccurrence

Original

• Data must

be preserved in its unaltered state

• If not, why

not

• Certified

copies

Accurate

• Data must

correctly reflect the action /

• bservation

made

• Data

checked where necessary

• Modification

s explained if not self- evident

Plus

• Complete
• Consistent
• Enduring
• Available

4

Designing paper systems which reduce opportunities for falsification Attributable Legible / Permanent Contemporaneous Original Accurate

System design: documents in right place at right time, clocks on wall, control

• f blank forms

Signatures, Aliases; signature logs

Workbooks, forms controlled, verified ‘true copy’ scans Reflective of the

• bservation; Data

checking, raw data verification No pencil, white-

• ut, soluble ink,

SOP for corrections and archiving

5

Designing electronic systems which reduce opportunities for falsification Attributable Legible / Permanent Contemporaneous Original Accurate

Auto-saving; step- wise recording; System clock synchronisation User access control; e- signatures; metadata Metadata which permits reconstruction Data capture; manual data entry; source data & audit trail review Data security, audit trails; back- up; sys. validation

6

Overarching DMDI policy

• Procedures
• Audit Trails
• Secure data retention
• Time/date stamps
• Traceable
• Correct movement of data
• Employee adherence
• Period review
• Security
• Detecting non-compliance
• Validation
• Accurate and complete data
• Unique user names/password/biometrics
• Prevent unauthorised changes
• Independent review
• Detecting wrongful acts
• Training
• Control of outsourced activities
• Corrective actions

7

Analytical laboratories – Common concerns & controls

• DI expectations:

– Lab electronic systems – Data review processes – Audit review processes – Manual integration – Analyst training – Spreadsheet management – Test Injections

8

Laboratory electronic systems

E-signatures Audit trail review External calculation tools Raw data verification Data review SOPs

Data management

System administrator Defined user privileges Individual user access SOPs for user access control

User Access

Test method configuration Data back- up/archiving OS security Audit Trails

Configuration

Periodic system review Change management Configuration management Hardware qualification Software validation

Validation

9

Data review processes

• Consider electronic and paper-based records
• Clear SOP required for data review

– frequency, roles, responsibilities and approach to risk-based review of data (and metadata, including audit trails as relevant) – Ensure that the entire set of data is considered in the reported data, should include checks of all locations where data may have been stored, including locations where voided, deleted, invalid or rejected data may have been stored

• Who-When-What-How:

– Who collected/when was it collected/what was collected/how was the data collected? – Who, when, what, how……data processed? – Who, when, what, how…...data reviewed? – Who, when, what, how…....data reported?

10

Audit trail review processes

• Technical Controls to aid secondary review of Audit Trails

– Identifying data that has been changed or modified – Review by exception – only look for anomalous or unauthorized activities – By limiting permissions to change recipes/methods/parameters, or locking access to specific parameters or whole recipes/methods may negate the need to examine associated audit trails in detail as changes can be easily observed, restricted or prohibited – Whatever activities are left open to modification need to be checked

11

Audit trail reviews - Common issues

Reasons for change or deletion of GMP-relevant data not documented Annex 11 §9: …for change or deletion of GMP-relevant data the reason should be documented

• Method to record legitimate changes to data that needs to be considered

when doing audit trail review, or covered by SOP i.e. allowable changes to methods.

• Explanation for ALL data recorded (complete data) including results that

aren’t reported

• Deviations from standard procedures or atypical results should be

investigated

12

Reasons for changes to data…

May be valid reason for invalidating data and repeating acquisition e.g. equipment malfunction, incorrect sample/standard preparation, power failure

• Need to be recorded, investigated and potentially implement CAPA for invalid

runs, failures, repeats and other atypical data

• All data should be included in the dataset unless there is a documented

scientific explanation for their exclusion

• Possibly reviewed for trends at some timepoint
• May need a new SOP for laboratories in addition to standard OOS/OOT

procedures

13

Audit trail challenges

Use of paper copies for review of electronic systems Many laboratories create paper copies of electronic records in the form of reports and rely on conduct the audit trail review. These reports can be huge (> 80 pages per chromatographic run when audit trails are included) and the risk that critical notifications in the sea of data may be lost is a significant risk.

14

Recording of audit trail review

How do manufacturers demonstrate they have reviewed audit trails?

• Documentation of audit trail reviews should be performed in a

similar way to documenting any review process. Typically done by signing the results as ‘reviewed’ or ‘approved’, following a data review SOP.

• For electronic records, this is typically signified by

electronically signing the electronic data set that has been reviewed and approved

15

Recording of audit trail review

What about manufacturers who don’t have electronic signatures available?

• Hybrid approach, which is not the preferred approach, using

paper printouts of original electronic records – Requirements for original electronic records must be met. – To rely upon these printed summaries of results for future decision-making, a second person would have to review the

• riginal electronic data and any relevant metadata such as

audit trails, to verify that the printed summary is representative.

• It seems unreasonable to require specific evidence of exactly which records

and metadata were looked at or opened (this would constitute an audit trail of the audit trail review)

16

Manual integration controls

• Automatic integration should be default manual only

where absolutely required

• SOP for integration required

– Define methods that can and cannot be adjusted – Document which actions are permissible, or restrict access to only allow the desirable actions – Document both original and manually integrated chromatograms – Electronic signatures/audit trail for manually integrated peaks

• Review results to ensure compliance

– Review reported data against electronic raw data (including audit trails)

17

Staff Training

• Essential that reviewers have a good knowledge
• f how the computer system or software works, and

how the audit trail is designed and works together with the data

• This may require specific training in evaluating the configuration settings and

reviewing electronic data and metadata, such as audit trails, for individual computerized systems used in the generation, processing and reporting of data.

• Include training on system vulnerabilities such as overwritten or obscured

through the use of hidden fields or data annotation tools

18

Excel spreadsheets

Spreadsheets for managing and presenting data, and their versatility and ease of use has led to wide application. When data contained within the spreadsheet cannot be reconstructed elsewhere and is essential to GMP activity then the data governance measures need to be rigorous.

Issues

• If the spreadsheet has multiple users it may be impossible to ascertain who (Attributable)

made an entry, whether entries have been over-written and replaced (permanent), and when the data entries had been made (Contemporaneous).

• If the spreadsheet is not version-controlled and managed as a controlled document, then

there may be different versions in use (Original).

• Where formulae and other functions are used there is potential for these to be corrupted

without being detected (Accurate).

19

Test injections

• All chromatographic systems need to equilibrate before they are ready for analysis. The time taken will typically

depend on factors such as the complexity of the analysis, the age and condition of the column, and detector lamp warm-up time.

• Generally there will be an idea of how long this will be from the method

development/validation/verification/ transfer work performed in the laboratory and this should be documented in the analytical procedure.

• Prepare an independent reference solution of analyte(s) that will be used for the sole purpose of system evaluation.

The solution container label needs to be documented to GMP standards and clearly identified for the explicit purpose

• f evaluating if a chromatography system is ready for a specific analysis.
• The analytical procedure needs to allow the use of system evaluation injections. Staff need to be trained in the

procedure.

• Inject one aliquot from the evaluation solution and compare with the SST criteria. Clearly label the vial in the

sequence file as a system evaluation injection. If the SST criteria are met then the system is ready for the analysis.

• Upon completion of the analysis, document the number of system evaluation injections as part of the analytical report

for the run.

20

Microbiological laboratories – common concerns and controls

• General issues
• DI expectations

– Note: All previous comments regarding computerised systems apply – Microbiology may present a greater DI risk

21

General issues observed

Competence/supervision Lack of effective controls Secondary Checks Computerised system configuration Organisational Culture / resources

Manipulation of data

No testing conducted Not counting all colonies OOL data not being investigated Resampling/retesting without justification

Incomplete Testing

Samples not taken or “lost” in transit No reconciliation of samples Incubation conditions incorrect Using unvalidated test methods

Poor test records

Not recording all key test data Worksheets ripped up and replaced No reconciliation of forms used Lack of proper computerised system security Colony morphology not matching identification results

Contributing causes

22

DI controls – Manual test methods

Sampling Procedures

Sampling schedule/plans Training of technicians Sample forms Detailed collection methods Identity of sampler recorded

Test methods

Test volumes/weights recorded Calibrated equipment used Reference to all reagents Reference to validated methods/dilution factors Samples processed under clean conditions, e.g. LAF Negative controls for processed samples Identity of tester/equipment recorded

Incubation

Incubation records maintained Min/max incubation time defined and validated All transfers/sub-culturing recorded All incubated samples tagged and identified

Reading results

Technicians trained in detection, enumeration and morphology – clear SOPs, photos Controlled environment for reading, light, magnification Counting device used for colonies Clear acceptance criteria/limits OOL & ID policy for manual recording All samples reconciled Results recorded Calculations applied correctly Second checks and verification in accordance with quality risk management

23

Summary

• Review DMDI guidance and TGA policy
• Develop DMDI policy for your organisation
• Risk based approach to systems and data

– Data criticality – Review capabilities of electronic systems

• Incorporate DMDI controls into QMS (and review!)

24

Where are you now?

• Do not know about the issue

and unaware of the gap UNCONSCIOUS INCOMPETENCE

• Aware of the gap but not

yet able to deal with it CONSCIOUS INCOMPETENCE

• Getting a handle on the

problem but only with effort CONSCIOUS COMPETENCE

• Good practice becomes

automatic UNCONSCIOUS COMPETENCE

25