efficiency A proof-of-concept in paediatric oncology Jane Shrapnel - - PowerPoint PPT Presentation

Harnessing data from electronic medical records to improve patient care and health efficiency

Jane Shrapnel Data Scientist, Sydney Children’s Hospital Network October, 2018

A proof-of-concept in paediatric oncology

A day in the life of a doctor: the technical story

Doctors are spending a small amount

• f time with patients:
• 1 and half hours in rounds
• 1 hour with patients

Mamykina, L et al. ‘How Do Residents Spend Their Shift Time? A Time and Motion Study With a Particular Focus on the Use of Computers’, Acad Med 2016 Definition of medical doctor from the International Labour Organisation

Role of a doctor is to apply the principles and procedures of medicine to prevent, diagnose, care for and treat patients with illness, disease and injury and to maintain physical and mental health. 6 hours a day are spent using computers

• 2 hours updating eMR, including

note writing

• 1 hour looking up and assessing

patient data

365 148 207 Other Patient Computer

Minutes % of day

29% 21% 51%

How can we harness the value of this data while assisting clinicians to provide high levels of care?

Three problems to solve:

1 2 3

Utilise the vast quantities of information to attain value through improved patient care and increased health efficiency Have real-time access to all data within the electronic medical records without impacting the production system Display these insight in the eMR to inform clinicians decision making at the right time in their workflow

Real-time access to electronic medical records

Benefits:

• Large quantities of

data can accessed any time as no impact of production performance

• In real-time
• Apply complex

queries on structured and unstructured data

Product- ion Mirrored Copy

2

eMR Data

Clinicians enters blood test results at 12:01pm Blood results are updated at 12:01pm

How can we harness the value of this data while assisting clinicians to provide high levels of care?

Three problems to solve:

1 2 3

Utilise the vast quantities of information to attain value through improved patient care and increased health efficiency Have real-time access to all data within the electronic medical records without impacting the production system Display these insight in the eMR to inform clinicians decision making at the right time in their workflow

Harnessing value from the eMR is complex

The eMR database contains all the encounters that a patient has had with the hospital as well as unnecessarily information for analysis. This information includes:

• Recording vital parameters & diagnostic tests
• Treatments including procedures and medication prescribed
• All notes recorded for a patient
• Transactional information such as system changes and system rules

70% of this data is unstructured, in the form of images or text The structured data requires quality assessment and rules for cleaning and interpretation Given this complexity, how to we go about showing we can extract value?

2

Start with a proof of concept to show impact

Question: how can we improving the model of care for oncology patients that present with a fever and a low white blood cell count, known as febrile neutropenia?

• Validate internationally recognised clinical decision rules
• These patients are at a higher risk of adverse events resulting from a

serious infection as they are immune compromised

• However, only 9% have a serious infection so there is room to change

model of care to home care for those at low risk of infection

• Currently treatment means immediate antibiotics and at least 48 hours

in hospital

2

Potential Impact:

• Reduce costs
• Higher quality of life
• Patients and families spending less time in hospital
• Free up critical bed space
• 1. Haeusler GM, Thursky KA, Slavin MA et al. (2017) External validation of six paediatric fever and

neutropenia clinical decision rules. Pediatric Infectious Disease Journal 1

Internationally published risk stratification models: PICNICC and SPOG

PICNICC SPOG Outcome What the model is trying to predict Risk of microbiological defined infection Risk of adverse outcome Model performance Prediction rate of infection out of 10 patients Number of countries data used to develop model 15 2 Time period to predict

• utcome

Predicted at admission x Predicted within 24 hours of admission x

1 Phillips RS, Sung L, Amman, RA et al. (2016) Predicting microbiologically defined infection in febrile neutropenic episodes in children: global individual participants data multivariable meta-analysis. British Journal of Cancer 2 Ammann RA, Bodmer N, Hirt A et al. (2010) Predicting adverse events in children with fever and chemotherapy-induced neutropenia: the prospective multicentre SPOG 2003 FN study. J Clin Oncol 28(12) 3 Adverse outcomes include serious medical complication, microbiological defined infection, radiologically confirmed pneumonia 4 Haeusler GM, Thursky KA, Slavin MA et al. (2017) External validation of six paediatric fever and neutropenia clinical decision rules. Pediatric Infectious Disease Journal 5 Haeusler GM, Thursky KA et al. (2017) Predicting Infectious Complications in Children with Cancer: an external validation study. British Journal of Cancer

The Predicting Infectious Complications in Children with Cancer (PICNICC) study1 and the Swiss Paediatric Oncology Group (SPOG) study2 have developed multivariate regression models to predict if a patient is at risk of contracting an infection. The table below details the similarities and differences between the studies.

1 2

3

2

Additionally, current Australian research shows promising results. Validating seven validations rule in retrospective study showed:

• SPOG rule showed sensitivity of 78% and specificity of 46.3%4
• PICNICC showed sensitivity 78.4% and specificity of 39.8%5

PICNICC: The Australian predicting infectious complications in children with cancer project:

• Prospective observational study, over 8 paediatric hospitals sampling 845 children
• Validation of a large number of clinical decision rules

Defining the cohort in the eMR

Defining febrile neutropenia patients:

• Admitted to Oncology speciality
• Temperature of >= 38.0oC & neutrophil count < 1.0 X 109 cells/L within first 24 hours
• Visit reason includes reference to a ‘fever’ or ‘febrile’ in lieu of a high temperature

Final sample: Date range is from 3 April – 31 December 2017 for Children’s Hospital Westmead’s data1

• 202 encounters
• At admission: 155 encounters2
• At day 2: 129 encounters
• 98 patients

1 Children’s Hospital Westmead went live for online between the flags from 3 April 2017 2 If any predictor values were missing, these individuals were removed from the analysis

2

Source data to validate clinical decisions rules

Variable Quality

Source

SPOG

Criteria for inclusion in CDR

Neutropenia ANC of < 1.0 X 109 cells/L Medium Neutrophil count Fever of ≥ 38.0oC Medium Temperature

CDR

Preceding chemotherapy more intensive than ALL maintenance Unknown Named protocol Haemoglobin ≥90 G/L Medium Haemoglobin Total white cell count < 0.3 G/L Medium White Cell Count Platelet <50 G/L Medium Platelet count

PICNICC

CDR

Clinical description ’severely unwell' Unknown RR & clinical notes Total white cell count (X109/L) Medium White Cell Count Temperature (C) Medium Temperature Haemoglobin (g/L) Medium Haemoglobin Absolute monocyte count (X109 /L) Medium Monocyte count Malignancy type Low Active problems Microbiologically defined infection (MDI) Positive bacterial High Microbiology report

Quality scale Low – high amounts of missing data, high potential for human error Medium – low amounts of missing data, small to medium potential for human error High – No missing data, small potential for human error (DQ steps in place) Variable creation required

2

Categorising MDIs from pathology results required development of a classification algorithm

Microbiologically defined infection (MDI) is defined as an infection that is clinically detectable and microbiologically proven. Examples of what is in the report:

• Code 0 – No growth at 5 days.
• Code 1 - Staphylococcus epidermidis was isolated from the anaerobic bottle.

Probable contaminant . Aerobic bottle

• Code 2 - Two strains of Staphylococcus epidermidis was isolated from the

aerobic bottle.

• Code 3 – E.Coli was isolated

Categorisation Coded value No MDI Possible MDI, possible contaminant 1 Probable MDI 2 Confirmed MDI 3

2

Categorisation decision flow of MDI (bacteraemia)

MDI is defined as an infection that was clinically detectable and microbiologically proven. For this study the focus is on Bacteraemia which is defined as a recognised pathogen from >= 1 blood culture or common commensals from >= 2 blood cultures drawn on separate occasions

Categorised as 1: Possible MDI, possible infection Categorised as 2: Probably MDI

2

This categorisation reduces the hours a research nurse would spend reading the eMR and categorising the presence

• f an MDI

Category # Negative blood culture or unconfirmed MDI 42 Confirmed MDI 6

MDI categorisation was 93.8% accurate

Prospective PICNICC study

Category # No MDI 39 Possible MDI, possible contaminant 6 Probable MDI Confirmed MDI 3

eMR PICNICC study

MDI is defined as an infection that was clinically detectable and microbiologically proven. For this study the focus is on Bacteraemia which is defined as a recognised pathogen from >= 1 blood culture or common commensals from >= 2 blood cultures drawn on separate occasions

3 of the ‘possible MDI, possible contaminant’ where incorrectly coded and should be probable MDI

3 3

2

An example texts: Nursing Progress Note Free text <name> admitted to ward from OTC at 1500hrs.

gm- sepsis. Mildly tachycardic to 125

with fever Other observations remain within normal ranges IVTKVO via both lumens between medications

feeling unwell with fevers, states she

feels much better after paracetamol. PRN IV paracetamol changed to 4/24. Same given with good effect.

White lumen accessed using aseptic technique.

Mother at bedside. Nil complaints voiced 2

‘gm- sepsis’ ‘Aseptic technique’ Septic = 1

Patient is ‘severely unwell’

Algorithm for defining severely unwell

1 2 3 Steps

Identifying sepsis

PICNICC showed more favourable results for detecting high risk and low risk patients

Sensitivity: 65% Specificity: 31%

AE TRUE FALSE Predicted AE TRUE 19 79 FALSE 10 35 AE TRUE FALSE Predicted AE TRUE 19 84 FALSE 10 35

Sensitivity: 65% Specificity: 29%

At admission After 24 hours

Sensitivity: 81% Specificity: 41%

MDI TRUE FALSE Predicted MDI TRUE 22 60 FALSE 5 42 MDI TRUE FALSE Predicted MDI TRUE 21 81 FALSE 6 47

Sensitivity: 78% Specificity: 37%

PICNICC SPOG

The PICNICC model would send 48 of 129 patients (30%) to hospital in the home, with 5 having an MDI.

2

Retrospectively implementing this rule would have sent 44 patient to Hospital in the Home

237

29 MDIs

80

8 MDIs

157

21 MDIs

57

4 MDIs

44

1 MDI

13

3 MDIs

100

17 MDIs Total FN episodes 1 April 2017 – 30 April 2018 No morning blood results Eligible for rule to be applied Predicted high risk, not eligible to go home Eligible to be considered for HITH Don’t meet HITH criteria Meet HITH criteria*

*Some criteria cannot be verified easily by system HITH criteria developed by Peter Mac and Royal Children’s Hospital Total patients Patients remaining in hospital Patients going to HITH

2

Benefits of implementing this rule cover patient, family and hospital needs

• Costing saving to the hospital is $700,000 a year
• Average length of stay for the low risk group is 5.2 days. Treating patients at

home will free up half a bed day a year per patient

• These patients are often in isolation rooms so this will free up critical bed

space

• Lowers the risk of hospital acquired complications, particularly serious

infections

• Improved quality of life for patients and their families1

2

2 Cheng S, et al. (2011) Health-related quality of life anticipated with different management strategies for paediatric febrile neutropaenia, Br J Cancer 2011 105(5):606-11

How can we harness the value of this data while assisting clinicians to provide high levels of care?

Three problems to solve:

1 2 3

Utilise the vast quantities of information to attain value through improved patient care and increased health efficiency Have real-time access to all data within the electronic medical records without impacting the production system Display these insight in the eMR to inform clinicians decision making at the right time in their workflow

Implementing this rule in the eMR requires a near- real time solution

eMR PROD eMR Mirrored Copy eMR Analytics DWH ETL tool

Clinical decision tool

Analysis tool

3

Reasons for this design:

• 1. Code is code is

complex so could put strain

• n system if in

production

• 2. Written in

python, a well known language with high

• growth. It runs

efficiently and will have skills to maintain compared to CCL

Technical detail of these data flows to the eMR

ETL tool Python code to:

• Select FN

patients

• Create features

for model

• Run PICNICC

algorithm on FN patients Python code to identify new patients and variables – one row & predicted value Select

• ncology

patients admitted yesterday eMR Database Outcome

• f low or

high risk

• f

infection Store PROC New table HL7 messages to eMR

From eMR to new table is 10 mins

3

To be completed

Acknowledgements

Clinicians

• Dr Gabrielle Haeusler, Peter Mac, Infectious

Diseases Consultant

• Dr Luciano Dalla-Pozza, SCHN, HoD Oncology

CHW

• Meredith Faggotter, SCHN, EMR Oncology

Nurse

• Dr Alison Kesson, SCHN, HoD Infectious

Disease and Microbiology

• Dr Tracey O’Brien, SCHN, HoD Oncology SCH
• Renee Byrne, Oncology CNE SCH

eMR expertise

• Matthew Huynh, SCHN, eMR Developer
• Paul Ngo, Systems, SCHN, Data and Database

Manager

• Herman Phung, SCHN, Project Co-ordinator
• Micheline Maddaford, SCHN, Costing & Analysis

Manager Funding

• Paediatrio

Sponsorship

• Cheryl McCullagh, SCHN, Director Clinical

Integration

• Christine Fan, SCHN, Performance Unit

Manager Data Scientists/Engineers

• Dr Charmaine Tam, USYD Centre of

Translational Data Science, NHMRC Early Career Fellow

• Dr Aldo Saavedra, USYD Centre of

Translational Data Science, Senior Research Scientist

• Dr Hayim Dar, USYD Sydney Informatics Hub,

Research Engineer

• Matthew Strasiotto, USYD, Engineering

Student