Hospital Acquired Complications Paediatric Risk Adjustment HISA - - PowerPoint PPT Presentation

Hospital Acquired Complications

Paediatric Risk Adjustment HISA Health Data Analytics Conference – October 2018

What are HACs?

• Hospital Acquired Complications

(HACs) refers to a national list of 16 complications developed by the Australian Commission on Safety and Quality in Health Care.

• Defined based on clinically coded

admitted patient data and contribute to a funding reduction per episode if a HAC occurs.

What is risk adjustment?

• Risk adjustment refers to recognising that there are patient related

characteristics that will increase the likelihood of a HAC occurring and adjusting the funding impact accordingly (but not to zero).

Adjusting the funding impact of a HAC

• These factors are used to assign a complexity score between 0 and 100 –

this then categorises an episode as low, moderate or high complexity.

• Patients that are moderate or high complexity have the adjustment

“dampened” and hence receive a smaller NWAU adjustment.

Paediatric Risk Adjustment – Some Issues

• Are complex paediatric patients sufficiently risk adjusted?
• Is the Charlson Score – developed based on 1 year mortality rates in a

largely adult population of 607 patients from New York Hospital in 1984 – the best approach in predicting the likelihood of a HAC

• ccurring in paediatric populations?
• Are the current age groupings reflective of the range of paediatric patients?
• What about neonates? Does the likelihood of a HAC differ between a

newborn and a toddler?

Key Questions

• Are paediatric equivalents of the Charlson score better predictors of a HAC
• ccurring in a paediatric population?
• Does the use of more granular age groups improve performance when

predicting the likelihood of a HAC in a paediatric population?

• IHPA raise valid concerns regarding the relatively small volume of paediatric

data (and paediatric HACs) in the national dataset – could the use of some ML techniques (cross validation, bootstrap resampling and synthetic

• versampling) provide some validation of the robustness of these subset

models?

The Charlson Score

Condition Charlson Score Myocardial Infarction 1 Congestive Heart Failure Peripheral Vascular Disease Cerebrovascular Disease Dementia Chronic Pulmonary Disease Connective Tissue Disease-Rheumatic Disease Peptic Ulcer Disease Mild Liver Disease Diabetes without complications Paraplegia and Hemiplegia 2 Renal Disease Diabetes with complications Cancer Moderate or Severe Liver Disease 3 Metastatic Carcinoma 6 AIDS/HIV

• The Charlson Score is calculated by

adding together the scores of any conditions present in the table shown.

• For each decade of age over 40, one is

added to the Charlson score.

• Example: a 65 year old patient with

Dementia and Severe Liver Disease has a Charlson score of 1 + 3 + 2 = 6

Dementia Liver Disease 60-69

The Tai Score

Condition Tai Score Agranulocytosis 1 Arrhythmia Coagulopathy Congenital subaortic stenosis Lung contusion Pyrexia Respiratory failure Septicemia Ventricular septal defect Acidosis 2 Candidiasis Developmental delay Feeding problem Head injury Hypertension Pneumonitis Stroke Asphyxia 3 Heart failure Leukaemia Shock Brain cancer 4 Diabetes insipidus

The Rhee Score

Condition Rhee Score Acute myocardial infarction 1 Aortic or peripheral arterial embolism or thrombosis Aortic, peripheral, visceral artery aneurysms/dissection Birth trauma Cardiac or circulatory congenital anomalies Chronic obstructive pulmonary disease/bronchiectasis Chronic renal failure Coagulation or hemorrhagic disorders Coronary atherosclerosis/other ischemic heart disease Cystic fibrosis Diabetes mellitus or complications Drowning/submersion Gastrointestinal hemorrhage Hepatic tumors Hepatitis Immunity disorders (except AIDS) Influenza Liver disease (eg, cirrhosis, increased LFTs) Meningitis, encephalitis, or other CNS infection Motor vehicle traffic Peri-/endo-/myocarditis, cardiomyopathy,or tamponade Condition Rhee Score Peritoneal or intestinal abscess, peritonitis 1 Primary malignant bone or articular cartilage tumors Primary malignant tumor of adrenal or paraganglia Pulmonary vascular disease (eg, PE,pulmonary HTN) Respiratory distress syndrome Respiratory failure, insufficiency, arrest Septicemia (except in labor) Shock Short gestation, low birth wt, or fetal growth retardation Soft tissue sarcomas Suffocation Systemic lupus erythematosus or connective tissue disorder Thyroid disorders or other endocrine disorders Acute cerebrovascular disease 2 Acute renal failure CNS or miscellaneous intracranial or intraspinal neoplasms Coma, stupor, or brain damage Crushing injury or internal injury Firearm HIV infection Hypoxia, asphyxia, or aspiration during birth Leukemia Lymphomas or reticuloendothelial neoplasms Poisoning by nonmedicinal substances Suicide or intentional self-inflicted injury Cardiac arrest or ventricular fibrillation or flutter 3 Intracranial injury

Age Adjustment

• The current risk adjustment model accounts for the paediatric population

through the inclusion of age as a risk adjustor.

• 5 year age brackets 0-4, 5-9, 10-14, 15-18
• Majority of complexity score contributions are 0 or negative
• For some HACs, children essentially grouped with adults (e.g. GI

bleeding 0-24)

The Data

• Analysis dataset:
• 237,464 inpatient episodes from SCHN (153,425) and LCCH (84,039)

for patients discharged between 01/07/2015 and 30/06/2017

• Diagnosis information available to flag HACs and assign comorbidity

scores

• 223,458 HAC in scope episodes (excludes same day chemotherapy,

transfers etc.)

• 223,458 used in comorbidity modelling
• 148,898 used for age modelling – SCHN data only (to drill down to

age in months for first year of life) and excluded a very small number

• f episodes with patients aged 20 and above.

Methodology

• Single variable logistic regressions used as

method to fit models

• Response variable is episode has HAC v No

HAC

• HAC02, HAC08, HAC12 excluded (< 25 in

sample)

• Dependent variable is a comorbidity score

(Charlson v Tai v Rhee) or age groups (5 year v 1 year (< 5 yo) v 6 months (<1 yo))

• Separate logistic regression per HAC per

dependent variable

• The area under the ROC curve (AUROC) used

to evaluate model performance

Cross Validation

• Cross validation used to confirm findings are valid in out of sample predictions
• 10 fold cross validation used
• Stratified folds – i.e. proportions of response and dependent variables

preserved in each fold

. . .

AUC1 AUC2 AUC10 Average AUC

. . .

TRAINING DATASET TESTING DATASET

Over Sampling for Class Imbalance

• The number of episodes with a HAC are extremely small – less than 1%. Such class

imbalance can be addressed through over sampling methods to investigate the impact on model fitting.

• Two over sampling techniques considered:
• Bootstrap
• Additional HAC episodes are generated by resampling data with replacement
• SMOTE (Synthetic Minority Over-Sampling Technique)
• Additional HAC episodes are generated by creating new data points
• The SMOTE algorithm adopts a k-nearest neighbours approach in the minority class
• Synthetic points are determined as a randomly selected point between an
• bservation and one of it’s k-nearest neighbours

Over Sampling with Cross Validation

• The over sampling process is applied in each fold only with the training

dataset. CV TRAINING DATASET TESTING DATASET

NO HAC EPISODES

HAC EPISODES

NO HAC EPISODES OVER SAMPLED HAC EPISODES

NEW TRAINING DATASET

BOOTSTRAP / SMOTE

Results

• In the comorbidity model, full sample results were consistent across all validation

conditions (cross validation, bootstrap and SMOTE).

• The Rhee score was the best performing comorbidity score for all but one of

the HACs modelled (Tai score was the best performing for remaining HAC).

• The Charlson score was the worst performing comorbidity score for all but one
• f the HACs modelled.
• In the age model, full sample results indicated that 6 month age groups (< 1 yo) was

the best performing age grouping.

• However, results were split between 1 year age groups (< 5 yo) and 6 month

age groups (< 1 yo) under the other modelling conditions.

• In all conditions, 5 year age groups was the worse performing predictor.

Any HAC – Full Sample Comorbidity Score Models

Any HAC – Full Sample Age Group Models

Cross Validated AUC – Comorbidity Scores

Charlson Tai Rhee Charlson Tai Rhee AnyHAC 0.601 0.662 0.698 0.600 0.661 0.700 HAC01 0.580 0.674 0.702 0.579 0.673 0.705 HAC03 0.619 0.683 0.710 0.619 0.683 0.711 HAC04 0.576 0.590 0.724 0.575 0.590 0.727 HAC06 0.582 0.737 0.699 0.581 0.736 0.700 HAC07 0.586 0.643 0.865 0.588 0.638 0.865 HAC09 0.696 0.697 0.721 0.696 0.697 0.722 HAC10 0.627 0.616 0.705 0.626 0.614 0.706 HAC11 0.620 0.703 0.704 0.619 0.703 0.703 HAC13 0.653 0.673 0.759 0.651 0.670 0.758 HAC14 0.568 0.611 0.726 0.565 0.610 0.727 Entire Sample AUC CV Average AUC

Cross Validated AUC – Age Groups

5 year bands 1 year bands (< 5yo) 6 mth bands (< 1 yo) 5 year bands 1 year bands (< 5yo) 6 mth bands (< 1 yo) AnyHAC 0.555 0.605 0.610 0.556 0.602 0.604 HAC01 0.576 0.643 0.659 0.574 0.601 0.632 HAC03 0.554 0.595 0.599 0.548 0.576 0.578 HAC04 0.638 0.742 0.753 0.631 0.727 0.737 HAC06 0.523 0.602 0.606 0.448 0.538 0.523 HAC07 0.676 0.776 0.778 0.664 0.746 0.732 HAC09 0.575 0.640 0.644 0.546 0.617 0.611 HAC10 0.617 0.660 0.665 0.616 0.625 0.627 HAC11 0.638 0.670 0.670 0.620 0.634 0.636 HAC13 0.547 0.594 0.595 0.489 0.507 0.490 HAC14 0.617 0.679 0.685 0.616 0.665 0.671 Entire Sample AUC CV Average AUC

Over sampling AUC – Comorbidity Scores

Charlson Tai Rhee Charlson Tai Rhee AnyHAC 0.600 0.661 0.700 0.600 0.661 0.700 HAC01 0.574 0.674 0.703 0.569 0.663 0.703 HAC03 0.619 0.683 0.711 0.618 0.683 0.711 HAC04 0.573 0.580 0.720 0.564 0.578 0.719 HAC06 0.570 0.733 0.689 0.575 0.726 0.683 HAC07 0.568 0.617 0.865 0.559 0.593 0.852 HAC09 0.696 0.698 0.711 0.682 0.684 0.700 HAC10 0.619 0.608 0.703 0.617 0.601 0.702 HAC11 0.611 0.695 0.687 0.612 0.671 0.687 HAC13 0.652 0.660 0.753 0.630 0.659 0.731 HAC14 0.567 0.609 0.727 0.568 0.599 0.727 Bootstrap SMOTE

Over sampling AUC – Age Groups

5 year bands 1 year bands (< 5yo) 6 mth bands (< 1 yo) 5 year bands 1 year bands (< 5yo) 6 mth bands (< 1 yo) AnyHAC 0.556 0.602 0.604 0.556 0.604 0.606 HAC01 0.574 0.604 0.633 0.574 0.602 0.625 HAC03 0.548 0.578 0.581 0.548 0.578 0.580 HAC04 0.630 0.730 0.740 0.631 0.730 0.741 HAC06 0.470 0.540 0.520 0.470 0.543 0.527 HAC07 0.673 0.718 0.709 0.664 0.709 0.709 HAC09 0.564 0.615 0.612 0.564 0.621 0.609 HAC10 0.616 0.628 0.628 0.616 0.628 0.628 HAC11 0.634 0.652 0.649 0.602 0.650 0.644 HAC13 0.497 0.523 0.498 0.487 0.535 0.503 HAC14 0.618 0.665 0.671 0.618 0.670 0.675 Bootstrap SMOTE

Next steps

• Further refinement and review of the ICD code list used in calculating the various

comorbidity scores. This includes review from clinical coding teams to validate mapping

• f different versions (over time and across countries) to ICD10V9 AM / ICD10V10 AM

and review by medical staff to ensure that these scores are clinically meaningful.

• Replicating this analysis on a broader paediatric population. It will be interesting to see

the performance of the comorbidity scores for a population of children outside of specialist / tertiary paediatric facilities.

• Assessing the performance of these comorbidity scores in the full national risk

adjustment model to see if the other risk factors account for the explanatory power suggested by these results.

Appendix

Comorbidity Score References

Paediatric Alternatives for Comorbidity Scoring

• Whilst the Charlson score is commonly used as a tool for assessing comorbidity in

adult populations, similar approaches have been considered using paediatric populations to develop comorbidity scores for children

• Two such approaches that have been considered are:
• Tai D, Dick, P & To, T et al. 2006, Development of Paediatric Comorbidity

Prediction Model, University of Toronto and the Research Institute, The Hospital for Sick Children, Toronto

• Rhee D, Salazar, JH & Zhang, Y et al. 2013, A Novel Multispecialty Surgical

Risk Score for Children, John Hopkins University School of Medicine, Baltimore

• For simplicity, these will be referred to as the Tai score and the Rhee score

respectively.

The Tai Score

• 339, 077 hospital discharges from April 1, 1991 to March 31, 2002
• Population consisted of children aged between 1 and 14 in Ontario, Canada
• Logistic regression used to predict 1 year mortality post discharge
• For comparison with the Charlson score, an integer score for each condition

has been assigned based on rounding the regression coefficients in the study (adopted approach from Rhee et al.)

The Rhee Score

• 2,087,915 patients aged under 18 years that underwent an inpatient surgical

procedure between 1988 and 2006

• Two national data sources from the US:
• The National (Nationwide) Inpatient Sample (NIS) – 1988 to 2005
• The Kid’s Inpatient Database (KID) – 2006 and validation sets
• Logistic regression used to predict in-hospital mortality
• For comparison with the Charlson score, an integer score for each condition

has been assigned based on rounding the regression coefficients in the study.

• The Rhee score also adds 1 for patients under 24 months of age

Acknowledgements

• I wish to acknowledge the assistance provided by the following people
• Stuart Bowhay (Children’s Health Queensland) for providing data for

Lady Cilento.

• Eui-soo Choi and Michael Man (NSW Health) for providing details on

calculation of the Charlson score

• Sally Chung (SCHN) for collating the ICD code list for the Rhee

comorbidity score