Hospital Acquired Complications Paediatric Risk Adjustment HISA - PowerPoint PPT Presentation

Hospital Acquired Complications Paediatric Risk Adjustment HISA Health Data Analytics Conference – October 2018

What are HACs? • Hospital Acquired Complications (HACs) refers to a national list of 16 complications developed by the Australian Commission on Safety and Quality in Health Care. • Defined based on clinically coded admitted patient data and contribute to a funding reduction per episode if a HAC occurs.

What is risk adjustment? • Risk adjustment refers to recognising that there are patient related characteristics that will increase the likelihood of a HAC occurring and adjusting the funding impact accordingly (but not to zero).

Adjusting the funding impact of a HAC • These factors are used to assign a complexity score between 0 and 100 – this then categorises an episode as low, moderate or high complexity. • Patients that are moderate or high complexity have the adjustment “dampened” and hence receive a smaller NWAU adjustment.

Paediatric Risk Adjustment – Some Issues • Are complex paediatric patients sufficiently risk adjusted? • Is the Charlson Score – developed based on 1 year mortality rates in a largely adult population of 607 patients from New York Hospital in 1984 – the best approach in predicting the likelihood of a HAC occurring in paediatric populations? • Are the current age groupings reflective of the range of paediatric patients? • What about neonates? Does the likelihood of a HAC differ between a newborn and a toddler?

Key Questions • Are paediatric equivalents of the Charlson score better predictors of a HAC occurring in a paediatric population? • Does the use of more granular age groups improve performance when predicting the likelihood of a HAC in a paediatric population? • IHPA raise valid concerns regarding the relatively small volume of paediatric data (and paediatric HACs) in the national dataset – could the use of some ML techniques (cross validation, bootstrap resampling and synthetic oversampling) provide some validation of the robustness of these subset models?

The Charlson Score • The Charlson Score is calculated by Condition Charlson Score Myocardial Infarction adding together the scores of any Congestive Heart Failure conditions present in the table shown. Peripheral Vascular Disease Cerebrovascular Disease • For each decade of age over 40, one is Dementia 1 Chronic Pulmonary Disease added to the Charlson score. Connective Tissue Disease-Rheumatic Disease • Example: a 65 year old patient with Peptic Ulcer Disease Dementia and Severe Liver Disease has Mild Liver Disease Diabetes without complications a Charlson score of Paraplegia and Hemiplegia Renal Disease 60-69 2 Diabetes with complications Dementia Cancer 1 + 3 + 2 = 6 Moderate or Severe Liver Disease 3 Metastatic Carcinoma 6 AIDS/HIV Liver Disease

The Tai Score Condition Tai Score Agranulocytosis Arrhythmia Coagulopathy Congenital subaortic stenosis Lung contusion 1 Pyrexia Respiratory failure Septicemia Ventricular septal defect Acidosis Candidiasis Developmental delay Feeding problem 2 Head injury Hypertension Pneumonitis Stroke Asphyxia Heart failure 3 Leukaemia Shock Brain cancer 4 Diabetes insipidus

The Rhee Score Condition Rhee Score Condition Rhee Score Acute myocardial infarction Peritoneal or intestinal abscess, peritonitis Aortic or peripheral arterial embolism or thrombosis Primary malignant bone or articular cartilage tumors Aortic, peripheral, visceral artery aneurysms/dissection Primary malignant tumor of adrenal or paraganglia Birth trauma Pulmonary vascular disease (eg, PE,pulmonary HTN) Cardiac or circulatory congenital anomalies Respiratory distress syndrome Chronic obstructive pulmonary disease/bronchiectasis Respiratory failure, insufficiency, arrest Chronic renal failure Septicemia (except in labor) 1 Coagulation or hemorrhagic disorders Shock Coronary atherosclerosis/other ischemic heart disease Short gestation, low birth wt, or fetal growth retardation Cystic fibrosis Soft tissue sarcomas Diabetes mellitus or complications Suffocation 1 Drowning/submersion Systemic lupus erythematosus or connective tissue disorder Gastrointestinal hemorrhage Thyroid disorders or other endocrine disorders Hepatic tumors Acute cerebrovascular disease Hepatitis Acute renal failure Immunity disorders (except AIDS) CNS or miscellaneous intracranial or intraspinal neoplasms Influenza Coma, stupor, or brain damage Liver disease (eg, cirrhosis, increased LFTs) Crushing injury or internal injury Meningitis, encephalitis, or other CNS infection Firearm 2 Motor vehicle traffic HIV infection Peri-/endo-/myocarditis, cardiomyopathy,or Hypoxia, asphyxia, or aspiration during birth tamponade Leukemia Lymphomas or reticuloendothelial neoplasms Poisoning by nonmedicinal substances Suicide or intentional self-inflicted injury Cardiac arrest or ventricular fibrillation or flutter 3 Intracranial injury

Age Adjustment • The current risk adjustment model accounts for the paediatric population through the inclusion of age as a risk adjustor. • 5 year age brackets 0-4, 5-9, 10-14, 15-18 • Majority of complexity score contributions are 0 or negative • For some HACs, children essentially grouped with adults (e.g. GI bleeding 0-24)

The Data • Analysis dataset: • 237,464 inpatient episodes from SCHN (153,425) and LCCH (84,039) for patients discharged between 01/07/2015 and 30/06/2017 • Diagnosis information available to flag HACs and assign comorbidity scores • 223,458 HAC in scope episodes (excludes same day chemotherapy, transfers etc.) • 223,458 used in comorbidity modelling • 148,898 used for age modelling – SCHN data only (to drill down to age in months for first year of life) and excluded a very small number of episodes with patients aged 20 and above.

Methodology • Single variable logistic regressions used as method to fit models • Response variable is episode has HAC v No HAC • HAC02, HAC08, HAC12 excluded (< 25 in sample) • Dependent variable is a comorbidity score (Charlson v Tai v Rhee) or age groups (5 year v 1 year (< 5 yo) v 6 months (<1 yo)) • Separate logistic regression per HAC per dependent variable • The area under the ROC curve (AUROC) used to evaluate model performance

Cross Validation • Cross validation used to confirm findings are valid in out of sample predictions • 10 fold cross validation used • Stratified folds – i.e. proportions of response and dependent variables preserved in each fold TESTING DATASET TRAINING DATASET AUC 1 AUC 2 . Average . . . AUC . . AUC 10

Over Sampling for Class Imbalance • The number of episodes with a HAC are extremely small – less than 1%. Such class imbalance can be addressed through over sampling methods to investigate the impact on model fitting. • Two over sampling techniques considered: • Bootstrap • Additional HAC episodes are generated by resampling data with replacement • SMOTE (Synthetic Minority Over-Sampling Technique) • Additional HAC episodes are generated by creating new data points • The SMOTE algorithm adopts a k-nearest neighbours approach in the minority class • Synthetic points are determined as a randomly selected point between an observation and one of it’s k -nearest neighbours

Over Sampling with Cross Validation • The over sampling process is applied in each fold only with the training dataset. TESTING DATASET CV TRAINING DATASET NO HAC EPISODES HAC EPISODES BOOTSTRAP / SMOTE NO HAC EPISODES OVER SAMPLED HAC EPISODES NEW TRAINING DATASET

Results • In the comorbidity model, full sample results were consistent across all validation conditions (cross validation, bootstrap and SMOTE). • The Rhee score was the best performing comorbidity score for all but one of the HACs modelled (Tai score was the best performing for remaining HAC). • The Charlson score was the worst performing comorbidity score for all but one of the HACs modelled. • In the age model, full sample results indicated that 6 month age groups (< 1 yo) was the best performing age grouping. • However, results were split between 1 year age groups (< 5 yo) and 6 month age groups (< 1 yo) under the other modelling conditions. • In all conditions, 5 year age groups was the worse performing predictor.

Any HAC – Full Sample Comorbidity Score Models

Any HAC – Full Sample Age Group Models

Cross Validated AUC – Comorbidity Scores Entire Sample AUC CV Average AUC Charlson Tai Rhee Charlson Tai Rhee AnyHAC 0.601 0.662 0.698 0.600 0.661 0.700 HAC01 0.580 0.674 0.702 0.579 0.673 0.705 HAC03 0.619 0.683 0.710 0.619 0.683 0.711 HAC04 0.576 0.590 0.724 0.575 0.590 0.727 HAC06 0.582 0.737 0.699 0.581 0.736 0.700 HAC07 0.586 0.643 0.865 0.588 0.638 0.865 HAC09 0.696 0.697 0.721 0.696 0.697 0.722 HAC10 0.627 0.616 0.705 0.626 0.614 0.706 HAC11 0.620 0.703 0.704 0.619 0.703 0.703 HAC13 0.653 0.673 0.759 0.651 0.670 0.758 HAC14 0.568 0.611 0.726 0.565 0.610 0.727