Evidence for Use of the Surprise Question 1 A L V I N H . M O S S - - PDF document

1/23/2020 1

A L V I N H . M O S S , M D S E CT I O N S O F N E P H R O L O G Y A N D G E R I A TR I CS & P A L L I A TI V E M E D I CI N E D E P A R T M E N T O F M E D I CI N E W E S T V I R G I N I A U N I V E R S I TY M O R G A N T O W N , W V A M O S S @ H S C. W V U . E D U

Improving Care of the Seriously Ill:

Evidence for Use of the Surprise Question

No relevant financial relationships to disclose

1

The Patient Reluctantly Agreed to Dialysis

 81 yo frail widower c stage 5 CKD from hypertensive nephrosclerosis  Former heavy smoker with oxygen-dependent COPD  Severe PAD with claudication; oxycodone qhs for leg pain  eGFR dropped to 5 ml/ min but not uremic  Serum albumin 2.9 mg/ dL  Nephrologist urged dialysis with daughter’s agreement  Lived alone and valued his independence and freedom  Did not want to be tied down to a dialysis schedule 3X/ week  Started reluctantly to please his daughter  Would you be surprised if this patient died in the next year?

2

1/23/2020 2

Objectives

 Explain the value of the surprise question in clinical practice  Describe the surprise question and its role in prognostication,

shared decision-making and advance care planning

 Review the medical literature on outcomes with use of the

surprise question

 Suggest how to implement use of the surprise question to

improve care of patients with serious illness

3

Definition of Serious Illness

4

 one that carries a high risk of death over the course

• f a year, but cure may remain a possibility

 has a strong negative impact on one’s QOL and

functioning, independent of its impact on mortality

 highly burdensome to a person and his or her family Kelley AS. Defining “Serious Illness.” J Palliat Med 2014; 17: 985.

1/23/2020 3

Thinking about Serious Illness: Markers of Poor Prognosis in ESRD

• Age
• Comorbidity
• Functional impairment
• Frailty
• Cognitive impairment
• Malnutrition

PubMed Surprise Question articles per Year

1 1 1 1 2 1 1 4 1 2 2 1 1 2 4 1 1 1 4 3 3 4 4 3 4 4 3 3 5 4 10 7 4 8 10 14 11 19 11 3334 32 5 10 15 20 25 30 35 40 1976 1977 1978 1979 1980 1982 1983 1984 1985 1986 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 # articles Year

1/23/2020 4

The “Surprise Question” (SQ)

 Would I be surprised if this patient died in the next year?

 “No, I would not be surprised.”  “Yes, I would be surprised.”

 Variations 6-months to align with hospice criterion 30 days prior to discharge 2 years

7

Prom pts action

Problem the SQ Is Meant to Address

 Clinicians are inaccurate at prognostication.  Earlier referral to palliative care improves patient

• utcomes.

 Therefore…  Can we devise a simple, feasible, and effective

approach to make identification of patients with a serious illness and a poor prognosis routine and thereby aid shared decision-making and advance care planning discussions?

8

1/23/2020 5

Understanding the Surprise Question

 Allows clinicians to exercise skilled intuitive judgment

 if the environment is a sufficiently regular to be predictable  if there is an opportunity to learn these regularities through

experience (Kahneman D. Thinking, Fast and Slow , 2011)

 can be integrated with objective measures to improve accuracy

 Is NOT meant to be a tool to predict death  SQ may improve prognostic accuracy because it “allows

physicians to think in a new way about their patients.”

 The question prompts consideration of whether the

patient m ight be dying or is at risk of dying sooner.

9

Use of the Surprise Question

 Identify patients with a serious illness, those “who

have a greatly increased risk of mortality in the coming year.”

 A tool, trigger or prompt to identify patients “who

might benefit from palliative care.”

Moroni M. SQ in advanced cancer patients: A prospective study among GPs. Palliat Med 2014;28:959-964.

10

1/23/2020 6

Why is this important?

11

Three Key Steps in Innovation Assessment

• Identifying the target population
• Describing baseline performance
• Documenting the components of the evaluation

History of the Surprise Question (SQ)

 1998 Institute for Healthcare Improvement

 Collaborative-Improving Care at the End of Life  Franciscan Health System primary care clinics  6-fold increase in appropriate hospice referral Fewer hospital days More deaths outside of hospital More days in hospice Higher patient and family satisfaction with care

Pattison M. J Palliat Med 2001

12

1/23/2020 7

History of the SQ

 2001 Perspectives on Care at the End of Life.

 Lynn J. JAMA 2001;285:925-932.  Joanne Lynn books and presentations

 Moss AH. Dialysis patients. CJASN 2008  Moss AH. Cancer patients. J Palliat Med 2010  Subsequent patient populations studied

13

COPD Hospitalist patients Primary Care Surgery Heart failure Chronic kidney disease Neurology Emergency Department

Use of the SQ

 No differences found among physicians  WVU nephrologists and nurse practitioners equally

accurate

 Physicians are better at its use than experienced

nurses who are better than younger nurses

 Da Silva Gane M. Nephron Clin Pract 2013

 Higher sensitivity and odds ratios in sicker patients

 Advanced cancer  Intensive care 14

1/23/2020 8

Use of the SQ in Hemodialysis Patients N=147

 23% “No, I would not be surprised”

 Odds ratio of dying within 1 year =OR 3.5, P=.01  SQ identified sicker patients

Moss AH, et al. CJASN 2008

15

Variable No Yes P value Age (yrs) 72.5 64.5 .005 CCI-comorbidity 7.1 5.8 .004 Karnofsky-function 69.7 81.6 <.001 Albumin (g/ dL) 3.7 3.9 .046 1-Yr Mortality (%) 29.4 10.6 .032

16

 Boston Primary Care Clinics with 1,163 patients

Discipline SQ Response Dead Alive Total* Physician No 143 309 452 (31.6%) Yes 37 674 711 (5.2%) Total 180 983 1163 (15.5%)

Physicians classified 38.9% as “No” compared to 24.3% for nurses.

Lakin JR. J Gen Intern Med, 2019

1/23/2020 9

Terms to Evaluate SQ Performance

17

SQ response Died Alive Total SQ “No” a True Positive b False Positive a + b SQ “Yes” c False negative d True Negative c + d Total a + c b + d a + b + c+ d

• Sensitivity=a/ (a + c) probability of “No” response when patient died
• Specificity=d/ (b +d) probability of “Yes” response when patient alive
• Accuracy=a + d/ a+b+c+d overall probability that a patient will be correctly

classified by the SQ response

• Odds ratio=a/ b÷c/ d=a x d/ b x c odds that a patient who is SQ “No” will have

died compared to odds that a person who is SQ “Yes” will have died

Multivariate Logistic Regression with Outcome of Vital Status at 2 Years

18

Physician unadjusted OR 8.4 for death at 2 years for “No,” AUC=.74; Nurse OR 4.6 with AUC=0.67

1/23/2020 10

19

 “…

2-year SQ holds promise for identification of appropriate patients for serious illness conversations in the primary care setting.”

 “Future work should focus on pairing it with

appropriate analytical tools and

 Studying the effect of its use on conducting

discussions [goals of care conversations.]”

Use of the SQ in Cancer Patients

 231 stage 4 cancer patients; 54.5% “No”  42 general practitioners  1-yr mortality 45.8%  68.5 vs 16.2% (P<.001) “No” vs “Yes”  OR death within 1 year=11.6

Moroni M et al. Palliat Med 2014

20

1/23/2020 11

21

• Overall pooled accuracy was 75%.
• SQ may detect as many “false positives” as “true positives.”
• Not clear whether the SQ is a cost-effective way to identify

patients suitable for palliative care.

• Combination of SQ with other prognostic measures may be

more accurate than the SQ alone.

22

CMAJ 2017 April 3;189:E484‐93.

• For 17 cohorts, median incidence of death was 15.1%; AUC=.81
• Pooled results show poor-modest accuracy of SQ-12: 67% sensitivity

(95% CI, 55.7-76.7), 80.2% specificity (95% CI, 73.3-85.6%), and 37.1% positive predictive value (95% CI, 30.2-44.6).

• Possible that a false positive SQ response would be beneficial; ASCO

recommends early palliative care.

• Further studies will be needed to determine whether SQ with other

clinical indicators improves identification of patients appropriate for palliative care.

1/23/2020 12

Summary Thus Far

 SQ increases odds of identifying seriously ill patients.  Recent metaanalyses showed that it does not

perform well by itself as a predictive tool for death (White, BMC Med 2017;Downar, CMAJ 2017) .

 Despite its limitations the SQ has been found to

contribute to estimates of mortality over and above

• ther factors such as age and comorbidities (Lakin

JAMA Intern Med 2016).

 Might incorporating the SQ into an integrated

prognostic model improve the accuracy of predictions?

23

Using the SQ in the MICU

24

2017;14(10):1556–1561. PubMed: 28598196

1/23/2020 13

Integrated Prognostic Model for MICU Patients Using the SQ

25

 An integrated patient-specific prognostic model to

identify MICU patients with a predicted high 6- month mortality could guide shared decision-making and appropriate palliative care consultation.

 Currently validated ICU predictive scoring systems

are valuable to compare patients in clinical trials or to assess quality of care but not to assist in individual patient care.

Derivation study

 An integrated prediction model was established in MICU

population.

 A prospective cohort of consecutive 500 patients from Nov

2013 to April 2014.

 Odds of dying within 6-months for those for whom the SQ

was answered "No, I would not be surprised" were 7.29 times greater than those for whom the response was "Yes"

(95% CI; 4.515-11.770, P<0.001).  ROC or C-statistic was 0.832 (95% CI; 0.795-0.870, P<0.001).

26

1/23/2020 14 Multivariate regression model for the derivation cohort (N=500) Variables Odds Ratio P-value CCI score 1.094 0.033 APACHE III 1.021 <0.001 SQ, “No” vs “Yes” response 7.290 <0.001

APACHE: Acute Physiology and Chronic Health Evaluation; SQ: Surprise Question

27

Validation Cohort Methods

 Prospective cohort of 549 consecutive adult MICU patients  Time frame for validation study: November 2014 - May 2015  The protocol was approved by IRB  Data collection at the time of admission

 Staff intensivists’ (n=7) response to the SQ  Demographics: Age, Gender, BMI  APACHE III score  Charlson Comorbidity Index (CCI) score  Use of mechanical ventilation and/ or vasopressors

 Prospectively followed for survival status (dead or alive) at 6 mos

28

1/23/2020 15

Results

29 Characteristics Derivation cohort (n=50 0 ) Validation cohort (n=549) P-value

Age (yrs ± SD, m ean ) 61.1 ± 17.7 61.1 ± 16.2 0.895 Gender (Fem ale, %) 243 (48.6) 253 (46.1) 0.422 Race (White, %) 481 (96.2) 523 (96.0) 0.875 BMI (kg/ m 2 ± SD, m ean) 31.5 ± 11.2 30.7 ± 9.8 0.178

CCI Score (m ean) 4.6 6.1 <0.0001

APACHE III score (m ean) 72.4 69.2 0.110 Mechanical Ventilation (yes-n, %) 237 (47.4) 277 (50.5) 0.354

Vasopressors (yes-n, %) 116 (23.2) 169 (30.8) 0.007

SQ response (“No”, %) 238 (47.6) 253 (46.1) 0.665 Outcom e at 6 -m onths (Death, %) 180 (36) 190 (34.6) 0.651 BMI-Body Mass Index; APACHE-Acute Physiology and Chronic Health Evaluation; SQ-Surprise Question

Results for Validation Sample

Sensitivity= 73.9.2%, Specificity= 81.7% NPV= 85.5%, PPV= 68.1%

30

Actual outcom e SQ Dead Alive Total “No” 139 (73.9%)† 65 (18.3%) 204 (100%) “Yes” 49 (26.1%) 290 (81.7%)‡ 339 (100%) Total 188 (100%) 355 (100%) 543* (100%)

*N=543 due to missing data.

1/23/2020 16

Results

31

Discussion of MICU study

Strengths Strengths

 SQ: simple, quick and intuitive to

understand

 First study to validate integrated

prognostic model in MICU population

 Robust accuracy, good sensitivity

and specificity

Lim itations Lim itations

 Single center  Consecutive patients  Ethnically non-diverse

32

1/23/2020 17

Conclusion of MICU Study

 Second cohort validates the integrated 6-month

mortality prognostic model.

 Model accuracy is robust and highly predictive of

short-term mortality.

 It enables clinicians to identify patients appropriate for

palliative care and potentially improve quality of care for MICU patients.

33

Calculator for MICU Integrated Prognostic Model

34

1/23/2020 18

How Cool Is This? Calculator in Epic

35

Results We reviewed 21 593 titles to identify 16 indices that predict risk of mortality from 6 months to 5 years for older adults in a variety of clinical settings. Although 13 indices had C statistics of 0.70 or greater, none of the indices had C statistics of 0.90 or greater. Three indices had C statistics between 0 .8 0 and 0 .8 9 suggesting very good discrim ination. Discriminatory ability of these 16 indices is consistent with other indices that commonly drive clinical decisions, such as the CHADS2 index to help determine warfarin therapy (C statistic, 0.68-0.72); the Framingham risk score to help determine lipid therapy (C statistic, 0.63-0.83); and the TIMI risk score to help determine invasive therapy for unstable angina (C statistic, 0.65).

36

1/23/2020 19

Why else does the SQ matter?

37

 Prompt for advance care planning with patients  Trigger for POLST form completion if desired  Prompt to consider palliative care consultation

Stage 3

Convert patients’ wishes into medical

• rders (POLST) so they

are immediately actionable and patient receives desired treatment.

Adults for whom it would not be a surprise if they died in the next 12 months. Stage 2

Determine goals of treatment for likely contingencies in patients with progressive illness.

Stage 1

Create MPOA and consider a Living Will. Healthy adults between ages 18 and 65.

Stages of Advance Care Planning

• ver a Lifetime

Adults with progressive, life‐limiting illness

38

1/23/2020 20

SQ as Prompt for POLST Conversation

http:/ / polst.org/ about-the-national-polst-paradigm/ what-is-polst/

39

1/23/2020 21

J Pain Symptom Manage 2016;51:240-246.

Patients with a POST form with comfort measures orders were significantly more likely to have an out-of-hospital death (OHD) than those who completed an advance directive (88.4% vs. 56.9%, P < 0.001). The odds of OHD were significantly higher for patients with POST forms with comfort measures orders than for those with advance directives (OR 4.239, P < 0.001).

42

1/23/2020 22 J Oncol Pract 2017 July 20. Epub ahead of print.

Odds of out-of-hospital death 2.33 greater for pts c POST form vs advance directive. Odds of hospice admission 2.69 greater for pts c POST forms vs advance directive.

The Patient Reluctantly Agreed to Dialysis

 81 yo frail widower with stage 5 CKD from renovascular hypertension  Former heavy smoker with oxygen-dependent COPD  Severe PAD with claudication; on oxycodone qhs for leg pain  eGFR dropped to 5 ml/ min; not yet uremic  Serum albumin 2.9 mg/ dL  Nephrologist urged dialysis with daughter’s agreement  Lived alone and valued his independence and freedom  Did not want to be tied down to a schedule three times/ week  Started reluctantly to please his daughter  Would you be surprised if this patient died in the next 6 months?

44

1/23/2020 23 A Patient-Specific Estimate of Prognosis for Our Patient

http:/ / touchcalc.com/ calculators/ sq

45

Integrated Prognostic Model for Hemodialysis Patients

Cohen LM, et al. CJASN 2010

1/23/2020 24

N=374 Forzley B, et al. Palliative Medicine 2018

The Patient Reluctantly Agreed to Dialysis

 Completed Living Will and Medical Power of Attorney  Named daughter as health care representative  Completed POST form with DNR and comfort measures  Did not want to be admitted to the hospital  Wanted to die at home if possible  Continued on dialysis for six months  Suffered large stroke with aphasia and right hemiplegia;

unable to communicate and care for himself

 Daughter respected her father’s wishes in LW and POST  Stopped dialysis and admitted to inpatient hospice  Died comfortably 8 days later

48

1/23/2020 25

49

Intervention Com ponent Description Clinical tools Serious Illness Conversation Guide Clinical training Communication skills and SDM System changes Surprise question prompt EMR Automatic Reminders Coaching Serious illness conversations

• More
• Earlier
• Better
• Accessible

278 DFCI patients identified with the “surprise question” “Would you be surprised if the patient died in the next year?”

Conclusions

 Using the SQ, clinicians can identify patients who are at

high risk of death in 6-24 months.

 The SQ is more accurate in patients with worse prognoses:

advanced cancer and MICU populations.

 The SQ is the strongest predictor so far in logistic

regression models of patients who are dead in 6 months.

 Integration of SQ into prognostic model with objective

measures improves accuracy.

 Use of the SQ can improve patient care

 identify patients appropriate for goals of care discussions  identify those appropriate for POLST completion

 Other uses of the SQ remain to be established.

50