Time-dependent Predictive Accuracy In the Presence of Competing - - PowerPoint PPT Presentation

Time-dependent Predictive Accuracy

In the Presence of Competing Risks Paramita Saha psaha@u.washington.edu http://students.washington.edu/psaha/ ENAR Spring Meeting 2009

Background

ENAR, 2009

1

Background

Diagnostic Accuracy for Retrospective Studies

• Disease status - Case (D) / Control ( ¯

D)

• Marker - M (binary or continuous)
• Higher marker is more indicative of disease
• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

2

Background

Diagnostic Accuracy for Retrospective Studies - contd.

• Sensitivity or True Positive (TP) -

TP(c) = P(M > c | D)

• Specificity or True Negative (1 - False Positive (FP)) -

FP(c) = P(M > c | ¯ D)

• ROC curve - plot (FP, TP) for all c ∈ (−∞, ∞)
• AUC - Area under the curve
• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

3

Background

Unique aspects of Time-to-event settings

• Disease status can be defined in more than one way
• Disease status can vary with time
• Censored event time
• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

4

Background

Notation

Suppose, we are interested in the ability of the marker M to predict event-time T.

• Ti, Mi - survival time and marker for ith subject

⊲ Marker can be time-dependent - Mi(t) ⊲ Higher marker values ⇒ poor survival

• Xi - covariates
• Ci - (independent) censoring time
• We observe: Zi = min(Ti, Ci),

δi = 1 1 (Ti ≤ Ci)

• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

5

Background

Cumulative/dynamic definition

• Cumulative case - subject experienced event by t
• Dynamic control - subject did not experience event by t
• TP C

t (c) = P(M > c | T ≤ t)

• FP D

t (c) = P(M > c | T > t)

• At each time t, divide all the subjects as either a case or a control
• Summary measures

⊲ ROC curve at time t ⊲ AUC at time t

• Dept. of Biostatistics, University of Washington
• P. Saha

Competing Risks ROC

ENAR, 2009

6

Competing Risks ROC

Goal

• Answer the following question

⊲ How well the marker distinguishes between subjects who experience event

• f type 1 (type 2) by time t and those who do not experience any type
• f event by time t?
• Estimate time-dependent ROC and AUC
• Account for competing risks events
• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

7

Competing Risks ROC

• We observe time until first event and type of event

⊲ observed time until first event - Zi ⊲ δi = 0, 1, 2, . . . , C ⊲ δi: censored = 0; different event types = 1, 2, . . . , C

• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

8

Competing Risks ROC

What is different?

• The risk of censored subjects are redistributed to the subjects present in the

riskset

• The subjects experiencing a competing risks event at t are not at risk of

failure due to other causes after t ⇒ risk redistribution to the right is not meaningful in general

• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

9

Competing Risks ROC

Review: Cumulative/dynamic definition

• Cumulative case - subject experienced event by t
• Dynamic control - subject did not experience event by t
• TP C

t (c) = P(M > c | T ≤ t)

• FP D

t (c) = P(M > c | T > t)

• At each time t, divide all the subjects as either a case or a control
• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

10

Competing Risks ROC

TP and FP for Cause-specific survival

• Controls are free of all events

FP D

t (c) = P(M > c | T > t)

• Cases are cause-specific - partition the cases into finer groups based on

event-type TP C

t (c, d)

= P(M > c | T ≤ t, δ = d)

• e.g. marker given AIDS by time t (d = 1)
• e.g. marker given death by time t (d = 2)
• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

11

Competing Risks ROC

Estimation

P(M > c | T ≤ t, δ = d) = P(M > c, T ≤ t, δ = d) P(T ≤ t, δ = d)

• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

11

Competing Risks ROC

Estimation

P(M > c | T ≤ t, δ = d) = P(M > c, T ≤ t, δ = d) P(T ≤ t, δ = d) Numerator P(M > c, T ≤ t, δ = d) = ∞

c

P(T ≤ t, δ = d | M = m)P(M = m)dm = ∞

c

P(M = m)dm

• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

11

Competing Risks ROC

Estimation

P(M > c | T ≤ t, δ = d) = P(M > c, T ≤ t, δ = d) P(T ≤ t, δ = d) Numerator P(M > c, T ≤ t, δ = d) = ∞

c

P(T ≤ t, δ = d | M = m)P(M = m)dm = ∞

c

Cd(t | M = m)

• P(M = m)dm
• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

12

Competing Risks ROC

Estimation - contd.

• Cd(t|M = m)

=

• u≤t
• Sǫn(u|M = m)

λd(u|M = m)

• Sǫn(u|M = m)

: use locally weighted KM estimator

• λd(u|M = m)

: use local observed hazard

• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

13

Competing Risks ROC

Estimation - contd.

• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

13

Competing Risks ROC

Estimation - contd.

• Estimate TP

P(M > c, T ≤ t, δ = d) = ∞

c

P(T ≤ t, δ = d | M = m)P(M = m)dm = ∞

c

Cd(t | M = m)

• P(M = m)dm
• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

13

Competing Risks ROC

Estimation - contd.

• Estimate TP

P(M > c, T ≤ t, δ = d) = ∞

c

P(T ≤ t, δ = d | M = m)P(M = m)dm = ∞

c

Cd(t | M = m)

• P(M = m)dm

⊲ Use local cause-specific cumulative incidence to estimate Cd(t | M = m) ⊲ Use empirical estimator for P(M = m)

• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

14

Competing Risks ROC

Estimation - contd.

• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

14

Competing Risks ROC

Estimation - contd.

• Estimate FP - use previous expressions to estimate FP D

t (c)

• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

14

Competing Risks ROC

Estimation - contd.

• Estimate FP - use previous expressions to estimate FP D

t (c)

P(M > c|T > t) = P(M > c, T > t) P(T > t) Numerator = ∞

c

P(T > t|M = m)P(M = m)dm

• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

14

Competing Risks ROC

Estimation - contd.

• Estimate FP - use previous expressions to estimate FP D

t (c)

P(M > c|T > t) = P(M > c, T > t) P(T > t) Numerator = ∞

c

P(T > t|M = m)P(M = m)dm P(T > t|M = m) = 1 −

• d

P(T ≤ t, δ = d|M = m) = 1 −

• d

Cd(t|M = m)

• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

15

Competing Risks ROC

Estimation - contd.

• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

15

Competing Risks ROC

Estimation - contd.

• Estimate TP and FP

⊲ Use cause-specific conditional cumulative incidence ⊲ Use empirical distribution for marker

• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

16

0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0

Event Type: 1 (C/D)

FP TP

92 93 90 84

0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0

Event Type: 2 (C/D)

FP

93 92 92 89

True ROC Estimated ROC Pointwise 90% Confidence band

Figure 1: Bootstrapped ROC curves, confidence bands and coverage

• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

17 Event type: 1 log(Time) Ri(t) AUC Mean SD Coveragea 0.0 52.8 0.8446 0.8309 0.0344 89.20 Event type: 2 log(Time) Ri(t) AUC Mean SD Coveragea 0.0 52.8 0.6011 0.5899 0.0476 90.20 Table 1: Average of bootstrap mean, sd and coverage (percentile-based) of AUC

aNominal: 90.00

• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

18

Competing Risks ROC

Example

Multicenter AIDS Cohort Study

• N = 438 sero-convert patients
• Endpoints -

⊲ time-to-AIDS ⊲ time-to-death

• Predictive marker -

⊲ CD4 and CD8 measured at “baseline”

• Goal: evaluate markers as predictors of disease-progression
• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

19

0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0

F P T P

AIDS Death All Cause

Time: 5 Years 0.583 0.506 0.575

• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

20

Competing Risks ROC

Summary

• Two of the existing approaches can be modified to account for competing

risks (Heagerty et al. (2000), Biometrics; Heagerty and Zheng (2005), Biometrics)

• Answer the following question

⊲ How well the marker distinguishes between subjects who experience event

• f type 1 (type 2) by time t and those who do not experience any type of

event by time t? (Saha and Heagerty, 2009)

• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

21

Thank you!!

• Dept. of Biostatistics, University of Washington
• P. Saha

ENAR, 2009

22

References

Heagerty, P. J., Lumley, T., and Pepe, M. S. (2000). Time-dependent ROC curves for censored survival data and a diagnostic marker. Biometrics, 56 : 337–344. Heagerty, P. J. and Zheng, Y. (2005). Survival model predictive accuracy and ROC curves. Biometrics, 61 : 92–105. Saha, P. and Heagerty, P. J. (2009). Time-dependent Predictive Accuracy in the Presence of Competing Risks. (Working paper)

• Dept. of Biostatistics, University of Washington
• P. Saha