Quantitative Aspects of Screening Michael OReilly, MD, MPH - - PowerPoint PPT Presentation

Quantitative Aspects of Screening

Michael O’Reilly, MD, MPH Technical Advisor Field Epidemiology Training Program - Thailand

Objectives

• Understand the concepts of sensitivity,

specificity, and predictive values

• Be able to calculate the testing

characteristics above

• Know what factors drive predictive values

Questions

• How do we evaluate the ability of a

test to “do what it’s supposed to do?”

• How well does it identify those who

have disease

• How well does it exclude those who

don’t have disease

Sensitivity Sensitivity

What is the definition?

a (true +) b (false +) c (false -) d (true -) Disease

+

• Test

+

Sensitivity Sensitivity

The likelihood of a person with disease having a positive test:

_a_ a + c

a (true +) b (false +) c (false -) d (true -) Disease

+

• Test

+

Sensitivity Sensitivity

This is a vertical property I mmutable Concerned only with individuals who have disease

117 48 33 102 Influenza

+

• Rapid

Test

+

117 48 33 102 Influenza

+

• Rapid

Test

+

• Sens = ?

117 48 33 102 Influenza

+

• Rapid

Test

+

• Sens =

117/150 = 78%

Sensitivity Sensitivity

I s it affected by prevalence? What happens if we double the number of individuals with flu in our study?

234 48 66 102 Influenza

+

• Rapid

Test

+

• Sens =

234/300 = 78%

Specificity Specificity

What is the definition?

Specificity Specificity

The likelihood of a person without disease having a negative test:

_d_ d + b

a (true +) b (false +) c (false -) d (true -) Disease

+

• Test

+

Specificity Specificity

This is a vertical property I mmutable Concerned only with individuals who don’t have disease

117 48 33 102 Influenza

+

• Rapid

Test

+

• Spec = ?

117 48 33 102 Influenza

+

• Rapid

Test

+

• Spec =

102/150 = 67%

Relationship of Sensitivity and Relationship of Sensitivity and Specificity Specificity

How are these two characteristics related? Under ordinary circumstances, what happens to specificity if you increase sensitivity?

MI patients non MI patients

CK MB band

Sensitivity & Specificity: Establishing a Threshold Value

8.0

*

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

1 - specificity sensitivity

Sensitivity & Specificity are Inversely Related

MI patients non MI patients

CK MB band

Sensitivity & Specificity: Establishing a Threshold Value

8.0

*

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

MI patients non MI patients

CK MB band

Maximizing Sensitivity

4.0

*

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

1 - specificity sensitivity

Gains in Sensitivity mean losses in Specificity

MI patients non MI patients

CK MB band

Sensitivity & Specificity: Changing Threshold Values

12.0

*

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

1 - specificity sensitivity

Gains in Specificity mean losses in Sensitivity

Positive Predictive Value Positive Predictive Value

What is the definition?

Positive Predictive Value Positive Predictive Value

The likelihood of a person with positive test really having disease:

_a_ a + b

a (true +) b (false +) c (false -) d (true -) Disease

+

• Test

+

117 48 33 102 Influenza

+

• Rapid

Test

+

• PPV = ?

117 48 33 102 Influenza

+

• Rapid

Test

+

• PPV

=117/165 =71%

Positive Predictive Value Positive Predictive Value

I s it affected by prevalence? What happens if we double the number of individuals with flu in our study?

234 48 66 102 Influenza

+

• Rapid

Test

+

• PPV

= 234/282 = 83%

Positive Predictive Value Positive Predictive Value

This is a horizontal property Mutable Concerned both with individuals who have and don’t have disease

Positive Predictive Value: Example Positive Predictive Value: Example

You are offered a PSA test with sensitivity 90% and specificity 80% I n a cancer clinic population of 1,000 prevalence is 75% . What is PPV? I n an asymptomatic population of 1,000 prevalence is .5% . What is the PPV? Where do you use the test?

a b c d Prostate Cancer

+

• PSA

Test

+

• Total = 1,000

a b c d Prostate Cancer

+

• PSA

Test

+

• Total = 1,000

Prevalence: 75% 1,000 x .75 = 750 a + c = 750 750

a b c d Prostate Cancer

+

• PSA

Test

+

• Total = 1,000

a + c = 750 d + b = 1,000 - 750 = 250 750 250

675

b

75

d Prostate Cancer

+

• PSA

Test

+

• Total = 1,000

Sensitivity = .90 a = 750 x .90 =675 [c = 750 – 675] 750 250

675 50 75 200

Prostate Cancer

+

• PSA

Test

+

• Total = 1,000

Specificity = .80 d = 250 x .80 =200 [b = 250 – 200] 750 250

675 50 75 200

Prostate Cancer

+

• PSA

Test

+

• Total = 1,000

PPV = a/a+b a + b = 725 PPV = 675/725 = 93% 750 250

4.5 199 0.5 796

Prostate Cancer

+

• PSA

Test

+

• Total = 1,000

a + b = 203.5 PPV = 2.2% 5 995

Positive Predictive Value: Positive Predictive Value: Comments Comments

Affected greatly by prevalence One other factor has a great impact: is it sensitivity or specificity?

PPV: I nfluence of sensitivity and PPV: I nfluence of sensitivity and specificity specificity

I n a general population of 1,000 prevalence

• f prostate cancer is .5%

PSA Laboratories, inc. has heard your criticisms and has made 2 new PSA tests:

• Super Sensitive PSA [sens= .98; spec= .80]
• Super Specific PSA [sens= .80; spec= .98]

Which test has the higher PPV?

4.9 199 0.1 796

Prostate Cancer

+

• PSA

Test

+

• Total = 1,000

Super Sensitive Sens .98 Spec .80 PPV = 2.4% 5 995

4 19.9 1 975.1

Prostate Cancer

+

• PSA

Test

+

• Total = 1,000

Super Specific Sens .80 Spec .98 PPV = 16.7% 5 995

PPV: Summary PPV: Summary

Driven by specificity and prevalence Most powerful in settings of moderate disease prevalence

Can describe screening results more powerfully than sensitivity or specificity

A

(true +)

b

(false +)

C

(false -)

d

(true -) Disease highly prevalent

+

• Test

+

a (true +)

b

(false +) c (false -)

D

(true -) Test highly specific

+

• Test

+

Testing Characteristics Testing Characteristics

What property is most important for a pregnancy test?

What property is most important for a confirmatory colon cancer test?