Effective Layering of Defenses Nathaniel Boggs, Salvatore J. Stolfo - - PowerPoint PPT Presentation

▶
effective layering of defenses
SMART_READER_LITE
LIVE PREVIEW

Effective Layering of Defenses Nathaniel Boggs, Salvatore J. Stolfo - - PowerPoint PPT Presentation

May 21, 2023 440 likes •630 views

ALDR: A New Metric for Measuring Effective Layering of Defenses Nathaniel Boggs, Salvatore J. Stolfo Columbia University 12/6/2011 Layered Assurance Workshop 1 Motivation Buy security product X? Am I secure? 12/6/2011 Layered

slide-1
SLIDE 1

ALDR: A New Metric for Measuring Effective Layering of Defenses

Nathaniel Boggs, Salvatore J. Stolfo Columbia University

12/6/2011 Layered Assurance Workshop 1

slide-2
SLIDE 2

Motivation

  • Buy security product X?
  • Am I secure?

12/6/2011 Layered Assurance Workshop 2

slide-3
SLIDE 3

Current Answers

  • Compliance checklist
  • “Best Practices”
  • Evaluate class of products
  • Penetration testing
  • Defense in Depth
  • Can we do better?

12/6/2011 Layered Assurance Workshop 3

slide-4
SLIDE 4

Defense in Depth

AV IDS Logs

12/6/2011 Layered Assurance Workshop 4

slide-5
SLIDE 5

Defense in Depth

AV IDS Logs

12/6/2011 Layered Assurance Workshop 5

slide-6
SLIDE 6

Compare Different Layers

  • Compare apples to oranges
  • Measure detection of ‘Attacks’
  • ‘Attacks’

– Source domain – Network traffic – Executable – And many more…

12/6/2011 Layered Assurance Workshop 6

slide-7
SLIDE 7

12/6/2011 Layered Assurance Workshop 7

slide-8
SLIDE 8

All Layer Detection Rate (ALDR)

  • Test each security project
  • # attacks detected / total attacks
  • Total attacks detected by a set of products

12/6/2011 Layered Assurance Workshop 8

slide-9
SLIDE 9

ALDR: 0.875 (14/16)

AV IDS Logs

12/6/2011 Layered Assurance Workshop 9

slide-10
SLIDE 10

ALDR – Key Attributes

  • Products tested individually
  • Expandable framework

– Measure education benefit – Social engineering attacks – Any ‘attack’ representable

  • Evaluate products in context

12/6/2011 Layered Assurance Workshop 10

slide-11
SLIDE 11

Additional Metrics

  • False Positives
  • Redundancy

– Good redundancy vs bad – Classify detection method

12/6/2011 Layered Assurance Workshop 11

slide-12
SLIDE 12

What Should I Buy?

  • Calculate increase in TP, FP, redundancy
  • Organization specific
  • Testing not organization specific!
  • Measure/Predict relative security change

12/6/2011 Layered Assurance Workshop 12

slide-13
SLIDE 13

Am I secure?

  • Given a set of products
  • Specific attack dataset
  • Measure how many attacks evade
  • Find product(s) to fix
  • Increase relative security

12/6/2011 Layered Assurance Workshop 13

slide-14
SLIDE 14

Challenges – Data Sets

  • How to link ‘attacks’
  • Define ‘attacks’
  • Future attacks differ?

12/6/2011 Layered Assurance Workshop 14

slide-15
SLIDE 15

Challenges

  • Not all attacks equal
  • Past predicts future?
  • Create a future data set?

12/6/2011 Layered Assurance Workshop 15

slide-16
SLIDE 16

Future Work - Experiments

  • Require data sets

– Linked attacks – Ground truth

  • Drive-by downloads?
  • Historical data?

12/6/2011 Layered Assurance Workshop 16

slide-17
SLIDE 17

Conclusion

  • New metrics needed
  • Security products are not isolated
  • Many challenges, no show stoppers
  • Measure relative security

12/6/2011 Layered Assurance Workshop 17

slide-18
SLIDE 18

Questions?

12/6/2011 Layered Assurance Workshop 18