Using Rule-Based Activity Using Rule-Based Activity Using - - PowerPoint PPT Presentation

Using Rule-Based Activity Using Rule-Based Activity Using Rule-Based Activity Using Rule-Based Activity Descriptions to Evaluate Descriptions to Evaluate Descriptions to Evaluate Descriptions to Evaluate Intrusion Detection Systems Intrusion Detection Systems Intrusion Detection Systems Intrusion Detection Systems

Dominique Alessandri

<dal@zurich.ibm.com> September 2001

Motivation and Goals of this Work Motivation and Goals of this Work Motivation and Goals of this Work Motivation and Goals of this Work

Unresolved issues with respect to ID-architecture components, i.e. IDSes

High failure rate of IDSes (false positives and false negatives) Insufficient understanding of the semantics attached to alarms generated by diverse IDSes (alarm correlation & design of ID-architectures) Difficulty of IDS testing (specific to environment, static setup, heavy weight

procedures)

Goals of this work

Evaluate IDSes wrt. their potential detection capabilities and wrt. their potential to failure Identify combinations of IDSes that provide increased ID-coverage

Classification of Attacks Classification of Attacks Classification of Attacks Classification of Attacks

We need an attack classification to identify a representative set of attacks to evaluate IDSes. Issues with existing attack classifications (e.g. Howard,

Cohen, Neumann, Kumar and others):

None of the existing classifications allows the classification of attacks with respects to aspects relevant to an IDS (This namely includes all the

aspects of attacks that are potentially observable by an IDS).

Often aspects of attacks and vulnerabilities are not clearly separated. Classification categories are not distinct.

We need to develop a classification of attacks that classifies attacks according to criteria relevant to ID.

Classification of IDSes Classification of IDSes Classification of IDSes Classification of IDSes

We need a description scheme for IDSes that enables us to determine what an IDS is able to deduce from a given activity. Issues with existing taxonomies and classifications:

Existing classifications (e.g. Debar et al., Axelsson, Lunt, Jackson and others) are not sufficiently systematic and/or detailed.

We need to develop a description scheme for IDSes that is sufficiently systematic and detailed such that it becomes possible to analyze what IDSes are able to deduce from activities.

Our Approach to IDS Evaluation Our Approach to IDS Evaluation Our Approach to IDS Evaluation Our Approach to IDS Evaluation

IDS descriptions Description of activities (incl. attacks) IDS Evaluation Precision and coverage estimates for single IDSes combinations of IDSes Potential true/false positives/negatives Classification of attacks (using VulDa) IDS description scheme Selection

• f attacks
• Classif. scheme for

attacks and activities in general Analysis of alarm set semantics RIDAX tool (Rule-based Intrusion Detection Analysis and eXamination)

Comparison of paradigms Existing work by Lippmann et al. (and others) Our approach (rule-based evaluation) Goal Provide measures to judge quality of IDSes and to support selection of IDSes Provide coverage and precision estimates

• f single IDSes and IDS combinations to

support ID-architecture design Implementation Evaluation of specific versions and configurations of real IDSes Evaluation of the potential of IDSes based

• n a description of their capabilities

Realization Evaluation testbed; replay of recorded traffic Description of IDSes, attacks and benign activity using prolog rules What is evaluated? IDS implementation and configuration Potential of the technology used Environment specific and given (testbed) independent Input Real attacks and background activity (traffic) Description of classes of attacks and of benign activity Input variation Known variants of given attacks selected Input variation generated systematically Results wrt. true positives List of specific attacks the IDS can detect List of attack classes the IDS can potentially detect Results wrt. false positives List of false positives observed during evaluation process List of activity classes that potentially cause false positives Analysis of results Number and percentage of detected attacks and false positives Potential estimates and precision for a normalized input set

A new IDS Evaluation Paradigm A new IDS Evaluation Paradigm A new IDS Evaluation Paradigm A new IDS Evaluation Paradigm

0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 90.00% 0.00% 20.00% 40.00% 60.00% 80.00% 100.00%

Rating Precision Recall (Coverage)

Single IDS 2 IDSes 3 IDSes 4 IDSes 5 IDSes

Estimates Estimates Estimates Estimates

(Rating Precision vs. Attack Detection Recall) (Rating Precision vs. Attack Detection Recall) (Rating Precision vs. Attack Detect (Rating Precision vs. Attack Detection Recall) ion Recall)

Goal: find the combination of IDSes that meets our requirements (e.g.

80% coverage, 80% rating precision)

Optima at concurrent 100% coverage and 100% rating precision

Considered IDSes: Snort (simple configuration), Snort (all features), DaemonWatcher for httpd and ftpd (IBM) and WebIDS (IBM)