Jaal: Towards Network Intrusion Detection at ISP Scale A. Aqil, K. - - PowerPoint PPT Presentation

Jaal: Towards Network Intrusion Detection at ISP Scale

• A. Aqil, K. Khalil, A. Atya, E. Paplexakis,
• S. Krishnamurthy, KK. Ramakrishnan
• T. Jaeger
• P. Yu, A. Swami

1

University of California Riverside Penn State University US Army Research Lab

Is IDS Needed at ISP Scale?

Increasing number of network attacks

Distributed, span entire WANs Unnoticed until too late

Mirai botnet

Mirai exploits vulnerable devices spread across the internet to launch DDoS Sep 2016: Krebs on Security (620 Gbit/s), OVH (1Tbit/s) Oct 2016: multiple attacks on Dyn, affected Twitter, Github, Airbnb, Netflix, others Nov 2016: Liberia’s internet infrastructure

2

Is IDS Needed at ISP Scale?

Simple two step attack: scan then flood

Hardcoded default passwords control vulnerable devices (scanning a large set of IP addresses) Compromised devices also repeat the scan Launch coordinated attack on targets at the bot master signal

Inherently difficult to detect

Scanning activity observable only at ISP level “DDoS prevention works best deep in the network, where the pipes are the largest and the capability to identify and block the attacks is the most evident” Bruce Schneier, security expert

3

ISP Scale IDS is Challenging

State of the art NIDS (e.g., Snort, Bro) are effective

But expect to inspect all packets Works only at enterprise scale

Problematic at ISP scale:

Multiple ingress/egress points To create global view required for analysis, information collected from multiple vantage points needs to be aggregated

Challenge: how to aggregate?

4

Aggregation Approach I

Copy and forward to central engine

Simple, but lead to performance degradation

5

Avg decrease in throughput Worst decrease in throughput Drop in accuracy Percentage decrease 50 100 Percentage of traffic replicated 10 20 30 40 50 60 70 80 90 100 Performance degradation as traffic replication increases

70% Tput loss

Aggregation Approach II

Sample and forward to central engine

Already used by ISPs for heavy-hitter detection Efficient but achieves poor detection accuracy for general attacks

6

Attack Reservoir Sampling Distributed SYN Flood 54% Sock Stress 60% SSH Brute Force 42%

Aggregation Approach III

Create sketches and forward to central engine

Targeted measurement approach Strong resource/accuracy guarantees Lacks generality: need one sketch for every measurement task

For TCP/IP header, need 218 different sketches to capture all possible measurements

7

Jaal Design Goals

Design an ISP-scale NIDS that:

Can detect wide array of attacks requiring global view, using signatures similar to Snort’s

Focus on TCP/IP header-based attacks

Does not require copying and forwarding raw packets (minimizes bandwidth overhead)

8

Jaal Overview

9

en source

• lumes [32, 42].

eater than egularly; this (can result

• nly way

clusters for peak that, a large are inade- to copying

Packet Filtering Summatization Flow Assign. Inference NIDS Summaries Assignments Decision Load Monitor Packet Filtering Summatization Monitor Info. Load Info. Rules Summaries Assignments

I- Monitors:

• Filter target flows
• Process packet

batches, create summaries II- Inference engine:

• Collects summaries
• Performs pattern

matching III- Flow assignment:

• Assigns flows to monitors
• Load balancing

Summarization

Goal: produce a representative summary of packets

Enables high accuracy detection of attacks using general signature Light weight, low BW overhead

10

all flows all flows assigned flows packets batch batch summary fields mode packets mode

Summarization (cont.)

Two step summarization

SVD to reduce fields mode Clustering to reduce packets mode

11

batch summary SVD k-means p r n k

centroids counts

• sed to

¯ X = UΣVT ,

fields mode packets mode eliminate small singular values

Inference

12

counter- to d. ¯ X (3)

Translator NIDS question Similarity Summary Sm

1 or Sm 2

Config. τd, τc Alert, Q Postprocessor Alert h, τv Config. Q Estimator Aggregator Sa Rules vectors q Inference Engine Feedback

Individual summaries

Collect individual summaries (push or pull) Transform NIDS rules (normalization, marking irrelevant fields) Estimate similarity Feedback: request finer grained summary to improve performance Estimate variance (e.g. port scans, DDoS)

Flow Assignment

Requirements:

Cover all flows Each flow is processed by exactly one monitor (for correct operation) Balance load to the extent possible Simple/Fast algorithm

Challenge:

Flows can start/terminate at any time, vary in packet rate Packet rates unknown a priori

Solution:

Model as constrained online load balancing problem Simple greedy algorithm, (empirically) close to optimal

13

Evaluation

Implemented on in-house high performance SDN-testbed Two Realistic RocketFuel topologies (~350 routers)

Complex topologies created by instantiating Open vSwitches instances connected via virtual links

Two ISP backbone traces from MAWI group as background traffic + inject malicious traffic Five different attacks:

DoS: SYN flood DDoS: distributed SYN flood Port scans: distributed port scans Brute forcing: distributed SSH brute forcing Sockstress

14

most common attack classes

Evaluation (cont.)

r = 14 retains most information in fields mode n ≥ 600, k ≥ 0.2n, r ≥ 12 enables high detection accuracy

15

98% average TPR @ 9% FPR and only 35% BW overhead (with feedback) Summarization parameters n, k, r set by studying ROC curves Each point has a different

n: batch size, r: rank, k: centroids

Evaluation (cont.)

Simulating Mirai progression

Scanning on ports 23, 2323 Randomly select a source node + 150 vulnerable nodes Jaal detects the scan with 95% accuracy

16

Unchecked infections Remaining infected devices after Jaal Number of infected devices 50 100 150 Time (s) 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 Number of Infected devices vs time

Conclusion

ISP scale NIDS is needed in the face of large scale attacks State of the art NIDS inadequate at ISP scale Jaal presents a major step forward in developing ISP scale NIDS

Uses dimensionality reduction and clustering Centralized pattern matching on packet summaries Achieves high detection accuracy at low bandwidth

• verhead

17

Thanks!

18