I ntroduction I ntrusion Detection How prevalent are DoS attacks? - - PDF document

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

I ntrusion Detection – Backscatter and Global Analysis

Stefan Zota

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

I ntroduction

How prevalent are DoS attacks? Quantitative analysis Long term predictions and recurring patterns of attacks Measurement and Global Analysis

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Outline

Challenges Methods for Measuring DoS attacks Firewall Logs Network Telescopes I nternet Sinks Backscatter Background Radiation Conclusions

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Outline

Challenges Methods for Measuring DoS attacks Firewall Logs Network Telescopes I nternet Sinks Backscatter Background Radiation Conclusions

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Challenges

Attackers find ingenious ways of compromising remote hosts Attackers give public access to the tools used so the hacking community improves The size and complexity of the Internet make impossible to remove all vulnerabilities The sharing of information between networks is complicated due to privacy issues Very little understanding of intrusion activity on a global basis Very hard to detect the length of an attack or combined protocol attacks

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Examples of Flow Anomalies

Barford and Plonka identify three categories: Network Operation Anomalies Flash Crowd Anomalies Network Abuse Anomalies

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Network Operation Anomalies

Outages, configuration changes, environmental limits

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Flash Crowd Anomalies

Rapid rise in traffic flows to a particular destination with a gradual drop-off in time

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Network Abuse Anomalies

Identify DoS flow flood attacks and port scans They may not be apparent in bit or packet rate measurements

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Outline

Challenges Methods for Measuring DoS attacks Firewall Logs Network Telescopes I nternet Sinks Backscatter Background Radiation Conclusions

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Goals

Characterization of the “non-productive” or malicious traffic Develop a methodology for measuring intrusions Filtering large traffic volume Designing scalable flexible architectures Building responders

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Overview of Methods of Measuring DoS attacks

Firewall Logs

Starting from a dataset like DSHIELD

Network Telescopes

Large chunks of unused, globally routable IP space

Internet Sinks

Unsolicited traffic for unused addresses Passive and Active Monitoring

Backscatter

Analysis of source addresses for attacks

Background Radiation

Traffic to unused addresses (similar to Network Telescopes)

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

DSHI ELD

Distributed Intrusion Detection System An attempt to collect data about cracker activity from the Internet Data contains:

Tops of worst offenders Port scans Block lists Port report IP Info Subnet Report

Easy to filter packets

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Network Telescopes

Chunk of globally routed IP address space Little or no legitimate traffic Unexpected traffic arriving at the network telescope can imply remote network/security events It contains a lot of statistical and random data It is good for seeing explosions not small events

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

I nternet Sinks

Monitors unused or dark IP Packets for those addresses may be dropped by gateways or border routers The size of the address space monitored is very important

Usually class A and B

Includes an active component

Generates packets as response to incoming traffic

Extensible and scalable

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Backscatter

Most denial of service attacks select source addresses at random for each sent packet

Shaft, TFT, Trinoo, Stackeldraht, Mstream, Trinity

It detects only attacks that use spoofed IP’s A router or an intermediate device may generate an ICMP response to the attack Assumption

The victim responses are equi-probably distributed across the entire Internet space

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Backscatter

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Background Radiation

Monitor unused addresses Detect non-productive traffic

Malicious: flooding backscatter, scans, worms Benign: misconfigurations

What is all this nonproductive traffic trying to do? How can we filter and detect new types of malicious activity?

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Outline

Challenges Methods for Measuring DoS attacks Firewall Logs Network Telescopes I nternet Sinks Backscatter Background Radiation Conclusions

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Firewall Logs

Internet Intrusions: Global Characteristics and Prevalence

Data collected in 1600 networks in a 4 month period by DSHIELD.ORG Each entry is recorded by firewalls and port scan logs recorded by NIDS (primarily Snort) Asses the daily volume of intrusion attempts Use the results to project intrusion activity in the entire Internet Investigate utility of sharing intrusion detection information

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Scans

Vertical

Sequential or random scan of multiple ports (5 or more) of a single IP from the same source during one hour period Survey of well-known vulnerabilities (strobe scans)

Horizontal

Scan from a single source to multiple IP on the same port Looking for the same vulnerability

Coordinated

Scans from multiple sources (5 or more) aimed at a particular port of destinations in the same /24 during one hour period Aggressive, active collaborative peers

Stealth

Low frequency horizontal and vertical scans. Minimum threshold for average interscan distance

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Port Distribution

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Persistence of Worm Activity

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Top Sources (1)

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Top Sources (2)

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Top Sources (3)

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Scan Types

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Stealth Scan Types

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Global Prevalence

Highly dynamic scanning patterns How the volume of scans have changed over the last year? Project daily scans to entire Internet

Average scans per IP * Total Number of IP Assumption: uniformity Daily scan rates 25B/day Relatively steady rates for port 80 scans (decreasing) Relatively steady rates for non-worm scans (increasing 25%)

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

I mplications of Shared I nformation

Refinement extent provided by additional data Relative entropy Marginal utility metric

Reduction of uncertainty resulting from the next experiment added to the aggregate set Offline/Online

Experiments to evaluate the marginal utility of intrusion detection log sharing for worst offenders and port identification Select randomly days and logs from dataset and try to estimate the gain in aggregation

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Marginal Utility (1)

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Marginal Utility (2)

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Summary

1M – 3M scans per day Widely distributed sources Power law distribution for the number of events Large amounts of scans for port 80 60-70% of non-worm scans are horizontal A lot of daily vertical scan episodes Coordinated worst offenders are responsible for a significant fraction of all scanning activity The collaboration benefit is sensitive to the size and diversity of the peering group

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Outline

Challenges Methods for Measuring DoS attacks Firewall Logs Network Telescopes I nternet Sinks Backscatter Background Radiation Conclusions

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Network Telescopes (1)

Assume random IP generation scanning

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Network Telescopes (2)

Size of the telescope is important for:

Detect events that generate fewer packets Better accuracy in determining the attack interval

The probability of detecting events increases with the size of the telescope Increase the size by using distributed telescopes Advantages:

Reduces dependency on reaching a single block Traffic load may be distributed over multiple sites May avoid being skipped by some IP generation algorithms

Disadvantages

Synchronization Data distribution

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Network Telescopes Size(1)

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Network Telescopes – Code Red

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Daily Non-Worm Scan Rate

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Daily Port 80 Scan Rate

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Outline

Challenges Methods for Measuring DoS attacks Firewall Logs Network Telescopes I nternet Sinks Backscatter Background Radiation Conclusions

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

I nternet Sinks

iSink capabilities:

Trace packets Respond actively Masquerade as several applications Fingerprint source hosts Sample packets

Monitor 4 class B and one class A for 4 months

Stateless and sampling increases the scalability B classes - holes between active subnets

Main objective

a highly interactive scalable backplane for filtering attacks, misconfigurations and attacks

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Architecture

3 main camponents:

Argus - Passive Monitor

• generic libpcap based on IP network auditing tool
• flow level monitoring of sink traffic

Click - Active Sink

• Poll device
• IP Classifier for routing ARP, ICMP and TCP packets
• Windows Responder

NAT Filter

• Reduce traffic responder generated traffic volume
• Routes requests to appropriate responders
• Filter requests – connections to first N destinations IPs targeted by

the source

VMware Honeynets – commodity VMware systems NIDS – evaluate packet logs collected at the filter

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Architecture

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

iSink Deployment

Campus Enterprise Sink (CES)

iSink received unsolicited traffic for 100,000 IP Configure a “black-hole” intra-campus router to advertise the class B aggregate routes into the intra-campus OSPF iSink has not participated to intra-campus routing iSink is a destination of a static route Unsolicited traffic falls to /16 routes, iSink Occasionally traffic for used addresses may fall to iSink because of inexistent routes

Service Provider Sink (SPS)

Unsolicited traffic for 16 million IP (class A) ISP advertised class A via BGP to SNMP measurements at switch ports for computing Argus packet loss

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

CES I nbound Traffic

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

SPS I nbound Traffic

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Backscatter Packets

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Unique Periodic Probes

TCP flow periodicity can be isolated to sources scanning port 139 (Server Message Block over NetBIOS) and port 445 (SMB)

Scans involve 256 IP from a /24 Probes have an one hour period Small scale periodicity super imposed over a daily periodicity

They have built responders for NetBIOS and SMB

The scanning process was done by LovGate worm

• Email propagation, at execution, it copies itself to kernel66.dll,

iexplore.exe etc, Backdoor (dropping a trojan) waiting on port 20168

• Dictionary attack

Setup a controlled experiment

Deterministic scanning Small periods of synchronization

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

SMTP Host-spot

One IP attracting large number of SMTP scans

4,5 million scans from 14,000 unique IP in 10 days Uncommon TCP SYN fingerprint All were DSL and cable modem hosts

They have setup a SMTP responder The source was a misconfigured wireless router Uninitialized garbage value converted to IP address They have looked for the printed ASCII version of the IP address and it in all versions of firmware for the device

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Scalability

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Sampling

Reduced bandwidth Improved scalability Simplified data management and analysis Adaptation of “Heavy hitters” sampling Subnet selection Memory constrained Sample and Hold

Identifies flows larger threshold Random sampling (uniform class A traffic) Hash containing flow id and byte count

Sampling rate based on empirical observation of traffic Larger blacklists easier to estimate

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Summary

Clear evidence of well documented worms New worm detection Different overall characteristics between class B and A iSink commodity PC hardware has the ability to monitor and respond to 20,000 connection requests per second (peek class A traffic)

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Outline

Challenges Methods for Measuring DoS attacks Firewall Logs Network Telescopes I nternet Sinks Backscatter Background Radiation Conclusions

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Backscatter

Random source selection for each packet Attack tools: Shaft, TFN, trinoo, Stackeldraht, mstream, Trinity Equi probable distribution of victim responses across all the Internet space Assumptions

Address uniformity Reliable delivery Backscatter hypothesis

Ingress filtering Reflector attacks

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Flow Based Classification

Classification for individual attacks Fixed flow lifetime (5 minutes interval)

Conservative timeout suggests fewer longer attacks Shorter timeout suggests a large number of shorter attacks

Discard all flows with less 100 packets and a duration less than 60seconds

Used to avoid random Internet misconfigurations?

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Event Based Classification

Used for highly variable attacks Examine time-domain qualities on the victim IP

Number of simultaneous attacks Distribution of attack rates

Divide the trace in one minute periods An attack event = victim emits 10 backscatter packets during a minute

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Breakdown of response protocols

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Breakdown of victim port numbers

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Cumulative distributions of estimated attack rates

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Attack I mpact

No dominant mode for address distribution

A2 testing may be prevented

500 SYN packets overwhelm a server

38% of uniform random attacks 46% of event attacks

14,000 SYN packets overwhelm a specialized firewall

0.3% of uniform random attacks 2.4% of event attacks

They cannot asses the victim connectivity loss

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Cumulative Distribution of Attack Durations

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Probability Density of Attack Durations

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Victim Classification

Significant fraction directed against home machines (IRC channels) 2-3% target network infrastructure (name servers) 1-3% target routers .net, .com and .ro are the main TLD attacked Uniform AS distribution, more variation than TLD 95% of the victims were attacked less than 5 times A couple of victims were attacked more than 50 times

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Outline

Challenges Methods for Measuring DoS attacks Firewall Logs Network Telescopes I nternet Sinks Backscatter Background Radiation Conclusions

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Methodology of Background Radiation

Filtering

138 hosts scan more than half of LBL I P’s Can we include all unsuccessful connections? Separating unwanted traffic from benign or transient failure traffic Goal: provide a complete characterization of radiation = > construction of classifiers

Active Responders

Engage hosts Elicit particular intentions from remote sources

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Taming Traffic Volume

Scalability for responses on the order of billions of addresses Source Connection Filtering

Keep first N connections initiated by each source

Source Port Filtering

Keep N connections for each source/ destination port pair

Source Payload Filtering

Keep one instance for each type of activity per source

Source/ Destination Filtering

Keep N connections per each source/ destination pair

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Application Level Responders

Data Driven Approach Responders for the most common form of traffic

HTTP NetBI OS CI FS/ SMB DCE/ RPC Dameware Emulate few backdoors (MyDoom, Beagle)

Do not provide understanding of binary code

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Top Level Responders

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Honeynet Architecture

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Traffic Composition

Snapshots

80 hour traces collected at UW Campus on / 19 network One week trace at LBL on 10 contiguous / 24 networks One week trace at Class A with 1/ 10 sampling

99% of TCP packets are TCP/ SYN 8 ports (445, 80, 135) account for 83%

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Radiation activity at LBL

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Port Classification

Rank by the number of I P’s

Filter bias against sources that try to reach multiple destinations Assume destination symmetry Focus on the popularity Multi-source activity is intentional Per session activity

Analyze application semantic level background radiation distribution

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Port Activity(1)

TCP HTTP 80 - against Microsoft I I S:

WebDAV, Nimda, Code Red I I , Agobot

TCP DCE/ RPC 135/ 1025 – against Endpoint Mapper:

Blaster, Welchia, RPC170

TCP CI FS 139/ 445 – against NetBios Session Service for CI FS:

Locator, Epmapper, Samr-exe, W32-Xibo

TCP Dameware 135/ 1025 – against Dameware Remote Control TCP Virus Backdoors 3127/ 2745/ 4751 – MyDoom, Beagle (MZ marked files)

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Port Activity(2)

TCP Exploit Follow-Ups 1981/ 4444/ 9996 – two step worms: Blaster, Sasser, Agobot, Welchia UDP 53 – malformed DNS requests: UDP 137 – NetBios standard name queries UDP WM Pop-Up Spam 1026/ 2027 – DCE/ RPC exploits UDP 1434 – Slammer TCP 1433 – MS-SQL TCP 5000 – Universal Plug and Play

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Summary

Diurnal cycles in volume (bursty arrivals) Prevalence and variability of radiation Majority of traffic targets services with frequently exploited vulnerabilities Domination for TCP SYN/ RST packets Consistent source activities across ports Extremely dynamic traffic (daily)

For benign traffic, major shifts on lengthy times

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Outline

Challenges Methods for Measuring DoS attacks Firewall Logs Network Telescopes I nternet Sinks Backscatter Background Radiation Conclusions

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Conclusions (1)

Scalable architectures for large number of monitored IP’s (class A or multiple class B) Combination of passive and active measurements A large variety of filtering methods. Important assumptions Big differences between traces temporally and spatially A lot of place for improvement on data driven active responders

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Conclusions (2)

Large number of intrusions (scans, exploits, worms) – millions per day Widely distributed sources of attack Horizontal scans cover 70% of all scanning Dyurnal (daily cycles), extremely dynamic traffic Blacklists (worst offenders) can prevent majority

• f attacks

Frequent exploited vulnerabilities Prevalence of Internet DoS attacks

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

References

I nternet I ntrusions: Global Characteristics and Prevalence,

Vinod Yegneswaran, Paul Barford, Johannes Ullrich

On the Design and Use of I nternet Sinks for Network Abuse Monitoring, Vinod Yegneswaran, Paul Barford, Dave Plonka On the Marginal Utility of Network Topology Measurements, Paul Barford, Azer Bestavros, John Byers, Mark Crovella Characteristics of Network Traffic Flow Anomalies, Paul

Barford and David Plonka

Network Telescopes, David Moore I nferring I nternet Denial-of-Service Activity, David Moore I nferring I nternet Denial-of-Service Activity, David Moore Characteristics of I nternet Background Radiation, Ruoming Pang, Vinod Yegneswaran