1 Some Challenges Some Common False Positives What is normal - PDF document

Thanks… • To Anthony Joseph, Doug Tygar, Umesh Intrusion Detection Vazirani, and David Wagner for generously allowing me to use their slides as the basis for this set of slides. Fall 2008 CS 334: Computer Security 1 Fall 2008 CS 334: Computer Security 2 Outline Intrusion Detection History • History • Detecting attempts to penetrate our systems – Used for post-mortem activities • Network-based Host Compromise – Related problem of extrusion (info leaking out) • Host-based Network Intrusion Detection • In pre-network days (centralized mainframes)… – Signature-based – Primary concern is abuse and insider information – Anomaly-based access/theft • Distributed Network Intrusion Detection – Reliance on logging and audit trails – Honeypots • But, highly labor intensive to analyze logs – Tarpits – What is abnormal activity? • An attack against an IDS – Ex: IRS employees snooping records – Ex: Moonlighting police officers Fall 2008 CS 334: Computer Security 3 Fall 2008 CS 334: Computer Security 4 Host-based Network Intrusion Network-based Host Compromises Detection • At each host, monitor all incoming and outgoing network traffic – for each packet: – Analyze 4-tuple and protocol • How do remote intruders gain access? – Examine contents – … • They attempt network-based attacks that • Challenge: Separate “signal” from “noise” exploit OS & app bugs – Signal is an attack (intrusion) – Ex: Denial of service, spyware install, zombie,… – Noise is normal “background” traffic – Assumption: can separate signal and noise… Fall 2008 CS 334: Computer Security 5 Fall 2008 CS 334: Computer Security 6 1

Some Challenges Some Common False Positives • What is normal traffic? • Proximity probes – Server, desktop, PDA, PDA/phone, … – Website load balancers will probe your machine for proximity – My normal traffic? Your normal traffic? – Connect to website hosted by mirror-image.com, and • What if I’m hurt and work from home for a while? >10 load balancers in 6 countries probe your machine – Lots of data for servers • Stale IP caches • Why do we need sufficient signal and noise – Using dynamic IP addresses, you may get the “old” separation? address of someone who was running a P2P app – To avoid too many false alarms! – Peers continue to try to “re-connect” • What happens if signals are missed? • Web posts with dynamic IP addresses – Possible intrusion! – Spiders crawl machine currently using IP address Fall 2008 CS 334: Computer Security 7 Fall 2008 CS 334: Computer Security 8 Lots and Lots of Data!! Trace Analysis • Network trace from Win2K desktop Fall 2008 CS 334: Computer Security 9 Fall 2008 CS 334: Computer Security 10 Some Background: MS Blaster Analyzing Host-based Trace Data • Worm affected Win XP and Win 2K, began • TCP spreading August 13, 2003 connection • Programmed to “SYN flood” port 80 of probes on windowsupdate.com on August 15, thus creating port 445 a DDoS attack • Exploited buffer overflow vulnerability (no surprise there) in Microsoft’s DCOM RPC service (located at • Day 0 is port 135) March 4, – DCOM (Distributed Component Object Model) is 2003 proprietary MS technology for communicating among software components distributed throughout network Fall 2008 CS 334: Computer Security 11 Fall 2008 CS 334: Computer Security 12 2

Some Background: MS Blaster MSBlaster in Detail • Port 445: reserved for MS Directory services. • TCP 445 • MS silently installed Internet server into every probes/hr version of Win 2K, accessible via port 445 • Allows crackers to remotely log on to computers, then upload and run any program without computer owner being aware • Hour 0 is • One method for setting up a “Botnet” on July • In addition to port 135, Blaster also targeted 20, 2003 ports 139 and 445 • Blaster propagated by testing connections to random IP addresses using these ports (hours) Fall 2008 CS 334: Computer Security 13 Fall 2008 CS 334: Computer Security 14 MSBlaster in More Detail Example Common Attack • TCP 445 • Port scanning a host probes / – Trying to connect/send data to different ports/ 10 min protocols: sequential scan of host – Nmap tool (http://www.insecure.org/nmap/) • Determines OS/hostname/device type detection via service fingerprinting (ex: SGI IRIX has svc on TCP port 1) • Minute 0 is • Determines what svc is really listening on a port and 15:20 on can even determine app name and version July 20, • Operates in optional obfuscation mode 2003 • How to detect attack? (minutes) Fall 2008 CS 334: Computer Security 15 Fall 2008 CS 334: Computer Security 16 Intrusion Detection Using Signals Intrusion Detection Using Noise • This is an anomaly detection problem • This is a misuse detection problem – Need to learn normal behavior – Similar problem to virus detection – “Match what’s different” – “Match what you know” • High-level solution: • High-level solution: – Try to identify what is normal traffic – Collect info about attack methods and types • Common 4-tuple/protocol • 4-tuple/protocol – Heuristic: Look for major deviations (outliers) • Packet contents • Ex: unusual target port, source addr, or port sequence – Create and look for signatures (scan) • Slammer packet, port scan, … – Apply AI: Statistical Learning Techniques Fall 2008 CS 334: Computer Security 17 Fall 2008 CS 334: Computer Security 18 3

Signature Detection Signature Detection • Language to specify intrusion patterns – 4-tuple/protocol and potential intrusion values • Snort tool (http://www.snort.org/) • Ex: External host -> file server (port 110, 135, …) – 2 million downloads, 100,000+ active users, • Ex: Internal workstation -> external P2P host • Advantages – Packet contents – Very low false positive (alarm) rate • Could be single or multiple packets (stream reconstruction) – Sequence of 4-tuple/protocol and packets • Disadvantages • Also, model of protocol/app finite state machine – Only able to detect already known attacks • Lots of state in pattern matching engine – Simple changes to attack can defeat detection • Example rule: • Ex: Scan every even port, then every odd port… – alert tcp any -> my ip 21 (content:"site exec”; content:"%"; msg:"site exec buffer overflow attempt";) Fall 2008 CS 334: Computer Security 19 Fall 2008 CS 334: Computer Security 20 Anomaly Detection Anomaly Detection • Advantages • Analyze normal operation (behavior), look for – Can recognize “evolved” and new attacks anomalies • Disadvantages – Uses AI techniques: Statistical Learning Techniques – High false positive rate (alarms) – Compute statistical properties of “features” – May have delayed alarm • 4-tuple, protocol, packet contents, packets/sec, range – Some attacks can hide in “normal” traffic of port numbers, … – SLT requires training on known good data – Report errors if statistics are outside of “normal” – Hard to capture protocol state behavior (FSM) range – Problems when what’s “normal” changes • Ex: flash crowds Fall 2008 CS 334: Computer Security 21 Fall 2008 CS 334: Computer Security 22 Super Stealthy Port Scanning Distributed Intrusion Detection • Place appliance in the network at choke point or, share results across machines • Apply signature or anomaly detection across larger data set • Advantages: – Easier to detect stealth probes of large number of • Use many zombies (each scans a few ports/ machines hour of target) • Disadvantages: – Each zombie is assigned many machines to scan – Large amount of data to communicate • Fast to scan both one machine, and many • Very hard to detect at targets! Fall 2008 CS 334: Computer Security 23 Fall 2008 CS 334: Computer Security 24 4