VisFlowConnect-IP: A Link-Based Visualization of Netflows for - PowerPoint PPT Presentation

VisFlowConnect-IP: A Link-Based Visualization of Netflows for Security Monitoring William Yurcik <byurcik@ncsa.uiuc.edu > National Center for Supercomputing Applications (NCSA) University of Illinois at Urbana-Champaign FIRST’06 Baltimore Maryland USA

Slide 2/58 • Motivation • Network Visualization for Security • Our Approach: VisFlowConnect-IP • Use Examples • Future Work: Link-Based Clustering • Summary

Slide 3/58 • Motivation Motivation • • Network Visualization for Security • Our Approach: VisFlowConnect-IP • Use Examples • Future Work: Link-Based Clustering • Summary

Slide 6/58 More Lessons Learned from Castles • Even medieval castles have monitoring systems for their innermost keeps • Internet security should be designed like a castle, with multiple layers of defenses for an attacker to avoid detection – Reduces the space of actions that an attacker can take and remain undetected – Components of a security monitoring framework can monitor each other • Have clear observation points – Internet analogy are data source and process

Fort McHenry

Slide 8/58 OODA Loop

Slide 9/58 OODA Loop for Internet Security Data Sources Human Collaboration (empirical, simulation, analytical) (virtual presence, transparent) Inferences for Action Storage Processing (distributed, fast, convenient) (computation, data analysis, discovery)

Slide 10/58 Visualization in OODA Loop Data Sources Human Collaboration (empirical, simulation, analytical) (virtual presence, transparent) visualization visualization Inferences for Action display systems display systems Storage Processing (distributed, fast, convenient) (computation, data analysis, discovery)

Slide 11/58 What is Visualization? Visual 1.235 4.351 Visual 2.981 7.989 Representation 7.112 5.231 Representation 9.722 7.111 Model Model 1.562 7.544 Data Image

Slide 12/58 Visualization Can Help Empirical Data: Visual vs Numerical (Visual Wins!)* Visual vs Auditory (Visual Wins)* Visual vs Tactile (Visual Wins)* Visual Spatial vs Visual Color (Visual Spatial Wins!)* [Chris Wickens, National Academy of Sciences Workshop on Visualizing Uncertainty, March 3, 2005]

Slide 13/58 Visualization Can Help Empirical Data: Visual vs Numerical (Visual Wins!)* Visual vs Auditory (Visual Wins)* Visual vs Tactile (Visual Wins)* Visual Spatial vs Visual Color (Visual Spatial Wins!)* [Chris Wickens, National Academy of Sciences Workshop on Visualizing Uncertainty, March 3, 2005] How? 1) See Previously Obscured Things 2) See New Things Faster (I never saw that before) 3) Share Insights (Do you see what I mean?)

Slide 14/58 • Motivation • Network Visualization for Security Network Visualization for Security • • Our Approach: VisFlowConnect-IP • Use Examples • Future Work: Link-Based Clustering • Summary

Slide 15/58 Current Net Vis Security Ops Tools

Slide 16/58 Etherape by Juan Toledo can be found at http://etherape.sourceforge.net/ screenshot: http://www.solaris4you.dk/sniffersSS.html

Slide 17/58 Lumeta’s Peacock Diagrams

Slide 18/58 Caida’s Walrus

Slide 19/58 Research: Network Viz for Security • Link-based approaches • Host-based approaches – Represent each host by – Represent each host by a a point point – Fix each host at a certain – Fix each host at a certain position according to its position according to its IP IP – Visualize traffic between – Visualize statistics of hosts by linkages each host (NVisionIP- NCSA) (Teoh et al, 2004) (Elisha-Teoh et al)

Slide 20/58 AT&T’s Graphiz

Slide 21/58 Graphviz again

Slide 22/58 • Motivation • Network Visualization for Security • Our Approach: Our Approach: VisFlowConnect VisFlowConnect- -IP IP • • Use Examples • Future Work: Link-Based Clustering • Summary

Slide 23/58 Our Design Goals • Traffic dynamics over time • Filtering • Scalability • Expose hidden structures & patterns for further investigation

Slide 24/58 System Architecture Visualization Netflow Traffic Statistics Logs Host 1 agent Host 2 …… …… Host k Host Traffic Statistics

Slide 25/58 Reading Netflow Logs • An agent reads records log (or streaming) – send record to VisFlowConnect-IP when requested • Reorder NetFlow records with record buffer – records are not strictly sorted by time stamps – use a record buffer

Slide 26/58 VisFlowConnect- -IP IP VisFlowConnect

Slide 27/58 VisFlowConnect- -IP IP VisFlowConnect Main View Main View inside inside outside outside outside outside hosts hosts domains domains domains domains axis axis axis axis axis axis

Slide 28/58 VisFlowConnect- -IP IP VisFlowConnect Internal View Internal View Internal Internal Internal Internal network network network network receivers receivers sources sources

Slide 29/58 VisFlowConnect- -IP IP VisFlowConnect VisFlowConnect-IP NVisionIP Domain View Domain View see see activity activity within an within an external external network network domain domain

Slide 30/58 Creating Dynamic Animation • Visualizing traffic statistics with time – update visualization after each time unit • How to arrange domains/hosts? – 100s of domains/hosts; added/removed in time – fairly stable positioning • Solution: sort by IP – domain/hosts move up or down

Slide 31/58 Time Window • User is usually interested in most recent traffic (e.g., in last minute or last hour) • VisFlowConnect-IP only visualizes traffic in a user adjustable time window – Update traffic statistics when • A record comes into time window • A record goes out of time window

Slide 32/58 Time Dynamics analog analog clock clock time window time window time axis time axis timestamp timestamp

Slide 33/58 Filtering/Highlighting Capability • Approach – Filter out “good” traffic • User specifies a list of filters: + : (SrcIP=141.142.0.0 − 141.142.255.255), (SrcPort=1 − 1000) //keep all records from domain 141.142.x.x, from port 1 – 1000 − : (SrcPort=80) − : (DstPort=80) //discard records of http traffic – Highlight “traffic of interest” • traffic colored by port

Slide 34/58 Highlighting “Traffic of Interest” File I/O File I/O Net Net highlighted highlighted Domain Domain ports ports VCR controls VCR controls highlighted flow highlighted flow

Slide 35/58 Storing Traffic Statistics • Store traffic statistics Sorted tree of domains involving each domain by a sorted tree – only necessary information for visualization is stored – statistics for every Host statistics domain or host can be updated efficiently

Slide 36/58 Scalability Experiments Runtime & Memory wrt records Runtime & Memory wrt time window size

Slide 37/58 • Motivation • Network Visualization for Security • Our Approach: VisFlowConnect-IP • Use Examples Use Examples • • Future Work: Link-Based Clustering • Summary

Slide 38/58 Example 1: MS Blaster • MS Blaster virus causes machines to send out 92 byte pakcets to many machines

Slide 39/58 Example 2: ? multiple connections to NCSA multiple connections to NCSA cluster from same domain cluster from same domain (scan?, DoS (scan?, DoS?) ?)

Slide 40/58 Example 2: ? Destination: Source: Destination: Source: multiple connections to NCSA multiple connections to NCSA consecutive consecutive consecutive consecutive cluster from same domain cluster from same domain IP addresses IP addresses IP addresses IP addresses (scan?, DoS (scan?, DoS?) ?)

Slide 41/58 Example 2: Grid Networking cluster- cluster -to to- -cluster communications cluster communications Destination: Source: Destination: Source: multiple connections to NCSA multiple connections to NCSA consecutive consecutive consecutive consecutive cluster from same domain cluster from same domain IP addresses IP addresses IP addresses IP addresses (scan?, DoS (scan?, DoS?) ?)

Slide 42/58 Example 3: ?

Slide 43/58 Example 3: ? NCSA web servers NCSA web servers

Slide 44/58 Example 3: Web Crawlers muitiple crawlers indexing NCSA web server content muitiple crawlers indexing NCSA web server content Web crawlers Web crawlers NCSA web servers NCSA web servers

Slide 45/58 • Motivation • Network Visualization for Security • Our Approach: VisFlowConnect-IP • Use Examples • Future Work: Link Future Work: Link- -Based Clustering Based Clustering • • Summary

Slide 46/58 Visual Clustering of Hosts • Visual clustering of hosts by link analysis – represent each host by a point – arrange hosts so related hosts are clustered

Slide 47/58 Relationships between Hosts • Direct communications – traffic intensity between two hosts NBA • Indirect communications NCAA – eg two basketball fans ESPN • Port Activity (Services) IRC – Eg web servers/surfers, IRC IRC

Slide 48/58 Initialization of Nodes Colored points represent internal hosts, and gray points represent external ones. Size of a point is proportional to logarithm of traffic volume involving this host.