VisFlowConnect-IP: A Link-Based Visualization of Netflows for - - PowerPoint PPT Presentation
VisFlowConnect-IP: A Link-Based Visualization of Netflows for Security Monitoring William Yurcik <byurcik@ncsa.uiuc.edu > National Center for Supercomputing Applications (NCSA) University of Illinois at Urbana-Champaign FIRST06
William Yurcik
<byurcik@ncsa.uiuc.edu>
National Center for Supercomputing Applications (NCSA) University of Illinois at Urbana-Champaign FIRST’06 Baltimore Maryland USA
A Link-Based Visualization of Netflows for Security Monitoring
Slide 2/58
Slide 3/58
systems for their innermost keeps
castle, with multiple layers of defenses for an attacker to avoid detection
– Reduces the space of actions that an attacker can take and remain undetected – Components of a security monitoring framework can monitor each other
– Internet analogy are data source and process
Slide 6/58
Slide 8/58
Data Sources (empirical, simulation, analytical) Storage (distributed, fast, convenient) Processing (computation, data analysis, discovery) Human Collaboration (virtual presence, transparent)
Inferences for Action Slide 9/58
Inferences for Action
Processing (computation, data analysis, discovery) Data Sources (empirical, simulation, analytical) Storage (distributed, fast, convenient) Human Collaboration (virtual presence, transparent)
visualization visualization display systems display systems Slide 10/58
1.235 4.351 2.981 7.989 7.112 5.231 9.722 7.111 1.562 7.544
Visual Representation Model Visual Representation Model Data Image Slide 11/58
Empirical Data: Visual vs Numerical (Visual Wins!)* Visual vs Auditory (Visual Wins)* Visual vs Tactile (Visual Wins)* Visual Spatial vs Visual Color (Visual Spatial Wins!)*
[Chris Wickens, National Academy of Sciences Workshop on Visualizing Uncertainty, March 3, 2005]
Slide 12/58
Empirical Data: Visual vs Numerical (Visual Wins!)* Visual vs Auditory (Visual Wins)* Visual vs Tactile (Visual Wins)* Visual Spatial vs Visual Color (Visual Spatial Wins!)*
[Chris Wickens, National Academy of Sciences Workshop on Visualizing Uncertainty, March 3, 2005]
How?
1) See Previously Obscured Things 2) See New Things Faster (I never saw that before) 3) Share Insights (Do you see what I mean?)
Slide 13/58
Slide 14/58
Slide 15/58
Etherape by Juan Toledo can be found at http://etherape.sourceforge.net/ screenshot: http://www.solaris4you.dk/sniffersSS.html
Slide 16/58
Slide 17/58
Slide 18/58
approaches
– Represent each host by a point – Fix each host at a certain position according to its IP – Visualize statistics of each host
– Represent each host by a point – Fix each host at a certain position according to its IP – Visualize traffic between hosts by linkages
(NVisionIP- NCSA) (Teoh et al, 2004)
Slide 19/58
(Elisha-Teoh et al)
Slide 20/58
Slide 21/58
Slide 22/58
further investigation
Slide 23/58
Netflow Logs Traffic Statistics
Host 1 Host 2 Host k …… ……
Host Traffic Statistics Visualization
agent
Slide 24/58
– send record to VisFlowConnect-IP when requested
– records are not strictly sorted by time stamps – use a record buffer
Slide 25/58
Slide 26/58
domains domains axis axis inside inside hosts hosts axis axis
domains domains axis axis
Slide 27/58
Internal Internal network network sources sources Internal Internal network network receivers receivers
Slide 28/58
see see activity activity within an within an external external network network domain domain Slide 29/58
time
– update visualization after each time unit
domains/hosts?
– 100s of domains/hosts; added/removed in time – fairly stable positioning
– domain/hosts move up or down
Slide 30/58
traffic (e.g., in last minute or last hour)
in a user adjustable time window
– Update traffic statistics when
Slide 31/58
time axis time axis time window time window timestamp timestamp Slide 32/58 analog analog clock clock
– Filter out “good” traffic
+: (SrcIP=141.142.0.0−141.142.255.255), (SrcPort=1−1000) //keep all records from domain 141.142.x.x, from port 1 – 1000 −: (SrcPort=80) −: (DstPort=80) //discard records of http traffic
– Highlight “traffic of interest”
Slide 33/58
highlighted highlighted ports ports File I/O File I/O VCR controls VCR controls Net Net Domain Domain highlighted flow highlighted flow Slide 34/58
involving each domain by a sorted tree
– only necessary information for visualization is stored – statistics for every domain or host can be updated efficiently
Host statistics
Sorted tree
Slide 35/58
Runtime & Memory wrt records
Slide 36/58
Runtime & Memory wrt time window size
Slide 37/58
causes machines to send out 92 byte pakcets to many machines
Slide 38/58
multiple connections to NCSA multiple connections to NCSA cluster from same domain cluster from same domain (scan?, (scan?, DoS DoS?) ?) Slide 39/58
Source: Source: consecutive consecutive IP addresses IP addresses Destination: Destination: consecutive consecutive IP addresses IP addresses multiple connections to NCSA multiple connections to NCSA cluster from same domain cluster from same domain (scan?, (scan?, DoS DoS?) ?) Slide 40/58
Source: Source: consecutive consecutive IP addresses IP addresses Destination: Destination: consecutive consecutive IP addresses IP addresses cluster cluster-
to-
cluster communications multiple connections to NCSA multiple connections to NCSA cluster from same domain cluster from same domain (scan?, (scan?, DoS DoS?) ?) Slide 41/58
Slide 42/58
NCSA web servers NCSA web servers Slide 43/58
muitiple muitiple crawlers indexing NCSA web server content crawlers indexing NCSA web server content NCSA web servers NCSA web servers Web crawlers Web crawlers Slide 44/58
Slide 45/58
– represent each host by a point – arrange hosts so related hosts are clustered
Slide 46/58
– traffic intensity between two hosts
– eg two basketball fans
– Eg web servers/surfers, IRC
NBA NCAA ESPN IRC IRC
Slide 47/58
Colored points represent internal hosts, and gray points represent external ones. Size
Slide 48/58
– divide the space into many small grids – DBSCAN to find such dense grids – highlight dense grids and connect grids
Slide 49/58
These green nodes are from 141.142.44.2x, which should be a cluster. They have much traffic in port 90.
90 Slide 50/58
Slide 51/58
realtime for security monitoring purposes
specialized security domains
– storage systems, linux clusters, etc.
<http://security.ncsa.uiuc.edu/distribution/VisFlowConnectDownLoad.html>
<http://www.ncassr.org/projects/sift/papers/>
Slide 52/58
<http://www.projects.ncassr.org/sift/vizsec/>
Slide 53/58
Large Installation System Administration Conference (LISA), San Diego, CA USA, 2005.
Visualizing Netflows," FLOCON - Network Flow Analysis Workshop, Pittsburgh PA USA, 2005.
System for IP Security Situational Awareness," 3rd IEEE Intl. Workshop on Information Assurance (IWIA) University of Maryland USA, 2005.
Visualizations of Link Relationships for Security Situational Awareness," CCS Workshop on Visualization and Data Mining for Computer Security (VizSEC/DMSEC) held in conjunction with 11th ACM Conf. on Computer and Communications Security, 2004.
Security Situational Awareness by Visualizing Network Traffic Flows," 23rd IEEE Intl. Performance Computing and Communications Conference (IPCCC), 2004.
System and Network Views for Intrusion Detection," Workshop on Link Analysis, Counter-terrorism, and Privacy held in conjunction with the SIAM International Conference on Data Mining (ICDM), 2004.
Slide 54/58
<http://security.ncsa.uiuc.edu/distribution/VisFlowConnectDownLoad.html>
Slide 55/58
supported by the Office of Naval Research.
recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the Office of Naval Research.
Slide 56/58
NetFlows can identify connection-oriented attacks like DoS, DDoS, malware distribution, worm scanning, etc…
given time? (upgrades)
Slide 58/58