ICmyNet.Flow: NetFlow based traffic investigation analysis traffic - - PowerPoint PPT Presentation

ICmyNet.Flow: NetFlow based traffic investigation analysis traffic investigation, analysis, and reporting

Slavko Gajin slavko.gajin@ rcub.bg.ac.rs AMRES – Academic Network of Serbia RCUB - Belgrade University Computer Center ETF – Faculty of Electrical Engineering

NetFlow

Challenges:

Who is consuming the bandwidth and how? D i i ht i t t k t ffi Deep insight into network traffic Recognize traffic anomaly – security threats Network optimization

S l i N Fl TM S

• lution – NetFlow TM

Protocol developed by Cisco for exporting IP flow statistics Other vendors: J-Flow, NetS tream, sFlow, IPFIX...

TF-NOC, 11.10.2011

How it works?

Exported data: S rc/ dst IP S rc/ dst ports S rc/ dst ports Protocol Total bytes, packets, fllows QoS QoS BGP src/ dst AS Exporter IP I / t t

Router (Exporter)

In/ out ports Timestamp … .

( p )

… .

TF-NOC, 11.10.2011

Why to use?

Performance management based on S NMP

network traffic – who is using? CPU/ Memory usage – why is increased? who is talking with whom?

TF-NOC, 11.10.2011

NetFlow Analyzers

Collect, process, present and analyze NetFlow data Most popular commercial solutions: p p

S

• larwinds, MenageEngine, S

crutinyzer...

ICmyNet.Flow ICmyNet.Flow

AMRES participated the development with expertise, requirements, testing Competitive with other commercial solutions p Full free software available for NRENs and their members www.icmynet.com live demo download free trial user manual support contact

TF-NOC, 11.10.2011

pp

System architecture

Binary raw data files

Flows_2009-10-21-09.20.00

ICmyNet.Flow Collector

Flows_2009-10-21-09.25.00 Flows_2009-10-21-09.30.00

Collector ICmyNet.Flow Aggregator Database gg g ICmyNet.Flow Web

TF-NOC, 11.10.2011

Raw Data Files Archive

Parameters for traffic analysis

Detailed information about:

IP subnets traffic Hosts traffic Hosts traffic Network S ervices and applications based on TCP/ UDP ports Network Protocols (TCP, UDP, ICMP, GRE...) QoS markers (ToS , IP precedence or DS CP) BGP Autonomous S ystem Numbers

For each parameter counters for:

Traffic Bandwidth (in bits/ s, kbps, Mbps..) Traffic Volume (in MBytes, GB, TB...) Number of Packets, volume and time based diagrams (pps) Number of Flows, volume and time based diagrams (fps)

Configurable cut-off percentage or data amount for negligible consumers

TF-NOC, 11.10.2011

Overview

Web application is chosen for the user interface

De-facto standard for network management applications Accessibility, permanent development, flexibility

Java application working under Tomcat

JS F technologies

TF-NOC, 11.10.2011

Traffic Patterns

Traffic Pattern - Traffic of Interest, defined by user Matches the traffic between “ Internal” and “ External” network S tatistics IS NOT per interface Statistics IS per subnet in Traffic Pattern D fi d b Defined by

IP networks

• ther NetFlow parameters

Internal External network network

TF-NOC, 11.10.2011

Traffic Patterns

Internet

Exclude 10 0 0 0/8

Internal Network

10.0.0.0/8 Exclude 10.0.0.0/8

TF-NOC, 11.10.2011

Traffic Patterns

External Network Internal Network

TF-NOC, 11.10.2011

Traffic Patterns

Application Servers

172 16 0 0/24 172.16.0.0/24

Internal Network

10.0.0.0/8

TF-NOC, 11.10.2011

Traffic Pattern – basic element of analysis

Internal Network 10.0.0.0/8 External Network 10.0.0.0/8

TF-NOC, 11.10.2011

Traffic Patterns

Advanced Traffic Patterns can be configured with flexible matching

• f any supported NetFlow field

Examples: Examples: AMRES -> Facebook Internal address 147.91.0.0/ 16, S rc or Dst AS 32934 (Facebook) (Facebook) Router X Internal & External address: 0.0.0.0/ 0, Exporter 10.1.1.1 Potential attacks: Potential attacks: S rc or Dst port: 22, 135-139, 445, 1434,… “Weird” Protocols: Protocols: Exclude 6 (TCP) or 17 (UDP) Protocols: Exclude 6 (TCP) or 17 (UDP) Blocked Traffic: Out Interface: 0 (Null)

TF-NOC, 11.10.2011

Subnets

S ubnets Defined by name and IP y address range in Internal network View tab / Address S pace IP address hierarchy of IP address hierarchy of subnets in a tree structure IPv6 are fully supported!

TF-NOC, 11.10.2011

Subnet Sets

S ubnet S et User defined group of S ubnets and/ or other S ubnets S ets View tab / Custom S pace User defined hierarchy of S ubnet S ets and belonging S ubnets S ets and belonging S ubnets Any logical grouping of S ubnets:

Institutions Faculties Universities S chools Libraries Libraries etc...

TF-NOC, 11.10.2011

View Tab – Top N

TF-NOC, 11.10.2011

View Tab – Chart

TF-NOC, 11.10.2011

View Tab – List

TF-NOC, 11.10.2011

Archived raw data review Raw data are archived in the files created every 5 minutes

Compressed and archived in separate folder Every single flow is saved

Raw data View Raw data View

Access, review and explore raw data files S earching for a single flow or event that traversed the t k network

TF-NOC, 11.10.2011

Archived raw data review

TF-NOC, 11.10.2011

Searching and grouping raw data

Filter, group and sort by any meaningful column

TF-NOC, 11.10.2011

Case study

Analysis of traffic anomaly

TF-NOC, 11.10.2011

TF-NOC, 11.10.2011

TF-NOC, 11.10.2011

TF-NOC, 11.10.2011

TF-NOC, 11.10.2011

TF-NOC, 11.10.2011

TF-NOC, 11.10.2011

TF-NOC, 11.10.2011

TF-NOC, 11.10.2011

TF-NOC, 11.10.2011

TF-NOC, 11.10.2011

TF-NOC, 11.10.2011

TF-NOC, 11.10.2011

TF-NOC, 11.10.2011

Configuration issues – Interfaces

NetFlow configured in both directions on interfaces Exported data duplication

Host A

TF-NOC, 11.10.2011

NetFlow Collector

Configuration issues – Interfaces

NetFlow configured in ingress direction on all interfaces No data duplication

TF-NOC, 11.10.2011

Host A

Configuration issues – Interfaces

NetFlow configured in ingress direction on all interfaces with redundant links D t d li ti ! Data duplication!

Gi0/3 Gi0/1 Gi0/1 Gi0/2 Gi0/3

TF-NOC, 11.10.2011

Host A

Configuration issues – Interfaces

S

• lution:

Configure ingress direction on edge links (do not configure on core links) (do not configure on core links) Exclude interfaces on core links between exporters from Traffic Pattern

TF-NOC, 11.10.2011

Configuration issues - Timers

Timer – aging Long

Defines data export interval for long flows – 5 min

Bits/s Bits/s Received Flow Real Flow Bits/s Bits/s Time of export Time of export 20k 5K

TF-NOC, 11.10.2011

t 20 minutes t 5 minutes

Configuration issues - Timers

Fast

Defines data export criteria based on the threshold ( 100packets) (~100packets) Preserves memory overload

TF-NOC, 11.10.2011

Configuration issues - Aggregation

Receiving application is using 5 minute aggregation

TF-NOC, 11.10.2011

NetFlow statistics from non-netflow device?

L2 switches usually do not support NetFlow protocol Examples: Examples:

LAN networks NREN member connected to

FastEthernet 0/1 FastEthernet 0/2 Gigabit Ethernet 0/0

NREN backbone

S

• lution

Port mirroring S i h NIC

Mirrored Ports

S erver with two NIC S

• ftflowd

http:/ / www.mindrot.org/ proj ects / softflowd/

FastEthernet 0/23

/ softflowd/ http:/ / code.google.com/ p/ softflo wd/

Interfaces info disappears,

FastEthernet 0/24 Gigabit Ethernet 0/1

NetFlow Date Export

TF-NOC, 11.10.2011

but Traffic Patterns don’ t need it!

NetFlow Emulator SOFTFLOW DEAMON

Conclusions

ICmyNet.Flow Pros

Traffic Patterns Traffic Patterns S ubnets and S ubnet S ets hierarchy Works with non-netflow devices Raw data inspection Full IPv6 support Full IPv6 support Web based, j ava – OS independent

Cons

S

• me net admins prefer link based statistics

S

• me net admins prefer link based statistics

(physical infrastructure view) Lack of top conversations statistics (plan to support in new version, 2012)

Links

www.icmynet.com live.icmynet.com/ NetFlowWeb

TF-NOC, 11.10.2011

Questions

TF-NOC, 11.10.2011

slavko.gaj in@ rcub.bg.ac.rs