Monitoring Tool for Analysing the Use of the Internet Services in - PDF document

“Monitoring Tool for Analysing the Use of the Internet Services in the Spanish Academic Network” Jordi Domingo-Pascual Universitat Politècnica de Catalunya jordi.domingo@ac.upc.es Partners: UC3M, UPC and UPM Project Objectives ❐ Traffic Capture Subsystem ● High Speed ● AAL5 Reassembly ● Modular and scalable ● Low cost ❐ Support for many Traffic Analysis tools: ● Identification and aggregation of bi-directional flows ● Traffic classification by usage ● Traffic classification by origin / destination ● Internet header verification ● Detailed analysis (including contents for AUP audits) 1

Functional Architecture MEHARI System Analysis Subsystem Capture Subsystem Analysis Platform(s) Auto- regulation IP Biflows Capture Platform(s) ATM 1 PPS + symptoms Preprocessing PPS Application ATM 1 ATM 0 Module ATM 1 Modules ATM 0 Traffic ATM 0 Samples Data base -patterns - addresses ATM Statistics - ... and Reports Cells Operator ATM Backbone Capture point Capture Subsystem ❐ Modular and scalable ● N units over the same or different trunk links ● Requires high speed connection to the analysis subsystem ❐ Senses ALL VPI/VCI in the fiber ● Captures in promiscuous or filtered mode over VPI/VCI list ❐ Capture capacity for each unit ● Sustained Average of 8 Mbit/s for a 6,000 Euros unit ● 3,000% better price/performance than commercial protocol analyzers (2 Mbit/s on HP BSTS) ● Capture rate controled by analysis rate 2

Capture Subsystem: Hardware ❐ 1 PC Pentium 200 MHz ❐ 128M RAM ❐ 4G Hard Disc ❐ 1 Ethernet NIC (10/100) ❐ 2 ATM NICs PCI SC-155, Fore PCA-200EPC 256KB ❐ 2 passive f.o. splitters SC ❐ Cost: ~ 5000 $ Capture Subsystem: Software ❐ S.O. Unix FreeBSD 2.2.5 ❐ Software OC3MON/BSD (modified by DIT-UPM) ● fatm-driver ● capture ❐ dumpcap application developped by DIT-UPM ● VPI/VCI demultiplexing and filtering ● reassembly of AAL5 frames ● dump to capture files ❐ NFS client for downloading capture files to the analysis subsystem 3

Capture Files length (bytes) timestamp UNIX Truncated AAL5 (seg.µseg) VPI/VCI info field 0:893083746.654070:100/1:1064 :45000428E81B40002F062E36C600B... 1:893083746.654090:100/1:44:4500002C00AC400037069CF5CC4B3C... frame 2:893083746.654101:100/1:40:45000028455840003606052FCF4F2C1... seq_num 3:893083746.654280:103/224:1500:450005DC6C4B4000FD06142640... 4:893083746.654288:103/224:40:45000028240440007B06401E829FD... 5:893083746.654517:103/224:400:45000190B30340001D06B516238A... ...... 1668:893083746.813551:100/1:281:4500011976710000FB04BFFCE40... # init_time=893083746.652986 final_time=893083746.813582 cap_time=0.160596 Files with programmable granularity Functional Architecture MEHARI System Analysis Subsystem Capture Subsystem Analysis Platform(s) Auto- regulation IP Biflows Capture Platform(s) ATM 1 PPS + symptoms Preprocessing PPS Application ATM 1 ATM 0 Module ATM 1 Modules ATM 0 Traffic ATM 0 Samples Data base -patterns - addresses Statistics ATM - ... and Reports Cells Operator ATM Backbone Capture point 4

Analysis Subsystem: Hardware ❐ PC Pentium 200 MHz ❐ 128M RAM ❐ 4G Hard Disc ❐ 2 Ethernet NICs (10/100) ❐ Cost: ~ 2500 $ Analysis Subsystem: Software ❐ Modular and Scalable Architecture ● N analysis platforms connected to one or more capture platforms ● PC Linux RedHat 5.0 ❐ Pre-processing module: ● Process of the samples “on-the-fly” ● Extract “relevant” information ● Erase capture files ❐ Analysis tools or Application modules: ● Classify traffic by use ● Classify traffic by destination ● Packet header verification ● Server location tool ● Usage based pricing tool ● ... 5

Analysis Subsystem Architecture Common Configuration language Common format Generic Generic Analysis Analysis Preprocessing Analysis Analysis Module module Module Module module (filter) (filter) Specific Specific Analysis Analysis Module module Modularity and Scalability P 1.2 P 1.1.2 P 1.1 P 1.1.1 P 1.1.3 P 1.3 P 1.3.1 ❐ Process tree structure for information flow ❐ Interprocess Comunication using shared files ❐ May be distributed among several machines using NFS 6

Key Issues of the Architecture ❐ Modularity ❐ Common data types and file format definition ❐ Common configuration interface ❐ This allows: ● the insertion of new processes in the whole analysis chain ● to obtain partial results in the analysis chain ● to control processes and to browse results from a single GUI Pre-processing Module ❐ Main functions ● packet agreggation to flows ● packet analysis ● count of symptoms associated to each flow ❐ Produces flow list with associated information: ● flow desc with packet and byte count ● weighted list of symptoms ❐ Highly configurable: ● symptom definition and inter-relation ● aggregation period 7

Pre-processing Module CAP*.1 AFL*.1 AAF*.1 Anaflow Acum BFL* Base de datos Biflow de patrones AAF*.0 CAP*.0 AFL*.0 Anaflow Acum Flujos Flujos Flujos Paquetes IP Unidireccionales Unidireccionales Bidireccionales Acumulados ❐ anaflow: análisis de paquetes IP y extracción de síntomas ❐ accum: agregación de nº bytes, nº paquetes y síntomas en flujos unidireccionales, según período ❐ biflow: correlación de tráfico E/S para obtención de flujos bidireccionales, conservando síntomas Application Modules ❐ Traffic analysis by volume ❐ Traffic analysis by origin/destination ❐ Header analysis and verification ❐ Server location tool ❐ Traffic classification (explicit routing) ❐ Usage based pricing tool 8

Trial on Spanish Academic Network: RedIRIS RedIRIS: the Spanish NRN Splitters GIGACOM RedIRIS RedIRIS Telefónica ATM Core Regional Router Nodes Network ATM Access Switch 100 BaseT 1 STM-1 ATM Ethernet Internet Optical 0 (RedIris ) Interfaces NFS Remote Access Analysis PC Traffic Capture PC (LINUX) (FreeBSD) Spanish Academic Network Topology 9

Some applications of these tools ❐ Traffic monitoring ● Billing and charging models for NRN and Corporate Networks ● Network configuration - Resources dimensioning - Placing Proxies, ... ❐ Service usage control ● Control that the services are used responsibly, i. e. auditing the academic networks AUP (Acceptable Use Policy) ● Security Summary ❐ Modular, scalable and extensible architecture ❐ Capture systems with excellent price/performance ❐ Flow information aggregation with symptoms and bi- directional flow correlation ❐ Intermediate data base of patterns and addresses ❐ Application modules currently implemented: ● Classification by usage (AUP) ● Classification by origin/destination ● Packet Header analysis 10

Traffic Analysis by volume ❐ Bandwidth utilization ❐ Packet size distribution ❐ Protocol distribution ❐ Cumulative graphs Backbone Utilization 30% Entrada 25% Salida Utilización (%) 20% 15% 10% 5% 0% 00:00 02:00 04:00 06:00 08:00 10:00 12:00 14:00 16:00 18:00 20:00 22:00 00:00 11

Utilization per Access Node IP over ATM overhead (in) 50% Sobrecarga media en Entrada: 15,12% 45% 40% % Tráfico de sobrecarga 35% % Tráfico total 30% % Tráfico útil 25% 20% 15% 10% 5% 0% < 41 89-136 185-232 281-328 377-424 473-520 569-616 665-712 761-808 857-904 953-1000 1049-1096 1145-1192 1241-1288 1337-1384 1433-1480 > 1528 Longitud de paquetes IP (bytes) 12

% Paquetes 10% 20% 30% 40% 50% 60% % Tráfico total 0% 10% 15% 20% 25% 30% 35% 40% 45% 50% 0% 5% < 41 < 41 89-136 89-136 185-232 IP over ATM overhead (out) Packet Length Distribution 185-232 281-328 281-328 377-424 377-424 Sobrecarga media en Salida: 20,83% 473-520 Longitud de paquetes IP (bytes) 473-520 569-616 569-616 % % Longitud (bytes) Tráfico útil Tráfico de sobrecarga 665-712 665-712 761-808 761-808 857-904 857-904 953-1000 953-1000 1049-1096 1049-1096 1145-1192 1145-1192 1241-1288 Salida Entrada 1241-1288 1337-1384 1337-1384 1433-1480 1433-1480 > 1528 > 1528 13

Protocol Distribution (mean packet length) Protocol Distribution (percentage of packets) 14

Traffic Distribution 350 unknown Entrada others 300 multicastIP EIGRP 250 games irc Mbytes 200 snmp 150 dns telnet 100 smtp nntp 50 ftp http 0 0:00 2:00 4:00 6:00 8:00 10:00 12:00 14:00 16:00 18:00 20:00 22:00 0:00 Traffic origin/destination analysis module Traffic Origin/Destination Analysis Module (TODM) Official IRR Data Bases Processor NRN BGP other... Subnetwork, CIDR, ASs, ... Databases Summary Report Files Identification Pre-processing of AS Module (TCM) IP Biflows 15

Traffic Classification by AS ❐ Maps of AS by destination ❐ Traffic statistics by outside links of RedIRIS (Ibernet, TEN-155 y USA) classified by CCAA (regions) ❐ Statistics of the most visited subnetworks within the relevant AS ❐ This output may be used by other modules Sample of Results: Main traffic origin/destination (I) 100% % Bytes (Input traffic) 90% 80% 70% RedIRIS 60% TEN-34/155 Ibernet 50% Rest of Internet (through USA) 40% 30% 20% 10% 0% User Groups (17) 16