SMARTxAC & Sentinel Maria Isabel Gandía, Communications Service Manager, CESCA Maurizio Molina, CEO, 8th TF-NOC meeting, GRnet, Athens, 27-5-2013
Agenda ü SMARTxAC • Introduction • The evolution of the network and the evolution of the tool • Examples of use • Requirements for the new platform • Demo ü Sentinel • Traditional network monitoring approaches • Talaia Networks alternative vision • Sentinel Highlights • Demo • Talaia Networks Roadmap • Discussion
About CESCA & Anella Científica Commercial Internet
Symbiosis and collaboration ü SMARTxAC is the collaboration between UPC (CCABA) and CESCA. ü It is a platform made by the university where CESCA gets a useful and adaptable tool for the management of the network and for its users in Anella Científica and the university gets real material and feedback for the research and projects. ü Since 2003, SMARTxAC is daily used by CESCA to detect anomalies, attacks, performance problems, network faults, etc.
SMARTxAC ü SMA Я TxAC: Traffic Monitoring and Analysis System for Anella Científica ( “ Sistema de Monitoratge i Anàlisi de TRàfic per l ’ Anella Científica ” ) ü Main objectives • Low-cost platform • Continuous monitoring of high-speed links without packet loss • Detection of network anomalies and irregular usage • Multi-user system: Network operators and Institutions
Agenda ü SMARTxAC • Introduction • The evolution of the network and the evolution of the tool • Examples of use • Requirements for the new platform • Demo ü Sentinel • Traditional network monitoring approaches • Talaia Networks alternative vision • Sentinel Highlights • Demo • Talaia Networks Roadmap • Discussion
How was it born? The background ü Previous monitoring and analysis projects: • CASTBA • MEHARI • MIRA ü With the collaboration among several universities • UPM (Universidad Politécnica de Madrid) • UC3M (Universidad Carlos III de Madrid) • UPC (Universitat Politècnica de Catalunya) ü And the participation of: • RedIRIS • CESCA • Telefónica Investigación y Desarrollo • Institut Català de Tecnologia
From 1999 to 2001: The MIRA project 50% Internet 50% 50% 50% Development platform Traffic Analysis @UPC – CCABA System (Linux) ü The MIRA platform was mainly divided in two subsystems: • The Traffic Capture Subsystem was a modification of the OC3MON software for the PCA200 Fore ATM card adapter that provided periodic full IP packet samples (the traffic capture was done in a passive way). • The Traffic Analysis Subsystem had several modules that extracted different parameters of the network. ü The capture was: • Passive (using optical splitters) • Statistic (10% of real traffic maximum) • All the packet (header and payload)
2003: first stages of SMARTxAC 25% Internet 25% Private network Management network 25% 50% Result Capture platform Capture platform Traffic Analysis 50% Visualization @CESCA (DAG 4.3GE + @UPC - CCABA System (Linux) System GPS) Pentium IV 2.6 GHz. + 1 GB RAM Intel Xeon 2.4 GHz + 1 GB Pentium III 450 MHz Optical splitters RAM 2 x Endace DAG 4.3GE ü The capture was: GPS (Trimble Acutime 2000) • Passive (using optical splitters and the corresponding cards) • No sampling • Only the packet headers ü Initially, it was a measurement only of the connection between Anella Científica and RedIRIS ü The images were updated at the end of the day (not exactly real-time)
2004-2009: SMARTxAC with RedIRIS-Anella Científica flows 25% Internet 25% Private network Management network 25% 50% Capture platform Traffic Analysis Result Capture platform @CESCA (DAG 4.3GE + System (Linux) 50% Visualization @UPC - CCABA GPS) System Intel Xeon 2.4 GHz Pentium IV 2.6 GHz 1 GB RAM Optical splitters 1 GB RAM 2 x Endace DAG 4.3GE Pentium III 450 MHz GPS (Trimble Acutime 2000) The SMARTxAC platform was mainly divided in three subsystems: • The Traffic Capture Subsystem evolved from CAIDA Coralreef and UPC developments. Aggregated flows were sent to the Analysis System. • The Traffic Analysis Subsystem classified the flows (aggregation of 5-tuple flows into classified flows) . Classified flows: >1:1000 ( ≈ 60 GB/day à ≈ 50 MB/ day). Compared with header traces: > 1:250000 ( ≈ 13 TB/day) • The Visualization Subsystem was an apache website were graphics and reports were shown on demand.
2009-2011: all the external interfaces are captured 25% Internet 25% Private network Management network 25% 50% Result Capture platform Capture platform Traffic Analysis 50% Visualization @CESCA (DAG 4.3GE + @UPC - CCABA System (Linux) System GPS) Pentium IV 2.6 GHz 1 GB RAM Intel Xeon 2.4 GHz Pentium III 450 MHz Optical splitters 1 GB RAM 2 x Endace DAG 4.3GE GPS (Trimble Acutime 2000) ü After some time in the development platform, all the external interfaces captures (RedIRIS, Internet, CATNIX) were also available for the users
2011-2013: Netflow, new visualization 25% Internet 25% Private network Management network 25% 50% Result Capture platform Capture platform Traffic Analysis 50% Visualization @CESCA (DAG 4.3GE + @UPC - CCABA System (Linux) System GPS) Virtual machineUbuntu Virtual machine Server 64 bits, Intel Xeon 2.4 GHz 1 GB RAM Optical splitters 4 cpus 1 GB RAM 1 CPU 8 Gb de RAM 2 x Endace DAG 4.3GE 30 GB for logs GPS (Trimble Acutime 2000) ü The splitters are still used for the Deep Packet Inspection (DPI) ü All the internal and external interfaces are measured ü Offline analysis with DPI patterns, based on Machine Learning techniques.
2011-2013: Netflow, new visualization Port Number Machine learning 8.17% 0.30% A_UKNWN DNS 19.65% 18.47% 0.51% FTP 8.48% GAMES 1.52% IRC 0.10% 0.30% MAIL 0.43% 2.97% 0.48% MULTIMEDIA 2.43% NETFS 47.39% 9.67% NETWORK 10.34% NEWS NO_TCPUDP 1.22% OTHERS P2P 5.42% 7.97% T_UKNWN 2.26% TELNET 0.10% UNIX 40.07% 0.53% 2.48% WWW 6.04% 0.55% 0.08% 0.23% 1.84%
2011-2013: more points of capture Providers Backup ISP A ISP B ISP C REDIRIS VAL ORANGE BCN1 REDIRIS ARA ORANGE BCN2 º Optical splitters CESCA-CN CESCA-T Capture, analysis and monitoring servers Institutions @CESCA1 @CESCA2 @REDIRIS With netflow With splitters only
2011-2013: Netflow, new visualization ü New web interface using javascripts: • More statistics • More real anomalies • Zoomable • Filterable • Easier for the users • With usage statistics
The evolution of the network, the tool and the research 2001 End of the MIRA project 2002 Presentation at the TNC2002 Jordi Domingo, Josep Solé 2003 From 155 Mbps ATM to 1 Gbps Ethernet UPC-CESCA agreement New capture cards (Endace DAG 4.23) Presentation at the RedIRIS Conference (Jornadas Técnicas) Change of splitters (1st window -> 2nd) 2004 Final Theses (from perl to C) Eva Codina 1st prototype at CESCA Final Theses (Automatic detection of Adnan Berberovic anomalies) Stage at Endace Presentation at the Anella Científica Conference (Trobada de l’Anella Científica, TAC) Pere Barlet
The evolution of the network, the tool and the research (II) SMARTxAC was successfully tested on one of the NLANR OC192MON’s at the San Diego Supercomputing Center TeraGrid cluster 2005 1st Anella Científica users 1st workshop for the users 2006 Capture cards for the CATNIX and commercial connections (development) Presentation at the TNC2006 Pere Barlet Final Theses Derek Hossak Digital certificates (cards) ok 2007 New capture cards From 2 Gbps to 10 Gbps New splitters adapting the platform to the 10 Gbps link.
The evolution of the network, the tool and the research (III) 2008 Doctoral Theses Pere Barlet All the external lines presented 2009 2010 New DAG 2011 Netflow, testbed users New platform Master Theses Ismael Castell From 10 Gbps to 20 Gbps 2012 Doctoral Theses Josep Sanjuàs 2013 is born!
Agenda ü SMARTxAC • Introduction • The evolution of the network and the evolution of the tool • Examples of use • Requirements for the new platform • Demo ü Sentinel • Traditional network monitoring approaches • Talaia Networks alternative vision • Sentinel Highlights • Demo • Talaia Networks Roadmap • Discussion
An example from 2006: Port Scanning (I) ü Traffic profile per application (bps)
An example from 2006: Port Scanning (I) ü Traffic profile per application (flows/s)
An example from April 2013
An example from April 2013
Agenda ü SMARTxAC • Introduction • The evolution of the network and the evolution of the tool • Examples of use • Requirements for the new platform • Demo ü Sentinel • Traditional network monitoring approaches • Talaia Networks alternative vision • Sentinel Highlights • Demo • Talaia Networks Roadmap • Discussion
Our current requirements ü Classification of the traffic following the Institutions / Points of access by default ü Customizable filters available to users (with a maximum threshold) ü Correlated TopN for input and output ü CSV reports ü Multi-user and multi-view ü Integrated with other sources of alarms (for instance, T. Cymru) ü Integrated with our databases
Recommend
More recommend
