SMARTxAC & Sentinel Maria Isabel Ganda, Communications Service - - PowerPoint PPT Presentation

▶
smartxac sentinel
SMART_READER_LITE
LIVE PREVIEW

SMARTxAC & Sentinel Maria Isabel Ganda, Communications Service - - PowerPoint PPT Presentation

Jul 03, 2023 339 likes •617 views

SMARTxAC & Sentinel Maria Isabel Ganda, Communications Service Manager, CESCA Maurizio Molina, CEO, 8th TF-NOC meeting, GRnet, Athens, 27-5-2013 Agenda SMARTxAC Introduction The evolution of the network and the evolution of

slide-1
SLIDE 1

SMARTxAC & Sentinel

Maria Isabel Gandía, Communications Service Manager, CESCA Maurizio Molina, CEO, 8th TF-NOC meeting, GRnet, Athens, 27-5-2013

slide-2
SLIDE 2

Agenda

ü SMARTxAC

  • Introduction
  • The evolution of the network and the evolution of the tool
  • Examples of use
  • Requirements for the new platform
  • Demo

ü Sentinel

  • Traditional network monitoring approaches
  • Talaia Networks alternative vision
  • Sentinel Highlights
  • Demo
  • Talaia Networks Roadmap
  • Discussion
slide-3
SLIDE 3

About CESCA & Anella Científica

Commercial Internet

slide-4
SLIDE 4

Symbiosis and collaboration

ü SMARTxAC is the collaboration between UPC (CCABA) and CESCA. ü It is a platform made by the university where CESCA gets a useful and adaptable tool for the management of the network and for its users in Anella Científica and the university gets real material and feedback for the research and projects. ü Since 2003, SMARTxAC is daily used by CESCA to detect anomalies, attacks, performance problems, network faults, etc.

slide-5
SLIDE 5

SMARTxAC

ü SMAЯTxAC: Traffic Monitoring and Analysis System for Anella Científica (“Sistema de Monitoratge i Anàlisi de TRàfic per l’Anella Científica”) ü Main objectives

  • Low-cost platform
  • Continuous monitoring of high-speed links without packet loss
  • Detection of network anomalies and irregular usage
  • Multi-user system: Network operators and Institutions
slide-6
SLIDE 6

Agenda

ü SMARTxAC

  • Introduction
  • The evolution of the network and the evolution of the tool
  • Examples of use
  • Requirements for the new platform
  • Demo

ü Sentinel

  • Traditional network monitoring approaches
  • Talaia Networks alternative vision
  • Sentinel Highlights
  • Demo
  • Talaia Networks Roadmap
  • Discussion
slide-7
SLIDE 7

How was it born? The background

ü Previous monitoring and analysis projects:

  • CASTBA
  • MEHARI
  • MIRA

ü With the collaboration among several universities

  • UPM (Universidad Politécnica de Madrid)
  • UC3M (Universidad Carlos III de Madrid)
  • UPC (Universitat Politècnica de Catalunya)

ü And the participation of:

  • RedIRIS
  • CESCA
  • Telefónica Investigación y Desarrollo
  • Institut Català de Tecnologia
slide-8
SLIDE 8

ü The MIRA platform was mainly divided in two subsystems:

  • The Traffic Capture Subsystem was a modification of the OC3MON

software for the PCA200 Fore ATM card adapter that provided periodic full IP packet samples (the traffic capture was done in a passive way).

  • The Traffic Analysis Subsystem had several modules that extracted

different parameters of the network.

ü The capture was:

  • Passive (using optical splitters)
  • Statistic (10% of real traffic maximum)
  • All the packet (header and payload)

From 1999 to 2001: The MIRA project

50% 50% 50%

Development platform @UPC – CCABA

Internet 50%

Traffic Analysis System (Linux)

slide-9
SLIDE 9

ü The capture was:

  • Passive (using optical splitters and the corresponding cards)
  • No sampling
  • Only the packet headers

ü Initially, it was a measurement only of the connection between Anella Científica and RedIRIS ü The images were updated at the end of the day (not exactly real-time)

2003: first stages of SMARTxAC

50% 25% 25% 25%

Capture platform @CESCA (DAG 4.3GE + GPS) Capture platform @UPC - CCABA

Optical splitters Internet

Traffic Analysis System (Linux) Result Visualization System

Private network Management network

Intel Xeon 2.4 GHz + 1 GB RAM 2 x Endace DAG 4.3GE GPS (Trimble Acutime 2000) Pentium IV 2.6 GHz. + 1 GB RAM Pentium III 450 MHz

50%

slide-10
SLIDE 10

The SMARTxAC platform was mainly divided in three subsystems:

  • The Traffic Capture Subsystem evolved from CAIDA Coralreef and UPC
  • developments. Aggregated flows were sent to the Analysis System.
  • The Traffic Analysis Subsystem classified the flows (aggregation of 5-tuple

flows into classified flows). Classified flows: >1:1000 (≈ 60 GB/day à ≈ 50 MB/ day). Compared with header traces: > 1:250000 (≈ 13 TB/day)

  • The Visualization Subsystem was an apache website were graphics and

reports were shown on demand. 2004-2009: SMARTxAC with RedIRIS-Anella Científica flows

50% 25% 25% 25%

Capture platform @CESCA (DAG 4.3GE + GPS) Capture platform @UPC - CCABA

Optical splitters Internet

Traffic Analysis System (Linux) Result Visualization System

Private network Management network

Intel Xeon 2.4 GHz 1 GB RAM 2 x Endace DAG 4.3GE GPS (Trimble Acutime 2000) Pentium IV 2.6 GHz 1 GB RAM Pentium III 450 MHz

50%

slide-11
SLIDE 11

ü After some time in the development platform, all the external interfaces captures (RedIRIS, Internet, CATNIX) were also available for the users

2009-2011: all the external interfaces are captured

50% 25% 25% 25%

Capture platform @CESCA (DAG 4.3GE + GPS) Capture platform @UPC - CCABA

Optical splitters Internet

Traffic Analysis System (Linux) Result Visualization System

Private network

Intel Xeon 2.4 GHz 1 GB RAM 2 x Endace DAG 4.3GE GPS (Trimble Acutime 2000) Pentium IV 2.6 GHz 1 GB RAM Pentium III 450 MHz

50%

Management network

slide-12
SLIDE 12

ü The splitters are still used for the Deep Packet Inspection (DPI) ü All the internal and external interfaces are measured ü Offline analysis with DPI patterns, based on Machine Learning techniques.

2011-2013: Netflow, new visualization

50% 25% 25% 25%

Capture platform @CESCA (DAG 4.3GE + GPS) Capture platform @UPC - CCABA

Optical splitters Internet

Traffic Analysis System (Linux) Result Visualization System

Private network

Intel Xeon 2.4 GHz 1 GB RAM 2 x Endace DAG 4.3GE GPS (Trimble Acutime 2000) Virtual machineUbuntu Server 64 bits, 4 cpus 8 Gb de RAM 30 GB for logs Virtual machine 1 GB RAM 1 CPU

50%

Management network

slide-13
SLIDE 13

2011-2013: Netflow, new visualization Port Number Machine learning

47.39% 10.34% 0.43% 0.10% 19.65% 7.97% 0.08% 2.48% 0.55% 1.84% 2.26% 0.10% 0.53% 6.04% 0.23% 40.07% 2.43% 2.97% 18.47% 0.30% 8.17% 0.48% 9.67% 1.22% 0.51% 0.30% 1.52% 8.48% 5.42% A_UKNWN DNS FTP GAMES IRC MAIL MULTIMEDIA NETFS NETWORK NEWS NO_TCPUDP OTHERS P2P T_UKNWN TELNET UNIX WWW

slide-14
SLIDE 14

Providers Institutions

ISP A ISP B ISP C @CESCA1 @CESCA2 @REDIRIS

2011-2013: more points of capture

º

REDIRIS ARA ORANGE BCN2 CESCA-T REDIRIS VAL ORANGE BCN1

With netflow

Capture, analysis and monitoring servers CESCA-CN Optical splitters

With splitters only

Backup

slide-15
SLIDE 15

2011-2013: Netflow, new visualization

ü New web interface using javascripts:

  • More statistics
  • More real anomalies
  • Zoomable
  • Filterable
  • Easier for the users
  • With usage statistics
slide-16
SLIDE 16

End of the MIRA project Presentation at the TNC2002 From 155 Mbps ATM to 1 Gbps Ethernet UPC-CESCA agreement New capture cards (Endace DAG 4.23) Presentation at the RedIRIS Conference (Jornadas Técnicas) Change of splitters (1st window -> 2nd) Final Theses (from perl to C) 1st prototype at CESCA Final Theses (Automatic detection of anomalies) Stage at Endace Presentation at the Anella Científica Conference (Trobada de l’Anella Científica, TAC) 2001 2002 2003 2004 The evolution of the network, the tool and the research

Jordi Domingo, Josep Solé Eva Codina Adnan Berberovic Pere Barlet

slide-17
SLIDE 17

The evolution of the network, the tool and the research (II) SMARTxAC was successfully tested

  • n one of the NLANR OC192MON’s

at the San Diego Supercomputing Center TeraGrid cluster 1st Anella Científica users 1st workshop for the users Capture cards for the CATNIX and commercial connections (development) Presentation at the TNC2006 Final Theses Digital certificates (cards) ok New capture cards From 2 Gbps to 10 Gbps New splitters adapting the platform to the 10 Gbps link. 2005 2006 2007

Pere Barlet Derek Hossak

slide-18
SLIDE 18

Doctoral Theses All the external lines presented New DAG Netflow, testbed users New platform Master Theses From 10 Gbps to 20 Gbps Doctoral Theses is born! 2008 2009 2010 2011 2012 2013 The evolution of the network, the tool and the research (III)

Pere Barlet Ismael Castell Josep Sanjuàs

slide-19
SLIDE 19

Agenda

ü SMARTxAC

  • Introduction
  • The evolution of the network and the evolution of the tool
  • Examples of use
  • Requirements for the new platform
  • Demo

ü Sentinel

  • Traditional network monitoring approaches
  • Talaia Networks alternative vision
  • Sentinel Highlights
  • Demo
  • Talaia Networks Roadmap
  • Discussion
slide-20
SLIDE 20

An example from 2006: Port Scanning (I)

ü Traffic profile per application (bps)

slide-21
SLIDE 21

An example from 2006: Port Scanning (I)

ü Traffic profile per application (flows/s)

slide-22
SLIDE 22

An example from April 2013

slide-23
SLIDE 23

An example from April 2013

slide-24
SLIDE 24

Agenda

ü SMARTxAC

  • Introduction
  • The evolution of the network and the evolution of the tool
  • Examples of use
  • Requirements for the new platform
  • Demo

ü Sentinel

  • Traditional network monitoring approaches
  • Talaia Networks alternative vision
  • Sentinel Highlights
  • Demo
  • Talaia Networks Roadmap
  • Discussion
slide-25
SLIDE 25

Our current requirements

ü Classification of the traffic following the Institutions / Points of access by default ü Customizable filters available to users (with a maximum threshold) ü Correlated TopN for input and output ü CSV reports ü Multi-user and multi-view ü Integrated with other sources of alarms (for instance, T. Cymru) ü Integrated with our databases

slide-26
SLIDE 26

Agenda

ü SMARTxAC

  • Introduction
  • The evolution of the network and the evolution of the tool
  • Examples of use
  • Requirements for the new platform
  • Demo

ü Sentinel

  • Traditional network monitoring approaches
  • Talaia Networks alternative vision
  • Sentinel Highlights
  • Demo
  • Talaia Networks Roadmap
  • Discussion