Software Defined Monitoring: Research Platform for High Speed - - PowerPoint PPT Presentation

▶
software defined monitoring research platform for high
SMART_READER_LITE
LIVE PREVIEW

Software Defined Monitoring: Research Platform for High Speed - - PowerPoint PPT Presentation

Sep 15, 2023 430 likes •684 views

Software Defined Monitoring: Research Platform for High Speed Network Monitoring (31st NMRG Meeting Zrich, Switzerland) Luk Kekely , Viktor Pu, Jan Ko renek (kekely,pus,korenek@cesnet.cz) 14. 10. 2013 Czech NREN Cesnet PIONEER

slide-1
SLIDE 1

Software Defined Monitoring: Research Platform for High Speed Network Monitoring

(31st NMRG Meeting – Zürich, Switzerland) Lukáš Kekely, Viktor Puš, Jan Koˇ renek (kekely,pus,korenek@cesnet.cz)

  • 14. 10. 2013
slide-2
SLIDE 2

Czech NREN Cesnet

NIX TELIA PIONEER SANET ACONET GEANT AMS-IX metering points on the edges of the network (highlighted)

  • L. Kekely

SDM: Platform for Network Monitoring

  • 14. 10. 2013

1 / 19

slide-3
SLIDE 3

Current Metering Point

commodity server running Linux SW flow exporter (NetFlow/IPFIX) from SME Invea-Tech

support for creation of traffic processing plugins

  • ur own hardware probe from COMBOv2 family

PCI-Express card with two 10 GbE ports and Virtex5 FPGA HaNic over NetCope as firmware – packet capture, precise timestamps (ns), flow based traffic division . . .

  • L. Kekely

SDM: Platform for Network Monitoring

  • 14. 10. 2013

2 / 19

slide-4
SLIDE 4

Motivation

⇒ We want more than that!

  • L. Kekely

SDM: Platform for Network Monitoring

  • 14. 10. 2013

3 / 19

slide-5
SLIDE 5

Motivation

⇒ We want more than that!

1

Higher speed

constant advances in the network bandwidth monitored links are going to be upgraded to 40/100 Gbps

2

Higher quality

more than just classical NetFlow statistics flexible additional data according to actual need application protocol parsing and deep packet inspection

  • L. Kekely

SDM: Platform for Network Monitoring

  • 14. 10. 2013

3 / 19

slide-6
SLIDE 6

Motivation

⇒ We want more than that!

1

Higher speed

constant advances in the network bandwidth monitored links are going to be upgraded to 40/100 Gbps

2

Higher quality

more than just classical NetFlow statistics flexible additional data according to actual need application protocol parsing and deep packet inspection

Problem: Current CPUs are not fast enough to process whole traffic all alone!

  • L. Kekely

SDM: Platform for Network Monitoring

  • 14. 10. 2013

3 / 19

slide-7
SLIDE 7

Motivation

⇒ We want more than that!

1

Higher speed

constant advances in the network bandwidth monitored links are going to be upgraded to 40/100 Gbps

2

Higher quality

more than just classical NetFlow statistics flexible additional data according to actual need application protocol parsing and deep packet inspection

Problem: Current CPUs are not fast enough to process whole traffic all alone! Solution: We created new approach to monitoring acceleration called Software Defined Monitoring!

  • L. Kekely

SDM: Platform for Network Monitoring

  • 14. 10. 2013

3 / 19

slide-8
SLIDE 8

Software Defined Monitoring

What is it? new approach to acceleration of network monitoring brings HW accelerated, application controlled reduction of traffic (packet processing offload) still performs packet capture, precise timestamps, flow based traffic division What does it do? Hardware provides various methods of packet preprocessing and aggregation – The Muscles Software controls the actual usage of preprocessing on flow basis – The Controller User applications request the acceleration and perform advanced monitoring tasks – The Intelligence

  • L. Kekely

SDM: Platform for Network Monitoring

  • 14. 10. 2013

4 / 19

slide-9
SLIDE 9

Software Defined Monitoring

What is it? new approach to acceleration of network monitoring brings HW accelerated, application controlled reduction of traffic (packet processing offload) still performs packet capture, precise timestamps, flow based traffic division What does it do? Hardware provides various methods of packet preprocessing and aggregation – The Muscles Software controls the actual usage of preprocessing on flow basis – The Controller User applications request the acceleration and perform advanced monitoring tasks – The Intelligence Applications adjust acceleration of traffic processing according to their actual needs!

  • L. Kekely

SDM: Platform for Network Monitoring

  • 14. 10. 2013

4 / 19

slide-10
SLIDE 10

Traffic Preprocessing in Hardware

fully controlled by rules from software four basic methods of frames preprocessing:

Send – preserve the whole frame (with payload) Extract – preserve only basic data about the frame Aggregate – update selected flow (NetFlow) record maintained in HW memory Drop – simply ignore the frame

  • L. Kekely

SDM: Platform for Network Monitoring

  • 14. 10. 2013

5 / 19

slide-11
SLIDE 11

SDM Layered Scheme

Hardware Layer Firmware Layer Software Layer Basic Control Tools NetCOPE (100GbE) SDM Acceleration Firmware SZE Software Defined Monitoring User Applications PCAP SDM Controller Data Path Control Path libSDM

  • L. Kekely

SDM: Platform for Network Monitoring

  • 14. 10. 2013

6 / 19

slide-12
SLIDE 12

SDM Firmware

Frames HFE UH Action UH Action UH Export Data Path Control Path ETH Link Memory Arbiter External Memory Search Update SW Access TABLE1: Rules TABLE2: Flow Records Rules

  • L. Kekely

SDM: Platform for Network Monitoring

  • 14. 10. 2013

7 / 19

slide-13
SLIDE 13

SDM Use Cases

1

Basic NetFlow statistics

2

Application protocol parsing

3

Specific Non-NetFlow statistics

4

Lawfull interceptions

5

Forensic analysis of network traffic

"zoom-in" on suspicious data

6

Active SW networking device

accelerated switch, firewall, router . . .

7

Acceleration of your research application?

  • L. Kekely

SDM: Platform for Network Monitoring

  • 14. 10. 2013

8 / 19

slide-14
SLIDE 14

UC1: Details of SDM Usage

Basic NetFlow statistics useless payload of frames, but must have information about all incoming frames

default: use Extract on all traffic rules: use Aggregate for selected (the heaviest) flows

CPU performance savings:

no packet parsing at all NetFlow aggregation computed only partially

need to decide when to use NetFlow in HW based on the first X packets of flows

  • L. Kekely

SDM: Platform for Network Monitoring

  • 14. 10. 2013

9 / 19

slide-15
SLIDE 15

UC1: Results

5 10 15 20 25 30 35 40 45 50 10 20 30 40 50 60 70 80 90 100

Decision time [packets] Aggregated in HW [%]

Packets Flows

the number of frames reduced to 1

5 and data load to 1 100

  • L. Kekely

SDM: Platform for Network Monitoring

  • 14. 10. 2013

10 / 19

slide-16
SLIDE 16

UC2: Details of SDM Usage

Application protocol parsing needs payload of selected frames, but do not have to see all incoming frames

default: use Send on interesting traffic, Drop the rest rules: Drop rules for already processed flows

CPU performance savings:

processing of interesting flows only not all packets from interesting flows must be processed

easy deployment in combination with UC1

  • L. Kekely

SDM: Platform for Network Monitoring

  • 14. 10. 2013

11 / 19

slide-17
SLIDE 17

UC2: Results

NetFlow HTTP DNS 20 40 60 80 100

Packets [%]

Drop Aggregate Extract Send

HTTP+ NetFlow

HTTP: 1

4 of frames and 1 4 of data load

DNS:

1 100 of frames and 1 200 of data load

  • L. Kekely

SDM: Platform for Network Monitoring

  • 14. 10. 2013

12 / 19

slide-18
SLIDE 18

UC3: SDM Firmware as Processor

HFE Reserve Arbiter Instr 1 Instr 2

...

Instr n SDM Update Instruction Decoder Merge Output M e m

  • r

y Search

  • L. Kekely

SDM: Platform for Network Monitoring

  • 14. 10. 2013

13 / 19

slide-19
SLIDE 19

UC3: Monitoring Instructions

update of stored record based on the frame data

consist of operation code and record address delimited by 2 memory accesses (read and write back) update process can vary

new instructions without changes in existing modules new instructions created in C/C++ with HLS

consumes less time and allows faster implementation verification during implementation even software guy can create accelerated solutions

  • L. Kekely

SDM: Platform for Network Monitoring

  • 14. 10. 2013

14 / 19

slide-20
SLIDE 20

UC3: Demonstration Instructions

NetFlow (I1)

basic NetFlow aggregation (basic Aggregate) packet/byte counters, start/end timestamps, TCP flags part of the basic SDM infrastructure

NetFlow Extended (I2)

I1 with TCP flags of the first 5 packets of the flow demonstrates easy NetFlow extending using plain C

TCP Flag Counters (I3) (Non-NetFlow)

counts the number of observed TCP flags support advanced flow analysis

Timestamp Diff (I4) (Non-NetFlow)

inter-arrival times of the first 11 packets flow based classification or identification of L7 protocols

  • L. Kekely

SDM: Platform for Network Monitoring

  • 14. 10. 2013

15 / 19

slide-21
SLIDE 21

UC3: Results

Instruction Regs LUTs

  • Freq. [MHz]

(I1)NetFlow (handmade) 1754 325 425.134 (I1)NetFlow 1846 824 308.641 (I2)NetFlow Extended 2070 1113 308.641 (I3)TCP Flag Counters 1046 327.868 (I4)Timestamp Diff 5199 2556 306.748 all modules meet the frequency requirement for 100 Gb/s HLS do not beat hand-written VHDL, but is good enough instruction creation in C/C++ is very simple and fast even non-VHDL programmer can accelerate his monitoring

  • L. Kekely

SDM: Platform for Network Monitoring

  • 14. 10. 2013

16 / 19

slide-22
SLIDE 22

New Metering Point

commodity server running Linux SW flow exporter (NetFlow/IPFIX) from SME Invea-Tech

support for creation of traffic processing plugins plugins utilizing the SDM acceleration capabilities

  • ur own hardware probe for up to 100 GbE

new PCI-Express card with powerful Virtex7 FPGA 1 × 100 GbE or 2 × 40 GbE or 8 × 10 GbE interfaces SDM over NetCope as firmware

  • L. Kekely

SDM: Platform for Network Monitoring

  • 14. 10. 2013

17 / 19

slide-23
SLIDE 23

Future NREN Cesnet

NIX TELIA PIONEER SANET ACONET GEANT AMS-IX

SDM SDM SDM SDM SDM SDM SDM SDM SDM SDM SDM SDM SDM SDM

all metering points doubled (production and testing)

  • L. Kekely

SDM: Platform for Network Monitoring

  • 14. 10. 2013

18 / 19

slide-24
SLIDE 24

Thank you for your attention.

  • L. Kekely

SDM: Platform for Network Monitoring

  • 14. 10. 2013

19 / 19