A 10GE Monitoring System Arin Vijn arien@ams-ix.net Agenda - - - PDF document

▶
a 10ge monitoring system ari n vijn arien ams ix net
SMART_READER_LITE
LIVE PREVIEW

A 10GE Monitoring System Arin Vijn arien@ams-ix.net Agenda - - - PDF document

Jan 24, 2023 304 likes •890 views

A 10GE Monitoring System Arin Vijn arien@ams-ix.net Agenda - Introduction The role of an internet exchange (IX). - The problem to be solved. Real life examples - The chosen solution for that problem * The Force10's P10 IDS/IPS card *

slide-1
SLIDE 1

A 10GE Monitoring System Ariën Vijn arien@ams-ix.net

slide-2
SLIDE 2
  • Introduction

The role of an internet exchange (IX).

  • The problem to be solved.

Real life examples

  • The chosen solution for that problem

* The Force10's P10 IDS/IPS card * Photonic cross connects

Agenda

slide-3
SLIDE 3

ISP E ISP G ISP H ISP C ISP F ISP I ISP A ISP B ISP D

slide-4
SLIDE 4

ISP E ISP G ISP H ISP C ISP F ISP I ISP A ISP B ISP D

Tier 1 Tier 2 Tier 3

slide-5
SLIDE 5

AS1

switch park

AS2 AS5 AS6 AS3 AS4

slide-6
SLIDE 6

AS1 switch park AS2 AS5 AS6 AS3 AS4 BGP Router BGP Router BGP Router BGP Router BGP Router BGP Router

slide-7
SLIDE 7

AS1 switch park AS2 AS5 AS6 AS3 AS4 BGP Router BGP Router BGP Router BGP Router BGP Router BGP Router

slide-8
SLIDE 8

AS1 switch park AS2 AS5 AS6 AS3 AS4 BGP Router BGP Router BGP Router BGP Router BGP Router BGP Router

slide-9
SLIDE 9

AS1 switch park AS2 AS5 AS6 AS3 AS4 BGP Router BGP Router BGP Router BGP Router BGP Router BGP Router

slide-10
SLIDE 10

AS1 switch park AS2 AS5 AS6 AS3 AS4 BGP Router BGP Router BGP Router BGP Router BGP Router BGP Router

monitor

slide-11
SLIDE 11

Example #1

slide-12
SLIDE 12

switch park AS2 AS3 AS1

2 3 1

1 M

UNI..

1 M

UNI..

1 M

UNI..

slide-13
SLIDE 13

switch park AS2 AS3 AS1

2 3 1

1 M

UNI..

1 M

UNI..

1 M

UNI..

slide-14
SLIDE 14

switch park AS2 AS3 AS1

2 3 1

2 F

ARP

2 F

ARP

2 F

ARP

Ethernet Destination: ff:ff:ff:ff:ff:ff Source: 00:00:00:00:00:01 Type: ARP (0x0806) Address Resolution Protocol (request) Hardware type: Ethernet (0x0001) Protocol type: IP (0x0800) Hardware size: 6 Protocol size: 4 Opcode: request (0x0001) Sender MAC address: 01:00:00:00:00:02 Sender IP address: 195.69.144.2 Target MAC address: 00:00:00:00:00:00 Target IP address: 195.69.144.3

slide-15
SLIDE 15

switch park AS2 AS3 AS1

2 3 1

2 F

ARP

2 F

ARP

2 F

ARP

Ethernet Destination: ff:ff:ff:ff:ff:ff Source: 00:00:00:00:00:01 Type: ARP (0x0806) Address Resolution Protocol (request) Hardware type: Ethernet (0x0001) Protocol type: IP (0x0800) Hardware size: 6 Protocol size: 4 Opcode: request (0x0001) Sender MAC address: 01:00:00:00:00:02 Sender IP address: 195.69.144.2 Target MAC address: 00:00:00:00:00:00 Target IP address: 195.69.144.3

slide-16
SLIDE 16

switch park AS2 AS3 AS1

2 3 1

2 F

ARP

2 F

ARP

2 F

ARP

slide-17
SLIDE 17

AS1 switch park AS2 AS5 AS6 AS3 AS4 BGP Router BGP Router BGP Router BGP Router

Tuttle Buttle

BGP Router BGP Router

slide-18
SLIDE 18

AS1 switch park AS2 AS5 AS6 AS3 AS4 BGP Router BGP Router BGP Router BGP Router BGP Router BGP Router

monitor

slide-19
SLIDE 19

AS1 switch park AS2 AS5 AS6 AS3 AS4 BGP Router BGP Router BGP Router BGP Router BGP Router BGP Router

monitor

slide-20
SLIDE 20

AS1 switch park AS2 AS5 AS6 AS3 AS4 BGP Router BGP Router BGP Router BGP Router BGP Router BGP Router

monitor

slide-21
SLIDE 21

Example #2

slide-22
SLIDE 22

BGP router AS 1 AS 1 transit ixp

  • wn netwerk

transit switch AS 1 IXP shared link

slide-23
SLIDE 23

BGP router AS 1 ixp AS1 switch AS 1 IXP shared link BGP router AS 2 switch AS 2 shared link ixp AS2 private interconnect AS1 - AS2

slide-24
SLIDE 24

switch park BGP ROUTER AS1

1 D

1

slide-25
SLIDE 25

switch park BGP ROUTER AS1

S != 1 D

1

syslog source mac

slide-26
SLIDE 26

switch park BGP ROUTER AS1

1

S D

PAYL.

Ethernet Destination: 00:00:00:00:00:02 Source: 00:00:5e:00:01:11 (IETF-VRRP-virtual-router-VRID_11) IP (0x0800) Internet Protocol Protocol: ICMP (0x01) Header checksum Source: 10.0.0.1 Destination: 10.0.0.2 Internet Control Message Protocol Type: 0 (Echo (ping) reply) Code: 0 [...]

slide-27
SLIDE 27

Monitoring

slide-28
SLIDE 28

General purpose PC (libpcap)

switch park BGP ROUTER AS1

1

S == 1 D NIC

PCI-X

mirror port

slide-29
SLIDE 29

≤ 5Gbps switch fabric ≤ 5Gbps ≤ 5Gbps monitored port MAC PP TM

  • ptic

mirror port MAC PP TM

  • ptic
slide-30
SLIDE 30

Router Ethernet switch

slide-31
SLIDE 31

Router Ethernet switch MEMS based Fiber Switch AKA Photonic Cross Connect (PXC)

slide-32
SLIDE 32

Movable Mirror Array Fiber Lens Array Fixed Mirror Dry Air

slide-33
SLIDE 33

Movable Mirror Array Fiber Lens Array Fixed Mirror Dry Air

slide-34
SLIDE 34

core switch Telecity core switch science park

BGP router metrofiber

slide-35
SLIDE 35

core switch Telecity core switch science park

BGP router metrofiber

slide-36
SLIDE 36
slide-37
SLIDE 37

Router AMS-IX switch traffic generator / analyser Anritsu MD1230A

XENPAK XENPAK

slide-38
SLIDE 38

frame n 9.6 ns 64 byte, 51.2 ns 64 byte, 51.2 ns frame n+1 9.6 ns

slide-39
SLIDE 39

Solution

slide-40
SLIDE 40

Router AMS-IX switch General Purpose Computer (Linux) interface card XPAK XPAK PCI-X

slide-41
SLIDE 41

PCI-X interface 10GE 10GE Layer 1 repeater and tap Programmable Filter PCI-X bus

  • max. ~ 600kfps

eth2 'normal' ethernet interface Hardware software driver

  • max. 2 x 14.8 Mfps one way
  • max. 2 x 10 Gbps one way
  • max. 14.8Mfps full duplex
  • max. 10 Gbps full duplex
slide-42
SLIDE 42

CPLD SRAM JTAG XPAK XPAK front-end FPGA flash PCI JTAG back-end FPGA

slide-43
SLIDE 43

back-end FPGA XAUI XAUI

AND AND

64 128 64 128 XPAK Transceivers frame data ch1 frame data ch 2 block ch1 block ch2 ch1 ch2

slide-44
SLIDE 44

15 31 47 63 79 95 111 127

preamble / SFD

  • dest. MAC addr.

src

slide-45
SLIDE 45

1 2

15 31 47 63 79 95 111 127

preamble / SFD

  • dest. MAC addr.

src MAC addr. type

4 H L TOS total len. ID F/F TTL pro to.

n IP payload HCS

  • src. IP addr.
  • dst. IP addr

IP payload

slide-46
SLIDE 46

R0 R1 Rn reduction network clk valid

  • ffset

match memory CPLD front-end FPGA C0 C1 Cn frame data frame data register filtered data

slide-47
SLIDE 47

1 n

15 31 47 63 79 95 111 127

preamble / SFD

  • dest. MAC addr.

src MAC addr. type payload payload

slide-48
SLIDE 48

15 31 47 63 79 95 111 127

preamble / SFD

  • dest. MAC addr.

src Rx Cx reduction network

slide-49
SLIDE 49

1

15 31 47 63 79 95 111 127

MAC addr. type payload Ry reduction network Rz Cz Cy

slide-50
SLIDE 50

user defined functionality clk valid

  • ffset

match memory CPLD front-end FPGA frame data filtered data packet processor memory management host interface addr. data R/W block capture

slide-51
SLIDE 51

R0 R1 Rn reduction network clk valid

  • ffset

CPLD front-end FPGA C0 C1 Cn frame data frame data register

counter

host interface

slide-52
SLIDE 52

R0 R1 Rn reduction network C0 C1 Cn R0 R1 Rn reduction network clk valid

  • ffset

CPLD front-end FPGA C0 C1 Cn frame data frame data register

counter

host interface

slide-53
SLIDE 53

frame counter Cr reduction network

random number generator register

slide-54
SLIDE 54

1 2

15 31 47 63 79 95 111 127

preamble / SFD

  • dest. MAC addr.

src MAC addr. type

4 H L TOS total len. ID F/F TTL pro to.

n IP payload HCS

  • src. IP addr.
  • dst. IP addr

IP payload

slide-55
SLIDE 55

CPLD SRAM JTAG XPAK XPAK front-end FPGA flash PCI JTAG back-end FPGA

slide-56
SLIDE 56
slide-57
SLIDE 57

Thank you for your attention