pmacct: BMP and Streaming Telemetry Paolo Lucente NTT CommunicaDons - - PowerPoint PPT Presentation

▶
pmacct bmp and streaming telemetry
SMART_READER_LITE
LIVE PREVIEW

pmacct: BMP and Streaming Telemetry Paolo Lucente NTT CommunicaDons - - PowerPoint PPT Presentation

May 28, 2023 251 likes •509 views

pmacct: BMP and Streaming Telemetry Paolo Lucente NTT CommunicaDons | pmacct ALNOG 2, Tirana Nov 2018 whoami Paolo Lucente GitHub: paololucente LinkedIn: plucente Digging data out of networks worldwide for fun and profit for more than

slide-1
SLIDE 1

ALNOG 2, Tirana – Nov 2018

pmacct: BMP and Streaming Telemetry

Paolo Lucente

NTT CommunicaDons | pmacct

slide-2
SLIDE 2

whoami

Paolo Lucente GitHub: paololucente LinkedIn: plucente Digging data out of networks worldwide for fun and profit for more than 10 years

slide-3
SLIDE 3

pmacct

ALNOG 2, Tirana – Nov 2018

slide-4
SLIDE 4

libpcap

pmacct is open-source, free, GPL’ed soRware

maps IGP MySQL PgSQL SQLite MongoDB BerkeleyDB flat-files RabbitMQ Kafka memory tables sFlow tee NetFlow IPFIX NetFlow IPFIX

hSp://www.pmacct.net/

Streaming Telemetry GeoIP BGP sFlow BMP

slide-5
SLIDE 5

pmacct: a few simple use-cases

BMP flat-files tee NetFlow IPFIX sFlow Kafka IPFIX libpcap

slide-6
SLIDE 6

pmacct: a slightly more complex use-case

BGP flat-files tee NetFlow IPFIX Kafka MySQL

aggregation method #1 aggregation method #2

nfacctd

slide-7
SLIDE 7

The use-case for message brokers

slide-8
SLIDE 8

Key pmacct non-technical facts

§ 15+ years old project § Can’t spell the name aRer the second drink § Free, open-source, independent § Under acDve development § InnovaDon being introduced § Well deployed around, also in large SPs/IXPs § Close to the SP/IXP community needs

slide-9
SLIDE 9

BMP

ALNOG 2, Tirana – Nov 2018

“It is wise not to use the same protocol to monitor itself”

  • - Ancient italian proverb
slide-10
SLIDE 10

BMP

§ BGP Monitoring Protocol § RFC 7854:

  • first draR in 2008, sparse work unDl 2012;
  • stall between 2012 and 2015;
  • real tracDon kicks in: 10 draRs between 2015 and 2016;
  • RFC award in Jun 2016

§ Uncomplicated protocol design § Great effort but ..

  • .. industry evolved all these years
  • increased hunger for data

A DevOps guy during lunch break

slide-11
SLIDE 11

BGP monitoring with BMP (1/2)

Credits to: R. Bush (IIJ) @ BMP BoF, RIPE74

All Paths

Vantage Point

With BMP, I learn all the paths the peering router heard

Peering Router P0-4

Peers

P0 P1 P2 P3 P4

slide-12
SLIDE 12

BGP monitoring with BMP (2/2)

slide-13
SLIDE 13

BGP Peer-B Adj-Rib-In (Pre) Adj-Rib-In (Post)

Filters/Policy Filters/Policy

Adj-Rib-In (Post) BGP Peer-A Adj-Rib-In (Pre) We can see this And this

Problem statement

Credits to: T. Evens (Cisco), S. Bayraktar (Cisco), P. Lucente (NTT) @ GROW WG, IETF 98

slide-14
SLIDE 14

StaDc BGP Peer-B Adj-Rib-In (Pre) Adj-Rib-In (Post)

Filters/Policy Filters/Policy Accepted Accepted

Adj-Rib-In (Post) Adj-Rib-Out (Pre) Adj-Rib-Out (Post)

Filters/Policy

BGP Peer-A Adj-Rib-In (Pre) ISIS

We also want to see this And this And this

Loc-Rib

Credits to: T. Evens (Cisco), S. Bayraktar (Cisco), P. Lucente (NTT) @ GROW WG, IETF 98

Proposal: extend BMP to loc-RIB and Adj-RIB-Out (1/3)

slide-15
SLIDE 15

Proposal: extend BMP to loc-RIB and Adj-RIB-Out (2/3)

slide-16
SLIDE 16

Proposal: extend BMP to loc-RIB and Adj-RIB-Out (3/3)

slide-17
SLIDE 17

Credits to: T. Evens (Cisco), S. Bayraktar (Cisco), P. Lucente (NTT) @ GROW WG, IETF 98

§ Loc-RIB:

§ Monitor routes selected and used by the router:

  • ECMP
  • CorrelaDon with NetFlow/IPFIX
  • Next-hop preservaDon

§ Monitor locally originated and BGP routes without requiring a BGP peering § Policy verificaDon

§ Adj-RIB-Out:

§ Monitor routes adverDsed to peers § Policy verificaDon

draR-iep-grow-bmp-{local-rib,adj-rib-out} use-cases

slide-18
SLIDE 18

draR-iep-grow-bmp-{local-rib,adj-rib-out} standardizaDon status § Both draRs in their -02 version § draR-iep-grow-bmp-local-rib-00 -> -02:

§ Mainly text clarificaDons § Peer down VRF/Table name opDonal TLV [reduce state]

§ draR-iep-grow-bmp-adj-rib-out-00 -> -02:

§ Mainly text clarificaDons § Peer up Admin Label opDonal TLV [ie. to carry peer- group info]

§ Plenty of discussion took place on the GROW WG list at IETF. Further conversaDon is encouraged!

slide-19
SLIDE 19

Streaming Telemetry

ALNOG 2, Tirana – Nov 2018

slide-20
SLIDE 20

§ A scalable replacement for SNMP:

  • Push technology
  • Subscribing to data of interest

§ A long journey to standardizaDon ahead:

  • Models: Openconfig and vendor-specific
  • Transport: tradiDonal, Netconf and gNMI
  • RPC: Netconf (YANG Push) and gNMI
  • Encoding: JSON and GPB

Streaming Telemetry

slide-21
SLIDE 21

§ Mission statement:

  • Integrate Streaming Telemetry data with other

relevant technologies (ie. IPFIX, BMP, etc.)

  • Especially in the current pre-standardizaDon

stage, offer an efficient mulD-vendor collecDon layer for Streaming Telemetry

pmacct & Streaming Telemetry (1/3)

slide-22
SLIDE 22

flat-files Streaming Telemetry Kafka

Telemetry dump at regular time intervals

pmtelemetryd

Telemetry real-time log

pmacct & Streaming Telemetry (2/3)

slide-23
SLIDE 23

Streaming telemetry flat-files NetFlow IPFIX Kafka MySQL

aggregation method #1 aggregation method #2

nfacctd

pmacct & Streaming Telemetry (3/3)

slide-24
SLIDE 24

Paolo Lucente paolo@nS.net

hSp://www.gin.nS.net

Paolo Lucente paolo@pmacct.net

hSp://www.pmacct.net/ | hSps://github.com/pmacct/pmacct

pmacct: BMP and Streaming Telemetry

ALNOG2, Tirana – Nov 2018