pmacct: BMP and Streaming Telemetry Paolo Lucente NTT CommunicaDons - PowerPoint PPT Presentation

pmacct: BMP and Streaming Telemetry Paolo Lucente NTT CommunicaDons | pmacct ALNOG 2, Tirana – Nov 2018

whoami Paolo Lucente GitHub: paololucente LinkedIn: plucente Digging data out of networks worldwide for fun and profit for more than 10 years

pmacct ALNOG 2, Tirana – Nov 2018

pmacct is open-source, free, GPL’ed soRware MySQL Streaming PgSQL Telemetry sFlow SQLite libpcap MongoDB BerkeleyDB NetFlow IPFIX tee sFlow flat-files NetFlow IPFIX RabbitMQ BGP BMP IGP Kafka memory maps GeoIP tables hSp://www.pmacct.net/

pmacct: a few simple use-cases NetFlow flat-files IPFIX sFlow tee BMP Kafka libpcap IPFIX

pmacct: a slightly more complex use-case tee aggregation method #1 flat-files nfacctd NetFlow IPFIX MySQL aggregation method #2 Kafka BGP

The use-case for message brokers

Key pmacct non-technical facts § 15+ years old project § Can’t spell the name aRer the second drink § Free, open-source, independent § Under acDve development § InnovaDon being introduced § Well deployed around, also in large SPs/IXPs § Close to the SP/IXP community needs

BMP “It is wise not to use the same protocol to monitor itself” -- Ancient italian proverb ALNOG 2, Tirana – Nov 2018

BMP § BGP Monitoring Protocol § RFC 7854: • first draR in 2008, sparse work unDl 2012; • stall between 2012 and 2015; • real tracDon kicks in: 10 draRs between 2015 and 2016; • RFC award in Jun 2016 § Uncomplicated protocol design § Great effort but .. • .. industry evolved all these years • increased hunger for data A DevOps guy during lunch break

BGP monitoring with BMP (1/2) Peers With BMP, I learn all the paths the peering router heard P 0 Peering P 1 Router All Vantage P 2 Point P 0-4 P 3 Paths P 4 Credits to: R. Bush (IIJ) @ BMP BoF, RIPE74

BGP monitoring with BMP (2/2)

Problem statement BGP Peer-A BGP Peer-B We can see this Adj-Rib-In (Pre) Adj-Rib-In (Pre) Filters/Policy Filters/Policy And this Adj-Rib-In (Post) Adj-Rib-In (Post) Credits to: T. Evens (Cisco), S. Bayraktar (Cisco), P. Lucente (NTT) @ GROW WG, IETF 98

Proposal: extend BMP to loc-RIB and Adj-RIB-Out (1/3) BGP Peer-A BGP Peer-B StaDc ISIS Adj-Rib-In (Pre) Adj-Rib-In (Pre) Filters/Policy Filters/Policy Adj-Rib-In (Post) Adj-Rib-In (Post) Accepted Accepted And this We also want to see this Adj-Rib-Out (Pre) Loc-Rib Filters/Policy Adj-Rib-Out (Post) And this Credits to: T. Evens (Cisco), S. Bayraktar (Cisco), P. Lucente (NTT) @ GROW WG, IETF 98

Proposal: extend BMP to loc-RIB and Adj-RIB-Out (2/3)

Proposal: extend BMP to loc-RIB and Adj-RIB-Out (3/3)

draR-iep-grow-bmp-{local-rib,adj-rib-out} use-cases § Loc-RIB: § Monitor routes selected and used by the router : o ECMP o CorrelaDon with NetFlow/IPFIX o Next-hop preservaDon § Monitor locally originated and BGP routes without requiring a BGP peering § Policy verificaDon § Adj-RIB-Out: § Monitor routes adverDsed to peers § Policy verificaDon Credits to: T. Evens (Cisco), S. Bayraktar (Cisco), P. Lucente (NTT) @ GROW WG, IETF 98

draR-iep-grow-bmp-{local-rib,adj-rib-out} standardizaDon status § Both draRs in their -02 version § draR-iep-grow-bmp-local-rib-00 -> -02: § Mainly text clarificaDons § Peer down VRF/Table name opDonal TLV [reduce state] § draR-iep-grow-bmp-adj-rib-out-00 -> -02: § Mainly text clarificaDons § Peer up Admin Label opDonal TLV [ie. to carry peer- group info] § Plenty of discussion took place on the GROW WG list at IETF. Further conversaDon is encouraged!

Streaming Telemetry ALNOG 2, Tirana – Nov 2018

Streaming Telemetry § A scalable replacement for SNMP: • Push technology • Subscribing to data of interest § A long journey to standardizaDon ahead: • Models: Openconfig and vendor-specific • Transport: tradiDonal, Netconf and gNMI • RPC: Netconf (YANG Push) and gNMI • Encoding: JSON and GPB

pmacct & Streaming Telemetry (1/3) § Mission statement: • Integrate Streaming Telemetry data with other relevant technologies (ie. IPFIX, BMP, etc.) • Especially in the current pre-standardizaDon stage, offer an efficient mulD-vendor collecDon layer for Streaming Telemetry

pmacct & Streaming Telemetry (2/3) Telemetry dump at regular time intervals flat-files pmtelemetryd Streaming Telemetry Kafka Telemetry real-time log

pmacct & Streaming Telemetry (3/3) aggregation method #1 flat-files nfacctd NetFlow IPFIX MySQL aggregation method #2 Kafka Streaming telemetry

pmacct: BMP and Streaming Telemetry Paolo Lucente paolo@nS.net hSp://www.gin.nS.net Paolo Lucente paolo@pmacct.net hSp://www.pmacct.net/ | hSps://github.com/pmacct/pmacct ALNOG2, Tirana – Nov 2018