Packet-Level Analytics in Software without Compromises HotCloud 18, - - PowerPoint PPT Presentation

HotCloud ’18, July 9th, 2018, Boston, MA

Oliver Michel John Sonchack Eric Keller Jonathan M. Smith

Packet-Level Analytics in Software without Compromises

■ Security issues ■ Performance issues ■ Equipment failure ■ Misconfiguration

Network monitoring is important

2

Packet-Level Analytics in Software without Compromises — Oliver Michel

Switch + Telemetry Analytics Platform

Challenging environment

■ more traffic ■ more threats ■ encrypted traffic

Packet-Level Analytics in Software without Compromises — Oliver Michel

3

Fraction of encrypted HTTP traffic in Google Chrome

% encrypted 25 50 75 100

Jun 6 2015 Jun 4 2016 Jun 3 2017 Jun 2 2018

82 67 54 47

[Google Transparency Report 2018]

Total Ransomware Samples

Collected Samples [M]

5 10 15

Q4 2015 Q12016 Q2 2016 Q3 2016 Q4 2016 Q1 2017 Q2 2017 Q3 2017

12.1 10.8 9.5 8.9 8.4 7.8 6 4.4 [McAfee Labs Thread Report Dec. 2017]

Existing systems make compromises

4

Packet-Level Analytics in Software without Compromises — Oliver Michel

collector analytics

filtering sampling

6 4 2 aggregation

PFE

filter() groupby() zip()

loss of information loss of capability

■ Programmable Forwarding Engines

■ Marple [SIGCOMM 2017] ■ *flow [ATC 2018]

Programmable Forwarding Engines

Packet-Level Analytics in Software without Compromises — Oliver Michel

5

High-Performance Network Telemetry

?

~ 131 M packet records/s *flow technology

■ per-packet records ■ x86 / general purpose programming language ■ ~5M pps per core

The ideal network analytics system

Packet-Level Analytics in Software without Compromises — Oliver Michel

6

Is it possible to perform packet-level analytics on cloud-scale infrastructures without compromises?

Leveraging parallel architectures

source sink parallel operators

Packet-Level Analytics in Software without Compromises — Oliver Michel

7

Leveraging parallel architectures

NIC

Backend

(e.g., time series DB)

input stage processing stages aggregation stage

Packet-Level Analytics in Software without Compromises — Oliver Michel

8

■ Network attached input ■ Partitionability/aggregation ■ High rates, small, well-formed records

Characteristics of packet record workloads

Packet-Level Analytics in Software without Compromises — Oliver Michel

9

Can we use properties of packet analytics workloads to our advantage?

Network attached input

40G/100G NIC Switch/PFE

queue queue queue NIC DMA NIC DMA NIC DMA analytics pipeline analytics pipeline analytics pipeline

Packet-Level Analytics in Software without Compromises — Oliver Michel

10

■ Array vs. linked list ■ Lock-free design ■ Wait-free design ■ Zero-copy operations

Many small records

Packet-Level Analytics in Software without Compromises — Oliver Michel

11

throughput [M records/s]

4 8 12 16

lock-based, array lock-free, linked list lock-free,array

Programming Abstraction

1 int main(int argc, char** argv) 2 { 3 4 5 6 7 8 return 0; 9 } auto source = app.add_stage<source>(1, “enp6s0f0”); auto sink = app.add_stage<sink>(1, std::cout); app.connect<jetstream::pkt_t>(source, sink); app(); jetstream::app app;

source sink

port port

ring buffer

Packet-Level Analytics in Software without Compromises — Oliver Michel

12

Performance

Packet-Level Analytics in Software without Compromises — Oliver Michel

13

throughput [M packets/s] 2 4 6 8 10 12 intermediate processors 1 2 3 4 5 6 passthrough packets per source 2 3 4 5 6 parallel operators 1 sink source

■ Facebook web cluster: ~ 91M egress pps ■ ~32 cores for basic packet-level insight ■ 176 web servers — 1 analytics server: ~0.5%

• f cluster capacity

Performance

14

Packet-Level Analytics in Software without Compromises — Oliver Michel

[Arjun Roy, Hongyi Zeng, Jasmeet Bagga, George Porter, and Alex C. Snoeren. 2015. Inside the Social Network's (Datacenter) Network. SIGCOMM Comput. Commun. Rev. 45, 4 (August 2015), 123-137]]

~352 Gb/s

~88 Gb/s — 91M p/s

jetstream 32 cores

Conclusion / Discussion

15

Packet-Level Analytics in Software without Compromises — Oliver Michel

jetstream high-performance, software network analytics platform

Is it possible to perform packet-level analytics on cloud-scale infrastructures without compromises?

Q&A / DISCUSSION

Oliver Michel

• liver.michel@colorado.edu

http://nsr.colorado.edu/oliver

17

Packet-Level Analytics in Software without Compromises — Oliver Michel

The right approach for network monitoring and analytics? What data do we need for monitoring/debugging?

packet-level flow-level software hardware

PANEL OPENING SLIDE

source sink parallel operators

behavioral analysis packet level record generation software processing

Packet-Level Analytics in Software without Compromises

encrypted traffic

programmable forwarding engines

complex applications

Oliver Michel, John Sonchack, Eric Keller, Jonathan M. Smith

University of Colorado Boulder, University of Pennsylvania

BACKUP SLIDES

21

Packet-Level Analytics without Compromises — Oliver Michel

[Apache Flink] [StreamBox Miao ‘18]

Programming abstraction

22

Packet-Level Analytics without Compromises — Oliver Michel

1 explicit source(const std::string& iface_name_) : proc() { 2 add_out_port<jetstream::pkt_t>(0); 3 […] 4 } 1 class source : public jetstream::proc { 2 […] 3 }; 1 jetstream::signal operator()() override { 2 out_port<pkt_t>(0)->enqueue(read_from_nic(_pkt), jetstream::signal::continue); 3 return jetstream::signal::continue; 4 }

Processor definition

Jetstream architecture

23

Packet-Level Analytics without Compromises — Oliver Michel

NIC

Backend

(e.g., time series DB)

NUMA awareness pipeline 1→ CPU socket 1 pipeline 2→ CPU socket 2

Stream Processing

24

Packet-Level Analytics without Compromises — Oliver Michel

Filter

• nly TCP

Packet Packet TCP

Packet

TCP

Packet

Parallelize

group by IP Destination

ip_dst % 2 == 1 ip_dst % 2 == 0 Bin

by time (e.g,, 10sec)

Filter

> n Bytes per 10 sec

Alert

Reducing copy operations

25

Packet-Level Analytics without Compromises — Oliver Michel

Packet Buffer

Pointer Passing queue<pkt*> queue<pkt*>

Reducing copy operations

26

Packet-Level Analytics without Compromises — Oliver Michel

1 packet p; 2 p.ip_proto = 6; 3 q.enqueue(p); 1 auto p = q.enqueue(); 2 p->ip_proto = 6;

queue<pkt> pointer directly into queue

Pointer Passing

Technologies

27

• Programmable switches and PISA: Protocol Independent Switch

Architecture

• Reconfigurable match-action tables in hardware
• multiple stages with TCAM/ALU pair, fixed processing time,

guarantees line rate