Symbolic Analysis of Networked Systems Klaus Wehrle t Joint work - - PDF document

SLIDE 1

http://comsys.rwth-aachen.de

Symbolic Analysis

• f Networked Systems

Klaus Wehrle Joint work by the COMSYS team

klaus@comsys.rwth-aachen.de

t

http://comsys.rwth-aachen.de

2

Challenges in Softwarized Communication Systems

— Software plays an increasingly important role in networking

„ Protocols, billions of apps, etc. „ Network elements become flexible (SDN, NFV, In-network processing)

— Important: Analysis of real code – not models

Switch Switch Switch Data Center Data Center

Ne Netwo worked Systems (protocols, apps)

Sensors Actuators

Switch Switch Switch Cloud- based Control

wired network

Lat Latenc ency-cr critica cal networke ked co control

Networked Systems

„ Edge: Protocols, Apps, ... „ Core: Network Functions

In-Network Processing

„ Reducing latency „ Networked control

Predictable?

(performance, resources)

Reliability?

(bugs, loops) Goal: devise a new methodology for

Software Analysis of Interacting Systems?

Rigorous, automated and effective!

SLIDE 2

http://comsys.rwth-aachen.de

3

State of the Art in Distributed Systems Testing

Testbeds, Prototypes Emulation Random testing

Automatism Coverage Effectiveness Automatism Coverage Effectiveness Automatism Coverage Effectiveness

Model-based proofs

model

Simulation

model

Automatism Coverage Effectiveness

http://comsys.rwth-aachen.de

Traditional

Symbolic Execution

if (x<5) if (x == 0)

SLIDE 3

http://comsys.rwth-aachen.de

5

Symbolic Execution: A Simple Example

if (x == 0) if (x < 50) if (x > 10) Test 1: x = 0 Test 2: x = 22 Test 3: x = 5 Test 4: x = 99 int x = symbolic_input(); à x may be any value Path 1: { x = 0 } Path 2: { 10 < x < 50 } Path 3: { x ≠ 0 ∧ x ≤ 10 } Path 4: { 50 ≤ x }

int get_range(int x) { if (x == 0) return blue(); if (x < 50) { if (x > 10) return red(); return green(); } return orange(); }

x = 0 x ≠ 0 x ≠ 0 && x < 50 x > 10 && x < 50 x ≤ 10 && x ≠ 0 x ≥ 50

SMT

But, is Symbolic Execution able to analyze networked systems?

http://comsys.rwth-aachen.de

6

Symbolic Execution and Networked Systems — Symbolic analysis of networked systems?

if (x < 5) if (x == 0)

X Y P P

Additional influence factor: à Consider packets as additional input

SLIDE 4

http://comsys.rwth-aachen.de

7

Symbolic Execution and Networked Systems

if (x < 5) if (x == 0)

X P Y P

Additional influence factor: à Consider packets as additional input à It may arrive at various/any times

Need for a rigorous analysis of any input at any time — Symbolic analysis of networked systems?

http://comsys.rwth-aachen.de

Symbolic Analysis

• f Network Input

t

SLIDE 5

http://comsys.rwth-aachen.de

9

Symbolic Execution of Networked Systems

P P P P P P P P P P

?

— Symbolic analysis of network input

http://comsys.rwth-aachen.de

10

Symbolic Distributed Execution (SDE) Branching within a node causes branches in all other nodes

SLIDE 6

http://comsys.rwth-aachen.de

11

SDE: State Explosion

— Test scenarios

„ Grid with n2 nodes (example: 49) „ Transmissions via a static path „ Symbolic network failures „ 10s simulated time

— Results using the conservative approach (49 nodes)

>24 GB >22 h >779K states

Source Destination

The basic implementation of the formal model of SDE is not scalable

http://comsys.rwth-aachen.de

12

SDE: Elimination of Redundant States

— Test scenarios

„ Grid with n2 nodes (example: 49) „ Transmissions via static path „ Symbolic network failures „ 10s simulated time

— Results using conservative and lazy forking algorithms

1:49 min ~587 MB 533 states

Significant elimination of duplicates enables much more scalable analyses

Source Destination

SLIDE 7

http://comsys.rwth-aachen.de

Symbolic Analysis

• f Protocol Loops

t

Infinite Loop Detection

http://comsys.rwth-aachen.de

14

Liveness of a Protocol – Infinite Loop Detection

— Why are infinite loops an issue with protocols?

„ The outmost (protocol) loop should run infinitely (intended loop) „ The input handler should always finalize (non-intended loop) „ Infinite inner loop is a bug

— When is a loop infinite?

„ If it comes to the same state, again and again!

¾ maybe with (different) intermediate steps

— When is a loop erroneous

„ If it does not consume any input any more?

— How can we detect re-occurring same states?

Source: baynote.com

• r

SLIDE 8

http://comsys.rwth-aachen.de

15

Efficient Implementation of Same State Detection

— Two states are the same if all their memory is the same

„ Including call stack and instruction pointer

— Compare each new state SX to all its predecessor states

„ How can this be achieved efficiently? 1 bool x = false; 2 x = true; 3 while (x) { 4 x = true; 5 }

… … S0 S1 S2 S3 S2 S3 memcmp(NULL1, NULL2, 0xFF…FF);

„ For every predecessor state „ Naïve implementation is prohibitively expensive! http://comsys.rwth-aachen.de

16

Efficient Implementation of Same State Detection

— Two states are the same if all their memory is the same

„ Including call stack and instruction pointer

— Compare each new state SX to all its predecessor states

„ How can this be achieved efficiently? 1 bool x = false; 2 x = true; 3 while (x) { 4 x = true; 5 }

… … S0 S2 S3 S2 H0 H2 H3 H2 H3 S3

Efficient same state detection?

„ Compare hashes instead of states „ Hash after blocks, not instructions

SLIDE 9

http://comsys.rwth-aachen.de

17

Efficient Implementation of Same State Detection

— Two states are the same if all their memory is the same

„ Including call stack and instruction pointer

— Compare each new state SX to all its predecessor states

„ How can this be achieved efficiently? 1 bool x = false; 2 x = true; 3 while (x) { 4 x = true; 5 }

… … S0 H0 H1 H2 H3 H2 H3 H0

Efficient same state detection?

„ Compare hashes instead of states „ Hash blocks, not instructions „ Compute hashes iteratively http://comsys.rwth-aachen.de

18

Results

— So far, a total of seven previously undetected bugs were detected

„ Five bugs in the GNU Coreutils ¾ e.g. in “tail”: 130 line while(1) loop calling 2 functions „ Two bugs in busybox ¾ e.g. In a 490 line while(1) loop calling 2 functions „ All bugs have been reported, confirmed and fixed „ The coreutils bugs have existed for over 12 years!

• r

SLIDE 10

http://comsys.rwth-aachen.de

The Next Challenge

Symbolic Analysis

• f Temporal Uncertainty

t

20

A B t

Analyzing uncertain event times – Why is time so important?

„ State of system at arrival time of input determines the behavior „ Rigorous analysis requires analysis of all points in time! „ Moreover, time is continuous – not discrete!

SLIDE 11

21

A B t Symbolic Time: Symbolic analysis of uncertain event times

Problems Challenges „Time is continuous – not discrete à How to derive temporal equival. classes? „Temporal dependencies in code à How to detect them? „Deriving all combinations and dependencies à How to make sure to consider all cases?

Challenge: Automatic Generation of Temporal Equivalence Classes

http://comsys.rwth-aachen.de

22

Challenges in Softwarized Communication Systems

— Trend: Software plays an increasingly important role in networking

„ Protocols, billions of apps, etc. „ Network elements become flexible (SDN, NFV, In-network processing)

— Important: Analysis of real code – not models

Switch Switch Switch Data Center Data Center

Ne Netwo worked Systems (protocols, apps)

Sensors Actuators

Switch Switch Switch Cloud- based Control

wired network

Lat Latenc ency-cr critica cal networke ked co control

Networked Systems

„ Edge: Protocols, Apps, ... „ Core: Network Functions

In-Network Processing

„ Reducing latency „ Networked control

Predictable?

(performance, resources)

Reliability?

(bugs, loops)

Reliability!

(bugs, loops)

SLIDE 12

http://comsys.rwth-aachen.de

Symbolic Analysis

• f Protocol / NF Performance

If pkt > 40 do_this(pkt); else do_that(pkt): ...

?

http://comsys.rwth-aachen.de

24

Performance Prediction of Softwarized Network Functions

— Challenge: Prediction of Processing Effort/Time of a NF

„ Necessary processing resources? „ Expected/worst latency? „ Achievable data rate? „ Influence among NFs? „ Are we under attack? „ …

— Influence Factors

„ Code of the NF „ Input Traffic (Pattern, Volume) „ CPU Execution

¾ Superscalar execution ¾ Branch prediction ¾ Caching

Achieved throughput per processing effort of a NF

SLIDE 13

http://comsys.rwth-aachen.de

25 Performance- & Analysis-Feedback Prediction Fix bugs Analysis

• f Paths

Instruction Chains

Per Function Per Platform Performance Prediction Network Function

1 #include <bcc/proto.h> 2 #include <linux/pkt_cls.h> 3 #include <net/sock.h> 4 BPF_TABLE("hash", uint16_t, uint8_t, blocked_dports, 4096); 5 6 int act_main(struct __sk_buff *skb) { 7 u8 *cursor = 0; 8 struct ethernet_t *ethernet; 9 struct ip_t *ip; 10 struct tcp_t *tcp; 11 if (skb->len < sizeof(*ethernet)+sizeof(*ip)+sizeof(*tcp)) return TC_ACT_UNSPEC; 12 13 ethernet = cursor_advance(cursor, sizeof(*ethernet)); 14 if (!(ethernet->type == 0x0800)) return TC_ACT_UNSPEC; 15 ip = cursor_advance(cursor, sizeof(*ip)); 16 if (ip->nextp != 0x06) return TC_ACT_UNSPEC; 17 18 tcp = cursor_advance(cursor, sizeof(*tcp)); 19 uint16_t dport = tcp->dst_port; 20 uint8_t *blocked_p = blocked_dports.lookup(&dport); 21 if (!blocked_p) goto return TC_ACT_OK; 22 uint8_t blocked = *blocked_p; 23 if (blocked) return TC_ACT_PIPE; 24 else return TC_ACT_OK; 25 }

Execution Tree Instruction Chains

1 alloca 2 getelementptr 3 load 4 icmp 5 br 6 bpf_load_half 7 icmp 8 br 1 alloca 2 getelementptr 3 load 4 icmp 5 br 6 bpf_load_half 7 icmp 8 br 9 bpf_load_byte 10 icmp 11 br 12 bpf_load_half 13 trunc 14 store 15 bpf_pseudo 16 bpf_map_lookup 17 hash(2, 1) 18 icmp 19 br 20 load 21 icmp 22 br

Performance Prediction

. . .

Offline Calibration

Measure instruction costs

• n target platform, e.g.,

CPU cycles needed for an add instruction on our hardware (i7-870)

Per Platform Instruction Cost Databases

Cisco Juniper AWS Carrier Cloud Our HW Linux Intel i7

Symbolic Execution 1 Fix Bugs 2 Iterate Over All Paths 3 Store 4 Predict 5 6 Increase Packet Rate, Reduce Latency, Harden Against Attacks Per Function Per Platform Performance Prediction Network Function

1 #include <bcc/proto.h> 2 #include <linux/pkt_cls.h> 3 #include <net/sock.h> 4 BPF_TABLE("hash", uint16_t, uint8_t, blocked_dports, 4096); 5 6 int act_main(struct __sk_buff *skb) { 7 u8 *cursor = 0; 8 struct ethernet_t *ethernet; 9 struct ip_t *ip; 10 struct tcp_t *tcp; 11 if (skb->len < sizeof(*ethernet)+sizeof(*ip)+sizeof(*tcp)) return TC_ACT_UNSPEC; 12 13 ethernet = cursor_advance(cursor, sizeof(*ethernet)); 14 if (!(ethernet->type == 0x0800)) return TC_ACT_UNSPEC; 15 ip = cursor_advance(cursor, sizeof(*ip)); 16 if (ip->nextp != 0x06) return TC_ACT_UNSPEC; 17 18 tcp = cursor_advance(cursor, sizeof(*tcp)); 19 uint16_t dport = tcp->dst_port; 20 uint8_t *blocked_p = blocked_dports.lookup(&dport); 21 if (!blocked_p) goto return TC_ACT_OK; 22 uint8_t blocked = *blocked_p; 23 if (blocked) return TC_ACT_PIPE; 24 else return TC_ACT_OK; 25 }

Execution Tree Instruction Chains

1 alloca 2 getelementptr 3 load 4 icmp 5 br 6 bpf_load_half 7 icmp 8 br 1 alloca 2 getelementptr 3 load 4 icmp 5 br 6 bpf_load_half 7 icmp 8 br 9 bpf_load_byte 10 icmp 11 br 12 bpf_load_half 13 trunc 14 store 15 bpf_pseudo 16 bpf_map_lookup 17 hash(2, 1) 18 icmp 19 br 20 load 21 icmp 22 br

·

Performance Prediction

. . .

Offline Calibration

Measure instruction costs

• n target platform, e.g.,

CPU cycles needed for an add instruction on our hardware (i7-870)

Per Platform Instruction Cost Databases

Cisco Juniper AWS Carrier Cloud Our HW Linux Intel i7

Symbolic Execution 1 Fix Bugs 2 Iterate Over All Paths 3 Store 4 Predict 5 6 Increase Packet Rate, Reduce Latency, Harden Against Attacks

...

Symbolic Analysis

Per Function Per Platform Performance Prediction Network Function

1 #include <bcc/proto.h> 2 #include <linux/pkt_cls.h> 3 #include <net/sock.h> 4 BPF_TABLE("hash", uint16_t, uint8_t, blocked_dports, 4096); 5 6 int act_main(struct __sk_buff *skb) { 7 u8 *cursor = 0; 8 struct ethernet_t *ethernet; 9 struct ip_t *ip; 10 struct tcp_t *tcp; 11 if (skb->len < sizeof(*ethernet)+sizeof(*ip)+sizeof(*tcp)) return TC_ACT_UNSPEC; 12 13 ethernet = cursor_advance(cursor, sizeof(*ethernet)); 14 if (!(ethernet->type == 0x0800)) return TC_ACT_UNSPEC; 15 ip = cursor_advance(cursor, sizeof(*ip)); 16 if (ip->nextp != 0x06) return TC_ACT_UNSPEC; 17 18 tcp = cursor_advance(cursor, sizeof(*tcp)); 19 uint16_t dport = tcp->dst_port; 20 uint8_t *blocked_p = blocked_dports.lookup(&dport); 21 if (!blocked_p) goto return TC_ACT_OK; 22 uint8_t blocked = *blocked_p; 23 if (blocked) return TC_ACT_PIPE; 24 else return TC_ACT_OK; 25 }

Execution Tree

if (skb->len < sizeof(*ethernet)+sizeof(*ip)+sizeof(*tcp)) {} return TC_ACT_UNSPEC {len < 54} if (!(ethernet->type == 0x0800)) {len ≥ 54} return TC_ACT_UNSPEC {len ≥ 54, read (data + 12) ̸= 2048} uint16_t dport = tcp->dst_port; {len ≥ 54, read (data + 12) = 2048} uint8_t *blocked_p = blocked_dports.lookup(&dport); {len ≥ 54, read (data + 12) = 2048} if (!blocked_p) {len ≥ 54, read (data + 12) = 2048} return TC_ACT_OK {len ≥ 54, read (data + 12) = 2048, λ = 0} uint8_t blocked = *blocked_p; {len ≥ 54, read (data + 12) = 2048, λ ̸= 0} if (blocked) {len ≥ 54, read (data + 12) = 2048, λ ̸= 0} return TC_ACT_PIPE {len ≥ 54, read (data + 12) = 2048, λ ̸= 0, read (λ) ̸= 0} return TC_ACT_OK {len ≥ 54, read (data + 12) = 2048, λ ̸= 0, read (λ) = 0}

Instruction Chains

1 alloca 2 getelementptr 3 load 4 icmp 5 br 6 bpf_load_half 7 icmp 8 br 1 alloca 2 getelementptr 3 load 4 icmp 5 br 6 bpf_load_half 7 icmp 8 br 9 bpf_load_byte 10 icmp 11 br 12 bpf_load_half 13 trunc 14 store 15 bpf_pseudo 16 bpf_map_lookup 17 hash(2, 1) 18 icmp 19 br 20 load 21 icmp 22 br

Performance Prediction

. . .

Offline Calibration

Measure instruction costs

• n target platform, e.g.,

CPU cycles needed for an add instruction on our hardware (i7-870)

Per Platform Instruction Cost Databases

Cisco Juniper AWS Carrier Cloud Our HW Linux Intel i7

Symbolic Execution 1 Fix Bugs 2 Iterate Over All Paths 3 Store 4 Predict 5 6 Increase Packet Rate, Reduce Latency, Harden Against Attacks

Execution Tree

Per Function Per Platform Performance Prediction Network Function

1 #include <bcc/proto.h> 2 #include <linux/pkt_cls.h> 3 #include <net/sock.h> 4 BPF_TABLE("hash", uint16_t, uint8_t, blocked_dports, 4096); 5 6 int act_main(struct __sk_buff *skb) { 7 u8 *cursor = 0; 8 struct ethernet_t *ethernet; 9 struct ip_t *ip; 10 struct tcp_t *tcp; 11 if (skb->len < sizeof(*ethernet)+sizeof(*ip)+sizeof(*tcp)) return TC_ACT_UNSPEC; 12 13 ethernet = cursor_advance(cursor, sizeof(*ethernet)); 14 if (!(ethernet->type == 0x0800)) return TC_ACT_UNSPEC; 15 ip = cursor_advance(cursor, sizeof(*ip)); 16 if (ip->nextp != 0x06) return TC_ACT_UNSPEC; 17 18 tcp = cursor_advance(cursor, sizeof(*tcp)); 19 uint16_t dport = tcp->dst_port; 20 uint8_t *blocked_p = blocked_dports.lookup(&dport); 21 if (!blocked_p) goto return TC_ACT_OK; 22 uint8_t blocked = *blocked_p; 23 if (blocked) return TC_ACT_PIPE; 24 else return TC_ACT_OK; 25 }

Execution Tree Instruction Chains

1 alloca 2 getelementptr 3 load 4 icmp 5 br 6 bpf_load_half 7 icmp 8 br 1 alloca 2 getelementptr 3 load 4 icmp 5 br 6 bpf_load_half 7 icmp 8 br 9 bpf_load_byte 10 icmp 11 br 12 bpf_load_half 13 trunc 14 store 15 bpf_pseudo 16 bpf_map_lookup 17 hash(2, 1) 18 icmp 19 br 20 load 21 icmp 22 br

Performance Prediction

. . .

Offline Calibration

Measure instruction costs

• n target platform, e.g.,

CPU cycles needed for an add instruction on our hardware (i7-870)

Per Platform Instruction Cost Databases

Cisco Juniper AWS Carrier Cloud Our HW Linux Intel i7

Symbolic Execution 1 Fix Bugs 2 Iterate Over All Paths 3 Store 4 Predict 5 6 Increase Packet Rate, Reduce Latency, Harden Against Attacks

Network Function Code

Instruction- Cache- & CPU-Model Traffic Pattern

100 200 300 400 CPU Cycles 0.00 0.01 0.02 Frequency 0.00 0.25 0.50 0.75 1.00 CDF measured predicted 5 5 Rate [Million pkt/s] 250 500 750 1000 1250 CPU Cycles 0.000 0.001 0.002 0.003 Frequency 0.00 0.25 0.50 0.75 1.00 CDF measured predicted 5 5 4 3 2 Rate [Million pkt/s]

Performance Predictions

100 200 300 CPU Cycles 0.00 0.05 0.10 Frequency 0.00 0.25 0.50 0.75 1.00 CDF measured predicted 100 200 300 400 CPU Cycles 0.00 0.02 0.04 Frequency 0.00 0.25 0.50 0.75 1.00 CDF measured predicted 100 200 300 400 CPU Cycles 0.00 0.05 0.10 Frequency 0.00 0.25 0.50 0.75 1.00 CDF measured predicted

Pre-Deployment Performance Prediction of On-Path NFs

http://comsys.rwth-aachen.de

26

Challenges in Softwarized Communication Systems

— Trend: Software plays an increasingly important role in networking

„ Protocols, billions of apps, etc. „ Network elements become flexible (SDN, NFV, In-network processing)

— Important: Analysis of real code – not models

Switch Switch Switch Data Center Data Center

Ne Netwo worked Systems (protocols, apps)

Sensors Actuators

Switch Switch Switch Cloud- based Control

wired network

Lat Latenc ency-cr critica cal networke ked co control

Networked Systems

„ Edge: Protocols, Apps, ... „ Core: Network Functions

In-Network Processing

„ Reducing latency „ Networked control

Predictable?

(performance, resources)

Reliability?

(bugs, loops)

Reliability!

(bugs, loops)

Predictable!

(performance, resources)

SLIDE 14

http://comsys.rwth-aachen.de

Symbolic Analysis

• f Networked Systems

Klaus Wehrle Joint work by the COMSYS team

klaus@comsys.rwth-aachen.de

t

SYMBIOSYS project homepage à https://comsys.rwth-aachen.de/research/projects/symbiosys/