beehive software-de fj ned networking Soheil Hassas Yeganeh Yashar - PowerPoint PPT Presentation

Towards a simple abstraction for scalable beehive software-de fj ned networking Soheil Hassas Yeganeh Yashar ganjali University of Toronto

Traditional networks Hard to Program Distributed Systems Controller Controller Controller Switch Switch Switch 2

Software Defined Networking Easy Hard to Program Distributed Systems Centralized Application Controller Switch Switch Switch 3

Software Defined Networking Easy Hard to Program Distributed Systems Existing Distributed Controllers • Excellent in performance & Application Application scalability • Perfect fj t for some speci fj c Controller Controller scenarios Switch Switch Switch 4

Software Defined Networking s k r o w t e n l a n o i t i d a r t n a h t r e t t e b h c u M still Hard to Program Distributed Systems Existing Distributed Controllers • Don’t hide the boilerplates of Application Application distributed programming • Require signi fj cant e fg orts to Controller Controller instrument and optimize apps Switch Switch Switch 5

Our GOAL Easy similar to Hard to Program Distributed Systems centralized controllers optimized + Application Application placements Controller Controller application + analytics Switch Switch Switch 6

Our GOAL centralized Application 7

Our GOAL centralized can be automatically Application Application Application Application transformed into 8

Our GOAL distributed Application centralized can be automatically Application Application transformed into Application 9

Our goal centralized distributed Application Application Application Application = Machine Machine Machine Machine deployed on multiple physical machines. Very challenging for generic control applications. 10

Overview Application Application Application Abstraction Compiler Application Control Platform Machine Machine Machine 11

Abstraction what is a control application? Process async messages in application functions using state dictionaries Application Dictionaries Function msg Function msg Function 12

Abstraction how do applications communicate? Application Application async messages all functions msg Function Function state dictionaries msg Function Function functions of the msg same application Function Function 13

Example Tra ffj c Engineering s i Initializes dictionary Init SwitchJoined{s i } Timeout s i Statistics S Queries switches Query s StatQuery{s i } t a t s : s i Collects stat results Collect StatResult{s i } Topology T * Reroutes fm ows, Timeout Route * if needed FlowMod 14

Example Tra ffj c Engineering s i Init How to transform TE into a distributed application while s i S Query preserving state consistency? s i Collect T * Route * 15

Example The dictionary key is in the message For each entry Tra ffj c Engineering Functions create an implicit s i mapping between messages Init SwitchJoined{s i } and dictionary entries: s i S Query Timeout s i The entries a function needs Collect StatResult{s i } T to process a message. * Route Timeout * All entries 16

Example Tra ffj c Engineering s i Init SwitchJoined{s i } Init() , Query() and Collect() s i S access S on a per switch basis. Query Timeout s i Collect StatResult{s i } T Route 17

Example Tra ffj c Engineering s i Init s i Switch Entry S Query Init() , Query() and Collect() fm ow1 -> stat s i 1 access S on a per switch basis. fm ow2 -> stat Collect fm ow3 -> stat T 2 fm ow4 -> stat Route 18

Example Switch Entry Switch Entry Tra ffj c Engineering Tra ffj c Engineering fm ow1 -> stat fm ow3 -> stat 1 2 fm ow2 -> stat fm ow4 -> stat s 2 s 1 Init Init s 1 s 2 S S Query Query s 2 s 1 Collect Collect T T Route Route Machine 1 Machine 2 19

Example Tra ffj c Engineering Init() , Query() and Collect() s i Init access S on a per switch basis. s i Switch Entry S Query Route() accesses the whole fm ow1 -> stat s i 1 dictionary S to process the fm ow2 -> stat Collect fm ow3 -> stat timeout message. T 2 fm ow4 -> stat * Route Timeout 20

Example This will cause in consistency. Switch Entry Switch Entry Tra ffj c Engineering Tra ffj c Engineering fm ow1 -> stat fm ow3 -> stat 1 2 fm ow2 -> stat fm ow4 -> stat s 2 s 1 Init Init s 1 s 2 S S Query Query s 2 s 1 Collect Collect T T * * Route Route Machine 1 Machine 2 21

Example Switch Entry fm ow1 -> stat Tra ffj c Engineering Tra ffj c Engineering 1 fm ow2 -> stat fm ow3 -> stat 2 fm ow4 -> stat s i Init Init s i S S Query Query s i Collect Collect T T * Route Route Machine 1 Machine 2 22

consistency k1 msg 3 k2 k3 k5 msg 2 k2 k4 msg 1 Application k1 Function 1 k2 k3 k4 Function 2 k5 23

consistency Application Application k2 k1 Function 1 Function 1 msg 2 msg 3 k3 msg 1 Function 2 k4 Function 2 k5 Machine Machine 24

We need a runtime that steers messages among application instances while preserving consistency. Application Application Application Function 1 Function 1 Function 1 Function 2 Function 2 Function 2

control platform Hive + Cell + Bee Application Application F F F F Hive Hive 26

control platform • is the controller • provides the boilerplates (e.g., locking, consistency, …) Hive • can run on a separate machine Application Application F F F F Hive Hive 27

control platform Cell • an entry in a dictionary of a speci fj c application • e.g., (TE, S, s i , stats of s i ) Application Application F F F F Hive Hive 28

control platform • a lightweight thread of execution • process messages Bee • exclusively owns a set of cells Application Application F F F F Hive Hive 29

control platform TE TE C I C I Hive Hive m Switch Switch Switch Switch 30

control platform TE TE m C I C I Hive Hive Switch Switch Switch Switch 31

control platform How do we infer the cells? TE TE m C I C I Hive Hive Switch Switch Switch Switch 32

control platform How do we infer the cells? map(app, msg) is an application de fj ned function that maps a message to the set of cells used to process that message. func Collect(r, s): TE m s.append(flow stats in r) C I Beehive’s compiler on StatReply(r): Collect(r, S[r.switch]) Hive can automatically generate the map map StatReply(r): function. return (S, r.switch) Switch Switch 1-3 lines of code 33

control platform • Function Composition • Transactions (State + Messages) • Bee Migration • Fault tolerance TE m • Optimized Placement C I • Runtime Instrumentation • Feedback Hive • Proxied Hives • … Switch Switch 34

Migration TE TE C I C I Hive Hive Switch Switch Switch Switch 35

Migration TE TE C I C I Hive Hive m Switch Switch Switch Switch 36

Migration TE TE C I C I m Hive Hive Switch Switch Switch Switch 37

Migration TE TE m C I C I Hive Hive Switch Switch Switch Switch 38

Migration This is not optimal and can happen often. TE TE m C I C I Hive Hive Switch Switch Switch Switch 39

Migration TE TE m m m C I C I Hive Hive Switch Switch Switch Switch 40

Migration TE TE m m m C I C I Hive Hive Switch Switch Switch Switch 41

Migration TE TE m m m C I C I m m Hive Hive Switch Switch Switch Switch 42

Migration When/where should we migrate bees? • NP-Hard problem • We use a simple heuristic TE TE C I C I m m m m m Hive Hive Switch Switch Switch Switch 43

Optimized Placement Our heuristic A bee that receives the majority of its messages from bees on another hive is migrated to that hive. TE TE C I C I Hive Hive Switch Switch Switch Switch 44

Runtime instrumentation • tra ffj c matrix among bees • resource consumption • message provenance TE TE C I C I Hive Hive Switch Switch Switch Switch 45

Analytics & FeedBack TE TE Q C I R Q C I R Hive Hive Switch Switch Switch Switch 46

Analytics & FeedBack TE TE Q C I R Q C I R Hive Hive Switch Switch Switch Switch 47

Analytics & FeedBack TE TE Hives Q C I R Q C I R Hives Hive Hive centralized Switch Switch Switch Switch 48

Analytics & FeedBack TE TE m m Hives Q C I R Q C I R Hives Hive Hive centralized Switch Switch Switch Switch 49

Analytics & FeedBack TE TE m m Hives Q C I R Q C I R Hives Hive Hive well-balanced Switch Switch Switch Switch 50

Fault tolerance Colony of replicated bees all in consensus about their state. TE TE TE Q C I R Q C I R Q C I R Hive Hive Hive 51