Participatory Networking: An API for Application Control of SDNs - - PDF document

participatory networking
SMART_READER_LITE
LIVE PREVIEW

Participatory Networking: An API for Application Control of SDNs - - PDF document

Participatory Networking: An API for Application Control of SDNs Andrew Ferguson, Arjun Guha, Chen Liang, Rodrigo Fonseca, and Shriram Krishnamurthi Cornell 1 Participatory Networking 2 Participatory Networking integrates end-users and their


slide-1
SLIDE 1

Participatory Networking:

An API for Application Control of SDNs

Andrew Ferguson, Arjun Guha, Chen Liang, Rodrigo Fonseca, and Shriram Krishnamurthi

1

Cornell

slide-2
SLIDE 2

Participatory Networking

2

Participatory Networking integrates end-users and their applications directly into the management of the network.

slide-3
SLIDE 3

3

Motivation

  • 2. Ekiga
  • 3. ZooKeeper
  • 4. Hadoop
  • 1. SSHGuard

As a motivation, let’s consider four applications which might like to manage the network

slide-4
SLIDE 4

4

  • 2. Ekiga
  • 3. ZooKeeper
  • 4. Hadoop
  • 1. SSHGuard

blocks hosts in response to login attempts uses knowledge from host OS prefers to deny traffic close to source

SSHGuard SSHGuard SSHGuard SSHGuard SSHGuard SSHGuard SSHGuard

today: block bad traffic at end host “if it could…”

slide-5
SLIDE 5

5

  • 2. Ekiga
  • 3. ZooKeeper
  • 4. Hadoop
  • 1. SSHGuard
  • pen source VOIP client

network needs dictated by end-user prefers to reserve bandwidth

Ekiga Ekiga

Explain Ekiga’s traffic pattern “if it could…”

slide-6
SLIDE 6

6

  • 2. Ekiga
  • 3. ZooKeeper
  • 4. Hadoop
  • 1. SSHGuard

Paxos-like coordination service network needs dictated by placement prefers high-priority switch queues

ZooKeeper ZooKeeper ZooKeeper

Explain ZooKeeper’s traffic pattern … “control-traffic” “if it could…”

slide-7
SLIDE 7

7

  • 2. Ekiga
  • 3. ZooKeeper
  • 4. Hadoop
  • 1. SSHGuard
  • pen source data processing platform

network weights known by scheduler prefers to reserve bandwidth

Hadoop Hadoop Hadoop Hadoop Hadoop Hadoop Hadoop Hadoop Hadoop Hadoop Hadoop Hadoop

weights are used to express the relative priority of jobs. today these weights affect the amount of CPU and memory for the job “if it could…”

slide-8
SLIDE 8

8

SDN Controllers SSHGuard SSHGuard

SSHGuard Ekiga ZooKeeper Hadoop Ekiga

how could we do this today? file a ticket with the network operators every few minutes as they have frequently changing dynamic needs, precluding a single, static policy

  • r today, we could program the network by writing an SDN controller for each application.

Combining these controllers would be diffjcult: 1) have to run as root, and 2) would be afgected by the decisions of other controllers

slide-9
SLIDE 9

9

slide-10
SLIDE 10

10

  • 1. decompose control and visibility
  • 2. resolve conflicts between requests

Challenges

stepping back, we see there are two challenges we need to overcome to prevent this chaos. (read slide)

  • r, in other words,
  • 1. how do we keep programs from all running as root?
  • 2. how do we keep programs from being affected by one another?
slide-11
SLIDE 11

Participatory Networking

11

participatory networking is the approach we developed to solve these challenges. to do so, we need to reason about changes being made to the network. to make such reasoning tractable, we don’t allow general purpose programming. instead, we provide applications with a restricted control-plane API.

slide-12
SLIDE 12

PANE

Participatory Networking

12

  • 1. Requests
  • 2. Hints
  • 3. Queries

In our API, users, their hosts, and their applications send three types of messages to a logically centralized network controller, which we call PANE. The first are requests for resources, such as guaranteed minimum bandwidth, latency, path properties, or access control. The second are hints about future traffjc. And the third are queries for current or future properties of the network. The PANE controller serves as an arbiter for conflicting proposals, and ultimately performs the requested reconfigurations.

slide-13
SLIDE 13

13

Participatory Networking

  • End-user API for SDNs
  • Exposes existing mechanisms
  • No effect on unmodified applications

Participatory networking introduces an end-user API or system calls for software defined

  • networks. It does not propose any new mechanisms or network resources such as QoS,

routing, or access control -- it simply allows end-users and their applications to use them. Unmodified applications, or those which choose not to participate, continue to receive the same best-efgort performance of existing networks. In our vision, network operators set baseline policies that enforce fairness and security, while end-users and their applications propose new configurations to meet their needs.

slide-14
SLIDE 14

14

Decomposing Control

Let’s begin with the first challenge: how to decompose control and visibility of the network?

slide-15
SLIDE 15

15

  



 

    

Shares

Hadoop

To divide authority, PANE uses a hierarchy of network “shares” which describe WHO can say WHAT about WHICH flows in the network. First, each share has a list of principals (click), who are the end users and applications authorized to use the share. Second, each share refers to a particular flowgroup (click) -- a set of traffjc flows identified by standard attributes such as source and destination IP and MAC addresses, protocols, and port numbers. Finally, they have a list of privileges (click) indicating what can be performed using the

  • share. For example, traffjc can be allowed or denied, rate-limited, waypointed through a

particular switch, or provided with guaranteed minimum bandwidth. Shares can also authorize end-users to issue hints or make queries about particular traffjc

  • flows. These actions can also come with restrictions. For example, bandwidth reservations

may be restricted using a token bucket. (Pause)

slide-16
SLIDE 16

15

  



 

    

Shares

Hadoop

To divide authority, PANE uses a hierarchy of network “shares” which describe WHO can say WHAT about WHICH flows in the network. First, each share has a list of principals (click), who are the end users and applications authorized to use the share. Second, each share refers to a particular flowgroup (click) -- a set of traffjc flows identified by standard attributes such as source and destination IP and MAC addresses, protocols, and port numbers. Finally, they have a list of privileges (click) indicating what can be performed using the

  • share. For example, traffjc can be allowed or denied, rate-limited, waypointed through a

particular switch, or provided with guaranteed minimum bandwidth. Shares can also authorize end-users to issue hints or make queries about particular traffjc

  • flows. These actions can also come with restrictions. For example, bandwidth reservations

may be restricted using a token bucket. (Pause)

slide-17
SLIDE 17

15

  



 

    

Shares

Hadoop

To divide authority, PANE uses a hierarchy of network “shares” which describe WHO can say WHAT about WHICH flows in the network. First, each share has a list of principals (click), who are the end users and applications authorized to use the share. Second, each share refers to a particular flowgroup (click) -- a set of traffjc flows identified by standard attributes such as source and destination IP and MAC addresses, protocols, and port numbers. Finally, they have a list of privileges (click) indicating what can be performed using the

  • share. For example, traffjc can be allowed or denied, rate-limited, waypointed through a

particular switch, or provided with guaranteed minimum bandwidth. Shares can also authorize end-users to issue hints or make queries about particular traffjc

  • flows. These actions can also come with restrictions. For example, bandwidth reservations

may be restricted using a token bucket. (Pause)

slide-18
SLIDE 18

15

  



 

    

Shares

Hadoop

To divide authority, PANE uses a hierarchy of network “shares” which describe WHO can say WHAT about WHICH flows in the network. First, each share has a list of principals (click), who are the end users and applications authorized to use the share. Second, each share refers to a particular flowgroup (click) -- a set of traffjc flows identified by standard attributes such as source and destination IP and MAC addresses, protocols, and port numbers. Finally, they have a list of privileges (click) indicating what can be performed using the

  • share. For example, traffjc can be allowed or denied, rate-limited, waypointed through a

particular switch, or provided with guaranteed minimum bandwidth. Shares can also authorize end-users to issue hints or make queries about particular traffjc

  • flows. These actions can also come with restrictions. For example, bandwidth reservations

may be restricted using a token bucket. (Pause)

slide-19
SLIDE 19

16

  • Share Tree

A share’s principals also have the capability to delegate privileges by creating subshares (click). The creation of subshares is guided by the principle that you can’t give away more authority than you have. For example, a subshare’s flowgroup (click) must be contained within the parent share’s flowgroup (click). Here, the blue bar represents each flowgroup’s range of permitted source IP addresses. Furthermore, a subshare may not have a more permissive action set (click) than the parent (click), and initially, the subshare’s only principal is its creator (click). Other users can later be added as additional principals (click). This process of creating subshares develops a privilege hierarchy we call the “Share Tree” (click). The root of the share tree is “the rootShare” (click) -- a share which contains all traffjc in the network, comes with all privileges, and has a single root user as the principal.

slide-20
SLIDE 20

16

  • Share Tree

A share’s principals also have the capability to delegate privileges by creating subshares (click). The creation of subshares is guided by the principle that you can’t give away more authority than you have. For example, a subshare’s flowgroup (click) must be contained within the parent share’s flowgroup (click). Here, the blue bar represents each flowgroup’s range of permitted source IP addresses. Furthermore, a subshare may not have a more permissive action set (click) than the parent (click), and initially, the subshare’s only principal is its creator (click). Other users can later be added as additional principals (click). This process of creating subshares develops a privilege hierarchy we call the “Share Tree” (click). The root of the share tree is “the rootShare” (click) -- a share which contains all traffjc in the network, comes with all privileges, and has a single root user as the principal.

slide-21
SLIDE 21

16

  • Share Tree

A share’s principals also have the capability to delegate privileges by creating subshares (click). The creation of subshares is guided by the principle that you can’t give away more authority than you have. For example, a subshare’s flowgroup (click) must be contained within the parent share’s flowgroup (click). Here, the blue bar represents each flowgroup’s range of permitted source IP addresses. Furthermore, a subshare may not have a more permissive action set (click) than the parent (click), and initially, the subshare’s only principal is its creator (click). Other users can later be added as additional principals (click). This process of creating subshares develops a privilege hierarchy we call the “Share Tree” (click). The root of the share tree is “the rootShare” (click) -- a share which contains all traffjc in the network, comes with all privileges, and has a single root user as the principal.

slide-22
SLIDE 22

16

  • Share Tree

A share’s principals also have the capability to delegate privileges by creating subshares (click). The creation of subshares is guided by the principle that you can’t give away more authority than you have. For example, a subshare’s flowgroup (click) must be contained within the parent share’s flowgroup (click). Here, the blue bar represents each flowgroup’s range of permitted source IP addresses. Furthermore, a subshare may not have a more permissive action set (click) than the parent (click), and initially, the subshare’s only principal is its creator (click). Other users can later be added as additional principals (click). This process of creating subshares develops a privilege hierarchy we call the “Share Tree” (click). The root of the share tree is “the rootShare” (click) -- a share which contains all traffjc in the network, comes with all privileges, and has a single root user as the principal.

slide-23
SLIDE 23

16

  • bandwidth

50Mbps

Share Tree

A share’s principals also have the capability to delegate privileges by creating subshares (click). The creation of subshares is guided by the principle that you can’t give away more authority than you have. For example, a subshare’s flowgroup (click) must be contained within the parent share’s flowgroup (click). Here, the blue bar represents each flowgroup’s range of permitted source IP addresses. Furthermore, a subshare may not have a more permissive action set (click) than the parent (click), and initially, the subshare’s only principal is its creator (click). Other users can later be added as additional principals (click). This process of creating subshares develops a privilege hierarchy we call the “Share Tree” (click). The root of the share tree is “the rootShare” (click) -- a share which contains all traffjc in the network, comes with all privileges, and has a single root user as the principal.

slide-24
SLIDE 24

16

  • bandwidth

100Mbps bandwidth 50Mbps

Share Tree

A share’s principals also have the capability to delegate privileges by creating subshares (click). The creation of subshares is guided by the principle that you can’t give away more authority than you have. For example, a subshare’s flowgroup (click) must be contained within the parent share’s flowgroup (click). Here, the blue bar represents each flowgroup’s range of permitted source IP addresses. Furthermore, a subshare may not have a more permissive action set (click) than the parent (click), and initially, the subshare’s only principal is its creator (click). Other users can later be added as additional principals (click). This process of creating subshares develops a privilege hierarchy we call the “Share Tree” (click). The root of the share tree is “the rootShare” (click) -- a share which contains all traffjc in the network, comes with all privileges, and has a single root user as the principal.

slide-25
SLIDE 25

16

  • root

root

bandwidth 100Mbps bandwidth 50Mbps

Share Tree

A share’s principals also have the capability to delegate privileges by creating subshares (click). The creation of subshares is guided by the principle that you can’t give away more authority than you have. For example, a subshare’s flowgroup (click) must be contained within the parent share’s flowgroup (click). Here, the blue bar represents each flowgroup’s range of permitted source IP addresses. Furthermore, a subshare may not have a more permissive action set (click) than the parent (click), and initially, the subshare’s only principal is its creator (click). Other users can later be added as additional principals (click). This process of creating subshares develops a privilege hierarchy we call the “Share Tree” (click). The root of the share tree is “the rootShare” (click) -- a share which contains all traffjc in the network, comes with all privileges, and has a single root user as the principal.

slide-26
SLIDE 26

16

  • root

root adf

bandwidth 100Mbps bandwidth 50Mbps

Share Tree

A share’s principals also have the capability to delegate privileges by creating subshares (click). The creation of subshares is guided by the principle that you can’t give away more authority than you have. For example, a subshare’s flowgroup (click) must be contained within the parent share’s flowgroup (click). Here, the blue bar represents each flowgroup’s range of permitted source IP addresses. Furthermore, a subshare may not have a more permissive action set (click) than the parent (click), and initially, the subshare’s only principal is its creator (click). Other users can later be added as additional principals (click). This process of creating subshares develops a privilege hierarchy we call the “Share Tree” (click). The root of the share tree is “the rootShare” (click) -- a share which contains all traffjc in the network, comes with all privileges, and has a single root user as the principal.

slide-27
SLIDE 27

16

  • root

root adf

bandwidth 100Mbps bandwidth 50Mbps

Share Tree

A share’s principals also have the capability to delegate privileges by creating subshares (click). The creation of subshares is guided by the principle that you can’t give away more authority than you have. For example, a subshare’s flowgroup (click) must be contained within the parent share’s flowgroup (click). Here, the blue bar represents each flowgroup’s range of permitted source IP addresses. Furthermore, a subshare may not have a more permissive action set (click) than the parent (click), and initially, the subshare’s only principal is its creator (click). Other users can later be added as additional principals (click). This process of creating subshares develops a privilege hierarchy we call the “Share Tree” (click). The root of the share tree is “the rootShare” (click) -- a share which contains all traffjc in the network, comes with all privileges, and has a single root user as the principal.

slide-28
SLIDE 28

16

  • root

root adf

bandwidth 100Mbps bandwidth 50Mbps

Share Tree

A share’s principals also have the capability to delegate privileges by creating subshares (click). The creation of subshares is guided by the principle that you can’t give away more authority than you have. For example, a subshare’s flowgroup (click) must be contained within the parent share’s flowgroup (click). Here, the blue bar represents each flowgroup’s range of permitted source IP addresses. Furthermore, a subshare may not have a more permissive action set (click) than the parent (click), and initially, the subshare’s only principal is its creator (click). Other users can later be added as additional principals (click). This process of creating subshares develops a privilege hierarchy we call the “Share Tree” (click). The root of the share tree is “the rootShare” (click) -- a share which contains all traffjc in the network, comes with all privileges, and has a single root user as the principal.

slide-29
SLIDE 29

17

  • 
  • The share tree only sets the static context for configuring the network. The actual

configuration is performed by requests and hints to the PANE controller (click). Requests describe an action the principal would like to perform on a flowgroup during a given time interval (click). After evaluating the request, the PANE controller returns an immediate response indicating an accept or reject (click). Hints provide information about current or future traffjc patterns (click). The PANE controller is not required to respond to hints and may optionally choose an action to perform on the traffjc (click). Shares may also provide principals with the right to issue queries about given flowgroups (click), such as for traffjc statistics (click). To keep things simple, I’m going to focus on requests for the remainder of this talk. More details about hints and queries can be found in our paper. (Pause)

slide-30
SLIDE 30

17

  • 
  • PANE

The share tree only sets the static context for configuring the network. The actual configuration is performed by requests and hints to the PANE controller (click). Requests describe an action the principal would like to perform on a flowgroup during a given time interval (click). After evaluating the request, the PANE controller returns an immediate response indicating an accept or reject (click). Hints provide information about current or future traffjc patterns (click). The PANE controller is not required to respond to hints and may optionally choose an action to perform on the traffjc (click). Shares may also provide principals with the right to issue queries about given flowgroups (click), such as for traffjc statistics (click). To keep things simple, I’m going to focus on requests for the remainder of this talk. More details about hints and queries can be found in our paper. (Pause)

slide-31
SLIDE 31

17

  • 
  • PANE

R e s e r v e 2 M b p s f r

  • m

n

  • w

t

  • +

5 m i n ?

The share tree only sets the static context for configuring the network. The actual configuration is performed by requests and hints to the PANE controller (click). Requests describe an action the principal would like to perform on a flowgroup during a given time interval (click). After evaluating the request, the PANE controller returns an immediate response indicating an accept or reject (click). Hints provide information about current or future traffjc patterns (click). The PANE controller is not required to respond to hints and may optionally choose an action to perform on the traffjc (click). Shares may also provide principals with the right to issue queries about given flowgroups (click), such as for traffjc statistics (click). To keep things simple, I’m going to focus on requests for the remainder of this talk. More details about hints and queries can be found in our paper. (Pause)

slide-32
SLIDE 32

17

  • 
  • PANE

Y e s

The share tree only sets the static context for configuring the network. The actual configuration is performed by requests and hints to the PANE controller (click). Requests describe an action the principal would like to perform on a flowgroup during a given time interval (click). After evaluating the request, the PANE controller returns an immediate response indicating an accept or reject (click). Hints provide information about current or future traffjc patterns (click). The PANE controller is not required to respond to hints and may optionally choose an action to perform on the traffjc (click). Shares may also provide principals with the right to issue queries about given flowgroups (click), such as for traffjc statistics (click). To keep things simple, I’m going to focus on requests for the remainder of this talk. More details about hints and queries can be found in our paper. (Pause)

slide-33
SLIDE 33

17

  • 
  • PANE

is traffic will be short and bursty

The share tree only sets the static context for configuring the network. The actual configuration is performed by requests and hints to the PANE controller (click). Requests describe an action the principal would like to perform on a flowgroup during a given time interval (click). After evaluating the request, the PANE controller returns an immediate response indicating an accept or reject (click). Hints provide information about current or future traffjc patterns (click). The PANE controller is not required to respond to hints and may optionally choose an action to perform on the traffjc (click). Shares may also provide principals with the right to issue queries about given flowgroups (click), such as for traffjc statistics (click). To keep things simple, I’m going to focus on requests for the remainder of this talk. More details about hints and queries can be found in our paper. (Pause)

slide-34
SLIDE 34

17

  • 
  • PANE

OK

The share tree only sets the static context for configuring the network. The actual configuration is performed by requests and hints to the PANE controller (click). Requests describe an action the principal would like to perform on a flowgroup during a given time interval (click). After evaluating the request, the PANE controller returns an immediate response indicating an accept or reject (click). Hints provide information about current or future traffjc patterns (click). The PANE controller is not required to respond to hints and may optionally choose an action to perform on the traffjc (click). Shares may also provide principals with the right to issue queries about given flowgroups (click), such as for traffjc statistics (click). To keep things simple, I’m going to focus on requests for the remainder of this talk. More details about hints and queries can be found in our paper. (Pause)

slide-35
SLIDE 35

17

  • 
  • PANE

How much web traffic in the last hour?

The share tree only sets the static context for configuring the network. The actual configuration is performed by requests and hints to the PANE controller (click). Requests describe an action the principal would like to perform on a flowgroup during a given time interval (click). After evaluating the request, the PANE controller returns an immediate response indicating an accept or reject (click). Hints provide information about current or future traffjc patterns (click). The PANE controller is not required to respond to hints and may optionally choose an action to perform on the traffjc (click). Shares may also provide principals with the right to issue queries about given flowgroups (click), such as for traffjc statistics (click). To keep things simple, I’m going to focus on requests for the remainder of this talk. More details about hints and queries can be found in our paper. (Pause)

slide-36
SLIDE 36

17

  • 
  • PANE

67,560 bytes

The share tree only sets the static context for configuring the network. The actual configuration is performed by requests and hints to the PANE controller (click). Requests describe an action the principal would like to perform on a flowgroup during a given time interval (click). After evaluating the request, the PANE controller returns an immediate response indicating an accept or reject (click). Hints provide information about current or future traffjc patterns (click). The PANE controller is not required to respond to hints and may optionally choose an action to perform on the traffjc (click). Shares may also provide principals with the right to issue queries about given flowgroups (click), such as for traffjc statistics (click). To keep things simple, I’m going to focus on requests for the remainder of this talk. More details about hints and queries can be found in our paper. (Pause)

slide-37
SLIDE 37

18

  • bandwidth

100Mbps bandwidth 100Mbps bandwidth 100Mbps

PANE

Current: 0 Mbps Current: 0 Mbps Current: 0 Mbps

ShareA ShareB

By design, a share’s resources may be over-subscribed by its subshares. For example, a share which is permitted up to (click) 100 Mbps of guaranteed minimum bandwidth may permit each of its subshares (click) to make reservations up to the same limit. In order to ensure that these restrictions are never violated, new requests are recursively evaluated up the tree. For example, if a user of ShareA requests (click) 80 Mbps of guaranteed bandwidth, the PANE controller accepts the request (click) and accounts for the reservation in ShareA and the rootShare. If a user of ShareB then requests (click) 50 Mbps of guaranteed bandwidth, the PANE controller rejects the request (click) to prevent a violation on the rootShare. Finally, when accepted requests become active, the PANE controller uses OpenFlow (click) to reconfigure the network and implement the request. (Pause)

slide-38
SLIDE 38

18

  • bandwidth

100Mbps bandwidth 100Mbps bandwidth 100Mbps

PANE

Current: 0 Mbps Current: 0 Mbps Current: 0 Mbps

ShareA ShareB

By design, a share’s resources may be over-subscribed by its subshares. For example, a share which is permitted up to (click) 100 Mbps of guaranteed minimum bandwidth may permit each of its subshares (click) to make reservations up to the same limit. In order to ensure that these restrictions are never violated, new requests are recursively evaluated up the tree. For example, if a user of ShareA requests (click) 80 Mbps of guaranteed bandwidth, the PANE controller accepts the request (click) and accounts for the reservation in ShareA and the rootShare. If a user of ShareB then requests (click) 50 Mbps of guaranteed bandwidth, the PANE controller rejects the request (click) to prevent a violation on the rootShare. Finally, when accepted requests become active, the PANE controller uses OpenFlow (click) to reconfigure the network and implement the request. (Pause)

slide-39
SLIDE 39

18

  • bandwidth

100Mbps bandwidth 100Mbps bandwidth 100Mbps

PANE

Current: 0 Mbps Current: 0 Mbps Current: 0 Mbps

ShareA ShareB

By design, a share’s resources may be over-subscribed by its subshares. For example, a share which is permitted up to (click) 100 Mbps of guaranteed minimum bandwidth may permit each of its subshares (click) to make reservations up to the same limit. In order to ensure that these restrictions are never violated, new requests are recursively evaluated up the tree. For example, if a user of ShareA requests (click) 80 Mbps of guaranteed bandwidth, the PANE controller accepts the request (click) and accounts for the reservation in ShareA and the rootShare. If a user of ShareB then requests (click) 50 Mbps of guaranteed bandwidth, the PANE controller rejects the request (click) to prevent a violation on the rootShare. Finally, when accepted requests become active, the PANE controller uses OpenFlow (click) to reconfigure the network and implement the request. (Pause)

slide-40
SLIDE 40

18

  • bandwidth

100Mbps bandwidth 100Mbps bandwidth 100Mbps

PANE

Current: 0 Mbps Current: 0 Mbps Current: 0 Mbps Reserve 80 Mbps?

ShareA ShareB

By design, a share’s resources may be over-subscribed by its subshares. For example, a share which is permitted up to (click) 100 Mbps of guaranteed minimum bandwidth may permit each of its subshares (click) to make reservations up to the same limit. In order to ensure that these restrictions are never violated, new requests are recursively evaluated up the tree. For example, if a user of ShareA requests (click) 80 Mbps of guaranteed bandwidth, the PANE controller accepts the request (click) and accounts for the reservation in ShareA and the rootShare. If a user of ShareB then requests (click) 50 Mbps of guaranteed bandwidth, the PANE controller rejects the request (click) to prevent a violation on the rootShare. Finally, when accepted requests become active, the PANE controller uses OpenFlow (click) to reconfigure the network and implement the request. (Pause)

slide-41
SLIDE 41

18

  • bandwidth

100Mbps bandwidth 100Mbps bandwidth 100Mbps

PANE

Current: 0 Mbps Current: 0 Mbps Current: 0 Mbps Current: 80 Mbps Yes Current: 80 Mbps

ShareA ShareB

By design, a share’s resources may be over-subscribed by its subshares. For example, a share which is permitted up to (click) 100 Mbps of guaranteed minimum bandwidth may permit each of its subshares (click) to make reservations up to the same limit. In order to ensure that these restrictions are never violated, new requests are recursively evaluated up the tree. For example, if a user of ShareA requests (click) 80 Mbps of guaranteed bandwidth, the PANE controller accepts the request (click) and accounts for the reservation in ShareA and the rootShare. If a user of ShareB then requests (click) 50 Mbps of guaranteed bandwidth, the PANE controller rejects the request (click) to prevent a violation on the rootShare. Finally, when accepted requests become active, the PANE controller uses OpenFlow (click) to reconfigure the network and implement the request. (Pause)

slide-42
SLIDE 42

18

  • bandwidth

100Mbps bandwidth 100Mbps bandwidth 100Mbps

PANE

Current: 0 Mbps Current: 0 Mbps Current: 0 Mbps Current: 80 Mbps Current: 80 Mbps Reserve 50 Mbps?

ShareA ShareB

By design, a share’s resources may be over-subscribed by its subshares. For example, a share which is permitted up to (click) 100 Mbps of guaranteed minimum bandwidth may permit each of its subshares (click) to make reservations up to the same limit. In order to ensure that these restrictions are never violated, new requests are recursively evaluated up the tree. For example, if a user of ShareA requests (click) 80 Mbps of guaranteed bandwidth, the PANE controller accepts the request (click) and accounts for the reservation in ShareA and the rootShare. If a user of ShareB then requests (click) 50 Mbps of guaranteed bandwidth, the PANE controller rejects the request (click) to prevent a violation on the rootShare. Finally, when accepted requests become active, the PANE controller uses OpenFlow (click) to reconfigure the network and implement the request. (Pause)

slide-43
SLIDE 43

18

  • bandwidth

100Mbps bandwidth 100Mbps bandwidth 100Mbps

PANE

Current: 0 Mbps Current: 0 Mbps Current: 0 Mbps Current: 80 Mbps Current: 80 Mbps No

ShareA ShareB

By design, a share’s resources may be over-subscribed by its subshares. For example, a share which is permitted up to (click) 100 Mbps of guaranteed minimum bandwidth may permit each of its subshares (click) to make reservations up to the same limit. In order to ensure that these restrictions are never violated, new requests are recursively evaluated up the tree. For example, if a user of ShareA requests (click) 80 Mbps of guaranteed bandwidth, the PANE controller accepts the request (click) and accounts for the reservation in ShareA and the rootShare. If a user of ShareB then requests (click) 50 Mbps of guaranteed bandwidth, the PANE controller rejects the request (click) to prevent a violation on the rootShare. Finally, when accepted requests become active, the PANE controller uses OpenFlow (click) to reconfigure the network and implement the request. (Pause)

slide-44
SLIDE 44

18

  • bandwidth

100Mbps bandwidth 100Mbps bandwidth 100Mbps

PANE

Current: 0 Mbps Current: 0 Mbps Current: 0 Mbps Current: 80 Mbps Current: 80 Mbps

ShareA ShareB

By design, a share’s resources may be over-subscribed by its subshares. For example, a share which is permitted up to (click) 100 Mbps of guaranteed minimum bandwidth may permit each of its subshares (click) to make reservations up to the same limit. In order to ensure that these restrictions are never violated, new requests are recursively evaluated up the tree. For example, if a user of ShareA requests (click) 80 Mbps of guaranteed bandwidth, the PANE controller accepts the request (click) and accounts for the reservation in ShareA and the rootShare. If a user of ShareB then requests (click) 50 Mbps of guaranteed bandwidth, the PANE controller rejects the request (click) to prevent a violation on the rootShare. Finally, when accepted requests become active, the PANE controller uses OpenFlow (click) to reconfigure the network and implement the request. (Pause)

slide-45
SLIDE 45

19

Resolving Conflicts

To solve participatory networking’s second challenge -- how to resolve conflicts between requests -- we developed Hierarchical Flow Tables, or HFTs.

slide-46
SLIDE 46

20

root root adf

bandwidth 100Mbps bandwidth 50Mbps

Share Tree

In PANE, we have two hierarchies. (pause) The first is a static hierarchy of the privileges granted to users and applications. This hierarchy sets the stage ...

slide-47
SLIDE 47

21

Policy Trees

… for a dynamic hierarchy of policy requests. As users and applications make requests, (click) the policy trees evolve, always within the bounds set by the Share Tree. (pause)

slide-48
SLIDE 48

21

(dstPort = 22, Deny) (dstIP=10.0.0.2, GMB=30) (dstPort=80, GMB=10) (srcIP=10.0.0.1, Allow)

Policy Trees

(srcIP=10.0.0.2, GMB=20)

… for a dynamic hierarchy of policy requests. As users and applications make requests, (click) the policy trees evolve, always within the bounds set by the Share Tree. (pause)

slide-49
SLIDE 49

22

(dstPort = 22, Deny) (dstIP=10.0.0.2, GMB=30) (dstPort=80, GMB=10) (srcIP=10.0.0.1, Allow)

P a c k e t : s r c 1 . . . 1 d s t 1 . . . 2 : 8

Policy Trees

(srcIP=10.0.0.2, GMB=20)

Following the Ethane model, we imagine every packet is processed against a global policy by the central controller. Here, packet processing is the result of evaluating each packet using the current policy tree.

slide-50
SLIDE 50

23

(dstPort = 22, Deny) (dstIP=10.0.0.2, GMB=30) (dstPort=80, GMB=10) (srcIP=10.0.0.1, Allow)

P a c k e t : s r c 1 . . . 1 d s t 1 . . . 2 : 8

Packet Evaluation

(srcIP=10.0.0.1, GMB=20)

Hierarchical Flow Tables

First, we identify the matching policy atoms, shown here in green. Next, policy atoms emit their actions. (click) When multiple subtrees have produced actions, we apply user-defined operators (click) at each node in the tree to combine the actions. Here, the sibling operator was applied. (click) Next, we combine the children’s action with the parent’s using a parent operator. Note that in this case, the parent did produce any action, which we denote by “0”, a special “don’t care” action. (continue until GMB=30 is emitted)

slide-51
SLIDE 51

23

(dstPort = 22, Deny) (dstIP=10.0.0.2, GMB=30) (dstPort=80, GMB=10) (srcIP=10.0.0.1, Allow)

P a c k e t : s r c 1 . . . 1 d s t 1 . . . 2 : 8 Allow

?

Packet Evaluation

(srcIP=10.0.0.1, GMB=20)

GMB=10

Hierarchical Flow Tables

First, we identify the matching policy atoms, shown here in green. Next, policy atoms emit their actions. (click) When multiple subtrees have produced actions, we apply user-defined operators (click) at each node in the tree to combine the actions. Here, the sibling operator was applied. (click) Next, we combine the children’s action with the parent’s using a parent operator. Note that in this case, the parent did produce any action, which we denote by “0”, a special “don’t care” action. (continue until GMB=30 is emitted)

slide-52
SLIDE 52

23

(dstPort = 22, Deny) (dstIP=10.0.0.2, GMB=30) (dstPort=80, GMB=10) (srcIP=10.0.0.1, Allow)

P a c k e t : s r c 1 . . . 1 d s t 1 . . . 2 : 8 Allow

?

+S

Packet Evaluation

(srcIP=10.0.0.1, GMB=20)

GMB=10

Hierarchical Flow Tables

First, we identify the matching policy atoms, shown here in green. Next, policy atoms emit their actions. (click) When multiple subtrees have produced actions, we apply user-defined operators (click) at each node in the tree to combine the actions. Here, the sibling operator was applied. (click) Next, we combine the children’s action with the parent’s using a parent operator. Note that in this case, the parent did produce any action, which we denote by “0”, a special “don’t care” action. (continue until GMB=30 is emitted)

slide-53
SLIDE 53

23

(dstPort = 22, Deny) (dstIP=10.0.0.2, GMB=30) (dstPort=80, GMB=10) (srcIP=10.0.0.1, Allow)

P a c k e t : s r c 1 . . . 1 d s t 1 . . . 2 : 8 Allow

?

+S +P

Packet Evaluation

(srcIP=10.0.0.1, GMB=20)

GMB=10

Hierarchical Flow Tables

First, we identify the matching policy atoms, shown here in green. Next, policy atoms emit their actions. (click) When multiple subtrees have produced actions, we apply user-defined operators (click) at each node in the tree to combine the actions. Here, the sibling operator was applied. (click) Next, we combine the children’s action with the parent’s using a parent operator. Note that in this case, the parent did produce any action, which we denote by “0”, a special “don’t care” action. (continue until GMB=30 is emitted)

slide-54
SLIDE 54

23

(dstPort = 22, Deny) (dstIP=10.0.0.2, GMB=30) (dstPort=80, GMB=10) (srcIP=10.0.0.1, Allow)

P a c k e t : s r c 1 . . . 1 d s t 1 . . . 2 : 8 Allow

?

+S

GMB=10

+P

Packet Evaluation

(srcIP=10.0.0.1, GMB=20)

GMB=10

Hierarchical Flow Tables

First, we identify the matching policy atoms, shown here in green. Next, policy atoms emit their actions. (click) When multiple subtrees have produced actions, we apply user-defined operators (click) at each node in the tree to combine the actions. Here, the sibling operator was applied. (click) Next, we combine the children’s action with the parent’s using a parent operator. Note that in this case, the parent did produce any action, which we denote by “0”, a special “don’t care” action. (continue until GMB=30 is emitted)

slide-55
SLIDE 55

23

(dstPort = 22, Deny) (dstIP=10.0.0.2, GMB=30) (dstPort=80, GMB=10) (srcIP=10.0.0.1, Allow)

P a c k e t : s r c 1 . . . 1 d s t 1 . . . 2 : 8 Allow

?

+S

GMB=10 GMB=30

+P

Packet Evaluation

(srcIP=10.0.0.1, GMB=20)

GMB=10

+D

Hierarchical Flow Tables

First, we identify the matching policy atoms, shown here in green. Next, policy atoms emit their actions. (click) When multiple subtrees have produced actions, we apply user-defined operators (click) at each node in the tree to combine the actions. Here, the sibling operator was applied. (click) Next, we combine the children’s action with the parent’s using a parent operator. Note that in this case, the parent did produce any action, which we denote by “0”, a special “don’t care” action. (continue until GMB=30 is emitted)

slide-56
SLIDE 56

23

(dstPort = 22, Deny) (dstIP=10.0.0.2, GMB=30) (dstPort=80, GMB=10) (srcIP=10.0.0.1, Allow)

P a c k e t : s r c 1 . . . 1 d s t 1 . . . 2 : 8 Allow

?

+S

GMB=10 GMB=30

+P

GMB=30

Packet Evaluation

(srcIP=10.0.0.1, GMB=20)

GMB=10

+D

Hierarchical Flow Tables

First, we identify the matching policy atoms, shown here in green. Next, policy atoms emit their actions. (click) When multiple subtrees have produced actions, we apply user-defined operators (click) at each node in the tree to combine the actions. Here, the sibling operator was applied. (click) Next, we combine the children’s action with the parent’s using a parent operator. Note that in this case, the parent did produce any action, which we denote by “0”, a special “don’t care” action. (continue until GMB=30 is emitted)

slide-57
SLIDE 57

24

GMB=10 GMB=30 GMB=30

Conflict Resolution

GMB=10

(dstPort=80, GMB=10)

Allow

(srcIP=10.0.0.1, Allow) (srcIP=10.0.0.1, GMB=20) (dstIP=10.0.0.2, GMB=30)

+P

+D

+S

Hierarchical Flow Tables

Participatory networking uses three combination operators within each node to resolve conflicts. The first is the +S operator, which combines sibling actions. The second is the +D operator, which combines multiple actions inside a single node. Finally, the +P operator combines the previously resolved actions of a parent and child. (Pause) The requirements on these operators are very basic: (click) first, they must be associative -- this allows us to resolve conflicts in a pairwise

  • fashion. And second, they must support the 0 or “don’t care” action as their identity value.

With these minimal requirements, we can convert the HFT into an effjcient implementation.

slide-58
SLIDE 58

24

GMB=10 GMB=30 GMB=30

Conflict Resolution

Only Requirements: Associative, 0-identity GMB=10

(dstPort=80, GMB=10)

Allow

(srcIP=10.0.0.1, Allow) (srcIP=10.0.0.1, GMB=20) (dstIP=10.0.0.2, GMB=30)

+P

+D

+S

Hierarchical Flow Tables

Participatory networking uses three combination operators within each node to resolve conflicts. The first is the +S operator, which combines sibling actions. The second is the +D operator, which combines multiple actions inside a single node. Finally, the +P operator combines the previously resolved actions of a parent and child. (Pause) The requirements on these operators are very basic: (click) first, they must be associative -- this allows us to resolve conflicts in a pairwise

  • fashion. And second, they must support the 0 or “don’t care” action as their identity value.

With these minimal requirements, we can convert the HFT into an effjcient implementation.

slide-59
SLIDE 59

25

+D

+P +S Sibling

Parent-Sibling In node D and S identical. Deny overrides Allow. GMB combines as max Rate-limit combines as min Child overrides Parent for Access Control GMB combines as max Rate-limit combines as min

PANE’s Conflict Resolution Operators

The conflict resolution operators’ flexibility creates a design space for our system. This slide is a summary of the choices we made for PANE. When sensible, we strive to combine both requests. For example a request to guarantee a minimum bandwidth, can combine with one that limits below a maximum rate. In other cases, we need a single outcome. PANE’s +D and +S operators implement a basic policy in which Deny requests override Allow requests, take the maximum of two bandwidth guarantees, and the minimum of two rate-limits. With the +P operator, PANE allows access control requests in child shares to override those in parent shares. The HFT itself is agnostic to the specific policies of the operators, as long as they satisfy the identity and associativity requirements. For example, we could develop operators that resolve conflicts according to priority. (Pause)

slide-60
SLIDE 60

26

Implementation

So, how do we implement this system? (pause) In an ideal world, we could simply pass each new HFT to the switches...

slide-61
SLIDE 61

27

(d (d (d (s (d (d (d (s (d (d (d (s (d (d (d (s (d (d (d (s

PANE

… and when packets arrive, the switches would evaluate the tree just as we did on the previous slides. However, today’s switches aren’t capable performing this evaluation. Therefore, rather than send every packet to the controller …

slide-62
SLIDE 62

28

(dstPort = 22, Deny) (dstIP=10.0.0.2, GMB=30) (dstPort=80, GMB=10) (srcIP=10.0.0.1, Allow) (srcIP=10.0.0.2, GMB=20)

… we developed a compiler which linearizes (click) an HFT instance into traditional, flat OpenFlow tables that are collectively equivalent to the logical policy tree. This compilation process is quadratic in the size of the tree, as we explain in our paper.

slide-63
SLIDE 63

28

(dstPort = 22, Deny) (dstIP=10.0.0.2, GMB=30) (dstPort=80, GMB=10) (srcIP=10.0.0.1, Allow) (srcIP=10.0.0.2, GMB=20)

… we developed a compiler which linearizes (click) an HFT instance into traditional, flat OpenFlow tables that are collectively equivalent to the logical policy tree. This compilation process is quadratic in the size of the tree, as we explain in our paper.

slide-64
SLIDE 64

29

PANE

Our OpenFlow controller then installs these tables on the switches (click), allowing the network to implement the HFT with hardware support.

slide-65
SLIDE 65

29

PANE

Our OpenFlow controller then installs these tables on the switches (click), allowing the network to implement the HFT with hardware support.

slide-66
SLIDE 66

30

PANE

Our compiler works in two stages. (click) First, the compiler linearizes the HFT into a single table we call the "network flow table." If the network were connected by a single, big switch, we might install this network flow table directly onto that switch

slide-67
SLIDE 67

30

PANE

Our compiler works in two stages. (click) First, the compiler linearizes the HFT into a single table we call the "network flow table." If the network were connected by a single, big switch, we might install this network flow table directly onto that switch

slide-68
SLIDE 68

31

PANE

In the compiler's second stage, it translates the network flow table into individual flow tables for the distributed OpenFlow switches. During this stage, (click) the compiler relies on a Network Information Base or NIB. The design of our NIB is inspired by Onix, and it describes the state of the network, including host locations, link statuses, queue availability, switch configurations, and more.

slide-69
SLIDE 69

31

PANE

In the compiler's second stage, it translates the network flow table into individual flow tables for the distributed OpenFlow switches. During this stage, (click) the compiler relies on a Network Information Base or NIB. The design of our NIB is inspired by Onix, and it describes the state of the network, including host locations, link statuses, queue availability, switch configurations, and more.

slide-70
SLIDE 70

32

PANE

For example, the compiler uses the NIB to implement a bandwidth reservation by finding a circuit with the requested bandwidth, (click) sending commands to create the necessary queues, (click) and finally, updating the OpenFlow tables (click) with the required forwarding decisions.

slide-71
SLIDE 71

32

PANE

For example, the compiler uses the NIB to implement a bandwidth reservation by finding a circuit with the requested bandwidth, (click) sending commands to create the necessary queues, (click) and finally, updating the OpenFlow tables (click) with the required forwarding decisions.

slide-72
SLIDE 72

32

PANE

For example, the compiler uses the NIB to implement a bandwidth reservation by finding a circuit with the requested bandwidth, (click) sending commands to create the necessary queues, (click) and finally, updating the OpenFlow tables (click) with the required forwarding decisions.

slide-73
SLIDE 73

32

PANE

For example, the compiler uses the NIB to implement a bandwidth reservation by finding a circuit with the requested bandwidth, (click) sending commands to create the necessary queues, (click) and finally, updating the OpenFlow tables (click) with the required forwarding decisions.

slide-74
SLIDE 74

33

PANE

The NIB also allows PANE’s compiler to choose where in the network to implement desired policies. As a simple example, it places rules which drop traffic as close as possible to the traffic's ingress port. In this experiment, we have two wireless clients communicating. One suffers from an attack, (click) and the transfer rate drops. With a local firewall rule, (click) the transfer only slightly recovers. Using PANE to install the rule, (click) the transfer fully recovers. And as the source of the traffic moves ...

slide-75
SLIDE 75

33

24Mbps

PANE

The NIB also allows PANE’s compiler to choose where in the network to implement desired policies. As a simple example, it places rules which drop traffic as close as possible to the traffic's ingress port. In this experiment, we have two wireless clients communicating. One suffers from an attack, (click) and the transfer rate drops. With a local firewall rule, (click) the transfer only slightly recovers. Using PANE to install the rule, (click) the transfer fully recovers. And as the source of the traffic moves ...

slide-76
SLIDE 76

33

5Mbps

PANE

The NIB also allows PANE’s compiler to choose where in the network to implement desired policies. As a simple example, it places rules which drop traffic as close as possible to the traffic's ingress port. In this experiment, we have two wireless clients communicating. One suffers from an attack, (click) and the transfer rate drops. With a local firewall rule, (click) the transfer only slightly recovers. Using PANE to install the rule, (click) the transfer fully recovers. And as the source of the traffic moves ...

slide-77
SLIDE 77

33

8Mbps

PANE

The NIB also allows PANE’s compiler to choose where in the network to implement desired policies. As a simple example, it places rules which drop traffic as close as possible to the traffic's ingress port. In this experiment, we have two wireless clients communicating. One suffers from an attack, (click) and the transfer rate drops. With a local firewall rule, (click) the transfer only slightly recovers. Using PANE to install the rule, (click) the transfer fully recovers. And as the source of the traffic moves ...

slide-78
SLIDE 78

33

24Mbps

PANE

The NIB also allows PANE’s compiler to choose where in the network to implement desired policies. As a simple example, it places rules which drop traffic as close as possible to the traffic's ingress port. In this experiment, we have two wireless clients communicating. One suffers from an attack, (click) and the transfer rate drops. With a local firewall rule, (click) the transfer only slightly recovers. Using PANE to install the rule, (click) the transfer fully recovers. And as the source of the traffic moves ...

slide-79
SLIDE 79

34

24Mbps

PANE

… the rule can shift with it. (Pause)

slide-80
SLIDE 80

35

PANE

Updating the NIB is the responsibility of our OpenFlow controller, (click) and any updates are propagated back into our compiler service. (click) The compiler may then construct a new set of OpenFlow tables (click) which continue to implement the decisions of the *network* flow table in the new environment. Seen this way, you can think of the network flow table as a set of invariants we would like to maintain, and our compiler's second stage as a service which maintains those invariants. (Pause)

slide-81
SLIDE 81

35

PANE

Updating the NIB is the responsibility of our OpenFlow controller, (click) and any updates are propagated back into our compiler service. (click) The compiler may then construct a new set of OpenFlow tables (click) which continue to implement the decisions of the *network* flow table in the new environment. Seen this way, you can think of the network flow table as a set of invariants we would like to maintain, and our compiler's second stage as a service which maintains those invariants. (Pause)

slide-82
SLIDE 82

35

PANE

Updating the NIB is the responsibility of our OpenFlow controller, (click) and any updates are propagated back into our compiler service. (click) The compiler may then construct a new set of OpenFlow tables (click) which continue to implement the decisions of the *network* flow table in the new environment. Seen this way, you can think of the network flow table as a set of invariants we would like to maintain, and our compiler's second stage as a service which maintains those invariants. (Pause)

slide-83
SLIDE 83

35

PANE

Updating the NIB is the responsibility of our OpenFlow controller, (click) and any updates are propagated back into our compiler service. (click) The compiler may then construct a new set of OpenFlow tables (click) which continue to implement the decisions of the *network* flow table in the new environment. Seen this way, you can think of the network flow table as a set of invariants we would like to maintain, and our compiler's second stage as a service which maintains those invariants. (Pause)

slide-84
SLIDE 84

36

Evaluation

(pause) We have been running several prototype PANE-controlled networks ...

slide-85
SLIDE 85

37

… which carry traffic in our labs on several hardware and software switches. It provides our day-to- day development and internet connectivity.

slide-86
SLIDE 86

38

Evaluation

  • 2. Ekiga
  • 3. ZooKeeper
  • 4. Hadoop
  • 1. SSHGuard

access control bandwidth reservations queues for low latency centralized traffic weights

We also adapted each of the four applications I discussed earlier to use PANE. SSHGuard and Ekiga directly use our simple ASCII protocol, while ZooKeeper and Hadoop use an

  • bject-oriented Java library we developed.
slide-87
SLIDE 87

39

Three equal-sized sort jobs:

  • Two Low Priority with 25% weight
  • One High Priority with 50% weight

I want to briefly take a look at the Hadoop case: 1) (job mix) 2) (network topology: 20 slaves plus 2 masters) 3) (PANE rules) 4) (outcome: high pri 23% faster, lowpri 10% because of work-conservation)

slide-88
SLIDE 88

39

Three equal-sized sort jobs:

  • Two Low Priority with 25% weight
  • One High Priority with 50% weight

PANE

22

Hosts

I want to briefly take a look at the Hadoop case: 1) (job mix) 2) (network topology: 20 slaves plus 2 masters) 3) (PANE rules) 4) (outcome: high pri 23% faster, lowpri 10% because of work-conservation)

slide-89
SLIDE 89

39

Three equal-sized sort jobs:

  • Two Low Priority with 25% weight
  • One High Priority with 50% weight

Dynamically apply QoS to High Priority flows using PANE.

PANE

22

Hosts

I want to briefly take a look at the Hadoop case: 1) (job mix) 2) (network topology: 20 slaves plus 2 masters) 3) (PANE rules) 4) (outcome: high pri 23% faster, lowpri 10% because of work-conservation)

slide-90
SLIDE 90

39

Three equal-sized sort jobs:

  • Two Low Priority with 25% weight
  • One High Priority with 50% weight

0.25 0.5 0.75 1 1.25 HighPri Speedup

Default With PANE

Dynamically apply QoS to High Priority flows using PANE.

PANE

22

Hosts

I want to briefly take a look at the Hadoop case: 1) (job mix) 2) (network topology: 20 slaves plus 2 masters) 3) (PANE rules) 4) (outcome: high pri 23% faster, lowpri 10% because of work-conservation)

slide-91
SLIDE 91

40

Hadoop’s OpenFlow rules

x-axis: time y-axis: number of rules created by one job running across 22 hosts

slide-92
SLIDE 92

40

Hadoop’s OpenFlow rules

5 10 15 20 25 30 Time(min)

x-axis: time y-axis: number of rules created by one job running across 22 hosts

slide-93
SLIDE 93

40

Hadoop’s OpenFlow rules

5 10 15 20 25 30 5 10 15 20 25 30 Number of Resident Rules Time(min)

x-axis: time y-axis: number of rules created by one job running across 22 hosts

slide-94
SLIDE 94

40

Hadoop’s OpenFlow rules

5 10 15 20 25 30 5 10 15 20 25 30 Number of Resident Rules Time(min)

PANE

22

Hosts

x-axis: time y-axis: number of rules created by one job running across 22 hosts

slide-95
SLIDE 95

41

Conclusion

  • 1. For applications that know what they

want from the network

  • 2. Allows these applications to co-exist

In conclusion, PANE is designed for applications and users that know what they want from the network. PANE provides a way for applications to talk back to the control-plane and use any mechanisms exposed by network. So far we’ve explored bandwidth, access control, routing, and rate-limiting, and hope to support new mechanisms in the future. And second, PANE allows all of these application requests to co-exist with a single network by deterministically resolving conflicting requests into a single policy.

slide-96
SLIDE 96

42

Andrew Ferguson adf@cs.brown.edu

pane.cs.brown.edu

I’m happy to take your questions at this time…

slide-97
SLIDE 97

43

Andrew Ferguson adf@cs.brown.edu

  • Arjun Guha
  • Chen Liang
  • Rodrigo Fonseca
  • Shriram Krishnamurthi

Co-authors

pane.cs.brown.edu

Brown ↦ Cornell ↦ UMass Amherst Brown ↦ Duke Brown Brown

… or you can contact any of my collaborators as well. Thank you very much!

slide-98
SLIDE 98

Backup Slides

44

slide-99
SLIDE 99

45

Proof of Correctness

  • As we saw on the last slide, this is a complex, concurrent system.
  • And complex systems have bugs, even if you write them in Haskell, as we did.
  • I’d like to briefly tell you how we proved a key portion of the system correct.
slide-100
SLIDE 100

46

P a c k e t : s r c 1 . . . 1 d s t 1 . . . 2 : 8

(dstPort = 22, Deny) (dstIP=10.0.0.2, GMB=30) (dstPort=80, GMB=10) (srcIP=10.0.0.1, Allow)

Allow GMB=10

+S

GMB=10 GMB=30

+P

GMB=30

Hierarchical Flow Tables

  • As a starting point, we know what it means for a hierarchical flow table to process a packet:

the packet enters the switch, the policy tree nodes produce their actions, and a result action is produced after applying the combination operators.

slide-101
SLIDE 101

Compiler Correctness

47

(dstPort = 22, Deny) (dstIP=10.0.0.2, GMB=30) (dstPort=80, GMB=10) (srcIP=10.0.0.1, Allow)

Allow GMB=10

+S

GMB=10 GMB=30

+P

GMB=30

  • To make this effjcient, we’ve built a compiler

(click) from declarative, hierarchical policies to linear, flow tables.

  • How do we know that this compiler is actually correct?
  • Compilers are also notorious diffjcult to get right.
  • If this compiler has a bug,

(click) it’s not that a program may crash, but the entire network may go down.

  • Or, a more subtle error may occur, such as traffjc that should be blocked, may instead be

permitted.

slide-102
SLIDE 102

Compiler Correctness

47

(dstPort = 22, Deny) (dstIP=10.0.0.2, GMB=30) (dstPort=80, GMB=10) (srcIP=10.0.0.1, Allow)

Allow GMB=10

+S

GMB=10 GMB=30

+P

GMB=30

  • To make this effjcient, we’ve built a compiler

(click) from declarative, hierarchical policies to linear, flow tables.

  • How do we know that this compiler is actually correct?
  • Compilers are also notorious diffjcult to get right.
  • If this compiler has a bug,

(click) it’s not that a program may crash, but the entire network may go down.

  • Or, a more subtle error may occur, such as traffjc that should be blocked, may instead be

permitted.

slide-103
SLIDE 103

Compiler Correctness

47

(dstPort = 22, Deny) (dstIP=10.0.0.2, GMB=30) (dstPort=80, GMB=10) (srcIP=10.0.0.1, Allow)

Allow GMB=10

+S

GMB=10 GMB=30

+P

GMB=30

  • To make this effjcient, we’ve built a compiler

(click) from declarative, hierarchical policies to linear, flow tables.

  • How do we know that this compiler is actually correct?
  • Compilers are also notorious diffjcult to get right.
  • If this compiler has a bug,

(click) it’s not that a program may crash, but the entire network may go down.

  • Or, a more subtle error may occur, such as traffjc that should be blocked, may instead be

permitted.

slide-104
SLIDE 104

Coq Proof Assistant

48

(dstPort = 22, Deny) (dstIP=10.0.0.2, GMB=30) (dstPort=80, GMB=10) (srcIP=10.0.0.1, Allow)

Allow GMB=10

+S

GMB=10 GMB=30

+P

GMB=30

  • Using the Coq proof assistant, we modeled and wrote a proof of the translation from HFT to

OpenFlow tables

  • Coq lets us write programs in a functional language, similar to ML or Haskell, and gives us

the ability to prove properties of these programs.

slide-105
SLIDE 105

49

(dstPort = 22, Deny) (dstIP=10.0.0.2, GMB=30) (dstPort=80, GMB=10) (srcIP=10.0.0.1, Allow)

Allow GMB=10

+S

GMB=10 GMB=30

+P

GMB=30

Packet: src 10.0.0.1 dst 10.0.0.2:80

Theorem

  • So, let’s look at what we actually proved. Logically, the HFT processes a packet

(click) and produces an action (click) When we compile the HFT to a network flow table, (click) the flow table produces exactly the same action (click) on the same packet. Proving this theorem requires a formal semantics for Hierarchical Flow Tables, which you can find in detail in our paper. The paper also contains the precise statement of this theorem, and the mechanized Coq proofs are available on our website.

slide-106
SLIDE 106

49

(dstPort = 22, Deny) (dstIP=10.0.0.2, GMB=30) (dstPort=80, GMB=10) (srcIP=10.0.0.1, Allow)

Allow GMB=10

+S

GMB=10 GMB=30

+P

GMB=30

Packet: src 10.0.0.1 dst 10.0.0.2:80

Theorem

  • So, let’s look at what we actually proved. Logically, the HFT processes a packet

(click) and produces an action (click) When we compile the HFT to a network flow table, (click) the flow table produces exactly the same action (click) on the same packet. Proving this theorem requires a formal semantics for Hierarchical Flow Tables, which you can find in detail in our paper. The paper also contains the precise statement of this theorem, and the mechanized Coq proofs are available on our website.

slide-107
SLIDE 107

49

(dstPort = 22, Deny) (dstIP=10.0.0.2, GMB=30) (dstPort=80, GMB=10) (srcIP=10.0.0.1, Allow)

Allow GMB=10

+S

GMB=10 GMB=30

+P

GMB=30

Packet: src 10.0.0.1 dst 10.0.0.2:80

GMB 30

Theorem

  • So, let’s look at what we actually proved. Logically, the HFT processes a packet

(click) and produces an action (click) When we compile the HFT to a network flow table, (click) the flow table produces exactly the same action (click) on the same packet. Proving this theorem requires a formal semantics for Hierarchical Flow Tables, which you can find in detail in our paper. The paper also contains the precise statement of this theorem, and the mechanized Coq proofs are available on our website.

slide-108
SLIDE 108

49

(dstPort = 22, Deny) (dstIP=10.0.0.2, GMB=30) (dstPort=80, GMB=10) (srcIP=10.0.0.1, Allow)

Allow GMB=10

+S

GMB=10 GMB=30

+P

GMB=30

Packet: src 10.0.0.1 dst 10.0.0.2:80

GMB 30 compile

Theorem

  • So, let’s look at what we actually proved. Logically, the HFT processes a packet

(click) and produces an action (click) When we compile the HFT to a network flow table, (click) the flow table produces exactly the same action (click) on the same packet. Proving this theorem requires a formal semantics for Hierarchical Flow Tables, which you can find in detail in our paper. The paper also contains the precise statement of this theorem, and the mechanized Coq proofs are available on our website.

slide-109
SLIDE 109

49

(dstPort = 22, Deny) (dstIP=10.0.0.2, GMB=30) (dstPort=80, GMB=10) (srcIP=10.0.0.1, Allow)

Allow GMB=10

+S

GMB=10 GMB=30

+P

GMB=30

Packet: src 10.0.0.1 dst 10.0.0.2:80

GMB 30 compile

Theorem

  • So, let’s look at what we actually proved. Logically, the HFT processes a packet

(click) and produces an action (click) When we compile the HFT to a network flow table, (click) the flow table produces exactly the same action (click) on the same packet. Proving this theorem requires a formal semantics for Hierarchical Flow Tables, which you can find in detail in our paper. The paper also contains the precise statement of this theorem, and the mechanized Coq proofs are available on our website.

slide-110
SLIDE 110

50

Protocol

Now that we’ve explored PANE’s semantics, we’ll take a brief look at its protocol for interactively using and delegating network resources.

slide-111
SLIDE 111

51

PANE

As I described earlier, the privileges in PANE derive from the root user’s (click) access to the share tree. To allow a regular user, Alice, (click) to reserve bandwidth, Root first creates a subshare with an appropriate flowgroup and privilege (click). In this example, the subshare is for all traffjc sent or received by Alice, with the authority to reserve up to 10 Mbps of guaranteed minimum bandwidth. After checking that Root has the necessary authority to create this share, the PANE controller accepts the request (click). But Alice is not yet a principal in this share. Root must explicitly grant Alice the privilege to use the share (click). As the root user is a principal on this new share, the PANE controller accepts the command to add Alice as well (click). Alice now tries to make a reservation using this share (click). She requests 5 Mbps of guaranteed minimum bandwidth for the next 10 minutes. Her message explicitly indicates which share she is using to make the request (click). The PANE controller first checks that the FlowGroup on the request (click) is a subset of the FlowGroup on the specified share (click) and that Alice is an authorized principal. As both

slide-112
SLIDE 112

51

PANE

Root

As I described earlier, the privileges in PANE derive from the root user’s (click) access to the share tree. To allow a regular user, Alice, (click) to reserve bandwidth, Root first creates a subshare with an appropriate flowgroup and privilege (click). In this example, the subshare is for all traffjc sent or received by Alice, with the authority to reserve up to 10 Mbps of guaranteed minimum bandwidth. After checking that Root has the necessary authority to create this share, the PANE controller accepts the request (click). But Alice is not yet a principal in this share. Root must explicitly grant Alice the privilege to use the share (click). As the root user is a principal on this new share, the PANE controller accepts the command to add Alice as well (click). Alice now tries to make a reservation using this share (click). She requests 5 Mbps of guaranteed minimum bandwidth for the next 10 minutes. Her message explicitly indicates which share she is using to make the request (click). The PANE controller first checks that the FlowGroup on the request (click) is a subset of the FlowGroup on the specified share (click) and that Alice is an authorized principal. As both

slide-113
SLIDE 113

51

PANE

Root Alice

As I described earlier, the privileges in PANE derive from the root user’s (click) access to the share tree. To allow a regular user, Alice, (click) to reserve bandwidth, Root first creates a subshare with an appropriate flowgroup and privilege (click). In this example, the subshare is for all traffjc sent or received by Alice, with the authority to reserve up to 10 Mbps of guaranteed minimum bandwidth. After checking that Root has the necessary authority to create this share, the PANE controller accepts the request (click). But Alice is not yet a principal in this share. Root must explicitly grant Alice the privilege to use the share (click). As the root user is a principal on this new share, the PANE controller accepts the command to add Alice as well (click). Alice now tries to make a reservation using this share (click). She requests 5 Mbps of guaranteed minimum bandwidth for the next 10 minutes. Her message explicitly indicates which share she is using to make the request (click). The PANE controller first checks that the FlowGroup on the request (click) is a subset of the FlowGroup on the specified share (click) and that Alice is an authorized principal. As both

slide-114
SLIDE 114

51

NewShare aBW for (user=Alice) [reserve <= 10Mb]

  • n rootShare.

PANE

Root Alice

As I described earlier, the privileges in PANE derive from the root user’s (click) access to the share tree. To allow a regular user, Alice, (click) to reserve bandwidth, Root first creates a subshare with an appropriate flowgroup and privilege (click). In this example, the subshare is for all traffjc sent or received by Alice, with the authority to reserve up to 10 Mbps of guaranteed minimum bandwidth. After checking that Root has the necessary authority to create this share, the PANE controller accepts the request (click). But Alice is not yet a principal in this share. Root must explicitly grant Alice the privilege to use the share (click). As the root user is a principal on this new share, the PANE controller accepts the command to add Alice as well (click). Alice now tries to make a reservation using this share (click). She requests 5 Mbps of guaranteed minimum bandwidth for the next 10 minutes. Her message explicitly indicates which share she is using to make the request (click). The PANE controller first checks that the FlowGroup on the request (click) is a subset of the FlowGroup on the specified share (click) and that Alice is an authorized principal. As both

slide-115
SLIDE 115

51

NewShare aBW for (user=Alice) [reserve <= 10Mb]

  • n rootShare.

PANE

OK Root Alice

As I described earlier, the privileges in PANE derive from the root user’s (click) access to the share tree. To allow a regular user, Alice, (click) to reserve bandwidth, Root first creates a subshare with an appropriate flowgroup and privilege (click). In this example, the subshare is for all traffjc sent or received by Alice, with the authority to reserve up to 10 Mbps of guaranteed minimum bandwidth. After checking that Root has the necessary authority to create this share, the PANE controller accepts the request (click). But Alice is not yet a principal in this share. Root must explicitly grant Alice the privilege to use the share (click). As the root user is a principal on this new share, the PANE controller accepts the command to add Alice as well (click). Alice now tries to make a reservation using this share (click). She requests 5 Mbps of guaranteed minimum bandwidth for the next 10 minutes. Her message explicitly indicates which share she is using to make the request (click). The PANE controller first checks that the FlowGroup on the request (click) is a subset of the FlowGroup on the specified share (click) and that Alice is an authorized principal. As both

slide-116
SLIDE 116

51

NewShare aBW for (user=Alice) [reserve <= 10Mb]

  • n rootShare.

PANE

OK

Grant aBW to Alice.

Root Alice

As I described earlier, the privileges in PANE derive from the root user’s (click) access to the share tree. To allow a regular user, Alice, (click) to reserve bandwidth, Root first creates a subshare with an appropriate flowgroup and privilege (click). In this example, the subshare is for all traffjc sent or received by Alice, with the authority to reserve up to 10 Mbps of guaranteed minimum bandwidth. After checking that Root has the necessary authority to create this share, the PANE controller accepts the request (click). But Alice is not yet a principal in this share. Root must explicitly grant Alice the privilege to use the share (click). As the root user is a principal on this new share, the PANE controller accepts the command to add Alice as well (click). Alice now tries to make a reservation using this share (click). She requests 5 Mbps of guaranteed minimum bandwidth for the next 10 minutes. Her message explicitly indicates which share she is using to make the request (click). The PANE controller first checks that the FlowGroup on the request (click) is a subset of the FlowGroup on the specified share (click) and that Alice is an authorized principal. As both

slide-117
SLIDE 117

51

NewShare aBW for (user=Alice) [reserve <= 10Mb]

  • n rootShare.

PANE

OK

Grant aBW to Alice.

OK Root Alice

As I described earlier, the privileges in PANE derive from the root user’s (click) access to the share tree. To allow a regular user, Alice, (click) to reserve bandwidth, Root first creates a subshare with an appropriate flowgroup and privilege (click). In this example, the subshare is for all traffjc sent or received by Alice, with the authority to reserve up to 10 Mbps of guaranteed minimum bandwidth. After checking that Root has the necessary authority to create this share, the PANE controller accepts the request (click). But Alice is not yet a principal in this share. Root must explicitly grant Alice the privilege to use the share (click). As the root user is a principal on this new share, the PANE controller accepts the command to add Alice as well (click). Alice now tries to make a reservation using this share (click). She requests 5 Mbps of guaranteed minimum bandwidth for the next 10 minutes. Her message explicitly indicates which share she is using to make the request (click). The PANE controller first checks that the FlowGroup on the request (click) is a subset of the FlowGroup on the specified share (click) and that Alice is an authorized principal. As both

slide-118
SLIDE 118

51

NewShare aBW for (user=Alice) [reserve <= 10Mb]

  • n rootShare.

PANE

OK

Grant aBW to Alice.

OK

reserve(user=Alice, dstPort=80) = 5Mb on aBW from now to +10min.

Root Alice

As I described earlier, the privileges in PANE derive from the root user’s (click) access to the share tree. To allow a regular user, Alice, (click) to reserve bandwidth, Root first creates a subshare with an appropriate flowgroup and privilege (click). In this example, the subshare is for all traffjc sent or received by Alice, with the authority to reserve up to 10 Mbps of guaranteed minimum bandwidth. After checking that Root has the necessary authority to create this share, the PANE controller accepts the request (click). But Alice is not yet a principal in this share. Root must explicitly grant Alice the privilege to use the share (click). As the root user is a principal on this new share, the PANE controller accepts the command to add Alice as well (click). Alice now tries to make a reservation using this share (click). She requests 5 Mbps of guaranteed minimum bandwidth for the next 10 minutes. Her message explicitly indicates which share she is using to make the request (click). The PANE controller first checks that the FlowGroup on the request (click) is a subset of the FlowGroup on the specified share (click) and that Alice is an authorized principal. As both

slide-119
SLIDE 119

51

NewShare aBW for (user=Alice) [reserve <= 10Mb]

  • n rootShare.

PANE

OK

Grant aBW to Alice.

OK

reserve(user=Alice, dstPort=80) = 5Mb on aBW from now to +10min. reserve(user=Alice, dstPort=80) = 5Mb on aBW from now to +10min.

Root Alice

As I described earlier, the privileges in PANE derive from the root user’s (click) access to the share tree. To allow a regular user, Alice, (click) to reserve bandwidth, Root first creates a subshare with an appropriate flowgroup and privilege (click). In this example, the subshare is for all traffjc sent or received by Alice, with the authority to reserve up to 10 Mbps of guaranteed minimum bandwidth. After checking that Root has the necessary authority to create this share, the PANE controller accepts the request (click). But Alice is not yet a principal in this share. Root must explicitly grant Alice the privilege to use the share (click). As the root user is a principal on this new share, the PANE controller accepts the command to add Alice as well (click). Alice now tries to make a reservation using this share (click). She requests 5 Mbps of guaranteed minimum bandwidth for the next 10 minutes. Her message explicitly indicates which share she is using to make the request (click). The PANE controller first checks that the FlowGroup on the request (click) is a subset of the FlowGroup on the specified share (click) and that Alice is an authorized principal. As both

slide-120
SLIDE 120

51

NewShare aBW for (user=Alice) [reserve <= 10Mb]

  • n rootShare.

PANE

OK

Grant aBW to Alice.

OK

reserve(user=Alice, dstPort=80) = 5Mb on aBW from now to +10min. reserve(user=Alice, dstPort=80) = 5Mb on aBW from now to +10min.

Root Alice

As I described earlier, the privileges in PANE derive from the root user’s (click) access to the share tree. To allow a regular user, Alice, (click) to reserve bandwidth, Root first creates a subshare with an appropriate flowgroup and privilege (click). In this example, the subshare is for all traffjc sent or received by Alice, with the authority to reserve up to 10 Mbps of guaranteed minimum bandwidth. After checking that Root has the necessary authority to create this share, the PANE controller accepts the request (click). But Alice is not yet a principal in this share. Root must explicitly grant Alice the privilege to use the share (click). As the root user is a principal on this new share, the PANE controller accepts the command to add Alice as well (click). Alice now tries to make a reservation using this share (click). She requests 5 Mbps of guaranteed minimum bandwidth for the next 10 minutes. Her message explicitly indicates which share she is using to make the request (click). The PANE controller first checks that the FlowGroup on the request (click) is a subset of the FlowGroup on the specified share (click) and that Alice is an authorized principal. As both

slide-121
SLIDE 121

51

PANE

OK

Grant aBW to Alice.

OK

reserve(user=Alice, dstPort=80) = 5Mb on aBW from now to +10min. reserve(user=Alice, dstPort=80) = 5Mb on aBW from now to +10min. NewShare aBW for (user=Alice) [reserve <= 10Mb]

  • n rootShare.

Root Alice

As I described earlier, the privileges in PANE derive from the root user’s (click) access to the share tree. To allow a regular user, Alice, (click) to reserve bandwidth, Root first creates a subshare with an appropriate flowgroup and privilege (click). In this example, the subshare is for all traffjc sent or received by Alice, with the authority to reserve up to 10 Mbps of guaranteed minimum bandwidth. After checking that Root has the necessary authority to create this share, the PANE controller accepts the request (click). But Alice is not yet a principal in this share. Root must explicitly grant Alice the privilege to use the share (click). As the root user is a principal on this new share, the PANE controller accepts the command to add Alice as well (click). Alice now tries to make a reservation using this share (click). She requests 5 Mbps of guaranteed minimum bandwidth for the next 10 minutes. Her message explicitly indicates which share she is using to make the request (click). The PANE controller first checks that the FlowGroup on the request (click) is a subset of the FlowGroup on the specified share (click) and that Alice is an authorized principal. As both

slide-122
SLIDE 122

52

PANE

reserve(user=Alice, dstPort=80) = 5Mb on aBW from now to +10min.

… next examines the schedule of accepted reservations in the aBW share (click). As there are currently no reservations ...

slide-123
SLIDE 123

Time Bandwidth

Reservation Limit

t

52

PANE

reserve(user=Alice, dstPort=80) = 5Mb on aBW from now to +10min.

… next examines the schedule of accepted reservations in the aBW share (click). As there are currently no reservations ...

slide-124
SLIDE 124

Time Bandwidth

Reservation Limit

t

53

PANE

reserve(user=Alice, dstPort=80) = 5Mb on aBW from now to +10min.

… the controller then recursively checks for other reservations up the share tree.

slide-125
SLIDE 125

Time Bandwidth

Reservation Limit

t

54

PANE

reserve(user=Alice, dstPort=80) = 5Mb on aBW from now to +10min.

When the controller tries to install the reservation…

slide-126
SLIDE 126

Time Bandwidth

Reservation Limit

  • t

55

PANE

reserve(user=Alice, dstPort=80) = 5Mb on aBW from now to +10min.

… it detects a conflict with the existing reservations. (Pause)

slide-127
SLIDE 127

56

PANE

reserve(user=Alice, dstPort=80) = 5Mb on aBW from now to +10min.

NO Alice

Therefore, the controller denies Alice’s initial request. Next, Alice retrieves the schedule of accepted requests from the controller, and creates a new request (click) for the same bandwidth, now starting 20 minutes in the future.

slide-128
SLIDE 128

56

PANE

reserve(user=Alice, dstPort=80) = 5Mb on aBW from now to +10min.

NO

reserve(user=Alice, dstPort=80) = 5Mb on aBW from +20min to +30min.

Alice

Therefore, the controller denies Alice’s initial request. Next, Alice retrieves the schedule of accepted requests from the controller, and creates a new request (click) for the same bandwidth, now starting 20 minutes in the future.

slide-129
SLIDE 129

Time Bandwidth

Reservation Limit

  • t

57

PANE

reserve(user=Alice, dstPort=80) = 5Mb on aBW from +20min to +30min.

The controller takes the new request ...

slide-130
SLIDE 130

Time Bandwidth

Reservation Limit

  • t

58

PANE

reserve(user=Alice, dstPort=80) = 5Mb on aBW from +20min to +30min.

… and checks if it can be installed at the new time.

slide-131
SLIDE 131

Time Bandwidth

Reservation Limit

  • t

59

PANE

reserve(user=Alice, dstPort=80) = 5Mb on aBW from +20min to +30min.

Because accepting this reservation would no longer exceed the limit ...

slide-132
SLIDE 132

60

PANE

reserve(user=Alice, dstPort=80) = 5Mb on aBW from now to +10min.

NO

reserve(user=Alice, dstPort=80) = 5Mb on aBW from +20min to +30min.

OK Alice

the controller returns a successful confirmation to Alice. When the reservation begins in 20 minutes, the PANE controller will establish the appropriate queues on the switches and provide Alice’s traffjc with 5 Mbps of guaranteed minimum bandwidth. (Pause)

slide-133
SLIDE 133

61

PANE

Let’s now consider a second example. If Alice (click) wants to block some traffjc to her computer (click), she can ask the root user (click) to create a subshare (click) for her with the deny privilege (click). After creating this share, the root user grants use of the share (click) to Alice, as we saw previously (click). (Pause) If Alice’s computer …

slide-134
SLIDE 134

61

PANE

Alice

Let’s now consider a second example. If Alice (click) wants to block some traffjc to her computer (click), she can ask the root user (click) to create a subshare (click) for her with the deny privilege (click). After creating this share, the root user grants use of the share (click) to Alice, as we saw previously (click). (Pause) If Alice’s computer …

slide-135
SLIDE 135

61

PANE

10.0.0.2 Alice

Let’s now consider a second example. If Alice (click) wants to block some traffjc to her computer (click), she can ask the root user (click) to create a subshare (click) for her with the deny privilege (click). After creating this share, the root user grants use of the share (click) to Alice, as we saw previously (click). (Pause) If Alice’s computer …

slide-136
SLIDE 136

61

PANE

10.0.0.2 Alice Root

Let’s now consider a second example. If Alice (click) wants to block some traffjc to her computer (click), she can ask the root user (click) to create a subshare (click) for her with the deny privilege (click). After creating this share, the root user grants use of the share (click) to Alice, as we saw previously (click). (Pause) If Alice’s computer …

slide-137
SLIDE 137

61

NewShare aAC for (dstHost=10.0.0.2) [deny = True]

  • n rootShare.

PANE

10.0.0.2 Alice Root

Let’s now consider a second example. If Alice (click) wants to block some traffjc to her computer (click), she can ask the root user (click) to create a subshare (click) for her with the deny privilege (click). After creating this share, the root user grants use of the share (click) to Alice, as we saw previously (click). (Pause) If Alice’s computer …

slide-138
SLIDE 138

61

NewShare aAC for (dstHost=10.0.0.2) [deny = True]

  • n rootShare.

PANE

OK 10.0.0.2 Alice Root

Let’s now consider a second example. If Alice (click) wants to block some traffjc to her computer (click), she can ask the root user (click) to create a subshare (click) for her with the deny privilege (click). After creating this share, the root user grants use of the share (click) to Alice, as we saw previously (click). (Pause) If Alice’s computer …

slide-139
SLIDE 139

61

NewShare aAC for (dstHost=10.0.0.2) [deny = True]

  • n rootShare.

PANE

OK

Grant aAC to Alice.

10.0.0.2 Alice Root

Let’s now consider a second example. If Alice (click) wants to block some traffjc to her computer (click), she can ask the root user (click) to create a subshare (click) for her with the deny privilege (click). After creating this share, the root user grants use of the share (click) to Alice, as we saw previously (click). (Pause) If Alice’s computer …

slide-140
SLIDE 140

61

NewShare aAC for (dstHost=10.0.0.2) [deny = True]

  • n rootShare.

PANE

OK

Grant aAC to Alice.

OK 10.0.0.2 Alice Root

Let’s now consider a second example. If Alice (click) wants to block some traffjc to her computer (click), she can ask the root user (click) to create a subshare (click) for her with the deny privilege (click). After creating this share, the root user grants use of the share (click) to Alice, as we saw previously (click). (Pause) If Alice’s computer …

slide-141
SLIDE 141

62

PANE

10.0.0.2 Alice

… is being attacked by Eve (click), she can send a deny request (click) to the PANE controller to have Eve’s traffjc blocked for the next five minutes. Because Alice was previously granted this authority, the PANE controller accepts her request (click), and uses OpenFlow to reconfigure the switches and block traffjc from Eve’s computer destined to Alice’s (click). If Alice tried to block Eve’s traffjc to another computer by changing the dstHost parameter on her request, the request would be denied as the flow would no longer be contained within the FlowGroup of the aAC subshare. (possibly make this its own slide?) (Pause) This has been a short sample of the PANE protocol. Our prototype supports several additional commands, for example, to establish rate-limits, manage users, and query the state of the ShareTree. (Pause)

slide-142
SLIDE 142

10.0.0.3 Eve

62

PANE

10.0.0.2 Alice

… is being attacked by Eve (click), she can send a deny request (click) to the PANE controller to have Eve’s traffjc blocked for the next five minutes. Because Alice was previously granted this authority, the PANE controller accepts her request (click), and uses OpenFlow to reconfigure the switches and block traffjc from Eve’s computer destined to Alice’s (click). If Alice tried to block Eve’s traffjc to another computer by changing the dstHost parameter on her request, the request would be denied as the flow would no longer be contained within the FlowGroup of the aAC subshare. (possibly make this its own slide?) (Pause) This has been a short sample of the PANE protocol. Our prototype supports several additional commands, for example, to establish rate-limits, manage users, and query the state of the ShareTree. (Pause)

slide-143
SLIDE 143

10.0.0.3 Eve

62

PANE

10.0.0.2

deny(dstHost=10.0.0.2, srcHost=10.0.0.3) on aAC from now to +5min.

Alice

… is being attacked by Eve (click), she can send a deny request (click) to the PANE controller to have Eve’s traffjc blocked for the next five minutes. Because Alice was previously granted this authority, the PANE controller accepts her request (click), and uses OpenFlow to reconfigure the switches and block traffjc from Eve’s computer destined to Alice’s (click). If Alice tried to block Eve’s traffjc to another computer by changing the dstHost parameter on her request, the request would be denied as the flow would no longer be contained within the FlowGroup of the aAC subshare. (possibly make this its own slide?) (Pause) This has been a short sample of the PANE protocol. Our prototype supports several additional commands, for example, to establish rate-limits, manage users, and query the state of the ShareTree. (Pause)

slide-144
SLIDE 144

10.0.0.3 Eve

62

PANE

10.0.0.2

deny(dstHost=10.0.0.2, srcHost=10.0.0.3) on aAC from now to +5min.

OK Alice

… is being attacked by Eve (click), she can send a deny request (click) to the PANE controller to have Eve’s traffjc blocked for the next five minutes. Because Alice was previously granted this authority, the PANE controller accepts her request (click), and uses OpenFlow to reconfigure the switches and block traffjc from Eve’s computer destined to Alice’s (click). If Alice tried to block Eve’s traffjc to another computer by changing the dstHost parameter on her request, the request would be denied as the flow would no longer be contained within the FlowGroup of the aAC subshare. (possibly make this its own slide?) (Pause) This has been a short sample of the PANE protocol. Our prototype supports several additional commands, for example, to establish rate-limits, manage users, and query the state of the ShareTree. (Pause)

slide-145
SLIDE 145

10.0.0.3 Eve

62

PANE

10.0.0.2

deny(dstHost=10.0.0.2, srcHost=10.0.0.3) on aAC from now to +5min.

OK Alice

… is being attacked by Eve (click), she can send a deny request (click) to the PANE controller to have Eve’s traffjc blocked for the next five minutes. Because Alice was previously granted this authority, the PANE controller accepts her request (click), and uses OpenFlow to reconfigure the switches and block traffjc from Eve’s computer destined to Alice’s (click). If Alice tried to block Eve’s traffjc to another computer by changing the dstHost parameter on her request, the request would be denied as the flow would no longer be contained within the FlowGroup of the aAC subshare. (possibly make this its own slide?) (Pause) This has been a short sample of the PANE protocol. Our prototype supports several additional commands, for example, to establish rate-limits, manage users, and query the state of the ShareTree. (Pause)

slide-146
SLIDE 146

Netflix

63

slide-147
SLIDE 147

64

For example, I like to watch movies at home with Netflix. And while there are many reasons …

slide-148
SLIDE 148

65

… why Netflix may begin to bufger, one reason is because ….

slide-149
SLIDE 149

66

… a second laptop has begun a network backup. And while there are …

slide-150
SLIDE 150 TCP Nice: A Mechanism for Background Transfers Arun Venkataramani Ravi Kokku Mike Dahlin Laboratory of Advanced Systems Research Department of Computer Sciences University of Texas at Austin, Austin, TX 78712 arun, rkoku, dahlin @cs.utexas.edu Abstract Many distributed applications can make use of large background transfers transfers of data that humans are not waiting for to improve availability, reliability, latency or consistency. However, given the rapid fluc- tuations of available network bandwidth and changing resource costs due to technology trends, hand tuning the aggressiveness of background transfers risks (1) compli- cating applications, (2) being too aggressive and inter- fering with other applications, and (3) being too timid and not gaining the benefits of background transfers. Our goal is for the operating system to manage network resources in order to provide a simple abstraction of near zero-cost background transfers. Our system, TCP Nice, can provably bound the interference inflicted by back- ground flows on foreground flows in a restricted network
  • model. And our microbenchmarks and case study appli-
cations suggest that in practice it interferes little with foreground flows, reaps a large fraction of spare net- work bandwidth, and simplifies application construction and deployment. For example, in our prefetching case study application, aggressive prefetching improves de- mand performance by a factor of three when Nice man- ages resources; but the same prefetching hurts demand performance by a factor of six under standard network congestion control. 1 Introduction Many distributed applications can make use of large background transfers transfers of data that humans are not waiting for to improve service quality. For exam- ple, a broad range of applications and services such as data backup [29], prefetching [50], enterprise data dis- tribution [20], Internet content distribution [2], and peer- to-peer storage [16, 43] can trade increased network This work was supported in part by an NSF CISE grant (CDA- 9624082), the Texas Advanced Technology Program, the Texas Ad- vanced Research Program, and Tivoli. Dahlin was also supported by an NSF CAREER award (CCR-9733842) and an Alfred P. Sloan Re- search Fellowship. bandwidth consumption and possibly disk space for im- proved service latency [15, 18, 26, 32, 38, 50], improved availability [11, 53], increased scalability [2], stronger consistency [53], or support for mobility [28, 41, 47]. Many of these services have potentially unlimited band- width demands where incrementally more bandwidth consumption provides incrementally better service. For example, a web prefetching system can improve its hit rate by fetching objects from a virtually unlimited col- lection of objects that have non-zero probability of ac- cess [8, 10] or by updating cached copies more fre- quently as data change [13, 50, 48]; Technology trends suggest that “wasting” bandwidth and storage to im- prove latency and availability will become increasingly attractive in the future: per-byte network transport costs and disk storage costs are low and have been improv- ing at 80-100% per year [9, 17, 37]; conversely net- work availability [11, 40, 54] and network latencies im- prove slowly, and long latencies and failures waste hu- man time. Current operating systems and networks do not provide good support for aggressive background transfers. In particular, because background transfers compete with foreground requests, they can hurt overall performance and availability by increasing network congestion. Ap- plications must therefore carefully balance the benefits
  • f background transfers against the risk of both self-
interference, where applications hurt their own perfor- mance, and cross-interference, where applications hurt
  • ther applications’ performance. Often, applications at-
tempt to achieve this balance by setting “magic num- bers” (e.g., the prefetch threshold in prefetching algo- rithms [18, 26]) that have little obvious relationship to system goals (e.g., availability or latency) or constraints (e.g., current spare network bandwidth). Our goal is for the operating system to manage net- work resources in order to provide a simple abstrac- tion of zero-cost background transfers. A self-tuning background transport layer will enable new classes of applications by (1) simplifying applications, (2) reduc- ing the risk of being too aggressive, and (3) making

67

… many proposals for how to solve this problem, it still exists. With participatory networking ...

slide-151
SLIDE 151

68

the Netflix application can inform my home network of its bandwidth and latency requirements (click), and be guaranteed a level of service. (pause) Turning now to an enterprise network ...

slide-152
SLIDE 152

68

the Netflix application can inform my home network of its bandwidth and latency requirements (click), and be guaranteed a level of service. (pause) Turning now to an enterprise network ...

slide-153
SLIDE 153

69

Datacenter

slide-154
SLIDE 154

70

Based on “Delusional Boot: Securing Cloud Hypervisors without Massive Re-Engineering” (EuroSys 2012)

Production Platform

  • n the Azure cloud environment (click) a firewall is used to isolate untrusted (click) customer

virtual machines while booting. After boot-up (click), the VM configuration can be made more secure ...

slide-155
SLIDE 155

70

Based on “Delusional Boot: Securing Cloud Hypervisors without Massive Re-Engineering” (EuroSys 2012)

Production Platform

  • n the Azure cloud environment (click) a firewall is used to isolate untrusted (click) customer

virtual machines while booting. After boot-up (click), the VM configuration can be made more secure ...

slide-156
SLIDE 156

70

Based on “Delusional Boot: Securing Cloud Hypervisors without Massive Re-Engineering” (EuroSys 2012)

Production Platform Boot Service

  • n the Azure cloud environment (click) a firewall is used to isolate untrusted (click) customer

virtual machines while booting. After boot-up (click), the VM configuration can be made more secure ...

slide-157
SLIDE 157

70

Based on “Delusional Boot: Securing Cloud Hypervisors without Massive Re-Engineering” (EuroSys 2012)

Production Platform Boot Service

  • n the Azure cloud environment (click) a firewall is used to isolate untrusted (click) customer

virtual machines while booting. After boot-up (click), the VM configuration can be made more secure ...

slide-158
SLIDE 158

71

Based on “Delusional Boot: Securing Cloud Hypervisors without Massive Re-Engineering” (EuroSys 2012)

Production Platform Boot Service

... the firewall lowered ...

slide-159
SLIDE 159

72

Based on “Delusional Boot: Securing Cloud Hypervisors without Massive Re-Engineering” (EuroSys 2012)

Production Platform Boot Service

… and the VM image transferred to the production-side of the cloud.

slide-160
SLIDE 160

72

Based on “Delusional Boot: Securing Cloud Hypervisors without Massive Re-Engineering” (EuroSys 2012)

Production Platform Boot Service

… and the VM image transferred to the production-side of the cloud.

slide-161
SLIDE 161

73

Based on “Delusional Boot: Securing Cloud Hypervisors without Massive Re-Engineering” (EuroSys 2012)

Production Platform Boot Service

Lacking a practical API for managing the firewall via the virtual machine boot service, the implementation uses programmable MAC addresses on the servers, a static configuration on the firewall, and the usual duck tape we find in networks to achieve the result. So again we can ask, why is this knowledge about managing the network trapped inside the end-hosts?

slide-162
SLIDE 162

74

Enterprise

slide-163
SLIDE 163

75

... we see shared links supporting many hosts. And if one host sufgers from a denial of service attack...

slide-164
SLIDE 164

76

… we may need more than a local firewall rule to protect the network. Today, we can call …

slide-165
SLIDE 165

76

… we may need more than a local firewall rule to protect the network. Today, we can call …

slide-166
SLIDE 166

77

… the network administrator, or with participatory networking, the victim host ...

slide-167
SLIDE 167

78

… can install a network firewall rule on its own. (pause) Furthermore, in Microsoft datacenters ...

slide-168
SLIDE 168

78

… can install a network firewall rule on its own. (pause) Furthermore, in Microsoft datacenters ...

slide-169
SLIDE 169

A problem in the datacenter

79

The final problem I want to look at exists in current proposals for hybrid optical-electrical networks.

slide-170
SLIDE 170

80

In these hybrid networks, connectivity is primarily provided by Ethernet running over the usual copper cables (click). In addition, the top-of-rack switches are also connected by a fully

  • ptical network (click).

The optical switch can create circuits between rack pairs (click), but cannot be reconfigured quickly because of physical delays when aligning the internal mirrors. In the current proposals...

slide-171
SLIDE 171

80

In these hybrid networks, connectivity is primarily provided by Ethernet running over the usual copper cables (click). In addition, the top-of-rack switches are also connected by a fully

  • ptical network (click).

The optical switch can create circuits between rack pairs (click), but cannot be reconfigured quickly because of physical delays when aligning the internal mirrors. In the current proposals...

slide-172
SLIDE 172

80

In these hybrid networks, connectivity is primarily provided by Ethernet running over the usual copper cables (click). In addition, the top-of-rack switches are also connected by a fully

  • ptical network (click).

The optical switch can create circuits between rack pairs (click), but cannot be reconfigured quickly because of physical delays when aligning the internal mirrors. In the current proposals...

slide-173
SLIDE 173

80

In these hybrid networks, connectivity is primarily provided by Ethernet running over the usual copper cables (click). In addition, the top-of-rack switches are also connected by a fully

  • ptical network (click).

The optical switch can create circuits between rack pairs (click), but cannot be reconfigured quickly because of physical delays when aligning the internal mirrors. In the current proposals...

slide-174
SLIDE 174

81

a management server monitors the traffjc matrix (click) on the copper Ethernet and uses a heuristic to detect large, long-lasting flows that would benefit from the higher bandwidth and lower latency of an all-optical path. When such flows are detected, the optical switch is reconfigured (click), and the heavy traffjc eventually moved to the new path. But such a detect-and-react strategy is not always necessary! There are many applications inside the datacenter that know in advance how much traffjc they will generate. For example, virtual machine migrations and shuffme stages in MapReduce-like frameworks. By now, I think you know the question to ask: why is this knowledge about managing the network trapped inside the end-hosts? (5 minutes)

slide-175
SLIDE 175

81

a management server monitors the traffjc matrix (click) on the copper Ethernet and uses a heuristic to detect large, long-lasting flows that would benefit from the higher bandwidth and lower latency of an all-optical path. When such flows are detected, the optical switch is reconfigured (click), and the heavy traffjc eventually moved to the new path. But such a detect-and-react strategy is not always necessary! There are many applications inside the datacenter that know in advance how much traffjc they will generate. For example, virtual machine migrations and shuffme stages in MapReduce-like frameworks. By now, I think you know the question to ask: why is this knowledge about managing the network trapped inside the end-hosts? (5 minutes)

slide-176
SLIDE 176

81

a management server monitors the traffjc matrix (click) on the copper Ethernet and uses a heuristic to detect large, long-lasting flows that would benefit from the higher bandwidth and lower latency of an all-optical path. When such flows are detected, the optical switch is reconfigured (click), and the heavy traffjc eventually moved to the new path. But such a detect-and-react strategy is not always necessary! There are many applications inside the datacenter that know in advance how much traffjc they will generate. For example, virtual machine migrations and shuffme stages in MapReduce-like frameworks. By now, I think you know the question to ask: why is this knowledge about managing the network trapped inside the end-hosts? (5 minutes)

slide-177
SLIDE 177

Participatory Networking

82

If we follow the analogy that software defined networks are developing an operating system for the network, Participatory Networking is building the end-user system calls -- an API for SDNs. (pause) Like previous work on operating systems ...

slide-178
SLIDE 178

83

Ken Thompson & Dennis Ritchie

... SDNs began by providing abstractions over the hardware; we believe it's time for SDNs to similarly evolve into arbiters that support multiple principals sharing and controlling those resources. (pause) One challenge, of course, is the development and implementation of a semantics which delegates authority ...

slide-179
SLIDE 179

Jon Postel

… from the network administrators ...

slide-180
SLIDE 180

85

… to the people, without sacrificing high-level requirements such as ...

slide-181
SLIDE 181

Safe? Secure? Fair? Loop freedom?

Participatory Networking

Black holes?

86

safety, security, and fairness, and low-level properties such as freedom from routing loops and traffic black holes.