Network Function Control Aaron Gember-Jacobson , Chaithan Prakash, - - PowerPoint PPT Presentation

▶
network function control
SMART_READER_LITE
LIVE PREVIEW

Network Function Control Aaron Gember-Jacobson , Chaithan Prakash, - - PowerPoint PPT Presentation

Nov 11, 2023 274 likes •1.16k views

OpenNF: Enabling Innovation in Network Function Control Aaron Gember-Jacobson , Chaithan Prakash, Raajay Viswanathan, Robert Grandl, Junaid Khalid, Sourav Das, Aditya Akella 1 Network functions (NFs) Perform sophisticated stateful actions

slide-1
SLIDE 1

Aaron Gember-Jacobson, Chaithan Prakash, Raajay Viswanathan, Robert Grandl, Junaid Khalid, Sourav Das, Aditya Akella

1

OpenNF: Enabling Innovation in Network Function Control

slide-2
SLIDE 2

Network functions (NFs)

  • Perform sophisticated stateful

actions on packets/flows

2

Intrusion detection system (IDS) Caching proxy WAN

  • ptimizer
slide-3
SLIDE 3

NF trends

  • Network Functions Virtualization (NFV)

3

Intrusion detection system (IDS) Caching proxy WAN

  • ptimizer
slide-4
SLIDE 4

NF trends

  • Network Functions Virtualization (NFV)

→ dynamically allocate NF instances

3

Hypervisor

slide-5
SLIDE 5

NF trends

  • Network Functions Virtualization (NFV)

→ dynamically allocate NF instances

  • Software-defined Networking

→ dynamically reroute flows

3

Hypervisor

slide-6
SLIDE 6

NF trends

  • Network Functions Virtualization (NFV)

→ dynamically allocate NF instances

  • Software-defined Networking

→ dynamically reroute flows Dynamic reallocation

  • f packet processing

3

Hypervisor

slide-7
SLIDE 7

Example: elastic NF scaling

  • 1. Satisfy performance SLAs

4

slide-8
SLIDE 8

Example: elastic NF scaling

  • 1. Satisfy performance SLAs

4

CPU Packet loss

slide-9
SLIDE 9

Example: elastic NF scaling

  • 1. Satisfy performance SLAs

4

CPU Packet loss

slide-10
SLIDE 10

Example: elastic NF scaling

  • 1. Satisfy performance SLAs

4

CPU Packet loss

slide-11
SLIDE 11

Example: elastic NF scaling

  • 1. Satisfy performance SLAs
  • 2. Minimize operating costs

4

CPU Packet loss

slide-12
SLIDE 12

Example: elastic NF scaling

  • 1. Satisfy performance SLAs
  • 2. Minimize operating costs

4

CPU Packet loss

slide-13
SLIDE 13

Example: elastic NF scaling

  • 1. Satisfy performance SLAs
  • 2. Minimize operating costs
  • 3. Accurately monitor traffic

4

CPU Packet loss

slide-14
SLIDE 14
  • 1. Satisfy performance SLAs
  • 2. Minimize operating costs
  • 3. Accurately monitor traffic

5

To simultaneously…

Problem: NFV+SDN is insufficient

slide-15
SLIDE 15

Why NFV + SDN falls short

  • 1. SLAs 2. Cost 3. Accuracy

Reroute new flows Reroute existing flows Wait for flows to die

6

Packet loss

slide-16
SLIDE 16

Why NFV + SDN falls short

  • 1. SLAs 2. Cost 3. Accuracy

Reroute new flows Reroute existing flows Wait for flows to die

6

Packet loss SLA: <1%

slide-17
SLIDE 17

Why NFV + SDN falls short

  • 1. SLAs 2. Cost 3. Accuracy

Reroute new flows Reroute existing flows Wait for flows to die

6

?

Packet loss SLA: <1%

slide-18
SLIDE 18

Why NFV + SDN falls short

  • 1. SLAs 2. Cost 3. Accuracy

Reroute new flows Reroute existing flows Wait for flows to die

6

?

Packet loss SLA: <1%

slide-19
SLIDE 19

Why NFV + SDN falls short

  • 1. SLAs 2. Cost 3. Accuracy

Reroute new flows Reroute existing flows Wait for flows to die

6

?

Packet loss

slide-20
SLIDE 20

Why NFV + SDN falls short

  • 1. SLAs 2. Cost 3. Accuracy

Reroute new flows Reroute existing flows Wait for flows to die

6

?

Packet loss

slide-21
SLIDE 21

Why NFV + SDN falls short

  • 1. SLAs 2. Cost 3. Accuracy

Reroute new flows Reroute existing flows Wait for flows to die

6

?

Packet loss

slide-22
SLIDE 22

Why NFV + SDN falls short

  • 1. SLAs 2. Cost 3. Accuracy

Reroute new flows Reroute existing flows Wait for flows to die

6

?

Packet loss

slide-23
SLIDE 23

SLAs + cost + accuracy: What do we need?

  • Quickly move, copy, or share internal NF state

alongside updates to network forwarding state

  • Guarantees: loss-free, order-preserving, …

7

   … 1 2 3 …

Also applies to other scenarios

slide-24
SLIDE 24

Outline

  • Motivation and requirements
  • Challenges
  • OpenNF architecture

– State export/import – State operations – Guarantees

  • Evaluation

8

slide-25
SLIDE 25
  • 1. Supporting many NFs with minimal changes
  • 2. Dealing with race conditions
  • 3. Bounding overhead

Challenges

9

slide-26
SLIDE 26
  • Virtual machine replication

– Cannot combine → limited rebalancing

  • Split/Merge [NSDI’13]

– State allocations and accesses occur via library – Addresses a specific problem → limited suitability – Packets may be dropped or re-ordered → wrong NF behavior

10

Existing approaches

slide-27
SLIDE 27

OpenNF overview

11

NF State Manager Flow Manager

OpenNF Controller

Control Application

move/copy/share state export/import State

slide-28
SLIDE 28

State created or updated by an NF applies to either a single flow or a collection of flows

NF state taxonomy

12

Connection Connection TcpAnalyzer HttpAnalyzer TcpAnalyzer HttpAnalyzer

Per-flow state

ConnCount

Multi-flow state All-flows state

Statistics

slide-29
SLIDE 29

NF API: export/import state

  • Functions: get, put, delete

13

No need to expose/change internal state organization! Filter

Per Multi All

Scope NF get put

slide-30
SLIDE 30

Control operations: move

14

NF State Manager Control Application Flow Manager Bro2 Bro1

slide-31
SLIDE 31

Control operations: move

14

NF State Manager Control Application

move (port=80, Bro1, Bro2)

Flow Manager Bro2 Bro1

slide-32
SLIDE 32

Control operations: move

14

NF State Manager Control Application

move (port=80, Bro1, Bro2) get(per, port=80)

Flow Manager Bro2 Bro1

slide-33
SLIDE 33

Control operations: move

14

NF State Manager Control Application

move (port=80, Bro1, Bro2) get(per, port=80) [Chunk1] [Chunk2]

Flow Manager Bro2 Bro1

slide-34
SLIDE 34

Control operations: move

14

NF State Manager Control Application

move (port=80, Bro1, Bro2) get(per, port=80) [Chunk1] del(per, port=80) [Chunk2]

Flow Manager Bro2 Bro1

slide-35
SLIDE 35

Control operations: move

14

NF State Manager Control Application

move (port=80, Bro1, Bro2) get(per, port=80) [Chunk1] put (per, Chunk1) del(per, port=80) [Chunk2] put (per, Chunk2)

Flow Manager Bro2 Bro1

slide-36
SLIDE 36

Control operations: move

14

NF State Manager Control Application

move (port=80, Bro1, Bro2) get(per, port=80) [Chunk1] put (per, Chunk1) del(per, port=80) [Chunk2] put (per, Chunk2) forward(port=80, Bro2)

Flow Manager Bro2 Bro1

slide-37
SLIDE 37

Control operations: move

14

NF State Manager Control Application

move (port=80, Bro1, Bro2) get(per, port=80) [Chunk1] put (per, Chunk1) del(per, port=80) [Chunk2] put (per, Chunk2) forward(port=80, Bro2)

Flow Manager Bro2 Bro1 Also provide copy and share

slide-38
SLIDE 38
  • 1. Supporting many NFs with minimal changes
  • 2. Dealing with race conditions
  • 3. Bounding overhead

Challenges

15

slide-39
SLIDE 39

detect- MHR

Lost updates during move

16

Bro2 Bro1

slide-40
SLIDE 40

detect- MHR

Lost updates during move

16

B1 R1

Bro2 Bro1

slide-41
SLIDE 41

detect- MHR

Lost updates during move

16

B1 R1

Bro2 Bro1

move(red,Bro1 ,Bro2 )

slide-42
SLIDE 42

detect- MHR

Lost updates during move

16

B1 R1

Bro2 Bro1

move(red,Bro1 ,Bro2 )

slide-43
SLIDE 43

detect- MHR

Lost updates during move

16

B1 R1 R2

Missing state

Bro2 Bro1

move(red,Bro1 ,Bro2 )

slide-44
SLIDE 44

detect- MHR

Lost updates during move

16

B1 R1 R2

Missing state

Bro2 Bro1

move(red,Bro1 ,Bro2 )

slide-45
SLIDE 45

detect- MHR

Lost updates during move

16

B1 R1 R2

Missing state

Bro2 Bro1

move(red,Bro1 ,Bro2 )

slide-46
SLIDE 46

detect- MHR

Lost updates during move

16

B1 R1 R2

Missing state

Bro2 Bro1

move(red,Bro1 ,Bro2 ) Missing updates

R3

slide-47
SLIDE 47

detect- MHR

Lost updates during move

16

B1 R1 R2

Missing state

Bro2 Bro1

move(red,Bro1 ,Bro2 ) Missing updates

R3

slide-48
SLIDE 48

detect- MHR

  • Split/Merge [NSDI ‘13]: pause traffic, buffer packets

– Packets in-transit when buffering starts are dropped

Lost updates during move

16

B1 R1 R2

Missing state

Bro2 Bro1

move(red,Bro1 ,Bro2 ) Missing updates

Loss-free: All state updates should be reflected in the transferred state, and all packets should be processed

R3

slide-49
SLIDE 49

NF API: observe/prevent updates using events

17

Only need to change an NF’s receive packet function!

R1

NF

slide-50
SLIDE 50

Use events for loss-free move

18

Bro2 Bro1

R1

slide-51
SLIDE 51
  • 1. enableEvents(red,drop) on Bro1

Use events for loss-free move

18

Bro2 Bro1

Drop R1

slide-52
SLIDE 52
  • 1. enableEvents(red,drop) on Bro1
  • 2. get/delete on Bro1

Use events for loss-free move

18

Bro2 Bro1

Drop R1

slide-53
SLIDE 53
  • 1. enableEvents(red,drop) on Bro1
  • 2. get/delete on Bro1

Use events for loss-free move

18

Bro2 Bro1

Drop R1 R2

slide-54
SLIDE 54
  • 1. enableEvents(red,drop) on Bro1
  • 2. get/delete on Bro1
  • 3. Buffer events at controller

Use events for loss-free move

18

Bro2 Bro1

Drop R1 R2

slide-55
SLIDE 55
  • 1. enableEvents(red,drop) on Bro1
  • 2. get/delete on Bro1
  • 3. Buffer events at controller
  • 4. put on Bro2

Use events for loss-free move

18

Bro2 Bro1

Drop R1 R2

slide-56
SLIDE 56
  • 1. enableEvents(red,drop) on Bro1
  • 2. get/delete on Bro1
  • 3. Buffer events at controller
  • 4. put on Bro2
  • 5. Flush packets in

events to Bro2

Use events for loss-free move

18

Bro2 Bro1

Drop R1 R1,R2

slide-57
SLIDE 57
  • 1. enableEvents(red,drop) on Bro1
  • 2. get/delete on Bro1
  • 3. Buffer events at controller
  • 4. put on Bro2
  • 5. Flush packets in

events to Bro2

  • 6. Update

forwarding

Use events for loss-free move

18

Bro2 Bro1

Drop R1 R1,R2

slide-58
SLIDE 58
  • 1. enableEvents(red,drop) on Bro1
  • 2. get/delete on Bro1
  • 3. Buffer events at controller
  • 4. put on Bro2
  • 5. Flush packets in

events to Bro2

  • 6. Update

forwarding

Use events for loss-free move

18

Bro2 Bro1

Drop R1 R1,R2 R1,R2,R3

slide-59
SLIDE 59
  • False positives from Bro’s weird script

Re-ordering of packets

19

Controller Switch Bro2 Bro1

slide-60
SLIDE 60
  • False positives from Bro’s weird script

Re-ordering of packets

19

Controller Switch Bro2

  • 5. Flush buffer

Bro1

R2 R2 R2

slide-61
SLIDE 61
  • False positives from Bro’s weird script

Re-ordering of packets

19

Controller Switch Bro2

  • 5. Flush buffer
  • 6. Request

forwarding update Bro1

R2 R2 R2

slide-62
SLIDE 62
  • False positives from Bro’s weird script

Re-ordering of packets

19

Controller Switch Bro2

  • 5. Flush buffer
  • 6. Request

forwarding update Bro1

R2 R2 R3 R2 R3

slide-63
SLIDE 63
  • False positives from Bro’s weird script

Re-ordering of packets

19

Controller Switch Bro2

  • 5. Flush buffer
  • 6. Request

forwarding update Bro1

R2 R2 R3 R3 R3 R2 R3 R3

slide-64
SLIDE 64
  • False positives from Bro’s weird script

Re-ordering of packets

19

Controller Switch Bro2

  • 5. Flush buffer
  • 6. Request

forwarding update Bro1

R2 R2 R4 R3 R3 R3 R2 R4 R3 R3

slide-65
SLIDE 65
  • False positives from Bro’s weird script

Re-ordering of packets

19

Order-preserving: All packets should be processed in the order they were forwarded by the switch

Controller Switch Bro2

  • 5. Flush buffer
  • 6. Request

forwarding update Bro1

R2 R2 R4 R3 R3 R3 R2 R4 R3 R3

slide-66
SLIDE 66

Order-preserving move

20

Drop R1 B2

slide-67
SLIDE 67
  • Flush packets in events to Inst2

Order-preserving move

20

Drop R1 R1,R2

slide-68
SLIDE 68
  • Flush packets in events to Inst2
  • enableEvents(red,buffer) on Inst2

Order-preserving move

20

Drop R1 R1,R2 Buf

slide-69
SLIDE 69
  • Flush packets in events to Inst2
  • enableEvents(red,buffer) on Inst2
  • Forwarding update: send to Inst1 & controller

Order-preserving move

20

Drop R1 R1,R2 Buf

slide-70
SLIDE 70
  • Flush packets in events to Inst2
  • enableEvents(red,buffer) on Inst2
  • Forwarding update: send to Inst1 & controller
  • Wait for packet from

switch (remember last)

  • Forwarding update:

send to Inst2

Order-preserving move

20

Drop R1 R1,R2 Buf R3 R3 R4

slide-71
SLIDE 71
  • Flush packets in events to Inst2
  • enableEvents(red,buffer) on Inst2
  • Forwarding update: send to Inst1 & controller
  • Wait for packet from

switch (remember last)

  • Forwarding update:

send to Inst2

  • Wait for event

for last packet from Inst2

Order-preserving move

20

Drop R1 R1,R2 R1,R2, R3 Buf R3 R3 R4

slide-72
SLIDE 72
  • Flush packets in events to Inst2
  • enableEvents(red,buffer) on Inst2
  • Forwarding update: send to Inst1 & controller
  • Wait for packet from

switch (remember last)

  • Forwarding update:

send to Inst2

  • Wait for event

for last packet from Inst2

  • Release buffer of packets on Inst2

Order-preserving move

20

Drop R1 R1,R2 R1,R2, R3 R1,R2, R3,R4

slide-73
SLIDE 73
  • Flush packets in events to Inst2
  • enableEvents(red,buffer) on Inst2
  • Forwarding update: send to Inst1 & controller
  • Wait for packet from

switch (remember last)

  • Forwarding update:

send to Inst2

  • Wait for event

for last packet from Inst2

  • Release buffer of packets on Inst2

Order-preserving move

20

Drop R1 R1,R2 R1,R2, R3 R1,R2, R3,R4

Assumes no loss or re-ordering on the links from switch to NFs

slide-74
SLIDE 74
  • 1. Supporting many NFs with minimal changes
  • 2. Dealing with race conditions
  • 3. Bounding overhead

Challenges

21

slide-75
SLIDE 75

Applications decide (based on NF & objectives):

  • 1. Granularity of
  • perations
  • 2. Guarantees

desired

Bounding overhead

22

Filter Per Multi All Scope

   …

LF LF+OP

1 2 3 …    …

+ None

slide-76
SLIDE 76
  • 1. Dealing with diversity
  • 2. Dealing with race conditions
  • 3. Bounding overhead

OpenNF: SLAs + cost + accuracy

23

Export/import state based

  • n its association with flows

Events Lock-step forwarding updates

+

Applications choose granularity and guarantees

slide-77
SLIDE 77

Implementation

  • Controller (3.8K lines of Java)
  • Communication library (2.6K lines of C)
  • Modified NFs (3-8% increase in code)

24

Bro IDS iptables Squid Cache PRADS

slide-78
SLIDE 78

Overall benefits for elastic scaling

  • Bro IDS processing 10K pkts/sec

– At 180 sec: move HTTP flows (489) to new IDS – At 360 sec: move back to old IDS

  • SLAs: 260ms to move (loss-free)
  • Accuracy: same log entries as using one IDS

– VM replication: incorrect log entries

  • Cost: scale in after state is moved

– Wait for flows to die: scale in delayed 25+ minutes

25

slide-79
SLIDE 79

Evaluation: state export/import

26

Serialization/deserialization costs dominate Cost grows with state complexity

slide-80
SLIDE 80

50 100 150 200 Average Maximum

Per-packet Latency Increase (ms)

100 200 300 400 500 NG NG PL LF PL+ER

Move Time (ms)

  • PRADS asset detector processing 5K pkts/sec
  • Move per-flow state for 500 flows

Evaluation: operations

27

Packets dropped! 686 462 881 packets in events

Operations are efficient, but guarantees come at a cost!

1120 pkts buffered 838 pkts in events +

NG NG PL LF PL+ER OP PL+ER

slide-81
SLIDE 81

Future work

  • Reduce buffering

– Allow packet processing during state transfer, then replay input to bring state “up to speed”

  • Improve scalability

– Peer-to-peer state transfer

  • (Semi) automatically modify NFs

– Static program analysis

28

slide-82
SLIDE 82
  • Dynamic reallocation of packet

processing enables new services

  • Realizing SLAs + cost + accuracy requires

quick, safe control of internal NF state

  • OpenNF provides flexible and efficient

control with few NF modification

Conclusion

29

Learn more and try it! http://opennf.cs.wisc.edu

slide-83
SLIDE 83

Backup

  • Copy and share
  • Example app: elastic NF scaling
  • Evaluation: controller scalability
  • Evaluation: importance of guarantees
  • Evaluation: benefits of granular control

30

slide-84
SLIDE 84

Copy and share operations

  • Used when multiple instances need some state
  • Copy – no or eventual consistency

– Once, periodically, based on events, etc.

  • Share – strong or strict consistency

– Events are raised for all packets – Events are released

  • ne at a time

– State is copied before releasing the next event

31

Copy (multi-flow): 111ms Share (strong): 13ms/packet

slide-85
SLIDE 85

Example app: elastic NF scaling

movePrefix(prefix,oldInst,newInst): copy(oldInst,newInst,{nw_src:prefix},multi) move(oldInst,newInst,{nw_src:prefix},per,LF+OP) while (true): sleep(60) copy(oldInst,newInst,{nw_src:prefix},multi) copy(newInst,oldInst,{nw_src:prefix},multi)

scan.bro vulnerable.bro weird.bro

32

slide-86
SLIDE 86

Evaluation: controller scalability

Improve scalability with P2P state transfers

33

slide-87
SLIDE 87

Evaluation: importance

  • f guarantees
  • Bro1 processing malicious trace @ 1K pkts/sec
  • After 14K packets: move active flows to Bro2

Alert Baseline NG LF LF+OP Incorrect file type 26 25 24 26 MHR Match 31 28 27 31 MD5 116 111 106 116 Total 173 164 157 173

slide-88
SLIDE 88

Evaluation: benefits

  • f granular control
  • HTTP requests from 2 clients (40 unique URLs)
  • Initially: both go to Squid1
  • 20s later: reassign Client1 to Squid2

Ignore Copy-client Copy-all Hits @ Squid1 117 117 117 Hits @ Squid2 Crash! 39 50 State transferred 0 MB 4 MB 54 MB