Reverse Engineering TCP/ IP Reverse Engineering TCP/ IP Steven Low - - PowerPoint PPT Presentation

Reverse Engineering TCP/ IP Reverse Engineering TCP/ IP

Steven Low EAS, Caltech Joint work with: Li J W P j T T M W Li, J. Wang, Pongsajapan, Tan, Tang, M. Wang

Outline

Background

L i ti i ti d iti Layering as optimization decomposition Reverse engineering TCP

Reverse engineering TCP/ IP Reverse engineering TCP/ IP

Delay insensitive utility Delay sensitive utility Delay sensitive utility How bad is single-path routing

• J. Wang, Li, Low, Doyle. ToN, 2005

Pongsajapan Low Infocom 2007 Pongsajapan, Low, Infocom 2007

• M. Wang, Tan, Tang, Low, pre-print, 2009

Layering as optimization decomposition

Each layer designed separately and evolves asynchronously evolves asynchronously Each layer optimizes certain

• bjectives
• bjectives

application transport Minimize response time (web layout)… Maximize utility (TCP/ AQM) transport network link

Minimize path costs (IP) Reliability, channel access, …

link physical

y, , Minimize SIR, max capacities, …

Layering as optimization decomposition

• Each layer is abstracted as an optimization problem
• Operation of a layer is a distributed solution
• Results of one problem (layer) are parameters of
• Results of one problem (layer) are parameters of
• thers
• Operate at different timescales

Application: utility

x U

i i

∑

) ( max

application transport

Application: utility

p c Rx

i i i x

≤

∑

≥

) ( to subj ) ( max

transport network link

Phy: power

X x∈

link physical

IP: routing Link: scheduling

A wireless example

Application objective Network objective

) ( ) ( max w V x U

l l i i

+ ∑

∑

) , ( ) ( to subj ) ( ) ( max P w c x G R

l l l i i i x

≤

∑ ∑

≥

) ( ) , ( ) ( to subj P P C x w c x G R ∈ ≤

IP: optimize route given network graph G Link: maximize channel capacity given link resources Rate also constrained by interaction capacity given link resources

w and desired error probability P

Rate also constrained by interaction

• f coding mechanism & ARQ

Layering as optimization decomposition

Each layer is abstracted as an optimization problem Operation of a layer is a distributed solution Results of one problem (layer) are parameters of Results of one problem (layer) are parameters of

• thers

Operate at different timescales

1) U d t d h l i i l ti i

application transport

1) Understand each layer in isolation, assuming

• ther layers are designed nearly optimally

2) Understand interactions across layers

transport network link

2) Understand interactions across layers 3) Incorporate additional layers 4) Ultimate goal: entire protocol stack as

link physical

) g p solving one giant optimization problem, where individual layers are solving parts of it

Layering as optimization decomposition

Network generalized NUM Layers subproblems Layering decomposition methods Layering decomposition methods Interface functions of primal or dual vars

1) U d t d h l i i l ti i

application transport

1) Understand each layer in isolation, assuming

• ther layers are designed nearly optimally

2) Understand interactions across layers

transport network link

2) Understand interactions across layers 3) Incorporate additional layers 4) Ultimate goal: entire protocol stack as

link physical

) g p solving one giant optimization problem, where individual layers are solving parts of it

Examples

Optimal web layer: Zhu, Yu, Doyle ’01 HTTP/ TCP: Chang, Liu ’04

application transport

TCP: Kelly, Maulloo, Tan ’98, ……

p network link

TCP/ IP: Wang et al ’05, …… TCP/ MAC: Chen et al ’05, ……

n physical

TCP/ power control: Xiao et al ’01, Chiang ’04, …… C / C C e e a 05, Rate control/ routing/ scheduling: Eryilmax et al ’05, Lin et al ’05, Neely, et al ’05, Stolyar ’05, Chen, et al ’05

Survey in Proc. of IEEE, 2006

Outline

Background

L i ti i ti d iti Layering as optimization decomposition Reverse engineering TCP

Reverse engineering TCP/ IP Reverse engineering TCP/ IP

Delay insensitive utility Delay sensitive utility Delay sensitive utility How bad is single-path routing

• J. Wang, Li, Low, Doyle. ToN, 2005

g y Pongsajapan, Low, Infocom 2007

• M. Wang, Tan, Tang, Low, pre-print, 2009

Network model: general

R x y F1 G1 R

TCP N t k AQM

FN GL

TCP Network AQM

RT q p

)) ( ), ( ( ) 1 ( t x t p R F t x

T

= +

Reno, Vegas, FAST

l i Rli link uses source if 1 =

IP routing

)) ( ), ( ( ) 1 ( t Rx t p G t p = +

DropTail, RED, …

Network model: example

f or ever y RTT { W += 1 }

Reno:

(AI)

{ W += 1 } f or ever y l oss { W : = W / 2 }

(MD)

Jacobson 1989

= +

∑

i

t p R x t x ) ( 1 ) 1 (

2

AI MD

⎟ ⎞ ⎜ ⎛ − = +

∑ ∑

l l li i i

t p R T t x ) ( 2 ) 1 (

2

MD

⎟ ⎠ ⎞ ⎜ ⎝ ⎛ = +

∑

i l i li l l

t p t x R G t p ) ( ), ( ) 1 (

TailDrop

Network model: example

FAST:

per i odi cal l y {

FAST:

Jin, Wei, Low 2004

α W RTT baseRTT : W + = { }

⎟ ⎠ ⎞ ⎜ ⎝ ⎛ − + = +

∑

l li i i i i i

t p R t x T t x t x ) ( ) ( ) ( ) 1 ( α γ ⎟ ⎞ ⎜ ⎛ − + = + ⎠ ⎝

∑

l i li l l l i

c t x R t p t p T ) ( 1 ) ( ) 1 ( ⎟ ⎠ ⎜ ⎝∑

i l i li l l l

c p p ) ( ) ( ) (

) , (

* * *

x p R F x

T

=

How to characterize equilibrium of TCP

) , ( ) , (

* * *

Rx p G p p = )) ( ), ( ( ) 1 ( t x t p R F t x

T

= +

Reno, Vegas, FAST

l i Rli link uses source if 1 =

IP routing

)) ( ), ( ( ) 1 ( t Rx t p G t p = +

DropTail, RED, …

Duality model of TCP

TCP

) ( ) , (

* * * * * *

Rx p G p x p R F x

T

= =

Equilibrium (x*,p*) primal-dual optimal:

) , ( Rx p G p =

∑

F determines utility function U

c Rx x U

i i x

≤

∑

≥

subject to ) ( max

y

G guarantees complementary slackness p* are Lagrange multipliers

Kelly, Maloo, Tan 1998 L L l 1999

p

g g p

Uniqueness of equilibrium

x* is unique when U is strictly concave

Low, Lapsley 1999

x* is unique when U is strictly concave p* is unique when R has full row rank

Duality model of TCP

TCP

) ( ) , (

* * * * * *

Rx p G p x p R F x

T

= =

Equilibrium (x*,p*) primal-dual optimal:

) , ( Rx p G p =

∑

F determines utility function U

c Rx x U

i i x

≤

∑

≥

subject to ) ( max

y

G guarantees complementary slackness p* are Lagrange multipliers

Kelly, Maloo, Tan 1998 L L l 1999

p

g g p

Low, Lapsley 1999

The underlying concave program also y g p g leads to simple dynamic behavior

Duality model of TCP

Equilibrium (x*,p*) primal-dual optimal: c Rx x U

i i x

≤

∑

≥

subject to ) ( max

Mo & Walrand 2000:

⎪ ⎨ ⎧ = = 1 if log ) ( α

i

x x U ⎪ ⎩ ⎨ ≠ − =

− −

1 if ) 1 ( ) (

1 1

α α

α i i i

x x U

V FAST STCP

α = 1 : Vegas, FAST, STCP α = 1.2: HSTCP α = 2 : Reno α 2 : Reno α = : XCP (single link only)

∞

Duality model of TCP

Equilibrium (x*,p*) primal-dual optimal: c Rx x U

i i x

≤

∑

≥

subject to ) ( max

Mo & Walrand 2000:

⎪ ⎨ ⎧ = = 1 if log ) ( α

i

x x U ⎪ ⎩ ⎨ ≠ − =

− −

1 if ) 1 ( ) (

1 1

α α

α i i i

x x U

i h h

α = 0: maximum throughput α = 1: proportional fairness α = 2: min delay fairness α 2: min delay fairness α = : maxmin fairness

∞

Some implications

E ilib i Equilibrium

Always exists, unique if R is full rank Bandwidth allocation independent of AQM or arrival pattern Can predict macroscopic behavior of large scale Can predict macroscopic behavior of large scale networks

Counter-intuitive throughput behavior g p

Fair allocation is not always inefficient Increasing link capacities do not always raise h h aggregate throughput

[ Tang, Wang, Low, ToN 2006]

FAST TCP FAST TCP

Design, analysis, experiments

[ Jin, Wei, Low, Hegde, ToN 2007]

Outline

Background

L i ti i ti d iti Layering as optimization decomposition Reverse engineering TCP

Reverse engineering TCP/ IP Reverse engineering TCP/ IP

Delay insensitive utility Delay sensitive utility Delay sensitive utility How bad is single-path routing

For joint congestion control and multipath routing:

Gallager (1977), Golestani & Gallager (1980), Bertsekas, Gafni & Gallager (1984), Kelly, Maulloo & Tan (1998), Kar, Sarkar & Tassiulas (2001), Lestas ( ) y ( ) ( ) & Vinnicombe (2004), Kelly & Voice (2005), Lin & Shroff (2006), He, Chiang & Rexford (2006), Paganini (2006)

Motivation

≤

∑

c Rx x U subject to ) ( max max

: Primal

⎟ ⎞ ⎜ ⎛ ⎟ ⎞ ⎜ ⎛ ≤

∑ ∑ ∑ ∑

≥ i i i x R

R U c Rx x U i ) ( i subject to ) ( max max

D l : Primal

⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ + ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ −

∑ ∑ ∑

≥ ≥ l l l i l l li R i i i x p

c p p R x x U

i i

min ) ( max min

: Dual

Motivation

≤

∑

c Rx x U subject to ) ( max max

: Primal

⎟ ⎞ ⎜ ⎛ ⎟ ⎞ ⎜ ⎛ ≤

∑ ∑ ∑ ∑

≥ i i i x R

R U c Rx x U i ) ( i subject to ) ( max max

D l : Primal

⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ + ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ −

∑ ∑ ∑

≥ ≥ l l l i l l li R i i i x p

c p p R x x U

i i

min ) ( max min

: Dual

Shortest path Shortest path routing!

Can TCP/IP maximize utility?

Assumptions

Two timescales

TCP converges instantly TCP converges instantly Route changes slowly

Single-path shortest path routing R(t) Single-path shortest path routing R(t) Link cost: pl(t) + b τl

prop delay queueing delay

TCP/ AQM

IP

p(0) p(1)

IP

R(0)

R(1) … R(t), R(t+1) ,

…

Assumptions

Two timescales

TCP converges instantly TCP converges instantly Route changes slowly

Single-path shortest path routing R(t) Single-path shortest path routing R(t) Link cost: pl(t) + b τl

prop delay queueing delay

TCP/ AQM

IP

p(0) p(1)

will only consider b=0 or b=1 IP

R(0)

R(1) … R(t), R(t+1) ,

…

TCP/ IP dynamic model

TCP

x U t x

i i x

=

∑

≥

) ( max arg ) ( c x t R

i x

≤

≥

) ( subject to

∑ ∑

⎟ ⎠ ⎞ ⎜ ⎝ ⎛ − =

≥ ≥ l li i i i x p

p t R x x U t p ) ( ) ( max min arg ) (

∑

+ ⎠ ⎝

≥ l l i l x p

p c

i

AQM

l

Link cost slow timescale

) ) ( ( min arg ) 1 (

l l l li R i

b t p R t R

li

τ + = +

∑

IP

Reverse engineering TCP/ IP

Does equilibrium routing Rb exist ?

H t h t i

R ? How to characterize Rb? Is Rb stable ? Can it be stabilized?

TCP/ AQM

IP

p(0) p(1)

IP

R(0)

R(1) … R(t), R(t+1) ,

…

Delay insensitive utility: b=0

Theorem

If b=0, Rb exists & solves NUM iff zero duality gap

Shortest-path routing is optimal with

i i congestion prices

No penalty for not splitting

∑

P i l

Kelly’s problem solved by TCP

⎟ ⎞ ⎜ ⎛ ⎟ ⎞ ⎜ ⎛ ≤

∑ ∑ ∑ ∑

≥ i i i x R

c Rx x U i ) ( i subject to ) ( max max

D l : Primal

⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ + ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ −

∑ ∑ ∑

≥ ≥ l l l i l l li R i i i x p

c p p R x x U

i i

min ) ( max min

: Dual

Delay insensitive utility: b=0

TCP-AQM IP TCP-AQM

Applications

Q

x U

i i i x R c

∑

≥

) ( max max max

Q

IP

TCP/ AQM

B c c Rx

T i x

≤ ≤

≥

α , subject to

IP Link

TCP/ IP (with fixed c): Equilibrium of TCP/ IP exists iff zero duality gap NP hard but subclass with zero duality gap is P NP-hard, but subclass with zero duality gap is P Equilibrium, if exists, can be unstable Can stabilize, but with reduced utility

, y

Nonzero duality gap: complexity, cost of not splitting

Delay insensitive utility: b=0

TCP-AQM IP TCP-AQM

Applications

Q

x U

i i i x R c

∑

≥

) ( max max max

Q

IP

TCP/ AQM

B c c Rx

T i x

≤ ≤

≥

α , subject to

IP Link

BUT…

b is never zero in practice b is never zero in practice

If b>0 then there are networks for which equilibrium routings exist but do not maximize q g any delay insensitive utility function

Outline

Background

L i ti i ti d iti Layering as optimization decomposition Reverse engineering TCP

Reverse engineering TCP/ IP Reverse engineering TCP/ IP

Delay insensitive utility Delay sensitive utility Delay sensitive utility How bad is single-path routing

• J. Wang, Li, Low, Doyle. ToN, 2005

Pongsajapan Low Infocom 2007 Pongsajapan, Low, Infocom 2007

• M. Wang, Tan, Tang, Low, pre-print, 2009

Delay sensitive utility: b=1

( ) ( )

= d x x V d x U (

) ( )

∑

= − =

l l li i i i i i i i i

R d d x x V d x U τ ,

Link prop delay Round-trip prop delay

l

prop delay prop delay

Round trip propagation delay depends on R Delay sensitive utility function Utility from throughput … balanced by Di tilit f d l Disutility from delay

Delay sensitive utility: b=1

Theorem

If b=1, Rb exists & solves NUM iff zero duality gap

Shortest-path routing is optimal No penalty for not splitting

≤

∑

c Rx d x U subject to ) ( max max

: Primal

⎟ ⎟ ⎞ ⎜ ⎜ ⎛ + ⎟ ⎞ ⎜ ⎛ + ≤

∑ ∑ ∑ ∑

≥ i i i i x R

c p p R x d x U c Rx d x U ) ( min ) ( max min subject to ) , ( max max

: Dual : Primal

τ ⎟ ⎟ ⎠ ⎜ ⎜ ⎝ + ⎟ ⎠ ⎜ ⎝ + −

∑ ∑ ∑

≥ ≥ l l l i l l l li R i i i i x p

c p p R x d x U

i i

) ( min ) , ( max min

: Dual

τ

Counter-intuitive behavior

With delay sensitive utility

Bottleneck links can be under-utilized

There exist networks such that the TCP/IP equilibrium (x* p* R*) is in the interior: equilibrium (x , p , R ) is in the interior:

R*x* < c

c c x 2

*

< =

Equilibrium rate:

( ) ( )

, − = τ x x V d x U

) ( ' , 2 c V c = τ

c c x 2 < =

Equilibrium rate:

( ) ( )

' , = − = ∂ ∂ τ τ c V c x U

Counter-intuitive behavior

With delay sensitive utility

Extra paths that will be utilized by delay- insensitive utility functions may not

It is sub-optimal to use the long path, even when traffic is allowed to distribute over

( ) ( )

when traffic is allowed to distribute over multiple paths

τ , c

( ) ( )

τ x x V d x U − = ,

,

Equilibrium routing: use short path only

) , ( , τ τ c x U ∂ ∂ + ∞

q g p y

Counter-intuitive behavior

Any delay sensitive utility that a TCP/IP ilib i i i il equilibrium maximizes necessarily possesses

• ne of 3 “strange” properties

The specific utility has two of the 3

( ) ( )

τ x x V d x U − = ,

In contrast to joint congestion control and multi-path routing

Counter-intuitive behavior

Routing stability

Given any network, suppose

link cost: apl(t) + τl

0<a<a# is small enough 0 a a# is small enough

If every SD pair has unique min prop delay path, If every SD pair has unique min prop delay path, then TCP/IP is asymptotically stable

Routing stability

Given any network, suppose

link cost: apl(t) + τl

0<a<a# is small enough 0 a a# is small enough

Otherwise, consider a modified network in which Otherwise, consider a modified network in which every SD pair has a unique min delay path, but

link cost: pl(t)

pl( )

Then the two networks have the same equilibrium q and stability properties

Routing stability

For any delay sensitive or insensitive utility function, there exists a network such that decreasing a can destabilize TCP/IP

link cost: apl(t) + τl

a a# a1 a2

stable unstable stable

# 1 2

Outline

Background

L i ti i ti d iti Layering as optimization decomposition Reverse engineering TCP

Reverse engineering TCP/ IP Reverse engineering TCP/ IP

Delay insensitive utility Delay sensitive utility Delay sensitive utility How bad is single-path routing

• J. Wang, Li, Low, Doyle. ToN, 2005

Pongsajapan Low Infocom 2007 Pongsajapan, Low, Infocom 2007

• M. Wang, Tan, Tang, Low, pre-print, 2009

Multi-path routing

Source can split its total rate into multiple paths paths

( )

ik i i

x x x

i

,..., : rate source Total

1

=

j ij i ij

x x x j i || || : path

• multi

: path

• n

rate s '

1= ∑ ij j i j

x x max || || : path

• single

=

∞

Multi-path routing

R x U

i i i x R

∑

≥

bj ) || (|| max max

1

: path

• Multi

U c Rx ≤

∑

) || (|| subject to

: path Single

c Rx x U

i i i x R

≤

∑

∞ ≥

subject to ) || (|| max max

: path

• Single

( )

ik i i

x x x

i

,..., : rate source Total

1

=

j ij i ij

x x x j i || || : path

• multi

: path

• n

rate s '

1= ∑ ij j i j

x x max || || : path

• single

=

∞

Multi-path routing

R x U

i i i x R

∑

≥

bj ) || (|| max max

1

: path

• Multi

U c Rx ≤

∑

) || (|| subject to

: path Single

c Rx x U

i i i x R

≤

∑

∞ ≥

subject to ) || (|| max max

: path

• Single

For multi-path routing Joint routing and congestion control is a concave program (polynomial-time solvable) Zero duality gap Zero duality gap Upper bounds max utility of single-path TCP/ IP

Multi-path routing

R x U

i i i x R

∑

≥

bj ) || (|| max max

1

: path

• Multi

U c Rx ≤

∑

) || (|| subject to

: path Single

c Rx x U

i i i x R

≤

∑

∞ ≥

subject to ) || (|| max max

: path

• Single

For single-path TCP/ IP: No longer concave program; primal is NP-hard Non-zero duality gap in general Z iff TCP/ IP ilib i i t Zero gap iff TCP/ IP equilibrium exists Duality gap = cost of not splitting

Multi-path routing

R x U

i i i x R

∑

≥

bj ) || (|| max max

1

: path

• Multi

U c Rx ≤

∑

) || (|| subject to

: path Single

c Rx x U

i i i x R

≤

∑

∞ ≥

subject to ) || (|| max max

: path

• Single

Theorem For any multi-path solution (R, x), there is a multi-path solution (R’, x’) That uses no more than N+L paths That uses no more than N+L paths Achieves the same utility

Multi-path routing

R x U

i i i x R

∑

≥

bj ) || (|| max max

1

: path

• Multi

U c Rx ≤

∑

) || (|| subject to

: path Single

c Rx x U

i i i x R

≤

∑

∞ ≥

subject to ) || (|| max max

: path

• Single

Theorem Duality gap is upper bounded by

max ) , min(

i i

N L ρ =

i

ρ

Multi-path routing

R x U

i i i x R

∑

≥

bj ) || (|| max max

1

: path

• Multi

U c Rx ≤

∑

) || (|| subject to

: path Single

c Rx x U

i i i x R

≤

∑

∞ ≥

subject to ) || (|| max max

: path

• Single

Corollary For Vegas/ FAST duality gap is bounded by

K N L log max ) min( α

i i i i

x x U log ) ( α =

i i i

K N L log max ) , min( α

Conclusion & open issues

Summary

E ilib i f TCP/ IP b i t t d Equilibrium of TCP/ IP can be interpreted as maximizing network utility over rates & routes

How to reconcile TCP utility maximization y and TCP/ IP utility maximization?

Given routing, TCP utility is increasing in throughput With TCP/ IP, this is no longer the case

In gene al can/ ho e ega d la e ing In general, can/ how we regard layering as optimization decomposition?