The Prices of Packets and Watts: Optimal Operation of Decentralized - - PowerPoint PPT Presentation

▶
the prices of packets and watts optimal operation of
SMART_READER_LITE
LIVE PREVIEW

The Prices of Packets and Watts: Optimal Operation of Decentralized - - PowerPoint PPT Presentation

Jun 13, 2023 35 likes •469 views

The Prices of Packets and Watts: Optimal Operation of Decentralized Stochastic Systems P. R. Kumar Dept. of Electrical and Computer Engineering Texas A&M University With Rahul Singh and Le Xie Rahul Singh 6th IFAC Workshop on

slide-1
SLIDE 1

1/43

The Prices of Packets and Watts:
 Optimal Operation of Decentralized Stochastic Systems

  • P. R. Kumar
  • Dept. of Electrical and Computer Engineering

Texas A&M University With Rahul Singh and Le Xie

6th IFAC Workshop on Distributed Estimation and Control in Networked Systems (NecSys’16), Sep 8, 2016 Tokyo, Japan

Email: prk.tamu@gmail.com Web: http://cesg.tamu.edu/faculty/p-r-kumar/

Rahul Singh

slide-2
SLIDE 2

2/43

Two problems

◆ Optimal scheduling of unreliable networks with hard end-

to-end deadlines

◆ Optimal pricing and scheduling of generators and loads ◆ Both are stochastic distributed control problems ◆ Both have a solution based on “price” ◆ PhD Thesis of Rahul Singh, 2015

slide-3
SLIDE 3

3/43

Unreliable multi-hop networks with
 end-to-end delay constraints

slide-4
SLIDE 4

4/43

Multi-hop network

◆ F flows ◆ Flow f has an end-to-end deadline τf ◆ Let rf = packet delivery rate of flow f (Timely throughput)

τf

Flow f Delivery rate rf

slide-5
SLIDE 5

5/43

Nodal constraints

◆ Node i has an average power constraint ◆ Packet transmission succeeds with probability pij ◆ No interference: Directional antennas

i ci

pij

j

lim

T→∞

1 T # of packets transmitted by node i at time t

( )

t=1 T

∑

≤ ci

slide-6
SLIDE 6

6/43

Challenge of scheduling a distributed system

◆ Does optimal scheduling require knowledge of the complete

network state?

◆ Obtaining network state instantaneously itself requires solving

end-to-end delay problem

◆ Even if each node could obtain complete state, DP is intractable

– Huge state space: (VΔ)FΔ

◆ Is optimal scheduling of this distributed system difficult?

Congestion No Congestion Flow 2 Flow 1

Transmit packet from
 Flow 2 since Flow 1 has
 downstream congestion

slide-7
SLIDE 7

7/43

Objective

Max α frf

f

∑

Where: rf = Throughput of packets of flow f that have an end-to-end delay ≤ τ f = Timely throughput of flow f

◆ The timely throughput of flow f is weighted by αf ◆ How to schedule the network?

slide-8
SLIDE 8

8/43

Solution

Max

π

limsup

T→∞

1 T α f (# of packets of flow f delivered in time at time t)

f

∑

t=1 T

∑

Subject to lim

T→∞

1 T # of packets transmitted by node i at time t

( )

t=1 T

∑

≤ ci Max

π

limsup

T→∞

1 T α f (# of packets of flow f delivered in time at time t)

f

∑

⎧ ⎨ ⎪ ⎩ ⎪

t=1 T

∑

− λi # of packets transmitted by node i at time t

( )

t=1 T

∑

⎫ ⎬ ⎭ + λici

i

∑

◆ Constrained optimization problem over stationary randomized policies ◆ Lagrangian ◆ Packet-by-Packet Decoupling

Max

π

limsup

T→∞

1 T α f1(Packet is delivered on time)

{

Packets of flow f released before time T

∑

f

∑

− λi1(Packet is transmitted by Node i)

i

∑

⎫ ⎬ ⎭

L(π,λ) π

slide-9
SLIDE 9

9/43

Packet level decision making

◆ Packet solves Dynamic Program offline ◆ Easy to solve: Packet state space size is VΔ

Packet state (i,τ) αf Pay λi? pij i

j

V(d,t) =α f for all t ≥ 0 V(i,τ ) = Min λi + pi, jV( j,τ −1)+(1− pi, j)V(i,τ −1), V(i,τ −1)

{ }

d

α f1(Packet is delivered on time)− λi1(Packet is transmitted by Node i)

i

∑

slide-10
SLIDE 10

10/43

Optimal distributed solution

◆ The optimal solution completely decouples! ◆ Each packet makes decision to be transmitted or not,

depending only on its own state (location, time to deadline)

◆ Optimal scheduling of a packet does not depend on

– State of other nodes – State of other flows – Even other packets within its own flow!

slide-11
SLIDE 11

11/43

How to obtain prices?

◆ If price λi is too low

– Too many packets ask to be transmitted – Average power λi constraint is exceeded

◆ If price λi is too high

– Too few packets ask to be transmitted – Average power available λi is not used

◆ Suggests tatonnement ◆ But even if price is exactly right, we will need to randomize

some flow’s decisions to get the power to be exactly used up

λi

n+1 = λi n +ε [Power consumed by node i −ci].

slide-12
SLIDE 12

12/43

Dual Problem

◆ The Dual function is ◆ “Max” is attained by Single Packet Transportation

Problem

◆ Dual Problem is ◆ No Duality Gap, since can be reduced to LP

D(λ) = maxπ L(π,λ) maxλ≥0 D(λ)

slide-13
SLIDE 13

13/43

Optimality condition

◆ Suppose λ* is price vector ◆ π(λ*) optimal randomized policy for single-packet

transportation problem for each flow f

◆ Suppose at every node i,

– Either power constraint is satisfied with equality by π(λ*) – Or λi*=0

◆ Then π(λ*) and λ* are optimal by Complementary

Slackness

lim

T→∞

1 T # of packets transmitted by node i at time t

( )

t=1 T

∑

= ci

slide-14
SLIDE 14

14/43

Combine Single-Packet Transportation Problems of all Flows

◆ Very tractable LP solution: Low complexity ◆ Just variables, constraints ◆ Reduction from exponential complexity

Af

j≠i

∑

s=0 τ f

∑

f ∈F

∑

ξ f (i, j,s)E ≤ P

i

∀i ∈ V,

Power Constraint

max Af

i∈V

∑

s=0 τ f

∑

f ∈F

∑

ξ f (i,d f ,s)pi,d f

Reward

ξ f

j∈V , j≠d f

∑

( j,i,s)pj,i +

m∈V

∑

ξ f (i,m,s)(1− pi,m) = ξ f

k∈V

∑

(i,k,s −1) ∀i ≠ d f ,1≤ s ≤ τ f

Balance equations

ξ f

j∈V

∑

(s f , j,0) =1 ∀f , ξ f (i, j,s) ≥ 0

Probabilities

(VΔ)FΔ

Use state-action probabilities

|V |+ |V | FΔ+ F+ |V |2 FΔ |V |2 FΔ

slide-15
SLIDE 15

15/43

Near Optimality for Peak Power Constraint

◆ Suppose Node i can only


transmit ci packets
 concurrently

◆ Simply truncate at ci

– Similar to Whittle’s relaxation for restless bandits

◆ Theorem ◆ Policy is asymptotically optimal as the total network


 capacity is scaled by N

i ci

pij

j

O 1 N ⎛ ⎝ ⎜ ⎞ ⎠ ⎟

slide-16
SLIDE 16

16/43

Comparison with Backpressure Policy

◆ Important development in scheduling over past 25 years ◆ Max Weight and Backpressure Policies

– Tassiulas and Ephremides (1992), Neely, Modiano and Rohrs (2003), Lin and Shroff (2004), Lin, Shroff and Srikant (2006)

◆ Backpressure Policy is based on a Lagrangian decomposition of

a fluid model

◆ Fluid model is appropriate for studying throughput

– Successful design of throughput optimal policies

◆ But Delay depends on stochastic variations – needs stochastic

model

– A la difference between LLN and CLT

◆ Resulting Lagrange Multipliers are very different

– Difference in queue lengths, i.e., backpressure, for fluid model – Price for energy of transmission in stochastic model

slide-17
SLIDE 17

17/43

Example: Explicit solution

◆ Optimal solution ◆ Optimal prices ◆ Optimal solution

τ1 = τ 2 = 3 β1 = 5 β2 = 2

p(1,2)=0.4 p(2,3)=0.3 p(3,2)=0.6 p(2,1)=0.7

3 2 1

P1=0.5 P2=0.4 P3=0.5 Flow 1 Flow 2

λ = 0.068,1.4,0

( )

⇡1(1, 3) = 0.5, ⇡1(1, 2) = 0, ⇡1(2, 2) = 1, ⇡1(2, 1) = 1, ⇡2(3, 3) = 1/13, ⇡2(3, 2) = 0, ⇡2(2, 2) = 1, ⇡2(2, 1) = 1.

  • ptimal single-packet transportation dynamic

Flow 1: Node 1 transmit with
 probability 0.5 if
 time-till-deadline is 3, else drop Flow 2: Node 3 transmit with
 probability 1/13 if
 time-till-deadline is 3, else drop Node 2: Both Flows Transmit

slide-18
SLIDE 18

18/43

β1 =1 β2 =1

p(5,6)= 0.5 p(3,4)= 0.5

3 1

Flow 1 Flow 2 p(1,2) = 0.5 p(2,5)= 0.5 p(6,4)= 0.5

2 4 5 6

p(2,3)= 0.5

3 4 5 6 7 8 9 10 11 12

Relative Deadline of Flow 2

0.1 0.2 0.3 0.4 0.5 0.6 0.7

Timely Throughput

Optimal Policy EDF-BP EDF-SP

Example: Numerical Computation and Simulation Comparison

◆ Comparison with


EDF-BP
 and
 EDF-SP

◆ A1=A2=1 ◆ C(i,j) = 1 ◆ Δ1 = Δ2+1

slide-19
SLIDE 19

19/43

Remarks

◆ Issues not considered here

– Contention, interference, coding

◆ FCC announcement on July 14, 2016 ◆ 10.85 GHz spectrum in millimeter band released by FCC

– 3.85 GHz licensed – 7 GHz unlicensed

◆ Another 18GHz unlicensed proposed to be released ◆ Advanced wireless initiative in US on city-scale testbeds ◆ Wireless information era coming?

slide-20
SLIDE 20

20/43

The Independent System Operator Problem in Power Systems

slide-21
SLIDE 21

21/43

p Supply Demand Price u Supply Price Demand Price

The ISO problem in a simple static context

ISO Generator Load

◆ ISO has to balance supply and

demand

◆ Generator and load bid their

supply and demand curves

◆ ISO intersects to find right price

slide-22
SLIDE 22

22/43

Uncertainties and dynamics in the era of renewables and demand response

Coal power plant Wind farm Nuclear power plant Hydropower plant ISO Commerci al load Industrial load Load serving entity Storage service

  • Dynamic constraints: ramping, thermal inertia
  • Uncertainty: Wind, temperature, water flow
  • All choices have costs/benefits
  • How can ISO ensure maximum social welfare?
  • How much should be generated, and balance
  • How should generators and loads bid?

Time Generation Time Consumption

slide-23
SLIDE 23

23/43

The ISO problem

Generator 1 Generator
 2 Storage
 5 Load
 4 Prosumer
 6

x1(t +1) = f1(x1(t),u1(t),w1(t)) (x1(t),u1(t)) ∈ G1(t) Max E U1

t=0 T

∑

(x1(t),u1(t))

Max E Ui(xi(t),ui(t))

t=0 T

∑

i=1 N

∑

How to assign ui(t)’s? ISO Without knowledge of:

  • States xi(t)
  • Models fi(xi,ui,wi)
  • Utilities Ui(xi,ui)

Balance: ui(t)

i=1 N

∑

= 0 for all t

Load 3

x3(t +1) = f3(x3(t),u3(t),w3(t)) (x3(t),u3(t)) ∈ G3(t) Max E U3

t=0 T

∑

(x3(t),u3(t))

slide-24
SLIDE 24

24/43

Coordinating multiple deterministic dynamical systems

◆ All generators, loads, storage and prosumers are

deterministic dynamical systems

◆ Goal

xi(t +1) = fi(xi(t),ui(t)) (xi(t),ui(t)) ∈ Gi(t)

Max Ui(xi(t),ui(t))

t=0 T

∑

i=1 N

∑

s.t. ui(t)

i=1 N

∑

= 0 for all t

slide-25
SLIDE 25

25/43

The solution

◆ Lagrange Dual ◆ ISO needs to announce p = (p(0), p(1), … , p(T))

– The entire sequence of prices for all future times

◆ Agent i distributedly chooses ui = (ui(0), ui(1), … , ui(T)) to

maximize its own utility

D(p) ! Max

u

Ui(xi(t),ui(t))

t=0 T

∑

i=1 N

∑

− p(t) ui(t)

i=1 N

∑

t=0 T

∑

⎛ ⎝ ⎜ ⎞ ⎠ ⎟ Ui(xi(t),ui(t))− p(t)ui(t)

( )

t=0 T

∑

= Max

ui

Ui(xi(t),ui(t))− p(t)ui(t)

t=0 T

∑

⎛ ⎝ ⎜ ⎞ ⎠ ⎟

i=1 N

∑

Decomposes

slide-26
SLIDE 26

26/43

How does ISO choose price sequence p?

◆ The ISO needs to solve dual problem: ◆ Subgradient iteration ◆ Now ◆ Price iteration ◆ Converges after weighted averaging under convexity

and compactness assumptions (“ergodic” method)

Min

p≥0 D(p)

pk+1 = pk − ε k ∂D(pk) ∂pk pk+1 = pk + ε k ui

k(0) i=1 N

∑

, ui

k(1) i=1 N

∑

,..., ui

k(T ) i=1 N

∑

⎛ ⎝ ⎜ ⎞ ⎠ ⎟ ∂D ∂p = − ui(0)

i=1 N

∑

, ui(1)

i=1 N

∑

,..., ui(T )

i=1 N

∑

⎛ ⎝ ⎜ ⎞ ⎠ ⎟

slide-27
SLIDE 27

27/43

Convergence?

◆ Example ◆ Social welfare problem is ◆ Optimal solution is ◆ But even if ISO declares the optimal price


generator is indifferent to -1 or 0,
 and could declare 0

◆ So how can the ISO determine the correct price?

Cost = − 2

5 u1

Generator

−1≤ u1 ≤ 0

Benefit =

Load

log(1+u2) 0 ≤ u2 ≤ 2 − 1 10 u1

Carbon tax =

Min − 2 5 u1 − 1 10 u1 −log(1+u2) ⎡ ⎣ ⎢ ⎤ ⎦ ⎥

−1≤ u1 ≤ 0 0 ≤ u2 ≤ 2 u1 +u2 = 0

Subject to:

(u1

!,u2 !) = (−1,1)

and λ ! = 1 2 λ ! = 1 2 ,

slide-28
SLIDE 28

28/43

The “averaging” solution

◆ Gustavsson et al 2015: Take ◆ Then ◆ In this example:

ui

k =

sθ

s=1 k−1

∑

sθ

s=1 k

∑

ui

k−1 + kθ

sθ

s=1 k

∑

ui

k;

ui

0 = ui

ui

k → ui !

{λk} :0,0,1,0.6667,0.5416,…→ 1 2 {uk} : −1 ⎡ ⎣ ⎢ ⎤ ⎦ ⎥, 2 ⎡ ⎣ ⎢ ⎤ ⎦ ⎥, −0.94 0.12 ⎡ ⎣ ⎢ ⎤ ⎦ ⎥, −0.99 0.41 ⎡ ⎣ ⎢ ⎤ ⎦ ⎥, −0..997 0.73 ⎡ ⎣ ⎢ ⎤ ⎦ ⎥…→ −1 1 ⎡ ⎣ ⎢ ⎤ ⎦ ⎥ θ ≥ 0 and λk → λ !

slide-29
SLIDE 29

29/43

Convergence

◆ General result ◆ ci, ei, hi convex, compactness, optimal exists ◆ Slater’s condition ◆ Then convergence after averaging

min [ci(ui)+ei(ui)] i=1 M

∑

F

iui ≤ gi,hi(ui) ≤ 0

Ei

i=1 M

∑

ui = d

slide-30
SLIDE 30

30/43

In the deterministic case:
 Price sequences and Bid sequences

ISO Coal power plant Wind farm Nuclear power plant Hydropower plant Commerci al load Industrial load Load serving entity Storage service

(p1(0),….,p1(T)) (p1(0),….,p1(T)) (p1(0),….,p1(T)) (p1(0),….,p1(T)) (p1(0),….,p1(T)) (p1(0),….,p1(T)) (p1(0),….,p1(T)) (p1(0),….,p1(T)) (u1(0),….,u1(T)) (u1(0),….,u1(T)) (u1(0),….,u1(T)) (u1(0),….,u1(T)) (u1(0),….,u1(T)) (u1(0),….,u1(T)) (u1(0),….,u1(T)) (u1(0),….,u1(T)) (p2(0),….,p2(T)) (p2(0),….,p2(T)) (p2(0),….,p2(T)) (p2(0),….,p2(T)) (p2(0),….,p2(T)) (p2(0),….,p2(T)) (p2(0),….,p2(T)) (p2(0),….,p2(T)) (u2(0),….,u2(T)) (u2(0),….,u2(T)) (u2(0),….,u2(T)) (u2(0),….,u2(T)) (u2(0),….,u2(T)) (u2(0),….,u2(T)) (u2(0),….,u2(T)) (u2(0),….,u2(T)) (p3(0),….,p3(T)) (p3(0),….,p3(T)) (p3(0),….,p3(T)) (p3(0),….,p3(T)) (p3(0),….,p3(T)) (p3(0),….,p3(T)) (p3(0),….,p3(T)) (p3(0),….,p1(T)) (u3(0),….,u3(T)) (u3(0),….,u3(T)) (u3(0),….,u3(T)) (u3(0),….,u3(T)) (u3(0),….,u3(T)) (u3(0),….,u3(T)) (u3(0),….,u3(T)) (u3(0),….,u3(T))

slide-31
SLIDE 31

31/43

…... until convergence

slide-32
SLIDE 32

32/43

Coordinating multiple stochastic dynamical systems with common uncertainty

◆ Generators, loads, storage, prosumers are stochastic


– dependent on a common uncertainty

◆ Ex: All loads depend on common temperature of the city ◆ Common uncertainty w(Ÿ) is observed by all loads/gens ◆ Goal

Max Ew(⋅) Ui(xi(t),ui(t))

t=0 T

∑

i=1 N

∑

⎛ ⎝ ⎜ ⎞ ⎠ ⎟ xi(t +1) = fi(xi(t),ui(t),w(t)) (xi(t),ui(t)) ∈ Gi(t)

slide-33
SLIDE 33

33/43

Uncertainty tree

◆ Uncertainty at each stage ◆ Tree policy: Choose action


u(v) for each vertex v

◆ Optimization problem

Min p(v)c(x(v,uv),uv)

v

∑

s.t. ui(v) = 0 for all v

i=1 N

∑

w(1) w(0) w(2) w(3)

v ◆ At time 0, bid-price iteration for all nodes until convergence ◆ Only needs to be done at time 0 ◆ Very complex since number of loads is exponentially large

slide-34
SLIDE 34

34/43

The stochastic case with private uncertainties

◆ Generators, loads, storage and prosumers are

stochastic having private uncertainties

◆ Goal: Max Ew1(⋅),w2 (⋅),...,wN (⋅)

Ui(xi(t),ui(t))

t=0 T

∑

i=1 N

∑

⎛ ⎝ ⎜ ⎞ ⎠ ⎟ xi(t +1) = fi(xi(t),ui(t),wi(t)) xi(t) ∈Xi(t) ui(t) ∈Ui(t)

slide-35
SLIDE 35

35/43

Bid-Price iteration for private stochastic case

◆ At time 0: bid-price iteration for all nodes ◆ At time 1: bid-price iteration for subtree ◆ … ◆ ISO needs to know and announce

– Labels of remaining subtree

◆ Agents need to know

– Laws of all labels – Labels could be hashed for confidentiality

◆ Agent i communicates


label of wi(t) to ISO at time t

◆ Same complexity as before ◆ Also repeated at each stage ◆ Not entirely satisfactory v v v

slide-36
SLIDE 36

36/43

A tractable solution:
 Generators and loads modeled as LQG systems

slide-37
SLIDE 37

37/43

Multiple LQG Systems with private uncertainty

◆ All generators and loads are LQG systems having

private uncertainties

◆ Goal:

Max E xi(t)T Qixi(t)+ui(t)T Riui(t)

t=0 T

∑

i=1 N

∑

⎛ ⎝ ⎜ ⎞ ⎠ ⎟ xi(t +1) = Aixi(t)+ biui(t)+ Ciwi(t) yi(t) = Dixi(t)+ Hivi(t) x0,wi(t),vi t

( ) ∼ N, mean 0, and independent

slide-38
SLIDE 38

38/43

Scheme for distributed LQG systems

◆ At each time t = 0, 1, 2, …

– Iterations k=1,2,3,…: » ISO announces future price sequence (pk (t),,, pk (T)) » Each Generator/Load i responds with optimal solution
 (ui

k (t), ui k (t+1), …, ui k (T)) of deterministic LQ problem

◆ Adequate number of iterations at each stage

pk = pk−1 + ε k−1 ui

k−1(t) i=1 N

∑

, ui

k−1(t +1) i=1 N

∑

,..., ui

k−1(T ) i=1 N

∑

⎛ ⎝ ⎜ ⎞ ⎠ ⎟ xi(t +1) = Aixi(t)+biui(t) Min xi(t)T Qixi(t)+ui(t)T Riui(t)+ pk(t)ui(t)

t=0 T

∑

⎛ ⎝ ⎜ ⎞ ⎠ ⎟

slide-39
SLIDE 39

39/43

Future price sequences and
 Future bid sequences at each time t

Coal power plant Wind farm Nuclear power plant Hydropower plant ISO Commerci al load Industrial load Load serving entity Storage service

(p1(t),….,p1(T)) (p1(t),….,p1(T)) (p1(t),….,p1(T)) (p1(t),….,p1(t)) (p1(t),….,p1(T)) (p1(t),….,p1(T)) (p1(t),….,p1(T)) (p1(t),….,p1(T))

slide-40
SLIDE 40

40/43

Future price sequences and
 Future bid sequences at each time t

Coal power plant Wind farm Nuclear power plant Hydropower plant ISO Commerci al load Industrial load Load serving entity Storage service

(u1(t),….,u1(T)) (u1(t),….,u1(T)) (u1(t),….,u1(T)) (u1(t),….,u1(T)) (u1(t),….,u1(T)) (u1(t),….,u1(T)) (u1(t),….,u1(T)) (u1(t),….,u1(T))

slide-41
SLIDE 41

41/43

Generator
 2 Storage
 5 Load
 4 Prosumer
 6 Generator 1

x1(t +1) = f1(x1(t),u1(t),w1(t)) u1(t) ∈U1(t), x1(t) ∈X1(t) Max E U1

t=1 T

∑

(x1(t),u1(t))

Load 3

x3(t +1) = f3(x3(t),u3(t),w3(t)) u3(t) ∈U3(t), x3(t) ∈X3(t) Max E U3

t=1 T

∑

(x3(t),u3(t))

Other electrical network constraints: Power flow, etc

Can handle multiple linear equalities h11(t)u1(t)+...+ h1NuN (t) = α1(t) ... hm1(t)u1(t)+...+ hmNuN (t) = α m(t)

◆ Can incorporate “DC” Power Flow Equations ◆ Provides Location Marginal Prices in dynamic stochastic

context

slide-42
SLIDE 42

42/43

Remarks

◆ With 15 min intervals, is such an iterative bidding feasible? ◆ Even if correct prices are known, we still need “exploration” to

determine allocations

◆ Is there a better alternative to get to optimal social welfare? ◆ Should we get to social welfare? ◆ Ongoing work

– Strategic considerations in bidding

◆ Many other issues

– Line limits – Line losses – AC power flow equations – Security constrained OPF

slide-43
SLIDE 43

43/43

Thank you