Bilevel Optimization, Pricing Problems and Stackelberg Games - - PowerPoint PPT Presentation

Bilevel Optimization, Pricing Problems and Stackelberg Games

Martine Labbé Computer Science Department Université Libre de Bruxelles INOCS Team, INRIA Lille

CO Workshop - Aussois - January 2017

1

Follower Leader

PART I: Bilevel optimization

CO Workshop - Aussois - January 2017

2

Bilevel Optimization Problem

CO Workshop - Aussois - January 2017

max

x,y

f(x, y) s.t. (x, y) ∈ X y ∈ S(x) where S(x) = argmax

y

g(x, y) s.t.(x, y) ∈ Y

3

Adequate framework for Stackelberg game

CO Workshop - Aussois - January 2017

• Leader: 1st level,
• Follower: 2nd level,
• Leader takes follower’s optimal reaction

into account.

4

(1905 - 1946)

Early papers on bilevel

• ptimization

CO Workshop - Aussois - January 2017

5

• Falk (1973): Linear min-max problem
• Bracken & McGill (1973): First bilevel model, structural

properties, military application.

Applications

CO Workshop - Aussois - January 2017

6

• Economic game theory
• Production planning
• Revenue management
• Security
• …

Example: a linear BP

CO Workshop - Aussois - January 2017

7

max

x,y

f1x + f2y s.t. max

y

g1x + g2y s.t.(x, y) ∈ Y

Y g

Inducible region (IR)

f x y OS

Coupling constraints

CO Workshop - Aussois - January 2017

8

Y g f x y X

Infeasible BP

The follower sees only the second level constraints

Y g f x y X

Multiple second level optima

CO Workshop - Aussois - January 2017

9

max

x≥0

xy s.t. max

y (1 − x)y

s.t.0 ≤ y ≤ 1

Y x y f

Y x y f

Y x y f

“Optimistic” “Pessimistic”

Multiple followers

independent

CO Workshop - Aussois - January 2017

10

max

x

f(x, y1, . . . , yn) s.t. (x, y1, . . . , yn) ∈ X max

yk g(x, yk), k = 1, . . . , n

s.t.(x, yk) ∈ Y k

Multiple followers

dependent

CO Workshop - Aussois - January 2017

11

max

x

f(x, y1, . . . , yn) s.t. (x, y1, . . . , yn) ∈ X max

yk g(x, yk, y k), k = 1, . . . , n

s.t.(x, yk, y k)) ∈ Y k

Linear BP

CO Workshop - Aussois - January 2017

12

[ ]

}S }T

max

x

c1x + d1y s.t. A1x + A1y ≤ b1 max

y

c2x + d2y, s.t.A2x + B2y ≤ b2

Linear BP

CO Workshop - Aussois - January 2017

13

• Linear BP is strongly NP-hard (Hansen et al. 1992)
• MILP is a special case of Linear BP
• IR is not convex and may be disconnected.

x ∈ {0, 1} ⇔ v = 0 and v = argmax

w

{w : w ≤ x, w ≤ 1 − x, w ≥ 0}

Linear BP

CO Workshop - Aussois - January 2017

14

• IR is the union of faces of S∩T
• If Linear BP is feasible, then there exists an optimal

solution which is a vertex of S∩T. (Bialas & Karwan(1982), Bard(1983)). K-th best algorithm

Linear BP- single level reformulation

CO Workshop - Aussois - January 2017

15

max

x

c1x + d1y s.t. A1x + B1y ≤ b1 max

y

d2y, s.t. B2y ≤ b2 − A2x (λ)

max

x

c1x + d1y s.t. A1x + B1y ≤ b1 B2y ≤ b2 − A2x λB2 = d2 λ(B2y − b2 + A2x) = 0 λ ≥ 0

• Branch & Bound (Hansen et al.1992)
• Branch & Cut (Audet et al. 2007)

Linear BP - Other approaches

CO Workshop - Aussois - January 2017

16

• Complementary pivoting (Bialas & Karwan 1984)
• Penalty functions (Anandalingam & White 1989)
• Reverse convex programming (Al-Khayyal 1992)

Mixed Integer Linear BP

CO Workshop - Aussois - January 2017

17

• -hard (Lodi et al. 2014)
• Branch & Bound (Moore & Bard 1990, Fischetti et al.

2016)

• Branch & Cut (DeNegre & Ralphs 2009, Fischetti et
• al. 2016)
• Branch & Cut & Price (Han & Zeng 2014)

PP

2

Some references

CO Workshop - Aussois - January 2017

18

• L.N. Vincente and P.H. Calamai (1994), Bilevel and multilevel programming : a

bibiliography review, J. Global Optim. 5, 291-306.

• S. Dempe. Foundations of bilevel programming. In Nonconvex optimization

and its applications, volume 61. Kluwer Academic Publishers, 2002.

• J. Bard. Practical Bilevel Optimisation: Algorithms and Applications.

KluwerAcademic Publishers, 1998

• B. Colson, P. Marcotte, and G. Savard. Bilevel programming: A survey.

4 OR, 3:87-107, 2005.

• M. Labbé and A. Violin. Bilevel programming and price setting problems.

4OR, 11:1-30, 2013.

PART II: Pricing problems

CO Workshop - Aussois - January 2017

19

Adequate framework for Price Setting Problem

CO Workshop - Aussois - January 2017

max

T ∈Θ,x,y

F(T, x, y) s.t. min

x,y f(T, x, y)

s.t.(x, y) ∈ Π

20

Applications

CO Workshop - Aussois - January 2017

21

Price Setting Problem with linear constraints

CO Workshop - Aussois - January 2017

22

max

T,x,y

Tx s.t. TC ≥ f min

x,y

(c + T)x + dy s.t. Ax + By ≥ b

• Π = {x, y : Ax + By ≥ b} is bounded
• {(x, y) ∈ Π : x = 0} is nonempty

Example: 2 variables in second level

CO Workshop - Aussois - January 2017

23

max

T,x,y

Ty s.t. min

x,y

c1x + (c2 + T)y s.t. (x, y) ∈ Π

CO Workshop - Aussois - January 2017

24

Example: 2 variables in second level

The first level revenue

CO Workshop - Aussois - January 2017

25

Network pricing problem

CO Workshop - Aussois - January 2017

• network with toll arcs (A1) and non toll arcs (A2)
• Costs ca on arcs
• Commodities (ok, dk, nk)
• Routing on cheapest (cost + toll) path
• Maximize total revenue

26

Example

CO Workshop - Aussois - January 2017

• UB on (T1 + T2) = SPL(T = ∞) − SPL(T = 0) = 22 − 6 = 16
• T2,3 = 5, T4,5 = 10

1 2 3 4 5 10 9 2 2 2 12

27

Example with negative toll arc

CO Workshop - Aussois - January 2017

28

6 2 2 1 2 3

T12 = 4 T23 = −2 T34 = 4

4

Network pricing problem

(Labbé et al., 1998, Roch at al., 2005)

CO Workshop - Aussois - January 2017

29

• Strongly NP-hard even for only one commodity.
• Polynomial for

– one commodity if lower level path is known – one commodity if toll arcs with positive flows are known – one single toll arc.

• Polynomial algorithm with worst-case guarantee of (log |A1|)/2 + 1

Network pricing problem

CO Workshop - Aussois - January 2017

max

T ≥0

X

a∈A1

Ta X

k∈K

nkxk

a

min

x,y

X

k∈K

( X

a∈A1

(ca + Ta)xk

a +

X

a∈A2

caya) s.t. X

a∈i+

(xk

a + yk a) −

X

a∈i−

(xk

a + yk a) = bk i

∀k, i xk

a, yk a ≥ 0,

∀k, a

30

CO Workshop - Aussois - January 2017

31

NPP: single level reformulation

31

max

T,x,y,λ

X

k∈K

nk X

a∈Ak

Taxk

a

s.t. X

a∈i+

(xk

a + yk a) −

X

a∈i−

(xk

a + yk a) = bk i

∀k, i λk

i − λk j ≤ ca + Ta

∀k, a ∈ A1, i, j λk

i − λk j ≤ ca

∀k, a ∈ A2, i, j X

a∈A1

(ca + Ta)xk

a +

X

a∈A2

caya = λk

• k − λk

dk

∀k xk

a, yk a ≥ 0

∀k, a Ta ≥ 0 ∀a ∈ A1

Solution approach by Branch & Cut

CO Workshop - Aussois - January 2017

32

• Formulate NPP as MIP
• Tight bound (Mak,Na) on tax, if arc used and if arc not used,

very effective

• Add valid inequalities to strengthen LP relaxation

Product pricing

CO Workshop - Aussois - January 2017

33

Seller Consumers pi nk Rk

i

Rk

i is the reservation price of consumer k for product i

Product pricing

CO Workshop - Aussois - January 2017

34

• PPP is Strongly NP-hard even if reservation price is

independent of product (Briest 2006)

• PPP is polynomial for one product or one customer.

PPP - bilevel formulation

CO Workshop - Aussois - January 2017

35

max

p≥0

X

k∈K

nk X

i∈I

pixk

i

s.t. max

xk

X

i∈I

(Rk

i − pi)xk i ,

k ∈ K s.t. X

i∈I

xk

i ≤ 1

xk

i ≥ 0

PPP - single level formulation

CO Workshop - Aussois - January 2017

36

max

p≥0

X

k∈K

nk X

i∈I

pixk

i

s.t. X

i∈I

(Rk

i − pi)xk i ≥ Rk j − pj,

j ∈ I, k ∈ K X

i∈I

(Rk

i − pi)xk i ≥ 0,

k ∈ K X

i∈I

xk

i ≤ 1

xk

i ≥ 0

PPP: MILP formulation

(Heilporn et al., 2010, 2011)

CO Workshop - Aussois - January 2017

37

• Linearize single level formulation⇒ MILP
• Add new FDI’s ⇒ convex hull for k=1
• FDI’s added ⇒ divides the gap by 2 to 4

PART III: Stakelberg games

CO Workshop - Aussois - January 2017

38

Bimatrix game

CO Workshop - Aussois - January 2017

39

Leader Follower

(2,1) (1,0) (4,0) (3,2) A B C D

0.5

A Stackelberg solution to the game (B,D) yielding a payoff of (3.5,1)

0.5 Follower Pure Strategy Leader Mixed Strategy

Stackelberg vs Nash

CO Workshop - Aussois - January 2017

40 Player 2 - A Player 2 - B Player 1 - A (2,2) (4,1) Player 1 - B (1,0) (3,1)

Nash equilibrium: Player 1-A and Player 2-A => (2,2) Stackelberg solution: Player 1-B and Player 2-B => (3,1) Nash equilibrium may not exist There is always an (optimistic) Stackelberg solution

CO Workshop - Aussois - January 2017

41

Stackelberg Games

Leader Follower

(R,C)

Stackelberg Game p-Followers Stackelberg Game

(Conitzer & Sandblom, 2006)

Leader Type 1

Objective of the Game

• Reward-maximizing strategy for the Leader.
• Follower will best respond to observable Leader’s strategy.

Follower Type 2 Type 3 Type p

…

Applications

(Tambe et al., USC)

CO Workshop - Aussois - January 2017

42

CO Workshop - Aussois - January 2017

43

The beauty of this approach comes from randomization

1-Follower General Stackelberg game

CO Workshop - Aussois - January 2017

44

• Follower optimally chooses one strategy j with probability 1
• For each possible strategy j of the follower, determine the probabilities xi

that leader chooses strategy i by solving the LP:

max X

i∈I

Rijxi s.t. X

i∈I

xi = 1 xi ≥ 0 X

i∈I

Cijxi ≥ X

i∈I

Cilxi, ∀l ∈ J

Modeling a p-Followers General Stackelberg Game

CO Workshop - Aussois - January 2017

45

Follower type k ∈ K and π ∈ [0, 1] xi = probability with which the Leader plays pure strategy i

x ∈ S|I| := {x ∈ [0, 1]|I| : X

i∈I

xi = 1}

qk ∈ S|J| := {q ∈ [0, 1]|J| : X

j∈J

qj = 1}, ∀k ∈ K

qk

j = probability with which type k Follower plays pure strategy j

Rk, Ck ∈ R|I|×|J|, ∀k ∈ K

Bilevel formulation

CO Workshop - Aussois - January 2017

46

(BIL-p-G) Maxx,q X

i∈I

X

j∈J

X

k∈K

πkRk

ijxiqk j

s.t. X

i∈I

xi = 1, xi ∈ [0, 1] ∀i ∈ I, qk = arg maxrk 8 < : X

i∈I

X

j∈J

Ck

ijxirk j

9 = ; ∀k ∈ K, rk

j ∈ [0, 1]

∀j ∈ J, ∀k ∈ K, X

j∈J

rk

j = 1

∀k ∈ K.

Bilinear formulation

Paruchuri et al.(2008)

CO Workshop - Aussois - January 2017

47

(QUAD) max

x,q,a

X

i∈I

X

j∈J

X

k∈K

πkRk

ijxiqk j

s.t. X

i∈I

xi = 1, X

j∈J

qk

j = 1

∀k ∈ K, 0 ≤ (ak − X

i∈I

Ck

ijxi) ≤ (1 − qk j )M

∀j ∈ J, ∀k ∈ K, xi ∈ [0, 1] ∀i ∈ I, qk

j ∈ {0, 1}

∀j ∈ J, ∀k ∈ K, ak ∈ R ∀k ∈ K.

Linearize

CO Workshop - Aussois - January 2017

48

xiqk

j = zk ij, ∀i ∈ I, j ∈ J, k ∈ K

• zk

ij ∈ [0, 1], ∀i ∈ I, j ∈ J, k ∈ K

• xi = P

j∈J

zk

ij, ∀i ∈ I, k ∈ K

• qk

j = P i∈I

zk

ij, ∀j ∈ J

12

MIP-p-G

CO Workshop - Aussois - January 2017

49

(MIP-p-G) max

x,q

X

i∈I

X

j∈J

X

k∈K

⇡kRk

ijzk ij

s.t. X

i∈I

X

j∈J

zk

ij = 1,

∀k ∈ K X

i∈I

(Ck

ij − Ck i`)zk ij ≥ 0

∀j, ` ∈ J, ∀k ∈ K, zk

ij ≥ 0

∀i ∈ I, ∀j ∈ J, ∀k ∈ K, X

i∈I

zk

ij ∈ {0, 1}

∀j ∈ J, ∀k ∈ K, X

j∈J

zk

ij =

X

j∈J

z1

ij

∀i ∈ I, ∀k ∈ K.

Computational comparison

Casorran et al.(2016)

CO Workshop - Aussois - January 2017

50

0.01 0.1 1 10 100 1000 10 20 30 40 50 60 70 80 90 100

Time vs. % of problems solved

0.01 0.1 1 10 20 30 40 50 60 70 80 90 100

LP TIme vs. % of problems solved

1 10 100 1000 10000 100000 1000000 10 20 30 40 50 60 70 80 90 100

Nodes vs. % of problems solved

0.01 0.1 1 10 100 1000 10 20 30 40 50 60 70 80 90 100

%Gap vs. % of problems solved

D2 FMD2 DOBSS FMDOBSS MIPpG

2: GSGs: I={10,20,30}, J={10,20,30}, K={2,4,6}–with variability.

(D2) (FMD2) (DOBSS) (FMDOBSS) (MIP-p-G) Mean Gap % 110.56 110.56 31.88 30.64 7.56

Stackelberg security game

CO Workshop - Aussois - January 2017

51

• Payoffs depend only on which target is attacked and whether it is covered
• r not
• Covered

Uncovered Defender Dk(j|c) Dk(j|u) Attacker Ak(j|c) Ak(j|u)

Compact representation of Stackelberg Security Games

CO Workshop - Aussois - January 2017

52

• Resources-Targets settings can be modeled as a Stackelberg Game BUT

if m ressources and n targets then n

m

• pure strategies!
• Stackelberg Security Games can be more compactly represented.
• Solve for optimal coverage probabilities of the targets.

Stackelberg security game:

“extended formulation”

CO Workshop - Aussois - January 2017

53

(QUAD) max

x,q,a

X

k∈K

πk X

j∈J

qk

j (Dk(j|c)

X

i∈I:j∈i

xi + Dk(j|u) X

i∈I:j / ∈i

xi) s.t. X

i∈I

xi = 1, X

j∈J

qk

j = 1

∀k ∈ K, 0 ≤ ak − (Ak(j|c) X

i∈I:j∈i

xi + Ak(j|u) X

i∈I:j / ∈i

xi) ≤ (1 − qk

j )M

∀j ∈ J, ∀k ∈ K, xi ∈ [0, 1] ∀i ∈ I, qk

j ∈ {0, 1}

∀j ∈ J, ∀k ∈ K, ak ∈ R ∀k ∈ K.

Stackelberg security game:

“extended formulation”

CO Workshop - Aussois - January 2017

54

cj 1 - cj

(QUAD) max

x,q,a

X

k∈K

πk X

j∈J

qk

j (Dk(j|c)

X

i∈I:j∈i

xi + Dk(j|u) X

i∈I:j / ∈i

xi) s.t. X

i∈I

xi = 1, X

j∈J

qk

j = 1

∀k ∈ K, 0 ≤ ak − (Ak(j|c) X

i∈I:j∈i

xi + Ak(j|u) X

i∈I:j / ∈i

xi) ≤ (1 − qk

j )M

∀j ∈ J, ∀k ∈ K, xi ∈ [0, 1] ∀i ∈ I, qk

j ∈ {0, 1}

∀j ∈ J, ∀k ∈ K, ak ∈ R ∀k ∈ K.

Stackelberg security game:

compact formulation

CO Workshop - Aussois - January 2017

55

(QUAD) max

x,q,a

X

k∈K

πk X

j∈J

qk

j (Dk(j|c)cj + Dk(j|u)(1 − cj))

s.t. X

i∈I

xi = 1, X

i:j∈i

xi = cj ∀j ∈ J, xi ∈ [0, 1] ∀i ∈ I, X

j∈J

qk

j = 1

∀k ∈ K, 0 ≤ ak − (Ak(j|c)cj + Ak(j|u)(1 − cj) ≤ (1 − qk

j )M

∀j ∈ J, ∀k ∈ K, qk

j ∈ {0, 1}

∀j ∈ J, ∀k ∈ K, ak ∈ R ∀k ∈ K.

Stackelberg security game

compact formulation

CO Workshop - Aussois - January 2017

56

(SECU-K-Quad) Maxc X

k∈K

X

j∈J

pk(qk

j (cjDk(j|c) + (1 − cj)Dk(j|u)))

s.t. cj ∈ [0, 1] ∀j ∈ J X

j∈J

cj ≤ m, qk

j (cjAk(j|c) − (1 − cj)Ak(j|u)) ≥ qk j (ctAk(t|c) − (1 − ct)Ak(t|u))

∀k ∈ K, qk

j ∈ {0, 1}

∀j ∈ J, ∀k ∈ K X

j∈J

qk

j = 1

∀k ∈ K.

Stackelberg security game: MIP3-compact

CO Workshop - Aussois - January 2017

57

(SECU-p-MIP) Maxy X

k∈K

X

j∈J

pk(Dk(j|c)yk

jj + Dk(j|u)(qk j − yk jj))

s.t. X

l∈J

yk

lj ≤ mqk j

∀k, j, 0 ≤ yk

lj ≤ qk j ,

∀k, j X

j∈J

qk

j = 1,

∀k, Ak(j|c)yk

jj + Ak(j|u)(qk j − yk jj) − A(l|c)yk lj

− A(l|u)(qk

l − yk lj) ≥ 0

∀j, l, k, X

l∈J

yk

lj ∈ {0, 1}

∀l, k, X

j∈J

yk

lj =

X

j∈J

y1

lj

∀l, k.

Link between MIP-p-G and SECU-p-MIP

CO Workshop - Aussois - January 2017

58

• A pure strategy of the defender is a set of at most m targets
• yk

hj = P i∈I:h∈i zk ij

• Proj(LP(PMIP 3)) ⊂ LP(PSECU−p−MIP )

Computational comparison

(Casorran et al. 2016)

CO Workshop - Aussois - January 2017

59

0.1 1 10 100 1000 10000 10 20 30 40 50 60 70 80 90 100

Time vs. % of problems solved

Ln(Time)

0.01 0.1 1 10 100 10 20 30 40 50 60 70 80 90 100

LP time vs. % of problems solved

Ln(LPTime) % of problems solved

1 10 100 1000 10000 100000 1000000 10000000 10 20 30 40 50 60 70 80 90 100

Nodes vs. % of problems solved

Ln(Nodes)

0.1 1 10 100 1000 10 20 30 40 50 60 70 80 90 100

Gap% vs. % of problems solved

Ln(Gap%) % of problems solved

ERASER SDOBSS MIPpS

(ERASER) (SDBOSS) (MIP-p-S) Mean Gap % 204.82 28.76 1.72

SSGs: |K| ∈ {4, 6, 8, 12}, |J| ∈ {30, 40, 50, 60, 70}, m ∈ {0.25|J|, 0.50|J|, 0.75|J|}

RECAP

CO Workshop - Aussois - January 2017

60

Pricing problems Second level: LP Second level: shortest path Second level: “One out of N” Bilevel bilinear optimization Stackelberg games

CO Workshop - Aussois - January 2017

61

RECAP

61

Pricing problems Second level: LP Second level: shortest path Second level: “One out of N” Bilevel optimization Stackelberg games

Single level nonlinear reformulation

CO Workshop - Aussois - January 2017

62

RECAP

62 62

Pricing problems Second level: LP Second level: shortest path Second level: “One out of N” Bilevel optimization Stackelberg games

Single level nonlinear reformulation

MIP

CO Workshop - Aussois - January 2017

63

2nd IWOBIP - International Workshop in Bilevel Programming Lille, June 18-22, 2018