[PPT] - Combinatorial Optimization at Work 2020 Traffic Optimization Part PowerPoint Presentation

SLIDE 1

Combinatorial Optimization at Work 2020 Traffic Optimization Part I: Paths & Lagrange Relaxation Part II: Vehicles & Crewsfdafdsafdsfl Part III: Pollsters & Vehiclesfdsafdkfll

Zuse Institute Berlin, 22.09.2020 Ralf Borndörfer Güvenç Şahin Luis Torres

1 Şahin | Traffic Optimization I | CO@Work 2020

SLIDE 2

Traffic of the Future

▪ Needs data and communication to assess and control system status ▪ Needs mathematics to find smart solutions

Şahin | Traffic Optimization I | CO@W 2020 2

no pollution reliable, comfortable intermodal fast, efficient information, control

SLIDE 3

Planning Problems in Public Transit

Şahin | Traffic Optimization I | CO@W 2020 3

Service Design Operational Planning Operations Control Passenger Information

SLIDE 4

DB Bahn Fernverkehr AG | Fahrplan und Zugfahrt | 4th of July 2013

4

Vehicle Rotation Planning is a Crucial Part of the Production Planning Process

Customer

◼ Market research ◼ Demand forecast ◼ Passenger counting

– … ◼ How can customer-friendly time tables be realised ◼ How can conflicts with other trains be avoided? ◼ How can vehicles be employed efficiently? ◼ How can plans be made maintenance friendly and

robust?

◼ How can maintenance capacity be used most

efficiently

◼ How can the availability of vehicles be maximised?

◼ Attractive timetable ◼ Reliability ◼ High product quality

◼ How can shifts be planned staff friendly? ◼ How can high productivity of personnel be

btained?

◼ Which connections correspond best to demand? ◼ What capacity is needed? ◼ What kind of product should be offered? ◼ How can vehicles be stabled, shunted, catered,

cleaned etc. between maintenance and commercial service?

Customer

Vehicle Rotation Planning Maintenance Planning Train Preparation Planning Marketing Train Path Planning Staff Planning

Slide of DB

SLIDE 5

Şahin | Traffic Optimization I | CO@W 2020 5

Slide of IVU

SLIDE 6

29 Sep 2005 Şahin | Traffic Optimization I | CO@W 2020 Optimization in NetLine/Crew Chart 6

Workflow Oriented and Integrated Optimization: How fast business processes can follow IT?

Pairing Generation

Control Planning

Pairing & Roster Maintenance Crew Assignment Hotel, DH, PickUp, Administration Crew Tracking Maintenance Planning

Operations Management Process

Tail Assignment Maintenance Control Flight Dispatch Movement Control Hub Control

Crew Management Process

Control Planning

Integrated Recovery

f

Aircraft and Crew Integrated Optimization

f the Ressource

Aircraft and Crew

Slide of LSB

Chart 6

SLIDE 7

Planning Problems in Public Transit

Şahin | Traffic Optimization I | CO@W 2020 7

Service Design Operational Planning Operations Control Passenger Information

SLIDE 8

The Shortest Path Problem

306 nodes, 445 edges

Şahin | Traffic Optimization I | CO@W 2020 8

Alexanderplatz ZIB

SLIDE 9

Shortest Path Problem

1 Def. (Single Source Shortest (st-)Path Problem (ShPP)): Let 𝐻 = (𝑊, 𝐹, 𝑑) be a (un)directed graph on 𝑜 nodes with edge weights 𝑑 ∈ ℝ≥0

𝐹 , 𝑡, 𝑢 ∈ 𝑊 two nodes, 𝑄 𝑡𝑢 𝐸 = 𝑄 𝑡𝑢 the set of all st-paths in D.

min 𝑑(𝑄) , 𝑄 ∈ 𝑄𝑡𝑢

shortet (st)-path problem (ShPP) a) ShPP conservative ∶⟺ 𝑑 𝐷 ≥ 0 ∀ (di)cycles 𝐷 ⊆ 𝐹 b) ShPP metric ∶⟺ 𝑑𝑣𝑤 + 𝑑𝑤𝑥 ≥ 𝑑𝑣𝑥 ∀𝑣𝑤, 𝑤𝑥, 𝑣𝑥 ∈ 𝐹 (Δ-inequality) c) ShPP Euclidean ∶⟺ 𝑑𝑣𝑤 = 𝑣 − 𝑤

2 ∀𝑣𝑤 ∈ 𝐹 ⊆ ℝ2

2 Obs. (Simplicity): A conservative ShPP has a simple optimal solution (no node repetitions).

Şahin | Traffic Optimization I | CO@W 2020 9

𝑣 𝑤 𝑥

Δ-inequality simple/non-simple path

cycle 𝐷

SLIDE 10

3 Alg. (Dijkstra's Algorithm, Dijkstra [1959]): Input: 𝐻 = 𝑊, 𝐹, 𝑑 , 𝑡, 𝑢 ∈ 𝑊, 𝑑 ∈ ℝ≥0

𝐹

Output: 𝑄 ∈ Argmin

𝑄∈𝑄𝑡𝑢

𝑑 𝑄

Data Structures: 𝑒 ∈ ℝ ∪ +∞

𝑊, pred ∈ 𝑊 ∪ nil 𝑊, 𝑆 ⊆ 𝑊

1. 𝑒[𝑤] ← +∞, pred 𝑤 ← nil ∀𝑤 ∈ 𝑊, 𝑒𝑡 ← 0, 𝑆 ← 𝑡 2. while 𝑆 ≠ ∅ do 3. 𝑣 ← argmin𝑤∈𝑆 𝑒 𝑤 , 𝑆 ← 𝑆 ∖ {𝑣} 4. forall 𝑣𝑤 ∈ 𝐹 do 5. if 𝑒 𝑣 + 𝑑𝑣𝑤 < 𝑒[𝑤] then 6. 𝑒 𝑤 ← 𝑒 𝑣 + 𝑑𝑣𝑤, 𝑆 ← 𝑆 ∪ 𝑤 , pred 𝑤 ← 𝑣 7. endif 8. endwhile

9. output (𝑢, pred 𝑢 , pred2 𝑢 , … , 𝑡 or nil)

4 Prop. (Correctness and Run Time of Dijkstra's Algorithm):

Alg. 3 is correct and runs in 𝑃(𝑜 log 𝑜 + 𝑛).

Dijkstra's Algorithm

Şahin | Traffic Optimization I | CO@W 2020 10

reached nodes

𝑃(𝑜 × log 𝑜) 𝑃 𝑛 (amortized)

SLIDE 11

Shortest Path Problem

306 nodes, 445 edges

Şahin | Traffic Optimization I | CO@W 2020 11

80 nodes, 122 edges

SLIDE 12

Dijkstra's Algorithm

Şahin | Traffic Optimization I | CO@W 2020 12

SLIDE 13

IP Formulation of the ShPP

5 Def. (IP Formulation of the ShPP): Let 𝐸 = (𝑊, 𝐵, 𝑑) be a directed graph, with arc weights 𝑑 ∈ ℝ≥0

𝐵 , 𝑡, 𝑢 ∈ 𝑊 two nodes.

ShPP min 𝑑𝑈𝑦

bjective

i 𝑦 𝜀+ 𝑤 − 𝑦 𝜀− 𝑤 = 0 ∀𝑤 ≠ 𝑡, 𝑢 flow conservation ii 𝑦 𝜀+ 𝑡 = 1 flow constraint iii 0 ≤ 𝑦 ≤ 1 bounds iv 𝑦 integer integrality a) 𝑄𝑇ℎ𝑄𝑄 ≔ conv {𝜓𝑄: 𝑄 ∈ 𝑄

𝑡𝑢}

st-path polytope b) 𝑄

𝐽 𝑇ℎ𝑄𝑄 ≔ conv {𝑦 ∈ ℝ𝐹: (ShPP) i − iv }

ShPP polytope c) 𝑄

𝑀𝑄 𝑇ℎ𝑄𝑄 ≔ conv {𝑦 ∈ ℝ𝐹: ShPP

i − iii } ShPP LP-relaxation 6 Prop. (Path Polytopes): 𝑄𝑇ℎ𝑄𝑄 = 𝑄

𝐽 𝑇ℎ𝑄𝑄 is in general not true, but

𝑄

𝐽 𝑇ℎ𝑄𝑄 = 𝑄 𝑀𝑄 𝑇ℎ𝑄𝑄; Argmin 𝑦∈𝑄𝑀𝑄

𝑇ℎ𝑄𝑄 𝑑𝑈𝑦 contains a path for conservative 𝑑. Proof:

𝑄

𝐽 𝑇ℎ𝑄𝑄 allows subtours, 𝑄 𝑀𝑄 𝑇ℎ𝑄𝑄 describes a flow. 

Şahin | Traffic Optimization I | CO@W 2020 13

SLIDE 14

Dijkstra's Algorithm – A* – Superoptimal Wind

Şahin | Traffic Optimization I | CO@W 2020 14

TXL-DXB

SLIDE 15

Flight Planning

Şahin | Traffic Optimization I | CO@W 2020 15

▪ 3D, fuel, time-dependent (wind) ▪ Cost, temporary flight restrictions

SLIDE 16

▪ The green trajectory takes better advantage of the strong jet stream (~300 km/h). ▪ It is worth to take a long detour. ▪ Besides saving fuel and time, the new route saves

verflight fees by avoiding the expensive

airspaces of Canada and Japan.

Min Cost Track Min Fuel Track

Taipei – New York

Boeing B777-300ER, 25 April 2017, great circle distance 12.565 km Şahin | Traffic Optimization I | CO@W 2020 16

distance flown (km) flight time

(hours)

fuel burn

(kg)

verflight

fees (USD) total cost

(USD)*

* based on: fuel price 500 USD / ton, flight time costs: 1400 USD / hour

13.385

using

ld

heuristic search space reduction

14:40 95.524 2291 76.453

using new dynamic search space reduction

14.635 13:55 89.859 1139 71.118

1250

GAIN

0:45 5665

= 17,8 t CO2

1152

5335

SLIDE 17

Constrained Shortest Path Problem

5 Def. (IP Formulation of the Constrained ShPP (CSP)): Let 𝐸 = (𝑊, 𝐵, 𝑑) be a directed graph, with arc weights 𝑑 ∈ ℝ≥0

𝐵 , 𝑡, 𝑢 ∈ 𝑊

two nodes, 𝐵𝑦 ≤ 𝑐 some linear constraints. CSP min 𝑑𝑈𝑦

bjective

i 𝑦 𝜀+ 𝑤 − 𝑦 𝜀− 𝑤 = 0 ∀𝑤 ≠ 𝑡, 𝑢 flow conservation ii 𝑦 𝜀+ 𝑡 = 1 flow constraint iii 0 ≤ 𝑦 ≤ 1 bounds iv 𝐵𝑦 ≤ 𝑐 path constraints v 𝑦 integer integrality a) 𝑄𝐷𝑇𝑄 ≔ conv {𝜓𝑄: 𝑄 ∈ 𝑄

𝑡𝑢, 𝐵𝜓𝑄 ≤ 𝑐 }

const. st-path polytope

b) 𝑄

𝐽 𝐷𝑇𝑄 ≔ conv {𝑦 ∈ ℝ𝐹: (ShPP) i − v }

CSP polytope c) 𝑄

𝑀𝑄 𝐷𝑇𝑄 ≔ conv {𝑦 ∈ ℝ𝐹: ShPP

i − i𝑤 } CSP LP-relaxation 6 Obs. (CSP): 𝑄𝐷𝑇ℎ𝑄 ⊆ 𝑄

𝐽 𝐷𝑇𝑄 ⊆ 𝑄 𝑀𝑄 𝐷𝑇𝑄; equality does in general not

hold. The CSP is NP-hard.

Şahin | Traffic Optimization I | CO@W 2020 17

SLIDE 18

IP Formulation of the Constrained ShPP

5 Def. (IP Formulation of the Constrained ShPP (CSP)): Let 𝐸 = (𝑊, 𝐵, 𝑑) be a directed graph, with arc weights 𝑑 ∈ ℝ≥0

𝐵 , 𝑡, 𝑢 ∈ 𝑊

two nodes, 𝐵𝑦 ≤ 𝑐 some linear constraints. CSP min 𝑑𝑈𝑦

bjective

i 𝑦 𝜀+ 𝑤 − 𝑦 𝜀− 𝑤 = 0 ∀𝑤 ≠ 𝑡, 𝑢 flow conservation ii 𝑦 𝜀+ 𝑡 = 1 flow constraint iii 0 ≤ 𝑦 ≤ 1 bounds iv 𝐵𝑦 ≤ 𝑐 path constraints v 𝑦 integer integrality 6 Obs. (CSP): 𝑄𝐷𝑇ℎ𝑄 ⊆ 𝑄

𝐽 𝐷𝑇𝑄 ⊆ 𝑄 𝑀𝑄 𝐷𝑇𝑄; equality does in general not

hold. The CSP is NP-hard. Proof: Solves knapsack problem min 𝑑𝑈𝑦 ,

𝑏𝑈𝑦 ≤ 𝑐, 𝑦 ∈ 0,1 𝑜: 

Şahin | Traffic Optimization I | CO@W 2020 18

1 2 𝑜

𝑑𝑣𝑤 = 0, 𝑏𝑣𝑤 = 0 𝑑𝑣𝑤 = 𝑑𝑗, 𝑏𝑣𝑤 = 𝑏𝑗 …

SLIDE 19

Constrained Shortest Path Problem

5 Def. (IP Formulation of the Constrained ShPP (CSP)): Let 𝐸 = (𝑊, 𝐵, 𝑑) be a directed graph, with arc weights 𝑑 ∈ ℝ≥0

𝐵 , 𝑡, 𝑢 ∈ 𝑊

two nodes, 𝐵𝑦 ≤ 𝑐 some linear constraints. CSP min 𝑑𝑈𝑦

bjective

i 𝑦 𝜀+ 𝑤 − 𝑦 𝜀− 𝑤 = 0 ∀𝑤 ≠ 𝑡, 𝑢 flow conservation ii 𝑦 𝜀+ 𝑡 = 1 flow constraint iii 0 ≤ 𝑦 ≤ 1 bounds iv 𝐵𝑦 ≤ 𝑐 path constraints v 𝑦 integer integrality a) 𝑄𝐷𝑇𝑄 ≔ conv {𝜓𝑄: 𝑄 ∈ 𝑄

𝑡𝑢, 𝐵𝜓𝑄 ≤ 𝑐 }

const. st-path polytope

b) 𝑄

𝐽 𝐷𝑇𝑄 ≔ conv {𝑦 ∈ ℝ𝐹: (ShPP) i − v }

CSP polytope c) 𝑄

𝑀𝑄 𝐷𝑇𝑄 ≔ conv {𝑦 ∈ ℝ𝐹: ShPP

i − i𝑤 } CSP LP-relaxation 6 Obs. (CSP): 𝑄𝐷𝑇ℎ𝑄 ⊆ 𝑄

𝐽 𝐷𝑇𝑄 ⊆ 𝑄 𝑀𝑄 𝐷𝑇𝑄; equality does in general not

hold. The CSP is NP-hard, even for acyclic digraphs and 𝑑 ≥ 0.

Şahin | Traffic Optimization I | CO@W 2020 19

SLIDE 20

(Acyclic) Constrained Shortest Path Problem

8 Def. (Acyclic Constrained Shortest Path Problem (ACSP)): A CSP on a acyclic digraph is acylic. 9 Obs. (Topological Sorting): The nodes of an acyclic digraph 𝐸 = (𝑊, 𝐵) can be topologically sorted s.t. 𝑣𝑤 ∈ 𝐵 ⟹ 𝑣 < 𝑤. 7 Ex. (ACSP): P min

𝑞∈𝑄19 𝑑𝑈 𝜓 𝑞 , 𝑏𝑈𝜓 𝑞 ≥ 6

1 < ⋯ < 9 sorts 𝑊 = [9] topologically.

Şahin | Traffic Optimization I | CO@W 2020 20

1 2 3 4 5 6 7 8 9

0|1 1|0 4|4 5|5 2|3 3|2 2|0 5|0 1|0 4|0 0|0 3|0 𝑑|𝑥 −𝐵𝑦 ≤ −𝑐

SLIDE 21

(Acyclic) Constrained Shortest Path Problem

7 Ex. (ACSP): P min

𝑞∈𝑄19 𝑑𝑈 𝜓 𝑞 , 𝑏𝑈𝜓 𝑞 ≥ 6

Şahin | Traffic Optimization I | CO@W 2020 21

1 2 3 4 5 6 7 8 9

0|1 1|0 4|4 5|5 2|3 3|2 2|0 5|0 1|0 4|0 0|0 3|0 𝑑|a −𝐵𝑦 ≤ −𝑐 Τ 𝑤 𝑏 1 2 3 4 5 6 7 8 9 10 1 −∞ −∞ −∞ −∞ −∞ −∞ −∞ −∞ −∞ −∞ 2 −∞ −∞ −∞ −∞ −∞ −∞ −∞ −∞ −∞ −∞ 3 −∞ 1 −∞ −∞ −∞ −∞ −∞ −∞ −∞ −∞ −∞ 4 2 −∞ −∞ −∞ −∞ −∞ −∞ −∞ −∞ −∞ −∞ 5 −∞ 1 −∞ 4 −∞ −∞ −∞ −∞ −∞ −∞ −∞ 6 −∞ 1 −∞ 4 −∞ 7 −∞ −∞ −∞ −∞ −∞ 7 7 −∞ −∞ −∞ −∞ −∞ −∞ −∞ −∞ −∞ −∞ 8 −∞ 5 −∞ 8 11 −∞ −∞ −∞ −∞ −∞ −∞ 9 −∞ 4 −∞ 7 −∞ 10 10 −∞ 13 16 −∞

SLIDE 22

(Acyclic) Constrained Shortest Path Problem

7 Ex. (ACSP): P min

𝑞∈𝑄19 𝑑𝑈 𝜓 𝑞 , 𝑏𝑈𝜓 𝑞 ≥ 6

Cor. (Pseudopolynomial Solution of the ACSP): The ACSP can

be solved in pseudopolynomial time of 𝑃(ς𝑗 𝐵𝑗⋅ 1 max 𝜀+ 𝑤 + 𝑛), and if 𝐵, 𝑐 ≥ 0, in 𝑃(ς|𝑐

𝑘| max 𝜀+ 𝑤 + 𝑛).

Proof: Sort 𝐸 topologically in linear time of 𝑃(𝑛), then fill the dynamic programming table in 𝑃(ς𝑗 𝐵𝑗⋅ 1 max 𝜀+ 𝑤 ).

Şahin | Traffic Optimization I | CO@W 2020 22

1 2 3 4 5 6 7 8 9

0|1 1|0 4|4 5|5 2|3 3|2 2|0 5|0 1|0 4|0 0|0 3|0 𝑑|a −𝐵𝑦 ≤ −𝑐

SLIDE 23

Lagrange Relaxation

3.1 Def. (Lagrange(an) Relaxation): Let 𝑑 ∈ ℝ𝑜, 𝐸 ∈ ℝ𝑛×𝑜, 𝑒 ∈ ℝ𝑛, 𝑌 ⊆ ℝ𝑜 closed, and consider the optimization problem P min 𝑑𝑈𝑦 𝐸𝑦 = 𝑒 complicated/ing 𝑦 ∈ 𝑌 tractable. Let 𝜇 ∈ ℝ𝑛 a vector of Lagrange multipliers. a) 𝑀𝑄

𝐸𝑦=𝑒 𝜇 = min 𝑦∈𝑌 𝑑𝑈𝑦 − 𝜇𝑈 𝐸𝑦 − 𝑒

∈ ℝ ∪ ±∞ Lagrange relaxation (of (P)) (w.r.t. 𝑬𝒚 = 𝒆) at 𝝁 b) 𝑀𝑄

𝐸𝑦=𝑒: ℝ𝑛 → ℝ ∪ ±∞ , 𝜇 ↦ min 𝑦∈𝑌 𝑑𝑈𝑦 − 𝜇𝑈 𝐸𝑦 − 𝑒

Lagrange function (of (P)) (w.r.t. 𝑬𝒚 = 𝒆) Notation: If (P), 𝐸𝑦 = 𝑒 are clear, we write 𝑀 instead of 𝑀𝑄

𝐸𝑦=𝑒 etc.

Şahin | Traffic Optimization I | CO@W 2020 23

SLIDE 24

Lagrange Relaxation

3.2 Thm. (Properties of the Lagrange Relaxation, Geoffrion [1974]) Let (P) as in Def. 3.1 and 𝜉 𝑄 = min 𝑑𝑈𝑦, 𝐸𝑦 = 𝑒, 𝑦 ∈ 𝑌 ∈ ℝ ∪ {±∞}. a) sup

Şahin | Traffic Optimization I | CO@W 2020 25

= 0

SLIDE 26

Lagrange Relaxation

3.2 Thm. (Properties of the Lagrange Relaxation, Geoffrion [1974]) b) Let 𝑌 = 𝐵𝑦 ≥ 𝑐 , 𝑌 ∩ 𝐸𝑦 = 𝑒 ≠ ∅ for 𝐵 ∈ ℝ𝑙×𝑜, 𝑐 ∈ ℝ𝑜. max

𝜇

𝑀 𝜇 = 𝜉 𝑄 . Proof: b) 𝜉 𝑄 = min 𝑑𝑈𝑦 = max 𝜇𝑈𝑒 + 𝜈𝑈𝑐 𝐸𝑦 = 𝑒 Duality 𝜇𝑈𝐸 + 𝜈𝑈𝐵 = 𝑑𝑈 𝐵𝑦 ≥ 𝑐 Thm. 𝜈 ≥ 0 = max 𝜇𝑈𝑒 + max 𝜈𝑈𝑐 𝜇 𝜈 𝜈𝑈𝐵 = 𝑑𝑈 − 𝜇𝑈𝐸 𝜈 ≥ 0 = max 𝜇𝑈𝑒 + min 𝑑𝑈 − 𝜇𝑈𝐸 𝑦 DT 𝜇 𝐵𝑦 ≥ 𝑐

Şahin | Traffic Optimization I | CO@W 2020 26

SLIDE 27

Lagrange Relaxation

3.2 Thm. (Properties of the Lagrange Relaxation, Geoffrion [1974]) Let (P) as in Def. 3.1 and 𝜉 𝑄 = min 𝑑𝑈𝑦, 𝐸𝑦 = 𝑒, 𝑦 ∈ 𝑌 ∈ ℝ ∪ {±∞}. c) Let 𝑌 = 𝐵𝑦 ≥ 𝑐 ∩ ℤ𝑜, 𝑌 ∩ 𝐸𝑦 = 𝑒 ≠ ∅ for 𝐵 ∈ ℝ𝑙×𝑜, 𝑐 ∈ ℝ𝑜. Then min

𝐸𝑦=𝑒,𝐵𝑦≥𝑐 𝑑𝑈𝑦 ≤ max 𝜇

𝑀𝑄

𝐸𝑦=𝑒 𝜇 ≤ 𝜉 𝑄

Proof: c) Follows from a) and b).

Şahin | Traffic Optimization I | CO@W 2020 27

SLIDE 28

Lagrange Relaxation

3.2 Thm. (Properties of the Lagrange Relaxation, Geoffrion [1974]) Let (P) as in Def. 3.1 and 𝜉 𝑄 = min 𝑑𝑈𝑦, 𝐸𝑦 = 𝑒, 𝑦 ∈ 𝑌 ∈ ℝ ∪ {±∞}. d) Let 𝑌 be a finite set polytope , 𝑌 ∩ 𝐸𝑦 = 𝑒 ≠ ∅. Then 𝑀 is i) concave, ii) piecewise affine, iii) bounded from above. Proof: d) Let 𝑌 be a 𝑦1, … , 𝑦𝑙 conv {𝑦1, … , 𝑦𝑙} . Then 𝑀 𝜇 = min

𝑦∈𝑌 𝑑𝑈𝑦 − 𝜇𝑈𝐸 𝑦 + 𝜇𝑈𝑒 = min 𝑗=1,…,𝑙 𝑑𝑈𝑦𝑗 − 𝜇𝑈𝐸 𝑦𝑗 + 𝜇𝑈𝑒

≤ 𝜉 𝑄 < ∞ as 𝑌 is bounded and 𝐸𝑦 = 𝑒 ∩ 𝑌 ≠ ∅. 

Şahin | Traffic Optimization I | CO@W 2020 28

affine in 𝜇 concave

SLIDE 29

Lagrange Relaxation (Inequality Version)

3.3 Cor. (Lagrange(an) Relaxation): Let 𝑑 ∈ ℝ𝑜, 𝐸 ∈ ℝ𝑛×𝑜, 𝑒 ∈ ℝ𝑛, 𝑌 ⊆ ℝ𝑜 closed, and consider the optimization problem P min 𝑑𝑈𝑦 𝐸𝑦 ≥ 𝑒 complicated/ing (standard form) 𝑦 ∈ 𝑌 tractable. Let 𝜇 ∈ ℝ≥0

𝑛 a vector of Lagrange multipliers.

a) 𝑀𝑄

𝐸𝑦=𝑒 𝜇 = min 𝑦∈𝑌 𝑑𝑈 − 𝜇𝑈𝐸 𝑦 + 𝜇𝑈𝑒 ∈ ℝ ∪ ±∞

Lagrange relaxation (of (P)) (w.r.t. 𝑬𝒚 ≥ 𝒆) at 𝝁 b) 𝑀𝑄

𝐸𝑦=𝑒: ℝ≥0 𝑛 → ℝ ∪ ±∞ , 𝜇 ↦

min 𝑦 ∈ 𝑌 𝑑𝑈 − 𝜇𝑈𝐸 𝑦 + 𝜇𝑈𝑒 Lagrange function (of (P)) (w.r.t. 𝑬𝒚 ≥ 𝒆) Proof: Ex. 

Şahin | Traffic Optimization I | CO@W 2020 29

SLIDE 30

Subgradient Optimization

3.4 Def. (Subgradient, Subdifferential): Let 𝑔: ℝ𝑜 → ℝ be concave, 𝜇0 ∈ ℝ𝑜. a) 𝑣 ∈ ℝ𝑜: 𝑔 𝜇 ≤ 𝑔 𝜇0 + 𝑣𝑈 𝜇 − 𝜇0 ∀𝜇 ∈ ℝ𝑜 𝒗 subgradient of 𝒈 at 𝝁𝟏 b) 𝜖𝑔 𝜇0 ≔ {𝑣 ∈ ℝ𝑜: 𝑣 subgradient of 𝑔 at 𝜇0} subdifferential of 𝒈 at 𝝁𝟏 3.5 Prop. (Sufficient Optimality Condition): Let 𝑔: ℝ𝑜 → ℝ be concave, 𝜇0 ∈ ℝ𝑛. Then 0 ∈ 𝜖𝑔 𝜇0 ⟹ 𝑔(𝜇0) = max 𝑔. Proof: Ex.  3.6 Prop. (Diff'able Case): Let 𝑔: ℝ𝑜 → ℝ be concave and diff'able at 𝜇0 ∈ ℝ𝑜. Then 𝜖𝑔 𝜇0 ≔ 𝑔′ 𝜇0 . Proof: Ex. 

Şahin | Traffic Optimization I | CO@W 2020 30

SLIDE 31

Subgradient Optimization

3.7 Prop. (Polyhedral Case): Let 𝑔 = 𝑀 be a Lagrange function (as in Def. 3.1) and let 𝑌 = 𝑦1, … , 𝑦𝑙 conv {𝑦1, … , 𝑦𝑙} . Then 𝜖𝑔 𝜇0 = conv − 𝐸𝑦𝑗 − 𝑒 : 𝑦𝑗 ∈ Argmin 𝑔(𝜇0) . Proof: Let 𝜇0 ∈ ℝ𝑜, 𝑌 𝜇0 ≔ Argmin 𝑔(𝜇0), 𝑣𝑗 ≔ − 𝐸𝑦𝑗 − 𝑒 , 𝑗 = 1, … , 𝑙. "⊇": ∀𝑦𝑘 ∈ 𝑌 𝜇0 holds 𝑔 𝜇0 + 𝑣𝑘

𝑈 𝜇 − 𝜇0 = 𝑑𝑈𝑦𝑘 − 𝜇0 𝑈 𝐸𝑦𝑘 − 𝑒 − 𝐸𝑦𝑘 − 𝑒 𝑈(𝜇 − 𝜇0)

= 𝑑𝑈𝑦𝑘 − 𝜇𝑈(𝐸𝑦𝑘 − 𝑒) ≥ min

𝑗=1,…,𝑙 𝑑𝑈𝑦𝑗 − 𝜇𝑈(𝐸𝑦𝑗 − 𝑒)

= 𝑔 𝜇0 .

Şahin | Traffic Optimization I | CO@W 2020 31

= 𝑔(𝜇0) = 𝑣𝑘

𝑈

SLIDE 32

Subgradient Optimization

3.7 Prop. (Polyhedral Case): Let 𝑔 = 𝑀 be a Lagrange function (as in Def. 3.1) and let 𝑌 = 𝑦1, … , 𝑦𝑙 conv {𝑦1, … , 𝑦𝑙} . Then 𝜖𝑔 𝜇0 = conv − 𝐸𝑦𝑗 − 𝑒 : 𝑦𝑗 ∈ Argmin 𝑔(𝜇0) . Proof: Let 𝜇0 ∈ ℝ𝑜, 𝑌 𝜇0 ≔ Argmin 𝑔(𝜇0), 𝑣𝑗 ≔ − 𝐸𝑦𝑗 − 𝑒 , 𝑗 = 1, … , 𝑙. "⊆": min

𝑦𝑗∉𝑌 𝜇0 𝑑𝑈𝑦𝑗 − 𝜇0 𝑈 𝐸𝑦𝑗 − 𝑒 > 𝑔 𝜇0

⟹ ∃𝜗 > 0: min

𝑦𝑗∉𝑌 𝜇0 𝑑𝑈𝑦𝑗 − 𝜇𝑈 𝐸𝑦𝑗 − 𝑒 > 𝑔 𝜇

∀𝜇 ∈ 𝑉𝜗 𝜇0 ⟹ 𝑌 𝜇 ⊆ 𝑌 𝜇0 ∀𝜇 ∈ 𝑉𝜗 𝜇0 .

Şahin | Traffic Optimization I | CO@W 2020 32

SLIDE 33

Subgradient Optimization

3.7 Prop. (Polyhedral Case): Let 𝑔 = 𝑀 be a Lagrange function (as in Def. 3.1) and let 𝑌 = 𝑦1, … , 𝑦𝑙 conv {𝑦1, … , 𝑦𝑙} . Then 𝜖𝑔 𝜇0 = conv − 𝐸𝑦𝑗 − 𝑒 : 𝑦𝑗 ∈ Argmin 𝑔(𝜇0) . Proof: Let 𝜇0 ∈ ℝ𝑜, 𝑌 𝜇0 ≔ Argmin 𝑔(𝜇0), 𝑣𝑗 ≔ − 𝐸𝑦𝑗 − 𝑒 , 𝑗 = 1, … , 𝑙. "⊆": ∃𝜗 > 0: 𝑌 𝜇 ⊆ 𝑌 𝜇0 ∀𝜇 ∈ 𝑉𝜗 𝜇0 . Let u ∉ conv 𝑣𝑗: 𝑦𝑗 ∈ 𝑌 𝜇0 ⟹ ∃𝜌 ∈ ℝ𝑛: 𝜌𝑈𝑣 < 𝜌𝑈𝑣𝑗 ∀𝑦𝑗 ∈ 𝑌 𝜇0 (by sep. hyperplane Thm.) ⟹ 𝑔 𝜇0 + 𝜗𝜌 ≥ min

𝑦𝑗∈𝑌 𝜇0 𝑑𝑈𝑦𝑗 − 𝜇0 + 𝜗𝜌 𝑈 𝐸𝑦𝑗 − 𝑒

= f 𝜇0 + min

𝑦𝑗∈𝑌 𝜇0 𝜗𝜌𝑈𝑣𝑗

> f 𝜇0 + 𝜗𝜌𝑈𝑣 = f 𝜇0 + 𝑣𝑈 𝜇0 + 𝜗𝜌 − 𝜇0  

Şahin | Traffic Optimization I | CO@W 2020 33

⊇ 𝑌(𝜇0 + 𝜗𝜌) = −𝜇0

𝑈 𝐸𝑦𝑗 − 𝑒 + 𝜗𝜌𝑈𝑣𝑗

= 𝜇 = 𝜇

SLIDE 34

Subgradient Algorithm

3.8 Alg. (Subgradient Algorithm): Input: 𝑔: ℝ𝑛 → ℝ concave (by func. & subgrad. oracle) 𝜇0 ∈ ℝ𝑛 starting point 𝛽𝑙 𝑙=1

∞

> 0∗ sequence of step lengths Output: 𝜇𝑙 𝑙=1

∞

∈ (ℝ𝑛)∗ iterates Data St.: 𝑙 ∈ ℕ0 iteration counter 𝑣𝑙 𝑙=1

∞

∈ (ℝ𝑛)∗ sequence of subgradients 1. 𝑙 ← 0, 𝑣0 ←

∈ 𝜖𝑔 𝜇0

2. 𝜇𝑙+1 ← 𝜇𝑙 + 𝛽𝑙𝑣𝑙, 𝑣𝑙+1 ←

∈ 𝜖𝑔 𝜇𝑙+1 , 𝑙 ← 𝑙 + 1

3. goto 2

Şahin | Traffic Optimization I | CO@W 2020 34

𝜇𝑙 𝑣𝑙

SLIDE 35

Convergence of the Subgradient Algorithm

3.9 Thm. (Convergence of the Subgradient Algorithm): Let 𝑔: ℝ𝑛 → ℝ be concave and a) 𝑔∗ = max 𝑔 < ∞ (in particular, the maximum exists) b) 𝑣

2 ≤ 𝑀

∀𝑣 ∈ 𝜖𝑔 for some 𝑀 ∈ ℝ c) σ𝑙=0

∞

𝛽𝑙

2 < ∞ ,

σ𝑙=0

∞

𝛽𝑙 → ∞. Then lim

𝑙→∞ max 𝑘=1,…,𝑙 𝑔(𝜇𝑘) = 𝑔∗.

Proof: Let 𝜇∗ ∈ Argmin 𝑔. Then 𝜇𝑙+1 − 𝜇∗

2 2 =

𝜇𝑙 + 𝛽𝑙𝑣𝑙 − 𝜇∗

2 2

= 𝜇𝑙 − 𝜇∗

2 2 + 2𝛽𝑙𝑣𝑙 𝜇𝑙 − 𝜇∗ + 𝛽𝑙 2 𝑣𝑙 2 2

≤ 𝜇0 − 𝜇∗

2 2 − σ𝑘=1 𝑙

2𝛽𝑙 𝑔∗ − 𝑔(𝜇𝑘) + σ𝑘=1

𝑙

𝛽𝑙

2 𝑣𝑙 2 2

Şahin | Traffic Optimization I | CO@W 2020 35

𝜇𝑙 𝑣𝑙 𝜇∗

≤ 𝑔 𝜇𝑙 − 𝑔∗

SLIDE 36

3.10 Ex. (Lagrange Relaxation) P min

𝑞∈𝑄19 𝑑𝑈 𝜓 𝑞 , 𝑥𝑈𝜓 𝑞 ≥ 6

Şahin | Traffic Optimization I | CO@W 2020 37

1 2 3 4 5 6 7 8 9

0|1 1|0 4|4 5|5 2|3 3|2 2|0 5|0 1|0 4|0 0|0 3|0 𝑑|𝑥

𝒒 nodes 𝒅(𝒒) 𝒙(𝒒) 𝒙 𝒒 − 𝟕 𝒅 𝒒 − 𝝁 𝒙 𝒒 − 𝟕 𝑞1 12369 4 1 −5 4 − 𝜇(−5) = 4 + 5𝜇 𝑞2 12569 7 3 −3 7 − 𝜇(−3) = 7 + 3 𝜇 𝑞3 12589 10 6 10 − 𝜇0 = 10 𝑞4 14569 10 5 −1 10 − 𝜇 −1 = 10 + 𝜇 𝑞5 14589 13 8 2 13 − 𝜇 −1 = 13 − 2𝜇 𝑞6 14789 16 9 3 16 − 𝜇 3 = 16 − 3𝜇

𝑌 𝐸𝑦 ≥ 𝑒

SLIDE 38

Lagrange Relaxation

3.10 Ex. (Lagrange Relaxation)

Şahin | Traffic Optimization I | CO@W 2020 38

𝒒 𝒅 𝒒 − 𝝁 𝒙 𝒒 − 𝟕 𝑞1 = 4 + 5𝜇 𝑞2 = 7 + 3 𝜇 𝑞3 = 10 𝑞4 = 10 + 𝜇 𝑞5 = 13 − 2𝜇 𝑞6 = 16 − 3𝜇

1 2 3 4 5 6

16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1

𝑞1 𝑞2 𝑞3 𝑞4 𝑞5 𝑞6

SLIDE 39

Lagrange Relaxation

3.10 Ex. (Lagrange Relaxation)

Şahin | Traffic Optimization I | CO@W 2020 39

𝑙 1 2 3 4 5 6 𝜇𝑙 5

ൗ 7 2 ൗ 5 2

2

ൗ 8 5 ൗ 19 15

𝑔(𝜇𝑙) 4 1

ൗ 11 2

8 9 10 10 𝑞k 𝑞1 𝑞6 𝑞6 𝑞5 𝑞5 𝑞5, 𝑞3 𝑞3 𝑥 𝑞𝑙 − 6

5

3 3 2 2

2, 0

𝑣𝑙 5

3
3
2
2

2,0 𝛽𝑙 1 ½

ൗ 1 3

¼

ൗ 1 5 ൗ 1 6 ൗ 1 7

1 2 3 4 5 6

𝒒 𝒅(𝒒) 𝒙(𝒒) 𝑞1 4 1 𝑞2 7 3 𝑞3 9 6 𝑞4 10 5 𝑞5 13 8 𝑞6 16 9

1 2 3 4 5 6

16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1

𝑞1 𝑞2 𝑞3 𝑞4 𝑞5 𝑞6 𝑔(𝜇0) 𝑔(𝜇1) 𝑔(𝜇2) 𝑔(𝜇3) 𝑔(𝜇4) 𝑔(𝜇5) 𝜇0 𝜇1 𝜇2 𝜇3 𝜇4 𝜇5 𝜇6 𝑔(𝜇6)

SLIDE 40

Thank you for your attention

Prof. Dr. Güvenç Şahin

Sabancı University Orta Mahalle Üniversite Caddesi No:27 Tuzla 34956 İstanbul Turkey guvencs@sabanciuniv.edu myweb.sabanciuniv.edu/guvencs/

Şahin | Traffic Optimization I | CO@W 2020 40