Integer Linear Programming
CONTACT@ADAMFURMANEK.PL HTTP://BLOG.ADAMFURMANEK.PL FURMANEKADAM
INTEGER LINEAR PROGRAMMING - ADAM FURMANEK 25.07.2020
1
Integer Linear Programming CONTACT@ADAMFURMANEK.PL - - PowerPoint PPT Presentation
Integer Linear Programming CONTACT@ADAMFURMANEK.PL HTTP://BLOG.ADAMFURMANEK.PL FURMANEKADAM 1 25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK About me Experienced with backend, frontend, mobile, desktop, ML, databases. Blogger,
CONTACT@ADAMFURMANEK.PL HTTP://BLOG.ADAMFURMANEK.PL FURMANEKADAM
INTEGER LINEAR PROGRAMMING - ADAM FURMANEK 25.07.2020
1
Experienced with backend, frontend, mobile, desktop, ML, databases. Blogger, public speaker. Author of .NET Internals Cookbook. http://blog.adamfurmanek.pl contact@adamfurmanek.pl furmanekadam
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
2
Declarative programming at a glance. Some theory. Real life example. Implementation consideration. Summary.
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
3
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
4
Paradigm that expresses the logic of a computation without describing its control flow. Declarative programming often considers programs as theories of a formal logic, and computations as deductions in that logic space. A high-level program that describes what a computation should perform. SELECT * FROM Orders
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
5
Decouples problem formulation from solving process. Solving part can be replaced without modifying the formulation. Easier to optimise algorithms for because „the goal” is understandable for the machine. Can be used with little to no programming skills. Sometimes we don’t know how to solve it.
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
6
We are playing RTS game. We are allowed to hire footmen and archers. Every footman costs 10 gold and 30 food. Every archer costs 20 gold, 25 food, and 10 wood. Footman’s attack is equal to 5, archer’s attack is equal to 7. Our population limit is set to 200 units. Every footman „costs” 1 unit, every archer „costs” 2 units. We have 1000 gold, 1000 food, and 200 wood. We want to get strongest possible army.
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
7
Gold Food Wood Footman 10 30 Archer 20 25 10
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
8
Attack Unit count Footman 5 1 Archer 7 2 Available gold Available food Available wood Max units count Constraint 1000 1000 200 200
Variables: 𝑔 − 𝑔𝑝𝑝𝑢𝑛𝑓𝑜 𝑏 − 𝑏𝑠𝑑ℎ𝑓𝑠𝑡 Constraints: 𝑔 + 2 ⋅ 𝑏 ≤ 200 // population 10 ⋅ 𝑔 + 20 ⋅ 𝑏 ≤ 1000 // gold 30 ⋅ 𝑔 + 25 ⋅ 𝑏 ≤ 1000 // food 10 ⋅ 𝑏 ≤ 200 // wood Target value: 5 ⋅ 𝑔 + 7 ⋅ 𝑏
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
9
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
10
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
11
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
12
Variables: 𝑇, 𝐹, 𝑂, 𝐸, 𝑁, 𝑃, 𝑆, 𝑍 ∈ 0, … , 9 and all different Constraints: 1000𝑇 + 100𝐹 + 10𝑂 + 𝐸 + 1000𝑁 + 100𝑃 + 10𝑆 + 𝐹 = 10000𝑁 + 1000𝑃 + 100𝑂 + 10𝐹 + 𝑍
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
13
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
14
RTS game = factory production allocation:
Send more money = pattern recognition:
Floor tiling = microchip transistor layout:
Others: BTS placement, power plant rooms layout, scheduling systems, items personalization and many more.
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
15
MIXED INTEGER LINEAR PROGRAMMING
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
16
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
17
Programming in mathematics means finding a solution to
Linear Programming is a class of a problems with only linear constraints and with linear cost function. Mixed Integer Linear Programming is a class of a problems with integer constraint for some of variables.
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
18
We can add two variables: 𝑏 + 𝑐 We can multiply variable by constant: 5 ⋅ 𝑏 We can constrain variable with lower/upper bound: 𝑏 ≤ 7 We cannot multiply variables: 𝑏 ⋅ 𝑐 We cannot use strict constraings (greater than, less than, not equal) 𝑏 < 10
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
19
Notice that not hiring anyone was also a solution! Problem may have zero, one, multiple or infinitely many solutions. We need to compare them. We use cost function to specify which one of them is the best.
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
20
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
21
Basic algorithms:
(continuous version of the problem), and next we cut the plane to get smaller problem
next we bound some variables and perform next iteration
MILP is NP-complete and we sometimes use heuristics:
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
22
WE WILL IMPLEMENT
WE WILL NOT IMPLEMENT
Arithmetic: division, remainder, exponentation, roots. Comparisons: min, max, absolute value. Number theory: factorial, GCD. Algorithms: if condition, sorting, loops, lexicographical comparisons, Gray’s code, linear regression. Set operators: SOS type 1 and 2, approximation. Graph operators: MST, vertex/edge cover, max flow, connectivity, shortest path, TSP. Turing machine.
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
23
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
24
If both 𝑏 and 𝑐 are 1 then 𝑦 must be 1. If 𝑦 was 0: 0 ≤ 1 + 1 − 2 ⋅ 0 ≤ 1 0 ≤ 2 − 0 ≤ 1 0 ≤ 2 ≤ 1
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
25
If both 𝑏 and 𝑐 are 1 then 𝑦 must be 1. If 𝑦 is 1: 0 ≤ 1 + 1 − 2 ⋅ 1 ≤ 1 0 ≤ 2 − 2 ≤ 1 0 ≤ 0 ≤ 1
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
26
If either 𝑏 or 𝑐 is 1 then 𝑦 must be 0. If 𝑦 was 1: 0 ≤ 1 + 0 − 2 ⋅ 1 ≤ 1 0 ≤ 1 − 2 ≤ 1 0 ≤ −1 ≤ 1
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
27
If either 𝑏 or 𝑐 is 1 then 𝑦 must be 0. If 𝑦 is 0: 0 ≤ 1 + 0 − 2 ⋅ 0 ≤ 1 0 ≤ 1 − 0 ≤ 1 0 ≤ 1 ≤ 1
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
28
If both 𝑏 or 𝑐 are 0 then 𝑦 must be 0. If 𝑦 was 1: 0 ≤ 0 + 0 − 2 ⋅ 1 ≤ 1 0 ≤ 0 − 2 ≤ 1 0 ≤ −2 ≤ 1
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
29
If both 𝑏 or 𝑐 are 0 then 𝑦 must be 0. If 𝑦 is 0: 0 ≤ 0 + 0 − 2 ⋅ 0 ≤ 1 0 ≤ 0 − 0 ≤ 1 0 ≤ 0 ≤ 1
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
30
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
31
Two variables conjunction: 0 ≤ 𝑏 + 𝑐 − 2𝑦 ≤ 1 𝑜 variables conjunction: 0 ≤ 𝑏1 + 𝑏2 + 𝑏3 + … + 𝑏𝑜 − 𝑜𝑦 ≤ 𝑜 − 1 Two variables disjunction: −1 ≤ 𝑏 + 𝑐 − 2𝑦 ≤ 0 𝑜 variables disjunction goes the same way
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
32
Negation: 𝑦 = 1 − 𝑏 Implication: 𝑏 ⇒ 𝑐 ≡ ~𝑏 ∨ 𝑐 Exclusive or: ~𝑏 ∧ 𝑐 ∨ (𝑏 ∧ ~𝑐)
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
33
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
34
Multiplication is possible when we know the maximum possible value of a variable. We set the upper bound and perform the long multiplication. It is rather slow approach, it requires 𝑃(𝑜2) temporary variables.
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
35
We decompose the variable to extract digits. We assume the maximum possible value, because we need to know the number of digits. Decomposition is straghtforward: 𝑐 = 𝑐0 + 2 ⋅ 𝑐1 + 4 ⋅ 𝑐2 + 8 ⋅ 𝑐3 + … + 2𝑜−1 ⋅ 𝑐𝑜−1
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
36
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
37
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
38
𝑏 ⋅ 𝑐 = 𝑏0 ∧ 𝑐0 + 2 ⋅ 𝑏0 ∧ b1 + 4 ⋅ a0 ∧ 𝑐2 + … + 2𝑜−1 ⋅ 𝑏0 ∧ 𝑐𝑜−1 +2 ⋅ 𝑏1 ∧ 𝑐0 + 2 ⋅ 𝑏1 ∧ 𝑐1 + 4 ⋅ 𝑏1 ∧ 𝑐2 + ⋯ 2𝑜−1 ⋅ (𝑏1 ∧ 𝑐𝑜−1) +4 ⋅ 𝑏2 ∧ 𝑐0 + 2 ⋅ 𝑏2 ∧ 𝑐1 + 4 ⋅ 𝑏2 ∧ 𝑐2 + … + 2𝑜−1 ⋅ 𝑏2 ∧ bn−1 + ⋯ + 2𝑜−1( 𝑏𝑜−1 ∧ 𝑐0 + 2 ⋅ 𝑏𝑜−1 ∧ 𝑐1 + 4 ⋅ 𝑏𝑜−1 ∧ 𝑐2 + … + 2𝑜−1 ⋅ 𝑏𝑜−1 ∧ 𝑐𝑜−1 ) It can be represented as: 𝑏 ⋅ 𝑐 =
𝑗=0 𝑜−1
2𝑗
𝑘=0 𝑜−1
2𝑘𝑏𝑗 ∧ 𝑐
𝑘
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
39
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
40
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
41
https://github.com/afish/MilpManager
REAL LIFE EXAMPLE
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
42
At the beginning of every term students must be assigned to classes. It is hard to make them happy because some of them work, some of them prefer to sleep long, some of them prefer to have lots of days off. We can try to represent the requirements as a MILP program and find the optimal solution.
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
43
We ask students to assign preferences points to every possible class. The more points assigned the more student wants to be assigned to that class. We need to take care of rooms occupancy limits, collisions etc.
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
44
We are given a schedule of classes in courses Every class has associated day (Monday – Friday), hour (e.g., 9:30 AM) and duration (e.g., 1:30 hrs). Every class has associated room with occupancy limit.
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
45
Variables. Exactly one class in one course. Collisions. Rooms occupancy limits. Cost function.
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
46
For every person, every course, every class we declare binary variable. Value 1 means that the student is assigned to this class. We define: 𝑦𝑑𝑝𝑣𝑠𝑡𝑓,𝑑𝑚𝑏𝑡𝑡,𝑡𝑢𝑣𝑒𝑓𝑜𝑢
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
47
Every student must attend exactly one class in course. For every course we need to assign student to exactly
ሥ
𝑑𝑝𝑣𝑠𝑡𝑓,𝑡𝑢𝑣𝑒𝑓𝑜𝑢
𝑑𝑚𝑏𝑡𝑡
𝑦𝑑𝑝𝑣𝑠𝑡𝑓,𝑑𝑚𝑏𝑡𝑡,𝑡𝑢𝑣𝑒𝑓𝑜𝑢 = 1
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
48
Student cannot be in two places at the same time. ሥ 𝑑𝑝𝑣𝑠𝑡𝑓1, 𝑑𝑚𝑏𝑡𝑡1 , 𝑑𝑝𝑣𝑠𝑡𝑓2, 𝑑𝑚𝑏𝑡𝑡2 𝑑𝑝𝑚𝑚𝑗𝑒𝑗𝑜 ሥ
𝑡𝑢𝑣𝑒𝑓𝑜𝑢
𝑦𝑑𝑝𝑣𝑠𝑡𝑓1,𝑑𝑚𝑏𝑡𝑡1,𝑡𝑢𝑣𝑒𝑓𝑜𝑢 + 𝑦𝑑𝑝𝑣𝑠𝑡𝑓2,𝑑𝑚𝑏𝑡𝑡2,𝑡𝑢𝑣𝑒𝑓𝑜𝑢 ≤ 1
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
49
Every room has an occupancy limit. ሥ
𝑑𝑝𝑣𝑠𝑡𝑓,𝑑𝑚𝑏𝑡𝑡
𝑡𝑢𝑣𝑒𝑓𝑜𝑢
𝑦𝑑𝑝𝑣𝑠𝑡𝑓,𝑑𝑚𝑏𝑡𝑡,𝑡𝑢𝑣𝑒𝑓𝑜𝑢 ≤ 𝑡𝑑𝑝𝑣𝑠𝑡𝑓,𝑑𝑚𝑏𝑡𝑡 where 𝑡𝑑𝑝𝑣𝑠𝑡𝑓,𝑑𝑚𝑏𝑡𝑡 means the size of the room
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
50
Every student assigned points to classes. We want to maximize the sum of those points.
𝑑𝑝𝑣𝑠𝑡𝑓,𝑑𝑚𝑏𝑡𝑡,𝑡𝑢𝑣𝑒𝑓𝑜𝑢
𝑞𝑑𝑝𝑣𝑠𝑡𝑓,𝑑𝑚𝑏𝑡𝑡,𝑡𝑢𝑣𝑒𝑓𝑜𝑢 ⋅ 𝑦𝑑𝑝𝑣𝑠𝑡𝑓,𝑑𝑚𝑏𝑡𝑡,𝑡𝑢𝑣𝑒𝑓𝑜𝑢 where 𝑞𝑑𝑝𝑣𝑠𝑡𝑓,𝑑𝑚𝑏𝑡𝑡,𝑡𝑢𝑣𝑒𝑓𝑜𝑢 means number of points
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
51
Students: 150 Courses: 15 Classes: 9 per course Problem size: ~40 000 variables. Solving time: 2 seconds. This solution is optimal, we won’t get any better than that!
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
52
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
53
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
54
Various approaches:
Licenses for universities and research work. Licenses for students.
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
55
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
56
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
57
https://neos-server.org/neos/solvers/index.html
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
58
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
59
Declarative programming allows you to focus on a problem, not on an algorithm! If something is slow — just replace the solver. There are many models, choose as powerful as you can (to make modelling easy) and as primitive as possible (to make solving fast).
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
60
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
61
Alexander Schrijver — „Theory of Linear and Integer Programming” Dennis Yurichev — „SAT/SMT by example” Adam Furmanek – „.NET Internals Cookbook” http://blog.adamfurmanek.pl/2015/08/22/ilp-part-1/ — a lot about ILP https://github.com/afish/MilpManager — library for modelling https://neos-server.org/neos/solvers/index.html — NEOS cloud for solving problems https://tore.tuhh.de/bitstream/11420/2548/1/201905-scopes-oehlert.pdf — Practical usage of ILP versus QP
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
62
CONTACT@ADAMFURMANEK.PL HTTP://BLOG.ADAMFURMANEK.PL FURMANEKADAM
25.07.2020 INTEGER LINEAR PROGRAMMING - ADAM FURMANEK
63