Problems That We . . . Linearization Is Often . . . An Example of a . . . How to Solve Linear . . . Chubanov’s Method – Khachiyan’s Algorithm . . . A New Polynomial-Time Karmarkar’s Algorithm . . . Constraint . . . Algorithm for Linear Sergei Chubanov’s Idea Why Chubanov’s . . . Programming Home Page Title Page Vladik Kreinovich Department of Computer Science ◭◭ ◮◮ University of Texas at El Paso ◭ ◮ El Paso, TX 79968, USA vladik@utep.edu Page 1 of 32 http://www.cs.utep.edu/vladik (based on a joint work with Olga Kosheleva) Go Back Full Screen Close Quit
Problems That We . . . Linearization Is Often . . . 1. Problems That We Solve in Real Life An Example of a . . . • In many practical situations, we need to maximize or How to Solve Linear . . . minimize some objective function. Khachiyan’s Algorithm . . . Karmarkar’s Algorithm . . . • When we select a plan for a company, we want to max- Constraint . . . imize profit. Sergei Chubanov’s Idea • When we select a route for a car, we want to minimize Why Chubanov’s . . . travel time. Home Page • When we select medical treatment, we want to mini- Title Page mize side effects, etc. ◭◭ ◮◮ • In all these situations, there are some constraints. ◭ ◮ • Pollution generated by a chemical plant cannot exceed Page 2 of 32 the legal limits. Go Back • A car cannot exceed the speed limit – unless it is an Full Screen emergency vehicle. Close Quit
Problems That We . . . Linearization Is Often . . . 2. Real-Life Problems (cont-d) An Example of a . . . • A medical treatment must satisfy a certain rate of cure, How to Solve Linear . . . etc. Khachiyan’s Algorithm . . . Karmarkar’s Algorithm . . . • In general, there are several parameters x 1 , . . . , x n pos- Constraint . . . sible alternatives. Sergei Chubanov’s Idea • The objective function f ( x 1 , . . . , x n ) depends on all Why Chubanov’s . . . these parameters. Home Page • A constraints means that some quantity g cannot ex- Title Page ceed the corresponding threshold t . ◭◭ ◮◮ • This quantity also depends on the parameters ◭ ◮ x 1 , . . . , x n : g = g ( x 1 , . . . , x n ). Page 3 of 32 • Thus, a constraint has the form g ( x 1 , . . . , x n ) ≤ t . Go Back • In general, we have a constraint optimization problem: Full Screen maximize f ( x 1 , . . . , x n ) under constraints Close g 1 ( x 1 , . . . , x n ) ≤ t 1 , . . . , g m ( x 1 , . . . , x n ) ≤ t m . Quit
Problems That We . . . Linearization Is Often . . . 3. Linearization Is Often Possible An Example of a . . . • In many practical situations, we know a reasonable How to Solve Linear . . . good solution x (0) = ( x (0) 1 , . . . , x (0) n ). Khachiyan’s Algorithm . . . Karmarkar’s Algorithm . . . • This usually means that the unknown optimal solution Constraint . . . x = ( x 1 , . . . , x n ) is close to x (0) . Sergei Chubanov’s Idea def = x i − x (0) • In other words, the differences v i are small. Why Chubanov’s . . . i Home Page • In physics and engineering, if the quantities v i are small, we can safely ignore terms quadratic in v i . Title Page • For example, if v i ≈ 10%, then v 2 i ≈ 1% ≪ 10%. ◭◭ ◮◮ ◭ ◮ Page 4 of 32 Go Back Full Screen Close Quit
Problems That We . . . Linearization Is Often . . . 4. Linearization (cont-d) An Example of a . . . • Thus, we can, e.g.: How to Solve Linear . . . Khachiyan’s Algorithm . . . – take the expression Karmarkar’s Algorithm . . . � � x (0) 1 + v 1 , . . . , x (0) f ( x 1 , . . . , x n ) = f n + v n ; Constraint . . . Sergei Chubanov’s Idea – expand it in Taylor series and keep only linear Why Chubanov’s . . . terms in this expansion: Home Page n Title Page f ( x 1 , . . . , x n ) ≈ y (0) + � c i · v i , ◭◭ ◮◮ j =1 ◭ ◮ = ∂f � � where y (0) def x (0) def 1 , . . . , x (0) = f and c j . n ∂x j Page 5 of 32 • Maximizing this expression for f ( x 1 , . . . , x n ) is equiva- Go Back n � lent to maximizing a linear function c i · v i . Full Screen j =1 Close Quit
Problems That We . . . Linearization Is Often . . . 5. Linearization (cont-d) An Example of a . . . • By applying a similar linearization to g i ( x 1 , . . . , x n ) = How to Solve Linear . . . � � x (0) 1 + v 1 , . . . , x (0) Khachiyan’s Algorithm . . . g i n + v n , we conclude that Karmarkar’s Algorithm . . . m Constraint . . . � g i ( x 1 , . . . , x n ) ≈ g i 0 + a ij · v j , Sergei Chubanov’s Idea j =1 Why Chubanov’s . . . = ∂g i � � def def x (0) Home Page 1 , . . . , x (0) where g i 0 = g i and a ij . n ∂x j Title Page • Thus, each constraint g i ( x 1 , . . . , x n ) ≤ t i takes the form ◭◭ ◮◮ n def � a ij · v j ≤ b i , where b i = t i − g i 0 . ◭ ◮ j =1 Page 6 of 32 • Thus, we arrive at the problem of maximizing a linear n n Go Back function � c i · v i under linear constraints � a ij · v j ≤ b i . j =1 j =1 Full Screen • Such problems are known as linear programming . Close Quit
Problems That We . . . Linearization Is Often . . . 6. Why the Name? An Example of a . . . • Why linear – clear, but why programming? How to Solve Linear . . . Khachiyan’s Algorithm . . . • The answer is simple: in the late 1940s, programming Karmarkar’s Algorithm . . . was all the range. Constraint . . . • If you called it programming, your changes of getting Sergei Chubanov’s Idea a grant drastically increased. Why Chubanov’s . . . Home Page • So we have dynamic programming, quadratic program- ming, etc. Title Page • All this has nothing to do with programming. ◭◭ ◮◮ • It is somewhat like now, when many folks processing ◭ ◮ kilobytes of data call it big data :-( Page 7 of 32 Go Back Full Screen Close Quit
Problems That We . . . Linearization Is Often . . . 7. An Example of a Linear Programming Problem An Example of a . . . • One of the first examples of linear programming was How to Solve Linear . . . developing meals plan for jails. Khachiyan’s Algorithm . . . Karmarkar’s Algorithm . . . • In this case, v 1 , . . . , v n are amounts of different prod- Constraint . . . ucts: beef, chicken, beans, bread, milk, etc. Sergei Chubanov’s Idea n � • The objective is to minimize cost c j · v j . Why Chubanov’s . . . j =1 Home Page • The main constraint is that the overall amount of calo- n Title Page ries should be sufficient: � a 1 j · v j ≥ b 1 . ◭◭ ◮◮ j =1 • Here, a 1 j is calories per pound for the j -th product. ◭ ◮ • We must also make sure that the folks get: Page 8 of 32 – enough proteins b 2 , Go Back – enough of different vitamins b 3 , . . . , Full Screen – enough of different micro-elements b i , etc. Close Quit
Problems That We . . . Linearization Is Often . . . 8. Jail Example: Comment An Example of a . . . • The solution, by the way, was indeed cheap. How to Solve Linear . . . Khachiyan’s Algorithm . . . • However, I would not advise students to use it: it does Karmarkar’s Algorithm . . . not take taste into account :-( Constraint . . . Sergei Chubanov’s Idea Why Chubanov’s . . . Home Page Title Page ◭◭ ◮◮ ◭ ◮ Page 9 of 32 Go Back Full Screen Close Quit
Problems That We . . . Linearization Is Often . . . 9. How to Solve Linear Programming Problems An Example of a . . . • Since linear programming problems are ubiquitous, How to Solve Linear . . . people have been trying to solve them. Khachiyan’s Algorithm . . . Karmarkar’s Algorithm . . . • It started with a simple mathematical analysis. Constraint . . . n • Each constraint � a ij · v j ≤ b i determines a half-space. Sergei Chubanov’s Idea j =1 Why Chubanov’s . . . • A half-space H is a convex set: if h ∈ H and h ′ ∈ H , Home Page then the whole straight line segment is in H : Title Page α · h + (1 − α ) · h ′ ∈ H for all α ∈ (0 , 1) . ◭◭ ◮◮ ◭ ◮ • The set of all v = ( v 1 , . . . , v n ) that satisfy all the con- straints is an intersection of several half-spaces. Page 10 of 32 • This intersection is thus also convex: a convex poly- Go Back tope. Full Screen • On each segment, a linear function is linear. Close Quit
Problems That We . . . Linearization Is Often . . . 10. Solving Linear Programming (cont-d) An Example of a . . . • The maximum of a linear function of a segment is at- How to Solve Linear . . . tained at the endpoints. Khachiyan’s Algorithm . . . Karmarkar’s Algorithm . . . • So, in our problem, the maximum of a linear function Constraint . . . is attained at one of the vertices. Sergei Chubanov’s Idea • A vertex is where n of m constraints are equalities. Why Chubanov’s . . . • Once we know which constraints are equalities, to find Home Page v , we solve a system of linear equations � a ij · v j = b i . Title Page • There are efficient algorithms for solving such systems; ◭◭ ◮◮ e.g., Gauss elimination takes time O ( n 3 ). ◭ ◮ • Problem: there are exponentially many size- n subsets. Page 11 of 32 • Idea: start with any vertex, and then replace one of the Go Back constraints so as to increase the objective function. Full Screen • This idea – known as simplex method – leads to a very efficient algorithm which is still used. Close Quit
Recommend
More recommend
