Chapter 4: Problem Statement CK Cheng Dept. of Computer Science and - - PowerPoint PPT Presentation

1

CSE203B Convex Optimization: Chapter 4: Problem Statement

CK Cheng

• Dept. of Computer Science and Engineering

University of California, San Diego

Convex Optimization Formulation

2

• 1. Introduction

I. Eliminating equality constants II. Slack variables

• III. Absolute values, softmax
• 2. Optimality Conditions

I. Local vs. global optimum II. Optimality criterion for differentiable 𝑔 i. Optimization without constraints ii.

• Opt. with inequality constraints
• iii. Opt. with equality constraints
• III. Quasi-convex optimization
• 3. Linear Optimization
• 4. Quadratic Optimization
• 5. Geometric Programming
• 6. Generalized Inequality Constraints

• 1. Introduction

3

Formulation: One of the most critical processes to conduct a

• project. min 𝑔

0(𝑦)

𝑡. 𝑢. 𝑔

𝑗 𝑦 ≤ 0 𝑗 = 1, … , 𝑛

ℎ𝑗 𝑦 = 0 𝑗 = 1, … , 𝑞 (𝐵𝑦 = 𝑐 Affine set) 𝑦 ∈ 𝑆𝑜 𝐸

𝑔

0 𝑔

0: 𝑆𝑜 → 𝑆

𝐸

𝑔𝑗 𝑔 𝑗: 𝑆𝑜 → 𝑆

𝐸ℎ𝑗 ℎ𝑗: 𝑆𝑜 → 𝑆 𝑔

0, 𝑔 𝑗, … , 𝑔 𝑛 𝑏𝑠𝑓 𝑑𝑝𝑜𝑤𝑓𝑦

𝐸 =∩𝑗=0,𝑛 𝐸

𝑔 ∩𝑗=0,𝑞 𝐸ℎ𝑗 Domain of functions, but not the

feasible set. Feasible Set: The set which satisfies the constraints (is convex for convex problems).

1.1 Introduction: Eliminating Equality Constraints

4

min 𝑔

0(𝑦)

𝑡. 𝑢. 𝑔

𝑗 𝑦 ≤ 0 𝑗 = 1, … , 𝑛

𝐵𝑦 = 𝑐 a. Convert 𝑦 𝐵𝑦 = 𝑐 𝑢𝑝 𝐺𝑨 + 𝑦0 𝑨 ∈ 𝑆𝑙

• b. We have a equivalent problem

min 𝑔

0(𝐺𝑨 + 𝑦0)

𝑡. 𝑢. 𝑔

𝑗 𝐺𝑨 + 𝑦0 ≤ 0

Remark: Matrix 𝐺 contains columns of null space basis

1.2 Introduction: Slack Variables

5

Add slack variables to convert to an equivalent problem a. Convert the objective function with variable t min 𝑢 𝑡. 𝑢. 𝑔

0 𝑦 − 𝑢 ≤ 0

𝑔

𝑗 𝑦 ≤ 0, 𝑗 = 1, … , 𝑛

𝐵𝑈𝑦 = 𝑐

• b. Convert the inequality with variables 𝑡𝑗

min 𝑔

0(𝑦)

𝑡. 𝑢. 𝑔

𝑗 𝑦 + 𝑡𝑗 = 0

𝐵𝑈𝑦 = 𝑐 𝑡𝑗 ∈ 𝑆+, 𝑗 = 1, … , 𝑛 min 𝑔

0(𝑦)

𝑡. 𝑢. 𝑔

𝑗 𝑦 ≤ 0, 𝑗 = 1, … , 𝑛

𝐵𝑦 = 𝑐

1.3 Introduction: Absolute values and Softmax

6

• a. Absolute values

𝑔

𝑗(𝑦) ≤ 𝑐

⇒ 𝑔

𝑗 𝑦 ≤ 𝑐 𝑏𝑜𝑒

−𝑔

𝑗 𝑦 ≤ 𝑐

• b. Maximum values

max{𝑔

1, 𝑔 2, … , 𝑔 𝑛}

Soft𝑛𝑏𝑦:

1 𝛽 log ( 𝑓𝛽𝑔

1 + 𝑓𝛽𝑔 1 + ⋯ + 𝑓𝛽𝑔 𝑛)

𝐹𝑦𝑏𝑛𝑞𝑚𝑓: max{1, 5, 10, 2, 3} ⇒ Softmax 1 𝛽 log(𝑓𝛽 + 𝑓5𝛽 + 𝑓10𝛽 + 𝑓2𝛽 + 𝑓3𝛽) ≈ 10

2.1 Optimality Conditions: Local vs. Global Optima

7

Definition: Local Optima Given a convex optimization problem and a point ҧ 𝑦 ∈ 𝑆𝑜 If there exists a 𝑠 > 0 𝑡. 𝑢. 𝑔

0 𝑨 ≥ 𝑔

ҧ 𝑦 for all 𝑨 ∈ Feasible Set, and 𝑨 − ҧ 𝑦 2 ≤ 𝑠 Then ҧ 𝑦 is a local optimum.

2.2 Optimality Conditions

8

Theorem: Given a convex opt. problem If ҧ 𝑦 is a local optimum, then ҧ 𝑦 is a global optimum Proof: By contradiction Suppose that ∃𝑧 ∈ 𝐺𝑓𝑏𝑡𝑗𝑐𝑚𝑓 𝑇𝑓𝑢 𝑡. 𝑢. 𝑔 ҧ 𝑦 > 𝑔

0 𝑧

We have 𝑔 ҧ 𝑦 > 1 − 𝜄 𝑔 ҧ 𝑦 + 𝜄𝑔

0 ത

𝑧 𝑐𝑧 𝑏𝑡𝑡𝑣𝑛𝑞𝑢𝑗𝑝𝑜 > 𝑔

0( 1 − 𝜄

ҧ 𝑦 + 𝜄ത 𝑧) 𝑔

0 𝑗𝑡 𝑑𝑝𝑜𝑤𝑓𝑦

And 1 − 𝜄 ҧ 𝑦 + 𝜄ത 𝑧 is feasible (Feasible set is convex) The inequality contradicts to the assumption of local optima.

2.2 Optimality Criterion for Differentiable 𝑔

0 𝑦

9

Theorem: If 𝛼𝑔

0 𝑦 𝑈 𝑧 − 𝑦 ≥ 0, for a given 𝑦 ∈Feasible Set

and for all 𝑧 ∈ Feasible Set, then 𝑦 is optimal. (i. e. 𝐿 = 𝑧 − 𝑦 𝑧 ∈ 𝑔𝑓𝑏𝑡𝑗𝑐𝑚𝑓 𝑡𝑓𝑢 , ∇𝑔

0 𝑦 ∈ 𝐿∗)

Proof: From the first order condition of convex function, we have 𝑔

0 𝑧 ≥ 𝑔 0 𝑦 + 𝛼𝑔 0 𝑦 𝑈(𝑧 − 𝑦).

Given the condition that 𝛼𝑔

𝑈 𝑦

𝑧 − 𝑦 ≥ 0, ∀𝑧 in feasible set. We have 𝑔

0 𝑧 ≥ 𝑔 0 𝑦 , ∀𝑧 in feasible set, which implies that 𝑦

is optimal. Remark: 𝛼𝑔

𝑈 𝑦

𝑧 − 𝑦 = 0 is a supporting hyperplane to feasible set at 𝑦.

2.2.1 Optimality Criterion without Constraints

10

Theorem: For problem min 𝑔

0 𝑦 , 𝑦 ∈ 𝑆𝑜, where 𝑔 0 is convex,

the optimal condition is ∇𝑔

0 𝑦 = 0.

Proof: (∇𝑔

0 𝑦 = 0 ⇒ Optimality)

Since 𝑔

0 𝑧 ≥ 𝑔 0 𝑦 + 𝛼𝑔 0 𝑦 𝑈 𝑧 − 𝑦 , ∀𝑦, 𝑧 ∈ 𝑆𝑜 (first order

condition of convex function) We have 𝑔

0 𝑧 ≥ 𝑔 0 𝑦 .

Therefore, x is an optimal solution. (∇𝑔

0 𝑦 = 0 ⇐ Optimality) By contradiction

2.2.2 Opt. with Inequality Constraints

11

Problem: Min 𝑔

0 𝑦

s.t. 𝐵𝑦 ≤ 𝑐, 𝐵 ∈ 𝑆𝑛×𝑜 Suppose that 𝐵 ҧ 𝑦 = 𝑐 (one particular case). Let 𝑦 = ҧ 𝑦 + 𝑣. We can write ቊmin 𝑔 ҧ 𝑦 + 𝑣 𝐵𝑣 ≤ 0

• Opt. condition: 𝛼𝑔

0 𝑦 𝑈𝑣 ≥ 0, ∀{𝑣|𝐵𝑣 ≤ 0} ≡ 𝐿

In other words, 𝛼𝑔 ҧ 𝑦 ∈ 𝐿∗ 𝑝𝑔 𝐿 = 𝑣 𝐵𝑣 ≤ 0 𝑏𝑜𝑒 𝐿∗ = {−𝐵𝑈𝑤|𝑤 ≥ 0} i.e. 𝛼𝑔 ҧ 𝑦 = −𝐵𝑈𝑤, ∃𝑤 ∈ 𝑆+

𝑛

𝛼𝑔

0( ҧ

𝑦) + 𝐵𝑈𝑤 = 0, 𝑤 ≥ 0.

2.2.3 Opt. with Equality Constraints

12

ቊ min 𝑔

0 𝑦

𝑡. 𝑢. 𝐵𝑦 = 𝑐 Let 𝑦 = ҧ 𝑦 + 𝑣 and 𝐵 ҧ 𝑦 = 𝑐, we have ቊmin 𝑔 ҧ 𝑦 + 𝑣 𝐵𝑣 = 0 , 𝐿 = {𝑣|𝐵𝑣 = 0} 𝛼𝑔 ҧ 𝑦 ∈ 𝐿∗, 𝐿∗ = {𝐵𝑈𝑤|𝑤 ∈ 𝑆𝑞} 𝛼𝑔 ҧ 𝑦 + 𝐵𝑈𝑤 = 0 Let 𝐿1= 𝑣 𝐵𝑣 ≥ 0 𝐿2 = 𝑣 −𝐵𝑣 ≥ 0 We have 𝐿1

∗ = 𝐵𝑈𝑤 𝑤 ≥ 0

𝐿2

∗ = −𝐵𝑈𝑤 𝑤 ≥ 0 = 𝐵𝑈𝑤 𝑤 ≤ 0

(𝐿1∩ 𝐿2)∗ = 𝐿1

∗ ∪ 𝐿2 ∗ = {𝐵𝑈𝑤|𝑤 ∈ 𝑆𝑞}

2.2.3 Opt. with Equality Constraints: Example

13

min

𝑦 𝑔 𝑦 = 𝑦1 2 + 𝑦2 2

𝑡. 𝑢. 2 1 𝑦1 𝑦2 = 3 We can derive 𝑦∗ = 𝑦1

∗, 𝑦2 ∗ = ( 6 5 , 3 5)

𝛼𝑔 𝑦∗ = 2𝑦1

∗

2𝑦2

∗ = 12 5 6 5

, 𝛼𝑔 𝑦∗ + 𝐵𝑈𝑤 =

12 5 6 5

+ 2 1 × −

6 5 = 0

New Problem: 𝛼𝑔 𝑦 + 𝐵𝑈𝑤 = 0 𝐵𝑦 = 𝑐 ⇒ 2𝑦1 2𝑦2 + 2 1 𝑤 = 0 2 1 𝑦1 𝑦2 = 3

2.3 Quasiconvex Functions

14

𝑔: 𝑆𝑜 → 𝑆 is called quasiconvex (unimodal) sublevel set 𝑇𝑢 = 𝑦 𝑦 ∈ 𝑒𝑝𝑛 𝑔, 𝑔 𝑦 ≤ 𝑢} if its domain and all sublevel sets 𝑇𝑢, ∀𝑢 ∈ 𝑆 are convex, 𝑔: 𝑆𝑜 → 𝑆 is called quasiconcave if −𝑔 is quasiconvex. 𝑔(𝑦) quasiconvex and quasiconcave → quasilinear Ex: log 𝑦, 𝑦 ∈ 𝑆++

15

Ex: Ceiling function 𝐷𝑓𝑗𝑚 𝑦 = inf 𝑨 ∈ 𝑎 𝑨 > 𝑦 : quasilinear Ex: 𝑔 𝑦1, 𝑦2 = 𝑦1𝑦2 =

1 2 𝑦1

𝑦2 1 1 𝑦1 𝑦2 is quasiconcave in 𝑆+

2, 𝑇𝑢 = 𝑦 ∈ 𝑆+ 2 𝑦1𝑦2 ≥ 𝑢}

Ex: 𝑔 𝑦 =

𝑏𝑈𝑦+𝑐 𝑑𝑈𝑦+𝑒 for 𝑑𝑈𝑦 + 𝑒 > 0

𝑇𝑢 = 𝑦 𝑑𝑈𝑦 + 𝑒 > 0, 𝑏𝑈 + 𝑐 ≤ 𝑢(𝑑𝑈𝑦 + 𝑒)}

• pen halfspace closed halfspace

→ 𝑇𝑢 is convex (𝑢 is given here) → 𝑔(𝑦) is ൠ quasiconvex quasiconcave → quasilinear

2.3 Quasiconvex Functions

2.3 Quasiconvex Optimization

16

min 𝑔

𝑝(𝑦) (𝑔 𝑝(𝑦) is quasiconvex, 𝑔 𝑗′𝑡 are convex.)

𝑡. 𝑢. 𝑔

𝑗 𝑦 ≤ 0, 𝑗 = 1, … , 𝑛

𝐵𝑦 = 𝑐 Remark: A locally opt. solution (𝑦, 𝑔

0 𝑦 ) may not be globally opt.

Algorithm: Bisection method for quasiconvex optimization. Given 𝑚 ≤ 𝑞∗ ≤ 𝑣, 𝜗 > 0 Repeat 1. 𝑢 = (𝑚 + 𝑣)/2

• 2. Find a feasible solution 𝑦:

𝑡. 𝑢. Φ𝑢 𝑦 ≤ 0 𝑔

0 𝑦 ≤ 𝑢 ⇔ Φt 𝑦 ≤ 0

𝑔

𝑗 𝑦 ≤ 0

𝐵𝑦 = 𝑐

• 3. If solution is feasible, 𝑣 = 𝑢, 𝑓𝑚𝑡𝑓 𝑚 = 𝑢

Until 𝑣 − 𝑚 ≤ 𝜗 Ex: 𝑔 𝑦 =

𝑞 𝑦 𝑟 𝑦 ≤ 𝑢 → 𝑞 𝑦 − 𝑢𝑟 𝑦 ≤ 0 (p is convex & q is

concave) Find a convex function

• 3. Linear Programming: Format

17

General Form : min 𝑑𝑈𝑦 𝑡. 𝑢. 𝐻𝑦 ≤ ℎ, 𝐻 ∈ 𝑆𝑛∗𝑜, 𝐵 ∈ 𝑆𝑞∗𝑜 𝐵𝑦 = 𝑐 Standard Form : min 𝑑𝑈𝑦 𝑡. 𝑢. 𝐵𝑦 = 𝑐 𝑦 ≥ 0 Remark: Figure out three possible situations

• 1. No feasible solutions
• 2. Unbounded solutions
• 3. Bounded solutions

18

min 𝑑𝑈𝑦 𝑡. 𝑢. 𝐵𝑦 = 𝑐 (1) No feasible solutions: 𝑐 ∉ 𝑆(𝐵) (b is not in the range of A) e.g. 1 1 1 2 2 3 𝑦1 𝑦2 = 2 2 3 (2) Unbounded solutions: 𝑐 ∈ 𝑆(𝐵) but 𝑑 ∉ 𝑆(𝐵𝑈) e.g. min 1 1 𝑦1 𝑦2 1 2 𝑦1 𝑦2 = 2 (The solution → −∞) (3) Bounded solutions: b∈ 𝑆 𝐵 , 𝑑 ∈ 𝑆 𝐵𝑈 e.g. min 1 1 𝑦1 𝑦2 1 1 1 2 𝑦1 𝑦2 = 2 2 Thus 𝑦∗ = 2 0 , 𝑔 𝑦∗ = 1 1 2 0 = 2

• 3. Linear Programming: Cases

• 3. Linear Fractional Programming

19

P1: min 𝑔

𝑝(𝑦) = 𝑑𝑈𝑦+𝑒 𝑓𝑈𝑦+𝑔 , 𝑒𝑝𝑛 𝑔 𝑝 = 𝑦 𝑓𝑈𝑦 + 𝑔 > 0}

𝑡. 𝑢. 𝐻𝑦 ≼ ℎ 𝐵𝑦 = 𝑐 P1⇒P2: Let 𝑧 =

𝑦 𝑓𝑈𝑦+𝑔 ,

𝑨 =

1 𝑓𝑈𝑦+𝑔

P2: min 𝑑𝑈𝑧 + 𝑒𝑨 𝑡. 𝑢. 𝐻𝑧 − ℎ𝑨 ≤ 0 𝐵𝑧 − 𝑐𝑨 = 0 𝑓𝑈𝑧 + 𝑔𝑨 = 1 𝑨 ≥ 0

• 4. Quadratic Opt. Problems (QP)

20

QP : min

1 2 𝑦𝑈𝑄𝑦 + 𝑟𝑈𝑦 + 𝑠

𝑡. 𝑢. 𝐻𝑦 ≼ ℎ 𝐵𝑦 = 𝑐 𝑄 ∈ 𝑇+

𝑜, 𝐻 ∈ 𝑆𝑛×𝑜, 𝐵 ∈ 𝑆𝑞×𝑜

QCQP : (Quadratically Constrained Quadratic Program) min

1 2 𝑦𝑈𝑄 𝑝𝑦 + 𝑟𝑝 𝑈𝑦 + 𝑠 𝑝

𝑡. 𝑢.

1 2 𝑦𝑈𝑄𝑗𝑦 + 𝑟𝑗 𝑈𝑦 + 𝑠 𝑗 ≤ 0, 𝑗 = 1, … , 𝑛

𝐵𝑦 = 𝑐 𝑄𝑗 ∈ 𝑇+

𝑜, 𝑗 = 0,1, … , 𝑛

• 4. Quadratic Opt. Problems (SOCP)

21

SOCP : (Second-Order Cone Program) min 𝑔𝑈𝑦 𝑡. 𝑢. 𝐵𝑗𝑦 + 𝑐𝑗

2 ≤ 𝑑𝑗 𝑈𝑦 + 𝑒𝑗, 𝑗 = 1, … , 𝑛

F𝑦 =g 𝑇𝑃𝐷𝑄: (𝐵𝑦 + 𝑐, 𝑑𝑈𝑦 + 𝑒) lies in the second order cone 𝑧, 𝑢 𝑧

2 ≤ 𝑢, 𝑧 ∈ 𝑆𝑙}

QCQP viewed as SOCP QCQP constraint: 𝑦𝑈𝐵𝑈𝐵𝑦 + 𝑐𝑈𝑦 + 𝑑 ≤ 0 can be expressed as a SOCP constraint: 1 + 𝑐𝑈𝑦 + 𝑑 2 𝐵𝑦

2

≤ (1 − 𝑐𝑈𝑦 − 𝑑)/2

• 4. Quadratic Opt. Problems (SOCP)

22

SOCP : (Second-Order Cone Program) min 𝑔𝑈𝑦 𝑡. 𝑢. 𝐵𝑗𝑦 + 𝑐𝑗

2 ≤ 𝑑𝑗 𝑈𝑦 + 𝑒𝑗, 𝑗 = 1, … , 𝑛

F𝑦 =g Example: SOCP constraint: 𝑦1 𝑦2

2 ≤ 2𝑦1 + 1, feasible region

• 4. Quadratic Opt. Problems (SOCP)

23

SOCP : (Second-Order Cone Program) min 𝑔𝑈𝑦 𝑡. 𝑢. 𝐵𝑗𝑦 + 𝑐𝑗

2 ≤ 𝑑𝑗 𝑈𝑦 + 𝑒𝑗, 𝑗 = 1, … , 𝑛

F𝑦 =g 𝑇𝑃𝐷𝑄: (𝐵𝑦 + 𝑐, 𝑑𝑈𝑦 + 𝑒) lies in the second order cone 𝑧, 𝑢 𝑧

2 ≤ 𝑢, 𝑧 ∈ 𝑆𝑙}

SOCP viewed as a Semidefinite Program Problem SOCP constraint: 𝐵𝑦 + 𝑐

2 ≤ 𝑑𝑈𝑦 + 𝑒

can be expressed as a Semidefinite Program constraint: 𝑑𝑈𝑦 + 𝑒 𝐽 𝐵𝑦 + 𝑐 𝐵𝑦 + 𝑐 𝑈 𝑑𝑈𝑦 + 𝑒 ≽ 0

• 5. Geometric Programming

24

𝑔 𝑦 = ෍

𝑙=1 𝑙

𝑑𝑙𝑦1

𝑏1𝑙𝑦2 𝑏2𝑙 … 𝑦𝑜 𝑏𝑜𝑙 ,

𝑑𝑙 > 0, 𝑏𝑗𝑙 ∈ 𝑆, 𝑦 ∈ 𝑆++

𝑜

Each term is called monomial 𝑔(𝑦) is called posynomial Geometric Program: min 𝑔

𝑝 𝑦

s.t. 𝑔

𝑗 𝑦 ≤ 1, 𝑗 = 1, … , 𝑛

ℎ𝑗 𝑦 = 1, 𝑗 = 1, … , 𝑞 𝑦 > 0 𝑔

𝑗s are posynomials

ℎ𝑗s are monomials

• 5. Geometric programing in convex form

25

monomial 𝑔 𝑦 = 𝑑𝑦1

𝑏1 … 𝑏𝑜 𝑏𝑜, 𝑦 ∈ 𝑆++ 𝑜

log 𝑔(𝑓𝑧1, … , 𝑓𝑧𝑜) = 𝑏𝑈𝑧 + 𝑐, 𝑐 = log 𝑑 polynomial 𝑔 𝑦 = σ𝑙=1

𝐿

𝑑𝑙𝑦1

𝑏1𝑙 … 𝑦𝑜 𝑏𝑜𝑙

log 𝑔 𝑓𝑧1 … 𝑓𝑧𝑜 = log σ𝑙=1

𝐿

𝑓𝑏𝑙

𝑈𝑧+𝑐𝑙 , 𝑐𝑙 = log 𝑑𝑙

Geometric program transform min log(σ𝑙=1

𝐿0 𝑓𝑏𝑝𝑙

𝑈 𝑧+𝑐𝑝𝑙)

𝑡𝑣𝑐𝑘𝑓𝑑𝑢 𝑢𝑝 log σ𝑙=1

𝐿𝑗

𝑓𝑏𝑗𝑙

𝑈 𝑧+𝑐𝑗𝑙 ≤ 0, 𝑗 = 1, … , 𝑛

𝐻𝑧 + 𝑒 = 0

• 6. Generalized Inequality Constraints

26

min 𝑔

𝑝(𝑦)

𝑡. 𝑢. 𝑔

𝑗 𝑦 ≼𝐿𝑗 0

𝐵𝑦 = 𝑐 (𝑦 ≼𝐿 𝑧 → 𝑧 − 𝑦 ∈ 𝐿) Semidefinite Programming (SDP) min 𝑑𝑈𝑦 𝑡. 𝑢. 𝑦1𝐺

1 + ⋯ + 𝑦𝑜𝐺 𝑜 + 𝐻 ≼ 0

𝐵𝑦 = 𝑐 𝐻, 𝐺

1, … , 𝐺 𝑜 ∈ 𝑇𝑙, 𝐵 ∈ 𝑆𝑞×𝑜

Standard Form SDP min 𝑢𝑠(𝐷𝑌) 𝑡. 𝑢. 𝑢𝑠 𝐵𝑗𝑌 = 𝑐𝑗, 𝑗 = 1, … , 𝑞 𝑌 ≽ 0 𝐷, 𝐵1, … , 𝐵𝑞 ∈ 𝑇𝑜, 𝑌 ∈ 𝑇𝑜

Summary

27

(1). 𝑀𝑄 ⊂ 𝑅𝑄 ⊂ 𝑅𝐷𝑅𝑄 ⊂ 𝑇𝑃𝐷𝑄 ⊂ 𝑇𝐸𝑄 (2). Software Tools (Examples) CVX: Matlab software for disciplined convex (Boyd) CPLEX: IP, LP, QP, SOCP (IBM) Gurobi: LP, QP, MILP, MIQP, MIQCP (Gu, Rothberg, Bixby) (3). Check if the problem is convex

Summary

28

(1). Format of the formulation

• a. Follow the format of the solver (software package)
• b. Find equivalent formulation for simpler approaches

(coding, complexity, accuracy) (2). Feasibility of the solution Check if the feasible set is not empty. (3). Boundness of the solution Check if the solution is bounded (reasonable, not −∞) (4). Optimality of the solution Check the supporting hyperplane of object function