Solving Minimax Problems with Feasible Sequential Quadratic - - PowerPoint PPT Presentation

▶
solving minimax problems with feasible sequential
SMART_READER_LITE
LIVE PREVIEW

Solving Minimax Problems with Feasible Sequential Quadratic - - PowerPoint PPT Presentation

Jun 01, 2023 379 likes •772 views

Qianli Deng (Shally) E-mail: dqianli@umd.edu Advisor: Dr. Mark A. Austin Department of Civil and Environmental Engineering Institute for Systems Research University of Maryland, College Park, MD 20742 E-mail: austin@isr.umd.edu Solving

slide-1
SLIDE 1

Solving Minimax Problems with Feasible Sequential Quadratic Programming

05/06/2014

Qianli Deng (Shally) E-mail: dqianli@umd.edu Advisor: Dr. Mark A. Austin Department of Civil and Environmental Engineering Institute for Systems Research University of Maryland, College Park, MD 20742 E-mail: austin@isr.umd.edu

slide-2
SLIDE 2

Constrained minimax problem

  • ( )

1,..., ( ) , 1,..., ( ) 1,..., ( ) , 1,...,

B j NI j j j LI j NE j j j LE

bl x bu n g x j n g x c x d j n h x j n h x a x b j n ≤ ≤   ≤ =   ≡ − ≤ =   = =   ≡ − = = 

slide-3
SLIDE 3

Constrained minimax problem

  • ( )

1,..., ( ) , 1,..., ( ) 1,..., ( ) , 1,...,

B j NI j j j LI j NE j j j LE

bl x bu n g x j n g x c x d j n h x j n h x a x b j n ≤ ≤   ≤ =   ≡ − ≤ =   = =   ≡ − = = 

* Objectives

slide-4
SLIDE 4

Constrained minimax problem

  • * Bounds

* Nonlinear inequality * Linear inequality * Nonlinear equality * Linear equality

( ) 1,..., ( ) , 1,..., ( ) 1,..., ( ) , 1,...,

B j NI j j j LI j NE j j j LE

bl x bu n g x j n g x c x d j n h x j n h x a x b j n ≤ ≤   ≤ =   ≡ − ≤ =   = =   ≡ − = = 

* Objectives

slide-5
SLIDE 5

FSQP Algorithm

A random point A point fitting all linear constraints A point fitting all constraints An optimal point fitting all constraints

slide-6
SLIDE 6

FSQP Algorithm

A random point A point fitting all linear constraints A point fitting all constraints An optimal point fitting all constraints

slide-7
SLIDE 7

FSQP Algorithm

A random point A point fitting all linear constraints

  • boundary + linear inequality + linear equality

A point fitting all constraints An optimal point fitting all constraints

slide-8
SLIDE 8

FSQP Algorithm

A random point A point fitting all linear constraints

  • boundary + linear inequality + linear equality

A point fitting all constraints

  • boundary + linear inequality + linear equality + nonlinear inequality

+ nonlinear equality

An optimal point fitting all constraints

slide-9
SLIDE 9

FSQP Algorithm

A random point A point fitting all linear constraints

  • boundary + linear inequality + linear equality

A point fitting all constraints

  • boundary + linear inequality + linear equality + nonlinear inequality

+ nonlinear equality

An optimal point fitting all constraints

slide-10
SLIDE 10

FSQP Algorithm

Step 1. Initialization Step 2. A search line Step 3. Line search Step 4. Updates

  • 1

k

x +

– direction – distance – a point; – Hessian matrix

k

x

k

H

k

d

k

t

1 k

H +

– an update point; – an update Hessian matrix

slide-11
SLIDE 11

Initialization

  • 0, j

x H p k

{ }

1

( , ) max ( ) ( ) 1,...,

NE f

n m i j j i I j NE

f x p f x p h x j n

∈ =

= − =

∑

= an initial point fitting all constraints = an identity matrix = a constant = 0

slide-12
SLIDE 12

Computation of a search line

1 min , '( , , ) 2 . . ( ) ( ), 0, 1,..., 1,..., ( ) ( ), 0, 1,..., ,

k k k d k j k j k i e j k j k e e j k j

d H d f x d p bl x d bu st g x g x d j t j n h x h x d j t n a x d b  +   ≤ + ≤    + ∇ ≤ =   = + ∇ ≤  = −  + = 

1

1 1 , 1 1 1 1 1 1

min , 2 . . '( , , ) ( ) ( ), 1,..., , 1,..., ( ) ( ), 1,..., , 1,...,

n

k k d k k k j k j k i j k j i i j k j k e j k j e e

d d d d s t bl x d bu f x d p g x g x d j n c x d d j t n h x h x d j n a x d b j t n

γ

η γ γ γ γ

∈ ∈

 − − +   ≤ + ≤   ≤   + ∇ ≤ =   + ≤ = −   + ∇ ≤ =   + = = −  

ℝ ℝ

|| || /(|| || )

k k k k

d d v

κ κ

ρ = +

1 1

max(0.5,|| || )

k k

v d

τ

=

slide-13
SLIDE 13

Line search

1

, '( , , ) '( , , ) max{ ( ) ( ), } ( ) ( ),

NE f f

k k k i e k k k k i e n i i j i I i I j

d H d n n f x d p n n f x d p f x f x d f x p h x d δ

∈ =

− + =  =  + ≠   = + ∇ − − ∇

∑

  • Armijo line search

( , ) ( , ) ( ) 0, 1,..., , , 1,..., ( ) 0, 1,...,

m k k k k m k k k j k k NI j k k j LI j k k NE

f x t d p f x p t g x td j n c x td d j n h x td j n α δ + ≤ +  + ≤ =   + ≤ =   + ≤ =  

No nonlinear constraints

1

( , ) max{ ( )} ( )

NE f

n m i j j i I j

f x p f x p h x

∈ =

= −∑

slide-14
SLIDE 14

Updates

  • 1

k k k k

x x t d

+ =

+

1 k

H + updated by BFGS formula with Powell’s modification

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

, ( , ) ( , ) 1, 0.2 0.8 (1 )

k k k k x k k x k k T T k k k k k T k k k k T T k k k k k k k k k k k T T k k k k k k k k T T k k k k k

x x L x p L x p H H

  • therwise

H H H H H H H η γ η γ η η θ η η η η η γ ξ θ γ θ γ η η ξ ξ η η η ξ

+ + + + + + + + + + + + + + + + + + + + + + + + + + + +

= − = ∇ − ∇  ≥  =   −  = ⋅ + − ⋅ = − +

1 +

, , 1 1, 1 ,

max{ , }

k j k j j k j j k j

p p p p

  • therwise

µ ε ε µ δ

+

+ ≥  =  − 

1 k k = +

slide-15
SLIDE 15

Implementation

  • JFSQP.java

Initial.java QP.java MiniMax.java Direction_d0.java QP.java KKT.java Direction_d1.java QP.java Arcsearch.java Check.java BFGS_Powell.java MiniMax.java Direction_d0.java QP.java KKT.java Direction_d1.java QP.java Arcsearch.java Check.java BFGS_Powell.java

slide-16
SLIDE 16

Implementation

  • JFSQP.java

Initial.java QP.java MiniMax.java Direction_d0.java QP.java KKT.java Direction_d1.java QP.java Arcsearch.java Check.java BFGS_Powell.java MiniMax.java Direction_d0.java QP.java KKT.java Direction_d1.java QP.java Arcsearch.java Check.java BFGS_Powell.java Loop 2. Find optimal point Loop 1. Find initial feasible point

slide-17
SLIDE 17

Implementation

  • JFSQP.java

Initial.java QP.java MiniMax.java Direction_d0.java QP.java KKT.java Direction_d1.java QP.java Arcsearch.java Check.java BFGS_Powell.java MiniMax.java Direction_d0.java QP.java KKT.java Direction_d1.java QP.java Arcsearch.java Check.java BFGS_Powell.java Loop 2. Find optimal point Loop 1. Find initial feasible point

slide-18
SLIDE 18

Implementation

  • JFSQP.java

Initial.java QP.java MiniMax.java Direction_d0.java QP.java KKT.java Direction_d1.java QP.java Arcsearch.java Check.java BFGS_Powell.java MiniMax.java Direction_d0.java QP.java KKT.java Direction_d1.java QP.java Arcsearch.java Check.java BFGS_Powell.java

Find a point fitting all linear constraints Find point fitting all constraints (linear + nonlinear) Find the

  • ptimal

point also fitting all constraints (linear + nonlinear)

slide-19
SLIDE 19

Implementation

  • Three open-source Java libraries were leveraged:

1) JAMA (Version 1.0.3; 11/2012)

http://math.nist.gov/javanumerics/jama/

2) Apache Commons.Lang (Version 3.3.2; 04/2014)

http://commons.apache.org/

3) Joptimizer (Version 3.3.0; 04/2014)

http://www.joptimizer.com/

slide-20
SLIDE 20

Validation

  • ,

JFSQP.java Initial.java QP.java MiniMax.java Direction_d0.java QP.java KKT.java Direction_d1.java QP.java Arcsearch.java Check.java BFGS_Powell.java

slide-21
SLIDE 21

Validation

  • ,

JFSQP.java Initial.java QP.java MiniMax.java Direction_d0.java QP.java KKT.java Direction_d1.java QP.java Arcsearch.java Check.java BFGS_Powell.java testJFSQP.java testInitial.java testQP.java testMiniMax.java testQP.java testKKT.java testQP.java testBFGS_Powell.java

slide-22
SLIDE 22

Validation

  • ,

JFSQP.java Initial.java QP.java MiniMax.java Direction_d0.java QP.java KKT.java Direction_d1.java QP.java Arcsearch.java Check.java BFGS_Powell.java testJFSQP.java testInitial.java testQP.java testMiniMax.java testQP.java testKKT.java testQP.java testBFGS_Powell.java

slide-23
SLIDE 23

Validation

  • ,

JFSQP.java Initial.java QP.java MiniMax.java Direction_d0.java QP.java KKT.java Direction_d1.java QP.java Arcsearch.java Check.java BFGS_Powell.java testJFSQP.java testInitial.java testQP.java testMiniMax.java testQP.java testKKT.java testQP.java testBFGS_Powell.java

slide-24
SLIDE 24

Validation

  • ,

JFSQP.java Initial.java QP.java MiniMax.java Direction_d0.java QP.java KKT.java Direction_d1.java QP.java Arcsearch.java Check.java BFGS_Powell.java testJFSQP.java testInitial.java testQP.java testMiniMax.java testQP.java testKKT.java testQP.java testBFGS_Powell.java

slide-25
SLIDE 25

Validation

  • ,

JFSQP.java Initial.java QP.java MiniMax.java Direction_d0.java QP.java KKT.java Direction_d1.java QP.java Arcsearch.java Check.java BFGS_Powell.java testJFSQP.java testInitial.java testQP.java testMiniMax.java testQP.java testKKT.java testQP.java testBFGS_Powell.java

slide-26
SLIDE 26

Validation

  • ,

testJFSQP.java testInitial.java testQP.java testMiniMax.java testQP.java testKKT.java testQP.java testBFGS_Powell.java

slide-27
SLIDE 27

Validation

  • ,

QP with LI, LE consts QP with LI, LE consts linear consts in QP QP with linear consts Same program checking with Matlab testJFSQP.java testInitial.java testQP.java testMiniMax.java testQP.java testKKT.java testQP.java testBFGS_Powell.java

slide-28
SLIDE 28

Validation

  • ,

LP, QP, NP NI, LI, NE, LE consts QP with LI, LE consts Feasible point, NP with consts QP with LI, LE consts linear consts in QP QP with linear consts Same program checking with Matlab testJFSQP.java testInitial.java testQP.java testMiniMax.java testQP.java testKKT.java testQP.java testBFGS_Powell.java

slide-29
SLIDE 29

Example 1

  • 1

2 1 2

1 1 x x x x − ≤   − ≤   − ≤   − ≤ 

Reference: Hock, Willi, and Klaus Schittkowski. "Test examples for nonlinear programming codes." Journal of Optimization Theory and Applications 30.1 (1980): 127-129.

2

1 2

min ( )

x

f x x x

∈

= − +

ℝ

Known minimizer:

,

* *

( 3,2) ; ( ) 5 (1,0) ; ( ) 1

T T

x f x x f x = − = = = −

slide-30
SLIDE 30

Example 1

  • 1

2 1 2

1 1 x x x x − ≤   − ≤   − ≤   − ≤ 

Reference: Hock, Willi, and Klaus Schittkowski. "Test examples for nonlinear programming codes." Journal of Optimization Theory and Applications 30.1 (1980): 127-129.

2

1 2

min ( )

x

f x x x

∈

= − +

ℝ

Known minimizer:

,

* *

( 3,2) ; ( ) 5 (1,0) ; ( ) 1

T T

x f x x f x = − = = = −

slide-31
SLIDE 31

Example 2

  • 1

2 3 3 1 2 3 1 2 3

, , 6 4 3 1 x x x x x x x x x ≥   − − + ≤   − − − = 

Reference: Hock, Willi, and Klaus Schittkowski. "Test examples for nonlinear programming codes." Journal of Optimization Theory and Applications 30.1 (1980): 127-129.

3

2 2 1 2 3 1 2

min ( ) ( 3 ) 4( )

x

f x x x x x x

∈

= + + + −

ℝ

Known global minimizer:

,

* *

(0,0,1) ; ( ) 1

T

x f x = =

slide-32
SLIDE 32

Example 2

  • ,
slide-33
SLIDE 33

Example 2

slide-34
SLIDE 34

Project Schedule

  • October
  • Literature review;
  • Specify the implementation module details;
  • Structure the implementation;

November

  • Develop the quadratic programming module;
  • Unconstrained quadratic program;
  • Strictly convex quadratic program;
  • Validate the quadratic programming module;

December

  • Develop the Gradient and Hessian matrix calculation module;
  • Validate the Gradient and Hessian matrix calculation module;
  • Midterm project report and presentation;
slide-35
SLIDE 35
  • January
  • Develop Armijo line search module;
  • Validate Armijo line search module;

February

  • Develop the feasible initial point module;
  • Validate the feasible initial point module;
  • Integrate the program;

March

  • Debug and document the program;
  • Validate and test the program with case application;

April

  • Add arch search variable

;

  • Compare calculation efficiency of line search with arch search methods;

May

  • Develop the user interface if time available;
  • Final project report and presentation;

d ɶ

slide-36
SLIDE 36

Deliverables

  • ,

Project proposal Algorithm description Well-documented Java codes Test cases: LP, QP, NP with NI, LI, NE, LE constraints Validation results Presentations Project reports

slide-37
SLIDE 37

Bibliography