ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS Jos e Mario Mart - - PowerPoint PPT Presentation

▶
order value optimization and new applications jos e mario
SMART_READER_LITE
LIVE PREVIEW

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS Jos e Mario Mart - - PowerPoint PPT Presentation

Feb 19, 2023 36 likes •602 views

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS Jos e Mario Mart nez www.ime.unicamp.br/ martinez UNICAMP, Brazil August 2, 2011 ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

slide-1
SLIDE 1

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS Jos´ e Mario Mart´ ınez www.ime.unicamp.br/∼martinez

UNICAMP, Brazil

August 2, 2011

slide-2
SLIDE 2

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Collaborators

Roberto Andreani (Applied Math - UNICAMP) Leandro Mart´ ınez (Chemistry - UNICAMP) Fl´ avio Yano (Ita´ u Bank) M´ ario Salvatierra (Fed. Univ. Amazonas) Giovane C´ esar (Applied Math - UNICAMP) Roberto Marcondes (Computer Science - USP) Paulo J. Silva (Computer Science - USP) Cibele Dunder (Ita´ u Bank) Lu´ ıs Felipe Bueno (Applied Math - UNICAMP) Lucas Garcia Pedroso (Applied Math - UNICAMP) Maria Aparecida Diniz (Applied Math - UNICAMP) Ernesto Birgin (Computer Science - USP)

slide-3
SLIDE 3

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Outline

Introduce Order-Value Optimization problems Review of Algorithms and Convergence results Applications

slide-4
SLIDE 4

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Order-Value Optimization (OVO) Problems

Let fi : Ω ⊂ Rn → R, i = 1, · · · , m J ⊂ {1, · · · , m} For all x ∈ Ω, we define i1(x), i2(x), . . . , im(x) by fi1(x)(x) ≤ fi2(x)(x) ≤ · · · ≤ fim(x)(x) The OVO problem is: minimize

  • j∈J

fij(x)(x)

x ∈ Ω

slide-5
SLIDE 5

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Examples

J = {m} − → min

x∈Ω max{f1(x), · · · , fm(x)}

J = {1} − → min

x∈Ω min{f1(x), · · · , fm(x)}

J = {p} − → min

x∈Ω fip(x)(x)

(VaR-Like) J = {1, · · · , p} − → min

x∈Ω p

  • j=1

fij(x)(x) (LOVO) J = {p + 1, · · · , m} − → min

x∈Ω m

  • j=p+1

fij(x)(x) (CVaR-Like) J = {q + 1, · · · , p} − → min

x∈Ω p

  • j=q+1

fij(x)(x)

slide-6
SLIDE 6

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Non-smoothness and Many local minimizers

slide-7
SLIDE 7

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

J = {m} (Minimax)

slide-8
SLIDE 8

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

J = {1} (Minimin)

slide-9
SLIDE 9

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

J = {p}, (VaR-like)

In this example, p = 3

slide-10
SLIDE 10

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

J = {1, · · · , p}, (LOVO)

min

x∈Ω

1 p

p

  • j=1

fij(x)(x) In this example, p = 2

slide-11
SLIDE 11

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

J = {p + 1, · · · , m}, (CVaR-like)

min

x∈Ω

1 m − p

m

  • j=p+1

fij(x)(x) In this example, p = 3

slide-12
SLIDE 12

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

J = {q + 1, · · · , p}

min

x∈Ω

1 p − q

p

  • j=q+1

fij(x)(x) In this example, q = 1, p = 4

slide-13
SLIDE 13

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Reformulation of CVaR-like

Fact b1 ≤ b2 ≤ · · · ≤ bm, p ≤ m − 1 ⇒ bp+1 + · · · + bm = Minimum (m − p)ξ +

m

  • i=1

max{0, bi − ξ}

ξ ∈ R

Minimizers= {ξ ∈ [bp, bp+1]} ⇓ Minimize fip+1(x)(x) + · · · + fim(x)(x)

x ∈ Ω

is equivalent to: Minimize (m − p)ξ +

m

  • i=1

max{0, fi(x) − ξ}

x ∈ Ω, ξ ∈ R

slide-14
SLIDE 14

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Reformulation of VaR-like

From the same “fact” Minimize fip(x)(x)

x ∈ Ω

is equivalent to: Minimize ξ (with respect to x ∈ Ω and ξ ∈ R) subject to ξ minimizes (m − p)ξ +

m

  • i=1

max{0, fi(x) − ξ} ( wrt ξ)

slide-15
SLIDE 15

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Consequence for the Reformulations of CVaR-like and VaR-like

CVaR-like is a Nonlinear Programming problem Convex if the fi are convex Linear-Programming if the fi are linear with many inequality constraints VaR-like is a Bilevel Programming problem with many Complementarity Constraints that come from the KKT conditions of the Lower-Level problem

slide-16
SLIDE 16

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Primal method for minimizing CVaR-like

Consider Minimize (m − p)ξ + m

i=1 max{0, fi(x) − ξ} x ∈ Ω, ξ ∈ R

Use smoothing to deal with of max and ordinary NLP for minimizing on Ω × R.

slide-17
SLIDE 17

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Primal Method for Minimizing VaR-like

Given the current point xk ∈ Ω (convex) take a sufficient descent direction dk for all j such that fip(xk)(xk) − ǫ ≤ fj(xk) ≤ fip(xk)(xk) + ǫ Line-search along dk → xk+1 = xk + αkdk Global convergence to (ǫ)stationary points Local Superlinear Quadratic Convex subproblems (linear or quadratic constraints)

slide-18
SLIDE 18

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Risk Minimization

m scenarios fi(x) =predicted loss caused by decision x ∈ Ω under scenario i fip(x)(x) = VaR associated with x 1 m − p

m

  • j=p+1

fij(x)(x) = CVaR associated with x minimize fip(x)(x) ≡ minimize VaR

x ∈ Ω

minimize

m

  • j=p+1

fij(x)(x) ≡ minimize CVaR

x ∈ Ω

slide-19
SLIDE 19

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Low Order-Value Optimization (LOVO)

Define, as always, i1(x), . . . , im(x) by: fi1(x)(x) ≤ · · · ≤ fim(x)(x); p ≤ m Then, the LOVO problem is: minimize

p

  • j=1

fij(x)(x)

x ∈ Ω

slide-20
SLIDE 20

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Fact

p

  • j=1

fij(x)(y) ≤

p

  • j=1

fij(x)(x) ⇒

p

  • j=1

fij(y)(y) ≤

p

  • j=1

fij(x)(x) ⇓ In order to decrease the LOVO function we may “fix” (i1(x), · · · , ip(x)) and “minimize”

p

  • j=1

fij(x)(y) with respect to y.

slide-21
SLIDE 21

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

g(y) =

p

  • j=1

fij(xk)(y) h(y) =

p

  • j=1

fij(xk+1)(y)

slide-22
SLIDE 22

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Methods for Unconstrained LOVO problems

Line-Search At iteration k Find a sufficient descent direction for

p

  • j=1

fij(xk)(x) take

p

  • j=1

fij(xk+αkd)(xk + αkd) <

p

  • j=1

fij(xk)(xk)

suf

Global Convergence to points x∗ such that ∇

p

  • j=1

fij(x∗)(x∗) = 0

slide-23
SLIDE 23

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Trust-Region methods for LOVO

Typical iteration Given xk, the trust region defined by ∆ and a quadratic approximation of

p

  • j=1

fij(xk)(x): Minimize the quadratic approximation on the trust region ∆ If the reduction of

p

  • j=1

fij(xk)(x) is sufficiently large with respect to the reduction of the quadratic approximation (Ared ≥ 0.1 Pred) accept the solution of the trust region subproblem as xk+1. Otherwise, reduce ∆.

slide-24
SLIDE 24

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Convergence of trust-region methods for LOVO

At every limit point x∗, ∇

p

  • j=1

fij(x∗)(x∗) = 0. Using the true Hessian to define the quadratic approximation: ∇2

p

  • j=1

fij(x∗)(x∗) ≥ 0 Local convergence: quadratic

slide-25
SLIDE 25

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Fitting with LOVO

Observations t y t1 y1 . . . . . . tm ym Model yj ≈ M(x, tj) fj(x) = [yj − M(x, tj)]2

slide-26
SLIDE 26

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Constrained LOVO problems

Minimize

p

  • j=1

fij(x)(x) subject to h(x) = 0, g(x) ≤ 0. Augmented Lagrangian (PHR-Like) (Code Algencan in www.ime.usp.br/∼egbirgin/tango) Minimize approx

p

  • j=1

fij(x)(x)+ρ 2

  • h(x) + λ

ρ

  • 2

+

  • g(x) + µ

ρ

  • +
  • 2

Update λ, µ ≥ 0, ρ. (a+ = max{0, a})

slide-27
SLIDE 27

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Convergence of Algencan-LOVO

Global minimization of subproblems ⇓ Global Minimization Limit points are either feasible or stationary points of Infeasibility Feasible limit points that satisfy the CPLD constraint qualification are “KKT” Boundedness of penalty parameter

slide-28
SLIDE 28

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Model fitting with Algencan-LOVO

Find the parameters of a Boundary Value Problem fitting a set of data that contains outliers

slide-29
SLIDE 29

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Fitting Nash-Equilibrium Models

Nash-Equilibrium Model: Given the parameters x ∈ Ω, the players 1, 2, · · · , m take , simultaneously, decisions y1, · · · , ym. Player j takes his/her decision minimizing fj(x, y1, · · · , yj−1, z, yj+1, · · · , ym) with respect to z.

slide-30
SLIDE 30

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Fitting Nash-Equilibrium Models

Nash-Equilibrium Model: Given the parameters x ∈ Ω, the players 1, 2, · · · , m take , simultaneously, decisions y1, · · · , ym. Player j takes his/her decision minimizing fj(x, y1, · · · , yj−1, z, yj+1, · · · , ym) with respect to z. Inverse Nash-Equilibrium: ¯ y1, · · · , ¯ ym are known Discover the parameters x.

slide-31
SLIDE 31

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Fitting Nash-Equilibrium Models

Nash-Equilibrium Model: Given the parameters x ∈ Ω, the players 1, 2, · · · , m take , simultaneously, decisions y1, · · · , ym. Player j takes his/her decision minimizing fj(x, y1, · · · , yj−1, z, yj+1, · · · , ym) with respect to z. Inverse Nash-Equilibrium: ¯ y1, · · · , ¯ ym are known Discover the parameters x. LOVO-Inverse-Nash-Equilibrium: ¯ y1, · · · , ¯ ym are known but only 90% of these observations are reliable.

slide-32
SLIDE 32

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Fitting Nash-Equilibrium Models

Minimize

p

  • j=1

(yij(x,y) − ¯ yij(x,y))2 subject to Player j minimizes fj(x, y1, . . . , yj−1, z, yj+1, . . . , ym) with respect to z, for all j = 1, . . . , m

slide-33
SLIDE 33

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Dealing with LOVO constraints

Minimize f(x) s/t x satisfies at least p constraints of the subset      g1(x) ≤ 0 . . . gm(x) ≤ 0 We define L(x, µ, ρ) = f (x) + ρ

p

  • j=1
  • gij(x)(x) + µij

ρ 2

+

where

  • gi1(x)(x) +

µi1 ρ

2

+ ≤ · · · ≤

  • gim(x)(x) + µim

ρ

2

+

minimize L(x, µ, ρ) update µ, ρ

slide-34
SLIDE 34

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

LOVO constraints

Example of Feasible Region for p = m − 1

slide-35
SLIDE 35

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Example of Minimization with LOVO constraints

Find the union of 2 ellipses with smallest area that contain 90% of the points {P1, · · · , Pm} We have one constraint Pi ∈ E1 ∪ E2 for each point Pi.

slide-36
SLIDE 36

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Example of Minimization with LOVO constraints

Given two sets of points A and B, find two ellipses with no intersection area, that contain, respectively, 80% of the points of A and 80% of the points of B.

slide-37
SLIDE 37

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Topics in computer vision

Curve detection        detection of lines detection of circles detection of ellipses

  • thers

[Comparisons with HT, Ransac, QMDPE, LKS in Cesar-Andreani-Marcondes-JMM & Silva 2007]

slide-38
SLIDE 38

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Lines

Given {(x1, y1), · · · , (xm, ym)} find θ and ρ such that

p

  • j=1

fij(θ,ρ)(θ, ρ) is minimal, where fi(θ, ρ) = [xi cos θ + yi sin θ − ρ]2 i = 1, · · · , m, and fi1(θ,ρ)(θ, ρ) ≤ · · · ≤ fim(θ,ρ)(θ, ρ) ∀θ and ρ > 0

slide-39
SLIDE 39

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Detection of lines

slide-40
SLIDE 40

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Detection of lines

slide-41
SLIDE 41

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Detection of circles

slide-42
SLIDE 42

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Detection of circles

slide-43
SLIDE 43

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Detection of circles

slide-44
SLIDE 44

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Current work

Tracking Parametric Curves in video sequences The tracked curve in frame t-1 should be used as initial solution for the LOVO problem to be solved in frame t.

slide-45
SLIDE 45

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Protein Alignment

Finding common 3D structures of two given proteins. Evaluating similarity.

slide-46
SLIDE 46

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Protein Alignment

Data: 3D coordinates of C − α atoms of both proteins. (Available, for example, in Protein Data Bank PDB) Protein A: (A1, · · · , AnA) Protein B: (B1, · · · , BnB) Ai, Bj ∈ R3

slide-47
SLIDE 47

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

To each admissible bijection between (a subset of) A and (a subset

  • f) B corresponds a score.

Example admissible bijection Φ Non-admissible bijection

slide-48
SLIDE 48

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Given fixed relative positions of A and B, the bijection with best score may be obtained using dynamic programming Changing the relative positions, both “best bijection” and score change.

slide-49
SLIDE 49

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

−fi(x) = score associated with “bijection i” under the movement defined by x.

⇓

Minimize fi1(x)(x) (LOVO, p = 1)

slide-50
SLIDE 50

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Objective of the alignment: maximize the score, with respect to bijection and relative position. Modelling as LOVO:

slide-51
SLIDE 51

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Objective of the alignment: maximize the score, with respect to bijection and relative position. Modelling as LOVO:

slide-52
SLIDE 52

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

LOVOALIGN PACKAGE

www.ime.unicamp.br/∼martinez/lovoalign

On-line alignment

  • f proteins
slide-53
SLIDE 53

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

www.ime.unicamp.br/∼martinez/lovoalign

slide-54
SLIDE 54

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

www.ime.unicamp.br/∼martinez/lovoalign

slide-55
SLIDE 55

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Conclusion

We presented: The general form of Order-Value functions and OVO problems Particular cases: VaR-like, CVaR-like, LOVO, . . . Discussion of nonsmoothness and many local minimizers Nonlinear Programming Reformulations Primal (trust-region and line-search) methods for unconstrained OVO problems with and without smoothing Using LOVO for Nonlinear Regression with Outliers Constrained LOVO problems Convergence of Algorithms for Constrained LOVO Application to Nash-Equilibrium fitting LOVO constraints Computer Vision Protein Alignment

slide-56
SLIDE 56

ORDER-VALUE OPTIMIZATION AND NEW APPLICATIONS

Some references

1 R. Andreani, J. M. Mart´

ınez, L. Mart´ ınez and F. Yano. Continuous Optimization Methods for Structure Alignments. To appear in Mathematical Programming.

2 R. Andreani, J. M. Mart´

ınez, M. Salvatierra and F. Yano. Quasi-Newton methods for order-value optimization and value-at-risk calculations. Pacific Journal of Optimization 2,

  • pp. 11-33 (2006).

3 R. Andreani, C. Dunder and J. M. Mart´

ınez. Nonlinear-Programming Reformulation of the Order-Value Optimization problem. Mathematical Methods of Operations Research 61, pp. 365-384 (2005).

4 R. Andreani, C. Dunder and J. M. Mart´

ınez. Order-Value Optimization: formulation and solution by means of a primal Cauchy method. Mathematical Methods of Operations Research 58, pp. 387-399 (2003).