Compromise Solutions in Multicriteria Optimization Kai-Simon - - PowerPoint PPT Presentation

Introduction Definitions and Notations Basic Properties Approximation

Compromise Solutions in Multicriteria Optimization

Kai-Simon Goetzmann

(Joint work with Christina B¨ using and Jannik Matuschke)

OR 2011 – August 31

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Multicriteria Optimization

Pareto optimal solutions: No objective can be improved without worsening another. f1(x) f2(x)

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Multicriteria Optimization

Pareto optimal solutions: No objective can be improved without worsening another. f1(x) f2(x)

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Multicriteria Optimization

Pareto optimal solutions: No objective can be improved without worsening another. f1(x) f2(x)

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Avoiding too many solutions

Scalarization: Maximize (weighted) sum of all objectives. f1(x) f2(x)

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Avoiding too many solutions

Scalarization: Maximize (weighted) sum of all objectives. f1(x) f2(x)

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Avoiding too many solutions

Scalarization: Maximize (weighted) sum of all objectives. f1(x) f2(x)

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Avoiding too many solutions

Scalarization: Maximize (weighted) sum of all objectives. f1(x) f2(x)

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Avoiding too many solutions

Scalarization: Maximize (weighted) sum of all objectives. f1(x) f2(x)

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Our Answer

Compromise Solutions: Minimize distance to ideal point (w.r.t. some metric). f1(x) f2(x)

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Our Answer

Compromise Solutions: Minimize distance to ideal point (w.r.t. some metric). f1(x) f2(x)

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Our Answer

Compromise Solutions: Minimize distance to ideal point (w.r.t. some metric). f1(x) f2(x)

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

1

Introduction

2

Definitions and Notations

3

Basic Properties

4

Approximation

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

1

Introduction

2

Definitions and Notations

3

Basic Properties

4

Approximation

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Definition (Ideal Point) Given a multicriteria optimization problem maxy∈Y y, the ideal point y∗ = (y∗

1, . . . , y∗ k) is defined by

y∗

i = max y∈Y yi

∀ i. y1 y2 Y y∗

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Definition (Compromise Solution, Yu 1973) Given a multicriteria optimization problem maxy∈Y y with the ideal point y∗ ∈ ◗k, the Compromise Solution w.r.t. the norm · on ◗k is yCS = min

y∈Y y∗ − y .

y1 y2 y∗

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Definition (Compromise Solution, Yu 1973) Given a multicriteria optimization problem maxy∈Y y with the ideal point y∗ ∈ ◗k, the Compromise Solution w.r.t. the norm · on ◗k is yCS = min

y∈Y y∗ − y .

y1 y2 y∗

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

The norms we consider: yp := k

• i=1

yp

i

1/p , p ∈ [1, ∞) (ℓp-Norm) y∞ := max

i=1,...,k yi

(Maximum (ℓ∞-)Norm) | | |y| | |p := y∞ + 1 p y1 , p ∈ [1, ∞] (Cornered p-Norm) ℓp-Norm

1 1

Cornered p-Norm

1 1

p = 1 p = 2 p = 5 p = ∞ Degree of balancing controlled by adjusting p.

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

The norms we consider: yp := k

• i=1

yp

i

1/p , p ∈ [1, ∞) (ℓp-Norm) y∞ := max

i=1,...,k yi

(Maximum (ℓ∞-)Norm) | | |y| | |p := y∞ + 1 p y1 , p ∈ [1, ∞] (Cornered p-Norm) Weighted version: For any norm and λ ∈ ◗k, λ ≥ 0, λ = 0 : yλ = (λ1y1, λ2y2, . . . , λkyk) .

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

1

Introduction

2

Definitions and Notations

3

Basic Properties

4

Approximation

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Known Properties

Notation: For a family of norms ·p , p ∈ [1, ∞], define CS(λ, p) := {Compromise Solution w.r.t. ·λ

p}

CS(p) := {CS(λ, p) : λ ∈ ◗k, λ ≥ 0, λ = 0} Gearhardt 1979: Pareto optimality: CS(p) ⊆ YP for p < ∞. Y discrete ⇒ YP = CS(p) for p big enough. y1 y2 YP

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Known Properties

Notation: For a family of norms ·p , p ∈ [1, ∞], define CS(λ, p) := {Compromise Solution w.r.t. ·λ

p}

CS(p) := {CS(λ, p) : λ ∈ ◗k, λ ≥ 0, λ = 0} Gearhardt 1979: Pareto optimality: CS(p) ⊆ YP for p < ∞. Y discrete ⇒ YP = CS(p) for p big enough. How big? y1 y2 YP

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

How to choose p

Consider the Cornered Norm | | | · | | |p. Assumptions: Integrality: Y ⊂ ◆k Boundedness: ∃ polynomial π : yi ≤ 2π(|I|) ∀y ∈ Y, i = 1, . . . , k Lemma (B¨ using,G.,Matuschke) Let M := 2π(|I|). For p > kM, it holds that YP = CS(p). Remarks: p can be chosen such that it has polynomial encoding length. ℓp-norm: p M log k suffices.

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

1

Introduction

2

Definitions and Notations

3

Basic Properties

4

Approximation

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Another way to cope with exponential size of YP : Definition (ε-approximate Pareto set) Let YP be the Pareto set of a given instance, and let ε > 0. YεP ⊆ Y is an ε-approximate Pareto set if for all y ∈ YP there is y′ ∈ YεP such that yi ≤ (1 + ε)y′

i

∀i = 1, . . . , k

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Another way to cope with exponential size of YP : Definition (ε-approximate Pareto set) Let YP be the Pareto set of a given instance, and let ε > 0. YεP ⊆ Y is an ε-approximate Pareto set if for all y ∈ YP there is y′ ∈ YεP such that yi ≤ (1 + ε)y′

i

∀i = 1, . . . , k

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Another way to cope with exponential size of YP : Definition (ε-approximate Pareto set) Let YP be the Pareto set of a given instance, and let ε > 0. YεP ⊆ Y is an ε-approximate Pareto set if for all y ∈ YP there is y′ ∈ YεP such that yi ≤ (1 + ε)y′

i

∀i = 1, . . . , k

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Another way to cope with exponential size of YP : Definition (ε-approximate Pareto set) Let YP be the Pareto set of a given instance, and let ε > 0. YεP ⊆ Y is an ε-approximate Pareto set if for all y ∈ YP there is y′ ∈ YεP such that yi ≤ (1 + ε)y′

i

∀i = 1, . . . , k

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Another way to cope with exponential size of YP : Definition (ε-approximate Pareto set) Let YP be the Pareto set of a given instance, and let ε > 0. YεP ⊆ Y is an ε-approximate Pareto set if for all y ∈ YP there is y′ ∈ YεP such that yi ≤ (1 + ε)y′

i

∀i = 1, . . . , k Theorem (Papadimitriou&Yannakakis,2000) There always exists an ε-approximate Pareto set with size polynomial in |I| and 1/ε.

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

How to find an ε-approximate Pareto set? ◗

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

How to find an ε-approximate Pareto set? Theorem (Papadimitriou&Yannakakis,2000) There is an efficient algorithm for constructing an ε-approximate Pareto set if and only if the Gap problem is tractable. Gap problem: Given y ∈ ◗k and ε > 0.

y1 y2 y

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

How to find an ε-approximate Pareto set? Theorem (Papadimitriou&Yannakakis,2000) There is an efficient algorithm for constructing an ε-approximate Pareto set if and only if the Gap problem is tractable. Gap problem: Given y ∈ ◗k and ε > 0.

y′ y1 y2 y

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

How to find an ε-approximate Pareto set? Theorem (Papadimitriou&Yannakakis,2000) There is an efficient algorithm for constructing an ε-approximate Pareto set if and only if the Gap problem is tractable. Gap problem: Given y ∈ ◗k and ε > 0.

y′ y1 y2 y y1 y2 (1 + ε)y no sol’n y

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Connection to Compromise Solutions: Theorem (B¨ using,G.,Matuschke) For p > kM 2 ε , an FPTAS for CS(p) solves the Gap problem. Remarks: p still has polynomial encoding length. ℓp-norm: p M2 log k

ε

suffices.

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Proof: Let y ∈ ◗k, ε > 0 be given. First consider y ∈ ◆k, w.l.o.g. y < y∗. For all i, set λi :=

1 y∗

i −yi . Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Proof: Let y ∈ ◗k, ε > 0 be given. First consider y ∈ ◆k, w.l.o.g. y < y∗. For all i, set λi :=

1 y∗

i −yi .

y∗ y

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Proof: Let y ∈ ◗k, ε > 0 be given. First consider y ∈ ◆k, w.l.o.g. y < y∗. For all i, set λi :=

1 y∗

i −yi .

y∗ y y′

Let y′ be returned by the FPTAS for CS(p, λ) for some ε′. Positive Case: | | |y∗ − y′| | |λ

p ≤ |

| |y∗ − y| | |λ

p p>kM

= ⇒ y′ ≥ y

• Return y′.

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Negative Case: | | |y∗ − y′| | |λ

p > |

| |y∗ − y| | |λ

p y∗ y y′

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Negative Case: | | |y∗ − y′| | |λ

p > |

| |y∗ − y| | |λ

p

⇒ ∄y′′ ∈ Y : | | |y∗ − y′′| | |λ

p ≤ 1 α|

| |y∗ − y| | |λ

p

(α = 1 + ε′).

no sol’n y∗ y y∗ − 1

α (y∗ − y) =:

y

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Negative Case: | | |y∗ − y′| | |λ

p > |

| |y∗ − y| | |λ

p

⇒ ∄y′′ ∈ Y : | | |y∗ − y′′| | |λ

p ≤ 1 α|

| |y∗ − y| | |λ

p

(α = 1 + ε′).

no sol’n y∗ y y∗ − 1

α (y∗ − y) =:

y

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Negative Case: | | |y∗ − y′| | |λ

p > |

| |y∗ − y| | |λ

p

⇒ ∄y′′ ∈ Y : | | |y∗ − y′′| | |λ

p ≤ 1 α|

| |y∗ − y| | |λ

p

(α = 1 + ε′).

no sol’n y∗ y y∗ − 1

α (y∗ − y) =:

y

⇒ Need y ≤ (1 + ε)y.

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Negative Case: | | |y∗ − y′| | |λ

p > |

| |y∗ − y| | |λ

p

⇒ ∄y′′ ∈ Y : | | |y∗ − y′′| | |λ

p ≤ 1 α|

| |y∗ − y| | |λ

p

(α = 1 + ε′).

no sol’n y∗ y y∗ − 1

α (y∗ − y) =:

y

⇒ Need y ≤ (1 + ε)y. True for α =

1 1−ε/M

• ε′ = 1 −

1 1−ε/M

Details Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Fractional y: Consider ⌊y⌋ instead. Similar calculations with more case distinctions, slightly smaller value for ε′, and choosing p > kM2

ε

enable us to answer Gap also in this case.

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Generic Approximation Results

Scalarization: maxy∈Y λTy yields a ρ-approximation to CS(p, λ) with ρ = k+kp

k+p ≤ min{k, p + 1}

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Generic Approximation Results

Scalarization: maxy∈Y λTy yields a ρ-approximation to CS(p, λ) with ρ = k+kp

k+p ≤ min{k, p + 1}

Aissi et al., ’06: If there are bounds L ≤ OPT ≤ U with U ≤ π1(|I|)L an algorithm for CS(∞) with running time π2(|I|, U), then there is an FPTAS for CS(∞).

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Generic Approximation Results

Scalarization: maxy∈Y λTy yields a ρ-approximation to CS(p, λ) with ρ = k+kp

k+p ≤ min{k, p + 1}

Aissi et al., ’06: If there are bounds L ≤ OPT ≤ U with U ≤ π1(|I|)L an algorithm for CS(∞) with running time π2(|I|, U), then there is an FPTAS for CS(∞). B¨ using, G, Matuschke, ’11: This also holds for CS(p) with p < ∞. Corollary For Shortest Path and Minimum Spanning Tree, there is an FPTAS for CS(p) w.r.t. the Cornered Norm.

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Generic Approximation Results

Scalarization: maxy∈Y λTy yields a ρ-approximation to CS(p, λ) with ρ = k+kp

k+p ≤ min{k, p + 1}

Aissi et al., ’06: If there are bounds L ≤ OPT ≤ U with U ≤ π1(|I|)L an algorithm for CS(∞) with running time π2(|I|, U), then there is an FPTAS for CS(∞). B¨ using, G, Matuschke, ’11: This also holds for CS(p) with p < ∞. Corollary For Shortest Path and Minimum Spanning Tree, there is an FPTAS for CS(p) w.r.t. the Cornered Norm. Problem: Algorithms based on approximating the set of all non-dominated regret-vectors ≈ Approximation of Pareto set.

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Conclusion

Results: Compromise Solutions have nice structural properties; approximating the Pareto set reduces to CS(p). FPTAS for some problems, but huge computational effort. Simple algorithms (Local Search, Greedy) for MST have no approximation guarantee better than Scalarization (though empirically good).

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Conclusion

Results: Compromise Solutions have nice structural properties; approximating the Pareto set reduces to CS(p). FPTAS for some problems, but huge computational effort. Simple algorithms (Local Search, Greedy) for MST have no approximation guarantee better than Scalarization (though empirically good). Open Questions: Can we approximate CS(p) for some problems directly (i.e. without approximating the Pareto set)? Is CS(p) strictly harder than approximating the Pareto set?

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Conclusion

Results: Compromise Solutions have nice structural properties; approximating the Pareto set reduces to CS(p). FPTAS for some problems, but huge computational effort. Simple algorithms (Local Search, Greedy) for MST have no approximation guarantee better than Scalarization (though empirically good). Open Questions: Can we approximate CS(p) for some problems directly (i.e. without approximating the Pareto set)? Is CS(p) strictly harder than approximating the Pareto set? Thank you for your attention.

Kai-Simon Goetzmann Compromise Solutions

AUGUST 19-24 www.ismp2012.org

THE 21ST INTERNATIONAL SYMPOSIUM ON MATHEMATICAL PROGRAMMING

Introduction Definitions and Notations Basic Properties Approximation

Choosing ε′

W.l.o.g. y < y∗ and ε < 1

yi ∀i. For simplicity, assume y > 0.

• yi = y∗

i − 1

α(y∗

i − yi) ≤ (1 + ε)yi

⇔ α ≤ y∗

i − yi

y∗

i − yi − εyi

◆

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Choosing ε′

W.l.o.g. y < y∗ and ε < 1

yi ∀i. For simplicity, assume y > 0.

• yi = y∗

i − 1

α(y∗

i − yi) ≤ (1 + ε)yi

⇔ α ≤ y∗

i − yi

y∗

i − yi − εyi

y∗

i − yi − εyi

y∗

i − yi

≤ 1 − εyi y∗

i − yi

≤ 1 − ε M (yi ≥ 1) ◆

Kai-Simon Goetzmann Compromise Solutions

Introduction Definitions and Notations Basic Properties Approximation

Choosing ε′

W.l.o.g. y < y∗ and ε < 1

yi ∀i. For simplicity, assume y > 0.

• yi = y∗

i − 1

α(y∗

i − yi) ≤ (1 + ε)yi

⇔ α ≤ y∗

i − yi

y∗

i − yi − εyi

y∗

i − yi − εyi

y∗

i − yi

≤ 1 − εyi y∗

i − yi

≤ 1 − ε M (yi ≥ 1) ⇒ α = 1 1 − ε/M

• r

ε′ = 1 − α = 1 − 1 1 − ε/M ⇒ For this ε′ and y ∈ ◆k, FPTAS for CS(p, λ) answers Gap.

Back Kai-Simon Goetzmann Compromise Solutions