A bisection method for generating random utility functions in SMAA - - PowerPoint PPT Presentation

A bisection method for generating random utility functions in SMAA

DA2PL 2018 Poznan, Nov. 22-23, 2018 Rudolf Vetschera1 Luis Dias2

1University of Vienna, Austria

rudolf.vetschera@univie.ac.at

2University of Coimbra, Portugal

lmcdias@fe.uc.pt

2

Outline

• Problem statement
• Arbitrary functjons

– Generatjon methods – Computatjonal results

• Concave functjons

– Generatjon methods – Computatjonal results

• Conclusions

3

Problem statement

Given a set of performance levels {x0, x1, x2, … xn} assign to them utility values {u(x0), u(x1), u(x2),... u(xn)} so that

• the resulting utility function is monotonic
• the entire range of possible utility functions is covered
• there is no bias toward a specific shape or
• (optional) the function has a predefined shape

u(x) x

4

A trivial problem?

1)Generate random values from uniform distributjon 2)Sort them 3)Assign to x values in increasing order

5

A trivial problem?

Data from: Kadzinski, M. and Tervonen, T. (2013). Robust multj-criteria ranking with additjve value models and holistjc pair-wise preference statements. European Journal of Operatjonal Research, 228(1):169 - 180.

1)Generate random values from uniform distributjon 2)Sort them 3)Assign to x values in increasing order

x

8 36 40 44 48 60 68 72 80 84 88 90 96

6

A trivial problem?

Data from: Kadzinski, M. and Tervonen, T. (2013). Robust multj-criteria ranking with additjve value models and holistjc pair-wise preference statements. European Journal of Operatjonal Research, 228(1):169 - 180.

1)Generate random values from uniform distributjon 2)Sort them 3)Assign to x values in increasing order

30 such runs:

• Distjnct convex shape
• Most functjons are in

narrow band x

8 36 40 44 48 60 68 72 80 84 88 90 96

20 40 60 80 0.0 0.2 0.4 0.6 0.8 1.0 x u(x)

7

Another problem

0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 x u(x) 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 x u(x)

10 performance levels 100 performance levels All data points drawn from uniform distributjon (not skewed)

8

The general idea: Bisection

1)Find point (aturibute value) between existjng points

9

The general idea: Bisection

1)Find point between existjng points 2)Assign utjlity to it

10

The general idea: Bisection

1)Find point between existjng points 2)Assign utjlity to it 3)and repeat for all new intervals

11

Main step 1: Find a split point

Given: Two points xlo and xup and their utjlity values, fjnd index s of split point: Approximate midpoint: s=argmin i|xi−(xlo+xup)/2| Median random choice: s=⌈(lo+up)/2⌉ s=⌊(lo+up)/2⌋ Random choice: s:lo<s<up Weighted random choice: pi= xi+1−xi−1 (xup−xlo)+(xup−1−xlo+1)

12

Main step 2: Determine u(xs)

Straightgorward: uniform distributjon between u(xlo) and u(xup) Correctjon if split point is not the midpoint p=0.5 p=0.5 xlo xup

13

Evaluation criteria (1): Coverage

0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 x u(x)

Below=∑

i=2 n u min(xi−1)+u min(xi)

2 (xi−xi−1) Above=∑

i=2 n

(1− u

max(xi−1)+u max(xi)

2

)(xi−xi−1)

Coverage=1−( Above+Below)

14

Evaluation criteria (2): Bias

• Average area below utjlity functjon:

Should be 0.5 for well-balanced shapes

• Absolute difgerence in fractjons of points below and above straight line (risk

neutral utjlity): Should be zero for balanced (symmetric) shapes

15

Compared approaches

• Splittjng points:

Next to mean (xB), median (xM), random (xR), random proportjonal (xP)

• Correctjon in generatjng u(x):

Direct random value (Dy), weighted (Wy) e.g. DM is direct with median split point

• As comparison:

– Direct assignment of Random Values (RV) – Using random values as difgerences (RD) – Creatjng utjlitjes at evenly spaced points using bisectjon, and linear

interpolatjon (BL)

16

Coverage: Effect of attribute levels

RV RD BL DB WB DM WM DR WR DP WP 0.0 0.2 0.4 0.6 0.8 1.0

cover

nUtil: 1000, nPoint: 10, Delta: 0.01 RV RD BL DB WB DM WM DR WR DP WP 0.0 0.2 0.4 0.6 0.8 1.0

cover

nUtil: 1000, nPoint: 100, Delta: 0.01

10 levels 100 levels

Simple methods degrade Random methods improve

17

Bias: Effects of skewness

RV RD BL DB WB DM WM DR WR DP WP 0.0 0.2 0.4 0.6 0.8 1.0

diffAbs

nUtil: 1000, nPoint: 100, Delta: 0.01 RV RD BL DB WB DM WM DR WR DP WP 0.0 0.2 0.4 0.6 0.8 1.0

diffAbs

nUtil: 1000, nPoint: 100, Delta: 2

Δ = 0.01 Δ = 2

Simple methods become completely biased Weighted random remains unbiased

18

Combining coverage and bias

0.2 0.4 0.6 0.8 0.35 0.40 0.45 0.50

100 points, delta=2

cover areaMean BL DB DM DP DR RD RV WB WM WP WR

19

Bisection for concave function

1) Randomly generate slope between slopes of adjacent segments

20

Bisection for concave function

1) Randomly generate slope between slopes of adjacent segments 2) Select split point 3) At split point, randomly generate new point between current segment and lower

• f the two slope lines

Possible range

21

Variants concave function

Code Descriptjon CD Benchmark: Sort random slopes, apply them sequentjally CB Split point closest to mean CM Split point median CR Random selectjon of split point CP Random selectjon with proportjonal probabilitjes

22

Results: Coverage

Delta = 0.01 Delta = 2 Points Method above below coverage above below coverage 10 CD 20.60% 53.50% 25.91% 27.24% 53.05% 19.71% CB 11.67% 50.03% 38.30% 17.22% 50.03% 32.75% CM 10.71% 50.03% 39.26% 21.11% 50.02% 28.88% CR 5.57% 50.02% 44.41% 14.93% 50.01% 35.06% CP 5.58% 50.02% 44.40% 14.85% 50.02% 35.13% 100 CD 29.16% 62.32% 8.52% 34.24% 59.41% 6.35% CB 11.28% 50.03% 38.69% 11.88% 50.03% 38.09% CM 11.34% 50.03% 38.63% 22.60% 50.02% 27.39% CR 0.82% 50.02% 49.16% 5.49% 50.01% 44.50% CP 0.83% 50.02% 49.16% 4.78% 50.02% 45.20%

23

Summary

• Criteria for "good" sampling of utjlity functjons:

– Coverage – Bias

• Directly generatjng utjlity values has problems in both criteria
• Bisectjon approach with (proportjonal) random split point and correctjon for
• fg-center values provides best results
• Approach can be modifjed to generate only functjons of specifjc shape

Thank you for your attention!

25

Why bias toward convexity?

• Distributjon of values skewed to the right

– Difgerence between values decreases

• Utjlity values drawn from uniform distributjon

– Expected difgerence between them is constant

• Expected slope increases - convex

26

Skewed density of performance

Density f(x) x 0≤Δ≤2 Performance levels between 0 and 1 1 d0

27

Generating skewed performance levels

Density: f (x)=d 0+Δ x F(x)=d0 x+ Δ 2 x

2

cdf F(1)=1 ⇒ d0=1−Δ/2 ⇒ F(x)=(1− Δ 2 ) x+ Δ 2 x

2

Inverse method: F(x)=r x= 1 2 − 1 Δ +√( 2 Δ−1)

2

/4+ 2r Δ

28

Simulation parameters

Performance levels 10, 25, 100 Skewness (Δ) 0.01, 0.5, 1, 1.5, 2 Functjons generated 100, 1000

29

Bias: Effects of skewness

RV RD BL DB WB DM WM DR WR DP WP 0.0 0.2 0.4 0.6 0.8 1.0

areaMean

nUtil: 1000, nPoint: 100, Delta: 0.01 RV RD BL DB WB DM WM DR WR DP WP 0.0 0.2 0.4 0.6 0.8 1.0

areaMean

nUtil: 1000, nPoint: 100, Delta: 2

Δ = 0.01 Δ = 2

30

Coverage: Mean values

10 points 100 points delta 0.01 0.5 1 1.5 2 0.01 0.5 1 1.5 2 RV 65.9% 65.1% 64.2% 62.0% 59.2% 25.1% 24.9% 24.2% 23.2% 21.5% RD 48.4% 47.6% 46.7% 44.7% 42.2% 14.8% 14.6% 14.2% 13.6% 12.4% BL 78.3% 77.7% 77.2% 75.8% 74.1% 86.3% 86.2% 86.2% 85.9% 84.6% DB 83.3% 82.7% 81.9% 80.0% 77.2% 86.8% 86.6% 86.6% 86.3% 84.9% WB 83.5% 82.8% 82.1% 80.1% 77.4% 86.8% 86.7% 86.7% 86.4% 85.1% DM 84.1% 83.1% 82.2% 79.6% 76.5% 86.8% 86.3% 84.7% 81.9% 76.5% WM 84.8% 83.9% 83.2% 81.2% 78.3% 87.1% 86.8% 86.2% 84.8% 81.8% DR 88.9% 88.0% 87.2% 85.0% 82.0% 98.4% 98.3% 98.0% 97.2% 94.0% WR 88.7% 87.8% 87.1% 84.8% 81.9% 97.9% 97.8% 97.5% 96.6% 93.5% DP 88.8% 88.0% 87.2% 85.0% 82.1% 98.4% 98.4% 98.1% 97.5% 94.6% WP 88.6% 87.8% 87.0% 84.8% 81.9% 97.9% 97.9% 97.6% 97.0% 94.2%

31

Difference in shares: Means

10 points 100 points delta 0.01 0.5 1 1.5 2 0.01 0.5 1 1.5 2 RV 33.6% 37.6% 41.6% 53.0% 64.2% 32.6% 51.9% 80.4% 92.2% 96.0% RD 42.2% 46.4% 50.8% 63.5% 75.3% 41.1% 61.8% 87.3% 95.5% 98.0% BL 6.4% 7.5% 8.3% 10.8% 13.8% 2.7% 3.9% 7.3% 10.8% 14.3% DB 12.2% 11.7% 12.1% 12.1% 12.5% 3.2% 4.2% 7.2% 10.8% 13.8% WB 7.1% 7.4% 8.0% 9.5% 11.4% 2.8% 3.9% 7.1% 10.7% 14.1% DM 21.3% 24.4% 27.1% 35.1% 43.8% 7.6% 12.9% 24.9% 36.0% 45.2% WM 6.0% 6.9% 7.6% 9.6% 11.9% 2.7% 3.9% 6.9% 10.1% 13.3% DR 17.8% 20.5% 22.8% 30.0% 37.8% 6.0% 9.9% 19.1% 28.4% 36.9% WR 6.0% 6.8% 7.6% 9.6% 12.1% 2.9% 4.1% 7.2% 10.6% 13.9% DP 9.2% 10.7% 11.4% 13.7% 16.9% 2.1% 2.2% 2.4% 2.8% 4.5% WP 3.5% 3.5% 3.6% 3.2% 3.4% 2.2% 2.5% 3.6% 4.7% 5.3%

32

Area under UF: Means

10 points 100 points delta 0.01 0.5 1 1.5 2 0.01 0.5 1 1.5 2 RV 49.8% 46.3% 43.2% 38.6% 34.7% 49.8% 46.1% 41.6% 37.5% 33.4% RD 49.8% 46.3% 43.2% 38.6% 34.7% 49.8% 46.1% 41.6% 37.5% 33.4% BL 50.0% 50.0% 50.0% 50.0% 50.0% 50.0% 50.0% 50.0% 50.0% 50.0% DB 50.0% 49.8% 49.5% 49.2% 48.6% 50.0% 50.0% 50.0% 50.0% 49.8% WB 50.0% 49.9% 49.8% 49.6% 49.3% 50.0% 50.0% 50.0% 50.0% 49.9% DM 49.8% 46.3% 43.2% 38.8% 34.8% 49.8% 46.1% 41.7% 37.5% 33.4% WM 49.9% 48.1% 46.4% 44.0% 41.8% 49.9% 48.0% 45.7% 43.4% 40.7% DR 49.9% 47.0% 44.3% 40.4% 36.9% 49.9% 47.0% 43.6% 40.1% 36.3% WR 49.9% 48.2% 46.8% 44.6% 42.5% 49.9% 48.2% 46.3% 44.3% 42.0% DP 50.0% 48.8% 47.8% 46.0% 44.0% 50.1% 49.9% 49.7% 49.2% 47.8% WP 50.0% 49.3% 48.8% 47.8% 46.7% 50.0% 49.9% 49.8% 49.6% 48.8%

33

Computational effort

• Find split point

– Median, random choice: constant – Approx. mean, weighted: O(n)

• Generate utjlity: constant
• All this is done exactly one for each point

For each point Overall: O(n) or O(n2)