Decision Making Beyond Interval . . . Multi-Agent . . . under - PowerPoint PPT Presentation

Decision Making: . . . The Notion of Utility From Utility to . . . Decision Making Beyond Interval . . . Multi-Agent . . . under Beyond Optimization Even Further Beyond . . . Interval Uncertainty Acknowledgments Home Page Vladik Kreinovich Title Page Department of Computer Science ◭◭ ◮◮ University of Texas at El Paso El Paso, TX 79968, USA ◭ ◮ vladik@utep.edu Page 1 of 55 http://www.cs.utep.edu/vladik Go Back Full Screen Close Quit

Decision Making: . . . 1. Decision Making: General Need and Traditional The Notion of Utility Approach From Utility to . . . Beyond Interval . . . • To make a decision, we must: Multi-Agent . . . – find out the user’s preference, and Beyond Optimization – help the user select an alternative which is the best Even Further Beyond . . . – according to these preferences. Acknowledgments Home Page • Traditional approach is based on an assumption that for each two alternatives A ′ and A ′′ , a user can tell: Title Page ◭◭ ◮◮ – whether the first alternative is better for him/her; we will denote this by A ′′ < A ′ ; ◭ ◮ – or the second alternative is better; we will denote Page 2 of 55 this by A ′ < A ′′ ; Go Back – or the two given alternatives are of equal value to the user; we will denote this by A ′ = A ′′ . Full Screen Close Quit

Decision Making: . . . 2. The Notion of Utility The Notion of Utility From Utility to . . . • Under the above assumption, we can form a natural Beyond Interval . . . numerical scale for describing preferences. Multi-Agent . . . • Let us select a very bad alternative A 0 and a very good Beyond Optimization alternative A 1 . Even Further Beyond . . . Acknowledgments • Then, most other alternatives are better than A 0 but Home Page worse than A 1 . Title Page • For every prob. p ∈ [0 , 1], we can form a lottery L ( p ) in which we get A 1 w/prob. p and A 0 w/prob. 1 − p . ◭◭ ◮◮ ◭ ◮ • When p = 0, this lottery simply coincides with the alternative A 0 : L (0) = A 0 . Page 3 of 55 • The larger the probability p of the positive outcome Go Back increases, the better the result: Full Screen p ′ < p ′′ implies L ( p ′ ) < L ( p ′′ ) . Close Quit

Decision Making: . . . 3. The Notion of Utility (cont-d) The Notion of Utility From Utility to . . . • Finally, for p = 1, the lottery coincides with the alter- Beyond Interval . . . native A 1 : L (1) = A 1 . Multi-Agent . . . • Thus, we have a continuous scale of alternatives L ( p ) Beyond Optimization that monotonically goes from L (0) = A 0 to L (1) = A 1 . Even Further Beyond . . . • Due to monotonicity, when p increases, we first have Acknowledgments Home Page L ( p ) < A , then we have L ( p ) > A . Title Page • The threshold value is called the utility of the alterna- tive A : ◭◭ ◮◮ def u ( A ) = sup { p : L ( p ) < A } = inf { p : L ( p ) > A } . ◭ ◮ Page 4 of 55 • Then, for every ε > 0, we have Go Back L ( u ( A ) − ε ) < A < L ( u ( A ) + ε ) . Full Screen • We will describe such (almost) equivalence by ≡ , i.e., Close we will write that A ≡ L ( u ( A )). Quit

Decision Making: . . . 4. Fast Iterative Process for Determining u ( A ) The Notion of Utility From Utility to . . . • Initially: we know the values u = 0 and u = 1 such Beyond Interval . . . that A ≡ L ( u ( A )) for some u ( A ) ∈ [ u, u ]. Multi-Agent . . . • What we do: we compute the midpoint u mid of the Beyond Optimization interval [ u, u ] and compare A with L ( u mid ). Even Further Beyond . . . • Possibilities: A ≤ L ( u mid ) and L ( u mid ) ≤ A . Acknowledgments Home Page • Case 1: if A ≤ L ( u mid ), then u ( A ) ≤ u mid , so Title Page u ∈ [ u, u mid ] . ◭◭ ◮◮ • Case 2: if L ( u mid ) ≤ A , then u mid ≤ u ( A ), so ◭ ◮ u ∈ [ u mid , u ] . Page 5 of 55 • After each iteration, we decrease the width of the in- Go Back terval [ u, u ] by half. Full Screen • After k iterations, we get an interval of width 2 − k which Close contains u ( A ) – i.e., we get u ( A ) w/accuracy 2 − k . Quit

Decision Making: . . . 5. How to Make a Decision Based on Utility Val- The Notion of Utility ues From Utility to . . . Beyond Interval . . . • Suppose that we have found the utilities u ( A ′ ), u ( A ′′ ), Multi-Agent . . . . . . , of the alternatives A ′ , A ′′ , . . . Beyond Optimization • Which of these alternatives should we choose? Even Further Beyond . . . Acknowledgments • By definition of utility, we have: Home Page • A ≡ L ( u ( A )) for every alternative A , and Title Page • L ( p ′ ) < L ( p ′′ ) if and only if p ′ < p ′′ . ◭◭ ◮◮ • We can thus conclude that A ′ is preferable to A ′′ if and ◭ ◮ only if u ( A ′ ) > u ( A ′′ ). Page 6 of 55 • In other words, we should always select an alternative with the largest possible value of utility. Go Back Full Screen • Interval techniques can help in finding the optimizing decision. Close Quit

Decision Making: . . . 6. How to Estimate Utility of an Action The Notion of Utility From Utility to . . . • For each action, we usually know possible outcomes Beyond Interval . . . S 1 , . . . , S n . Multi-Agent . . . • We can often estimate the prob. p 1 , . . . , p n of these out- Beyond Optimization comes. Even Further Beyond . . . • By definition of utility, each situation S i is equiv. to a Acknowledgments Home Page lottery L ( u ( S i )) in which we get: Title Page • A 1 with probability u ( S i ) and • A 0 with the remaining probability 1 − u ( S i ). ◭◭ ◮◮ ◭ ◮ • Thus, the action is equivalent to a complex lottery in which: Page 7 of 55 • first, we select one of the situations S i with proba- Go Back bility p i : P ( S i ) = p i ; Full Screen • then, depending on S i , we get A 1 with probability Close P ( A 1 | S i ) = u ( S i ) and A 0 w/probability 1 − u ( S i ). Quit

Decision Making: . . . 7. How to Estimate Utility of an Action (cont-d) The Notion of Utility From Utility to . . . • Reminder: Beyond Interval . . . • first, we select one of the situations S i with proba- Multi-Agent . . . bility p i : P ( S i ) = p i ; Beyond Optimization • then, depending on S i , we get A 1 with probability Even Further Beyond . . . P ( A 1 | S i ) = u ( S i ) and A 0 w/probability 1 − u ( S i ). Acknowledgments Home Page • The prob. of getting A 1 in this complex lottery is: n n � � Title Page P ( A 1 ) = P ( A 1 | S i ) · P ( S i ) = u ( S i ) · p i . ◭◭ ◮◮ i =1 i =1 ◭ ◮ • In the complex lottery, we get: � n Page 8 of 55 • A 1 with prob. u = p i · u ( S i ), and i =1 Go Back • A 0 w/prob. 1 − u . Full Screen • So, we should select the action with the largest value of expected utility u = � p i · u ( S i ). Close Quit

Decision Making: . . . 8. Non-Uniqueness of Utility The Notion of Utility From Utility to . . . • The above definition of utility u depends on A 0 , A 1 . Beyond Interval . . . • What if we use different alternatives A ′ 0 and A ′ 1 ? Multi-Agent . . . • Every A is equivalent to a lottery L ( u ( A )) in which we Beyond Optimization get A 1 w/prob. u ( A ) and A 0 w/prob. 1 − u ( A ). Even Further Beyond . . . Acknowledgments • For simplicity, let us assume that A ′ 0 < A 0 < A 1 < A ′ 1 . Home Page • Then, A 0 ≡ L ′ ( u ′ ( A 0 )) and A 1 ≡ L ′ ( u ′ ( A 1 )). Title Page • So, A is equivalent to a complex lottery in which: ◭◭ ◮◮ 1) we select A 1 w/prob. u ( A ) and A 0 w/prob. 1 − u ( A ); ◭ ◮ 2) depending on A i , we get A ′ 1 w/prob. u ′ ( A i ) and A ′ 0 w/prob. 1 − u ′ ( A i ). Page 9 of 55 Go Back • In this complex lottery, we get A ′ 1 with probability u ′ ( A ) = u ( A ) · ( u ′ ( A 1 ) − u ′ ( A 0 )) + u ′ ( A 0 ) . Full Screen • So, in general, utility is defined modulo an (increasing) Close linear transformation u ′ = a · u + b , with a > 0. Quit

Decision Making: . . . 9. Subjective Probabilities The Notion of Utility From Utility to . . . • In practice, we often do not know the probabilities p i Beyond Interval . . . of different outcomes. Multi-Agent . . . • For each event E , a natural way to estimate its subjec- Beyond Optimization tive probability is to fix a prize (e.g., $1) and compare: Even Further Beyond . . . – the lottery ℓ E in which we get the fixed prize if the Acknowledgments Home Page event E occurs and 0 is it does not occur, with – a lottery ℓ ( p ) in which we get the same amount Title Page with probability p . ◭◭ ◮◮ • Here, similarly to the utility case, we get a value ps ( E ) ◭ ◮ for which, for every ε > 0: Page 10 of 55 ℓ ( ps ( E ) − ε ) < ℓ E < ℓ ( ps ( E ) + ε ) . Go Back • Then, the utility of an action with possible outcomes Full Screen � n S 1 , . . . , S n is equal to u = ps ( E i ) · u ( S i ). Close i =1 Quit