ECE 4524 Artificial Intelligence and Engineering Applications - - PowerPoint PPT Presentation

ECE 4524 Artificial Intelligence and Engineering Applications

Lecture 21: Decisions and Utility Reading: AIAMA 16.1-16.3 Today’s Schedule:

◮ Introduction to Decision Theory - Maximum Expected Utility ◮ Utility Functions ◮ Examples

Introduction to Decision Theory

◮ A decision is a choice among alternatives. For stochastic

environments there is a probability (belief) of that alternative.

◮ If we consider our agent model as a discrete Bayesian

Network, the probability of the resulting state after applying action a given the action and evidence, e, is s′ P(result(a) = s′|a, e)

◮ To choose an action we might just

argmax

a

P(result(a) = s′|a, e) but that assumes all outcomes are equally desirable.

◮ Before we used a function to order the desirability of results,

how do we do this including the probability?

Maximum Expected Utility

◮ Define the utility of a state s′ as a function U(s′) ∈ R ◮ Written as a function of the action requires we weight the

utility of an outcome by the probability of its occurrence U(a|e) = P(result(a) = s′|a, e)U(s′)

◮ To choose an action we select the action with the highest

expected utility, EU(a|e) best action = argmax

a

EU(a|e)

Maximum Expected Utility

◮ For discrete state spaces

EU(a|e) =

• s′

P(result(a) = s′|a, e)U(s′)

◮ For continuous state spaces

EU(a|e) =

• f (result(a) = s′|a, e)U(s′) ds′

This is used in Optimal Control where result is the solution to a differential equation. It’s integral over time is called the ”cost to go”.

Example: Lottery tickets

Do you buy lottery tickets

◮ never ◮ rarely ◮ often ◮ whenever I have the money

Another Example

Recall our test for a disease P(D|T) = P(T|D)P(D) P(T)

◮ Consider the decision to take the test with no evidence

(screening)

◮ What is the state? outcomes?

Why AI is hard

So why is this course not just maximum expected utility?

◮ computing probabilities in general is #-Hard ◮ building models of the world is hard ◮ utility is subjective ◮ you might have to explore the state space to evaluate utility

Utility Functions

What constitutes a valid utility function?

◮ Define a preference as an ordering among outcomes, denoted

A ≻ B A ∼ B

◮ Define a lottery for outcomes Si with probability pi as

L = [(p1, S1); (p2, S2); · · · (pn, Sn); ]

The axioms of utility theory

◮ orderability ◮ transitivity ◮ continuity ◮ substitutability ◮ monotonicity ◮ Decomposibility, aka no fun in gambling

These axioms lead to a numerical relationship among utility as equivalent to preference.

Economic models of utility

◮ risk averse v/s risk seeking ◮ optimizer’s curse

Next Actions

◮ Reading on Learning (AIAMA 18.1-18.3) ◮ No warmup

Note: This concludes part three of the course. Remember PS 3 is due 4/5. Quiz 3 will be Thursday 4/12.