Lectur ture 2 e 23 Planni anning U ng Under er U Uncer ertai - - PowerPoint PPT Presentation

lectur ture 2 e 23 planni anning u ng under er u uncer
SMART_READER_LITE
LIVE PREVIEW

Lectur ture 2 e 23 Planni anning U ng Under er U Uncer ertai - - PowerPoint PPT Presentation

Computer Science CPSC 322 Lectur ture 2 e 23 Planni anning U ng Under er U Uncer ertai tainty nty a and Decision N on Networ works 1 Announ nouncem emen ents Final exam Mon, Dec. 18, 12noon Same general format as


slide-1
SLIDE 1

Computer Science CPSC 322

Lectur ture 2 e 23 Planni anning U ng Under er U Uncer ertai tainty nty a and Decision N

  • n Networ

works

1

slide-2
SLIDE 2

Announ nouncem emen ents

  • Final exam
  • Mon, Dec. 18, 12noon
  • Same general format as midterm

 Part short questions, part longer problems  List from which I will draw the short questions is posted on Connect (“Final” folder)  I will also post there some Practice problems

Covers material from the beginning of the course See list of posted learning goals for what you should know

  • Office hours will continue as usual after the end of

classes

slide-3
SLIDE 3

Lect cture re Overvi rview

  • Recap
  • Intro to Decision theory
  • Utility and expected utility
  • Decision Networks for Single-stage decision

problems

3

slide-4
SLIDE 4

Inf nference in n Gen eneral

) ( ) , ( ) | ( e E P e E Y P e E Y P = = = =

  • All we need to compute is the numerator: joint probability of the query variable(s)

and the evidence!

  • Variable Elimination is an algorithm that efficiently performs this operation by

casting it as operations between factors - introduced next

= =

Y

e E Y P e E Y P ) , ( ) , (

We need to compute this numerator for each value of Y, Y, yi We need to marginalize over all the variables Z1,…Zk not involved in the query

∑ ∑

= = … = = =

k

Z i k Z i

e) ,E y ,Y ,Z , P(Z e E y Y P

1

1

... ) , (

To compute the denominator, marginalize over Y

  • Same value for every P(Y=yi). Normalization

constant ensuring that ∑

= =

Y i

E y P(Y 1 ) | Def of conditional probability

  • Y: subset of variables that is queried (e.g. Temperature in previous example)
  • E: subset of variables that are observed . E = e (W = yes in previous example)
  • Z1, …,Zk remaining variables in the JPD (Cloudy in previous example)
slide-5
SLIDE 5

X Y Z val t t t 0.1 t t f 0.9 t f t 0.2 t f f 0.8 f t t 0.4 f t f 0.6 f f t 0.3 f f f 0.7

Fa Factors

  • A factor is a function from a tuple of random variables to the

real numbers R

  • We write a factor on variables X1,… ,Xj as f(X1,… ,Xj)
  • A factor denotes one or more (possibly partial) distributions
  • ver the given tuple of variables, e.g.,
  • P(X1, X2) is a factor f(X1, X2)
  • P(Z | X,Y) is a factor

f(Z,X,Y)

  • P(Z=f|X,Y) is a factor f(X,Y)
  • Note: Factors do not have to sum to one

Distribution Set of Distributions

One for each combination

  • f values for X and Y

Set of partial Distributions

f(X, Y ) Z = f

slide-6
SLIDE 6

Rec ecap: F Fac actors and and Ope perations on T

  • n Them

hem

If we assign variable A=a in factor f7(A,B), what is the correct form for the resulting factor?

  • f(B).

When we assign variable A we remove it from the factor’s domain If we marginalize variable A out from factor f7(A,B), what is the correct form for the resulting factor?

  • f(B).

When we marginalize out variable A we remove it from the factor’s domain If we multiply factors f4(X,Y) and f6(Z,Y), what is the correct form for the resulting factor?

  • f(X,Y,Z)
  • When multiplying factors, the resulting factor’s domain is the union
  • f the multiplicands’ domains
  • What is the correct form for ∑B f5(A,B) × f6(B,C)
  • As usual, product before sum: ∑B ( f5(A,B) × f6(B,C) )
  • Result of multiplication: f7 (A,B,C). Then marginalize out B: f8 (A,C)
slide-7
SLIDE 7

Th The e variable elimi mination algo gorithm,

1. Construct a factor for each conditional probability. 2. For each factor, assign the observed variables E to their observed values. 3. Given an elimination ordering, decompose sum of products 4. 4. Sum um out

  • ut all variables Zi not involved in the query (one a time)
  • Multiply factors containing Zi
  • Then marginalize out Zi from the product

5. Multiply the remaining factors (which only involve Y )

y

Y f ) (

To compute P(Y=yi| E1=e1, …, Ej=ej) = The JPD of a Bayesian network is Given: P(Y, E1…, Ej , Z1…,Zk )

) ) ( | ( ) , , P(

1 1

=

= …

n i i i n

X pa X P X X

)) ( , ( )) ( | (

i i i i i

X pa X f X pa X P =

∑∏ ∑

= = =

= = =

1 1 1

, , 1 1 1

) ( ) , , , (

Z e E e E n i i Z j j

j j k

f e E e E Y P

 

  • bserved

Other variables not involved in the query

=

= … = = = … = =

y Y j j j j i

e , E , e E y Y P e , E , e E y Y P ) , ( ) , (

1 1 1 1

slide-8
SLIDE 8

The variable elimination algorithm,

1. Construct a factor for each conditional probability. 2. For each factor, assign the observed variables E to their observed values. 3. Given an elimination ordering, decompose sum of products 4. Sum out all variables Zi not involved in the query (one a time)

  • Multiply factors containing Zi
  • Then marginalize out Zi from the product

5. Multiply the remaining factors (which only involve Y )

  • 6. Normalize by dividing the resulting factor f(Y) by ∑

y

Y f ) (

To compute P(Y=yi| E1=e1, …, Ej=ej) = The JPD of a Bayesian network is Given: P(Y, E1…, Ej , Z1…,Zk )

) ) ( | ( ) , , P(

1 1

=

= …

n i i i n

X pa X P X X

)) ( , ( )) ( | (

i i i i i

X pa X f X pa X P =

∑∏ ∑

= = =

= = =

1 1 1

, , 1 1 1

) ( ) , , , (

Z e E e E n i i Z j j

j j k

f e E e E Y P

 

  • bserved

Other variables not involved in the query

=

= … = = = … = =

y Y j j j j i

e , E , e E y Y P e , E , e E y Y P ) , ( ) , (

1 1 1 1

slide-9
SLIDE 9

Step 1: Construct a factor for each cond. probability

P(G,H) = ∑A,B,C,D,E,F,I P(A)P(B|A)P(C)P(D|B,C)P(E|C)P(F|D)P(G|F,E)P(H|G)P(I|G)

P(G,H) = ∑A,B,C,D,E,F,I f0(A) f1(B,A) f2(C) f3(D,B,C) f4(E,C) f5(F, D) f6(G,F,E) f7(H,G) f8(I,G)

Compute P(G | H=h1 ).

9

slide-10
SLIDE 10

Previous state: P(G,H) = ∑A,B,C,D,E,F,I f0(A) f1(B,A) f2(C) f3(D,B,C) f4(E,C) f5(F, D) f6(G,F,E) f7(H,G) f8(I,G) Observe H :

Step 2: assign to observed variables their observed values.

P(G,H=h1)=∑A,B,C,D,E,F,I f0(A) f1(B,A) f2(C) f3(D,B,C) f4(E,C) f5(F, D) f6(G,F,E) f9(G) f8(I,G)

Compute P(G | H=h1 ).

10

slide-11
SLIDE 11

Step 3: Decompose sum of products

Previous state: P(G,H=h1) = ∑A,B,C,D,E,F,I f0(A) f1(B,A) f2(C) f3(D,B,C) f4(E,C) f5(F, D) f6(G,F,E) f9(G) f8(I,G)

Elimination ordering A, C, E, I, B, D, F :

P(G,H=h1) = f9(G) ∑F ∑D f5(F, D) ∑B ∑I f8(I,G) ∑E f6(G,F,E) ∑C f2(C) f3(D,B,C) f4(E,C) ∑A f0(A) f1(B,A)

Compute P(G | H=h1 ).

11

slide-12
SLIDE 12

Step 4: sum out non query variables (one at a time)

Previous state: P(G,H=h1) = f9(G) ∑F ∑D f5(F, D) ∑B ∑I f8(I,G) ∑E f6(G,F,E) ∑C f2(C) f3(D,B,C) f4(E,C) ∑A f0(A) f1(B,A) Eliminate A: perform product and sum out A in P(G,H=h1) = f9(G) ∑F ∑D f5(F, D) ∑B f10(B) ∑I f8(I,G) ∑E f6(G,F,E) ∑C f2(C) f3(D,B,C) f4(E,C)

Elimination order: A,C,E,I,B,D,F

Compute P(G | H=h1 ).

f10(B) does not depend

  • n C, E, or I, so we can

push it outside of those sums.

12

slide-13
SLIDE 13

Step 4: sum out non query variables (one at a time)

Previous state: P(G,H=h1) = f9(G) ∑F ∑D f5(F, D) ∑B f10(B) ∑I f8(I,G) ∑E f6(G,F,E) ∑C f2(C) f3(D,B,C) f4(E,C) Eliminate C: perform product and sum out C in P(G,H=h1) = f9(G) ∑F ∑D f5(F, D) ∑B f10(B) ∑I f8(I,G) ∑E f6(G,F,E) f11(B,D,E)

Compute P(G | H=h1 ).

Elimination order: A,C,E,I,B,D,F

13

slide-14
SLIDE 14

Step 4: sum out non query variables (one at a time)

Previous state: P(G,H=h1) = P(G,H=h1) = f9(G) ∑F ∑D f5(F, D) ∑B f10(B) ∑I f8(I,G) ∑E f6(G,F,E) f11(B,D,E) Eliminate E: perform product and sum out E in P(G,H=h1) = P(G,H=h1) = f9(G) ∑F ∑D f5(F, D) ∑B f10(B) f12(B,D,F,G) ∑I f8(I,G)

Compute P(G | H=h1 ).

Elimination order: A,C,E,I,B,D,F

14

slide-15
SLIDE 15

Previous state: P(G,H=h1) = P(G,H=h1) = f9(G) ∑F ∑D f5(F, D) ∑B f10(B) f12(B,D,F,G) ∑I f8(I,G) Eliminate I: perform product and sum out I in P(G,H=h1) = P(G,H=h1) = f9(G) f13(G)∑F ∑D f5(F, D) ∑B f10(B) f12(B,D,F,G)

Elimination order: A,C,E,I,B,D,F Step 4: sum out non query variables (one at a time)

Compute P(G | H=h1 ).

15

slide-16
SLIDE 16

Previous state: P(G,H=h1) = P(G,H=h1) = f9(G) f13(G)∑F ∑D f5(F, D) ∑B f10(B) f12(B,D,F,G) Eliminate B: perform product and sum out B in P(G,H=h1) = P(G,H=h1) = f9(G) f13(G)∑F ∑D f5(F, D) f14(D,F,G)

Elimination order: A,C,E,I,B,D,F Step 4: sum out non query variables (one at a time)

Compute P(G | H=h1 ).

16

slide-17
SLIDE 17

Previous state: P(G,H=h1) = P(G,H=h1) = f9(G) f13(G)∑F ∑D f5(F, D) f14(D,F,G) Eliminate D: perform product and sum out D in P(G,H=h1) = P(G,H=h1) = f9(G) f13(G)∑F f15(F,G)

Elimination order: A,C,E,I,B,D,F Step 4: sum out non query variables (one at a time)

Compute P(G | H=h1 ).

Multiply remaining factors (all in G): P(G,H=h1) = f17(G)

17

slide-18
SLIDE 18

Previous state: P(G,H=h1) = P(G,H=h1) = f9(G) f13(G)∑F f15(F,G) Eliminate F: perform product and sum out F in P(G,H=h1) = f9(G) f13(G)f16(G)

Elimination order: A,C,E,I,B,D,F Step 4: sum out non query variables (one at a time)

Compute P(G | H=h1 ).

Multiply remaining factors (all in G): P(G,H=h1) = f17(G)

∑ ∑

∈ ∈

= = = = = = = = = = = =

) ( ' 17 17 ) ( ' 1

) ' ( ) ( ) , ' ( ) , ( ) ( ) , ( ) | (

1 1 1 1

G dom g G dom g

g f g f h H g G P h H g G P h H P h H g G P h H g G P

Normalize

18

slide-19
SLIDE 19

Variable elimination: pruning

  • Before running VE, we can prune all variables Z that are

conditionally independent of the query Y given evidence E: Z ╨ Y | E – They cannot change the belief over Y given E!

  • We can also prune unobserved leaf nodes

– Since they are unobserved and not predecessors of the query nodes, they cannot influence the posterior probability of the query nodes

19

Thus, if the query is P(G=g| C=c1, F=f1, H=h1) we only need to consider this subnetwork

slide-20
SLIDE 20

Compl

  • mplexi

xity of

  • f Var

ariabl able e Elimi minat nation

  • n (

(VE) (not not r requi equired ed)

  • The complexity of VE is exponential in the maximum

number of variables in any factor during its execution

  • This number is called the treewidth of a graph (along an ordering)
  • Elimination ordering influences treewidth
  • Finding the best ordering is NP complete
  • I.e., the ordering that generates the minimum treewidth
  • Heuristics work well in practice (e.g. least connected variables first)
  • Even with best ordering, inference is sometimes infeasible

 In those cases, we need approximate inference. See CS422 & CS540

slide-21
SLIDE 21

Lect cture re Overvi rview

  • Recap
  • Intro to Decision theory
  • Utility and expected utility
  • Decision Networks for Single-stage decision

problems

slide-22
SLIDE 22

Wher here are are w we? e?

Environm nment ent Problem Type Query Planning Deterministic Stochastic Constraint Satisfaction Search Arc Consistency Search Search Logics STRIPS Vars + Constraints Variable Elimination Belief Nets Decision Nets Static Sequential

Representation Reasoning Technique

Variable Elimination

This concludes the module

  • n answering queries in

stochastic environments

slide-23
SLIDE 23

What hat’s N Nex ext?

Environm nment ent Problem Type Query Planning Deterministic Stochastic Constraint Satisfaction Search Arc Consistency Search Search Logics STRIPS Vars + Constraints Variable Elimination Belief Nets Decision Nets Static Sequential

Representation Reasoning Technique

Variable Elimination

Now we will look at acting in stochastic environments

slide-24
SLIDE 24

Deci cisi sions U s Under U r Unce cert rtainty: y: Intro ro

  • Earlier in the course, we focused on decision

making in deterministic domains

  • Planning
  • Now we face stochastic domains
  • so far we've considered how to represent and update

beliefs

  • what if an agent has to make decisions (act) under

uncertainty?

  • Making decisions under uncertainty is important
  • We represent the world probabilistically so we can use
  • ur beliefs as the basis for making decisions
slide-25
SLIDE 25

Deci cisi sions s Under der Unce cert rtainty: y: I Intro ro

  • An agent's decision will depend on
  • What actions are available
  • What beliefs the agent has
  • Which goals the agent has
  • Differences between deterministic and stochastic setting
  • Obvious difference in representation: need to represent our

uncertain beliefs

  • Actions will be pretty straightforward: represented as decision

variables

  • Goals will be interesting: we'll move from all-or-nothing goals to a

richer notion: rating how happy the agent is in different situations.

  • Putting these together, we'll extend Bayesian Networks to make a

new representation called Decision Networks

slide-26
SLIDE 26

Del elive very R y Robot

  • bot E

Exam ampl ple

  • Robot needs to reach a certain room
  • Robot can go
  • the short way - faster but with more obstacles, thus more prone to

accidents that can damage the robot and prevent it from reaching the room

  • the long way - slower but less prone to accident
  • Which way to go? Is it more important for the robot to arrive fast, or to

minimize the risk of damage?

  • The Robot can choose to wear pads to protect itself in case of accident,
  • r not to wear them. Pads make it heavier, increasing energy

consumption

  • Again, there is a tradeoff between reducing risk of damage, saving

resources and arriving fast

  • Possible outcomes
  • No pad, no accident
  • Pad, no accident
  • Pad, Accident
  • No pad, accident
slide-27
SLIDE 27

Next

  • We’ll see how to represent and reason about situations
  • f this nature using Decision Trees, as well as
  • Probability to measure the uncertainty in action outcome
  • Utility to measure agent’s preferences over the various outcomes
  • Combined in a measure of expected utility that can be used to

identify the action with the best expected outcome

  • Best that an intelligent agent can do when it needs to act in

a stochastic environment

slide-28
SLIDE 28

Dec ecisi sion T

  • n Tree

ee for

  • r t

the he Del elive very y Robot

  • bot Exam

ampl ple

  • Decision variable 1: the robot can choose to wear pads
  • Yes: protection against accidents, but extra weight
  • No: fast, but no protection
  • Decision variable 2: the robot can choose the way
  • Short way: quick, but higher chance of accident
  • Long way: safe, but slow
  • Random variable: is there an accident?

Agent decides Chance decides

slide-29
SLIDE 29

Possibl ble w e worlds ds and d d decision v

  • n variabl

ables es

  • A possible world specifies a value for each random variable

and each decision variable

  • For each assignment of values to all decision variables:
  • the probabilities of the worlds satisfying that assignment

sum to 1.

0.2 0.8

slide-30
SLIDE 30

Possibl ble w e worlds ds and d d decision v

  • n variabl

ables es

0.01 0.99 0.2 0.8

  • A possible world specifies a value for each random variable

and each decision variable

  • For each assignment of values to all decision variables:
  • the probabilities of the worlds satisfying that assignment

sum to 1.

slide-31
SLIDE 31

Possibl ble w e worlds ds and d d decision v

  • n variabl

ables es

0.01 0.99 0.2 0.8 0.2 0.8

  • A possible world specifies a value for each random variable

and each decision variable

  • For each assignment of values to all decision variables:
  • the probabilities of the worlds satisfying that assignment

sum to 1.

slide-32
SLIDE 32

Possibl ble w e worlds ds and d d decision v

  • n variabl

ables es

0.01 0.99 0.2 0.8 0.01 0.99 0.2 0.8

  • A possible world specifies a value for each random variable

and each decision variable

  • For each assignment of values to all decision variables:
  • the probabilities of the worlds satisfying that assignment

sum to 1.

slide-33
SLIDE 33

Lect cture re Overvi rview

  • Recap
  • Intro to Decision theory
  • Utility and expected utility
  • Decision Networks for single-stage decision

problems

slide-34
SLIDE 34

Utility tility

  • Utility: a measure of desirability of possible worlds to

an agent

  • Let U be a real-valued function such that U(w) represents

an agent's degree of preference for world w

  • Expressed by a number in [0,100]
slide-35
SLIDE 35

Utility tility fo for th the R Robot E Example le

  • Which would be a reasonable utility function for
  • ur robot?
  • Which are the best and worst scenarios?

0.01 0.99 0.2 0.8 0.01 0.99 0.2 0.8

Utility probability

slide-36
SLIDE 36

Utility tility fo for th the R Robot E Example le

  • Which would be a reasonable utility function for
  • ur robot?

0.01 0.99 0.2 0.8 0.01 0.99 0.2 0.8

Utility probability

35 95 30 75 3 100 80

slide-37
SLIDE 37

Utility: Simple Goals

  • How can the simple (boolean) goal “reach the room”

be specified?

Which way Accident Wear Pads Utility long true true long true false long false true long false false short true true short true false short false true short false false 100 90

B. A. C .

  • D. Not possible

Which way Accident Wear Pads Utility long true true long true false long false true long false false short true true short true false short false true short false false 100 Which way Accident Wear Pads Utility long true true long true false long false true long false false short true true short true false short false true short false false 100 100 100 100

slide-38
SLIDE 38

Utility tility

  • Utility: a measure of desirability of possible worlds to an agent
  • Let U be a real-valued function such that U(w) represents an agent's

degree of preference for world w

  • Expressed by a number in [0,100]
  • Simple goals can still be specified
  • Worlds that satisfy the goal have utility 100
  • Other worlds have utility 0

Which way Accident Wear Pads Utility long true true long true false long false true long false false short true true short true false short false true short false false 100 100 100 100

e.g., goal “reach the room”

slide-39
SLIDE 39

Optimal decisions: combining Utility and Probability

  • Each set of decisions defines a probability distribution over possible
  • utcomes
  • Each outcome has a utility
  • For each set of decisions, we need to know their expected utility
  • the value for the agent of achieving a certain probability distribution
  • ver outcomes (possible worlds)
  • The expected utility of a set of decisions is obtained by
  • weighting the utility of the relevant possible worlds by their probability.

35 35 95 95 0. 0.2 0. 0.8

  • We want to find the decision with maximum expected utility

value of this scenario?

slide-40
SLIDE 40

Expected u utility tility o

  • f

f a decis isio ion

  • The expected utility of a specific decision D = d is indicated as

E(U | D = d ) and it is computed as follows

P(w1)×U(w1) + P(w2)×U(w2) + P(w3)×U(w3) +…+ P(wn)×U(wn) Where

  • w1 , w2 , w3 ..., wn are all the possible worlds in which d is true
  • P(w1), P(w2), ...P(wn) are their probabilities
  • U(w1), U(w2), ...U(wn) are the corresponding utilities for w1 , ..., wn

That is,

  • for each possible world wi in which the decision d is true, multiply the

probability of that world and its utility P(wi) × U(wi)

  • sum all these products together

In a formula E(U | D = di ) = ∑ w╞ (D = di )P(w) ×U(w)

This notation indicates all the possible worlds in which d is true

slide-41
SLIDE 41

Exam ampl ple of e of Expect pected U ed Utility

0.01 0.99 0.2 0.8 0.01 0.99 0.2 0.8

Utility

35 35 95

Probability E[U|D]

35 30 75 35 3 100 35 80

  • The expected utility of decision D = d is
  • What is the expected utility of Wearpads=yes, Way=short ?

E(U | D = d ) = ∑ w╞ (D = d )P(w) U(w) = P(w1)×U(w1) + ….+ P(wn)×U(wn)

  • A. 7
  • B. 83
  • C. 76
  • D. 157.55
slide-42
SLIDE 42

Expected u utility tility o

  • f

f a decis isio ion

0.01 0.99 0.2 0.8 0.01 0.99 0.2 0.8

Utility

35 35 95

Probability E[U|D]

35 30 75 35 3 100 35 80

  • The expected utility of decision D = d is
  • What is the expected utility of Wearpads=yes, Way=short ?
  • 0.2 * 35 + 0.8 * 95 = 83

E(U | D = d ) = ∑ w╞ (D = d )P(w) U(w) = P(w1)×U(w1) + ….+ P(wn)×U(wn)

slide-43
SLIDE 43

Expected u utility tility o

  • f

f a decis isio ion

0.01 0.99 0.2 0.8 0.01 0.99 0.2 0.8

Utility

35 35 95

Probability E[U|D]

83 35 30 75 35 3 100 35 80 74.55 80.6 79.2

  • The expected utility of decision D = d is

E(U | D = d ) = ∑ w╞ (D = d )P(w) U(w) = P(w1)×U(w1) + ….+ P(wn)×U(wn)

slide-44
SLIDE 44

Lect cture re Overvi rview

  • Recap
  • Intro to Decision theory
  • Utility and expected utility
  • Decision Networks for Single-stage decision

problems

slide-45
SLIDE 45

Singl gle e Action v

  • n vs. Sequen

quence o e of Actions

  • ns
  • Single Action (aka One-Off Decisions)
  • One or more primitive decisions that can be treated as a single macro

decision to be made before acting

  • E.g., “WearPads” and “WhichWay” can be combined into macro

decision (WearPads, WhichWay) with domain {yes,no} × {long, short}

  • Sequence of Actions (Sequential Decisions)
  • Repeat:

make observations decide on an action carry out the action

  • Agent has to take actions not knowing for sure what the future brings

This is fundamentally different from everything we’ve seen so far Planning was sequential, but agent could still think first and then act

slide-46
SLIDE 46

Op Optim timal s l sin ingle-stage d age decision

  • n

Given a single (macro) decision variable D

  • the agent can choose D=di for any value di ∈ dom(D)
slide-47
SLIDE 47

Opt ptimal dec decision i in n rob robot del delivery ex example

0.01 0.99 0.2 0.8 0.01 0.99 0.2 0.8

Utility

35 35 95

Conditional probability E[U|D]

83 35 30 75 35 3 100 35 80 74.55 80.6 79.2

Best decision: (wear pads, short way)

slide-48
SLIDE 48

Singl ngle-Stag age e dec decisi sion net

  • n networ
  • rks

Extend belief networks Random variables: same as in Bayesian networks

  • drawn as an ellipse
  • Arcs into the node represent probabilistic dependence
  • random variable is conditionally independent of its non-descendants gi

its parents

Decision nodes, that the agent chooses the value for

  • Parents: only other decision nodes allowed

represent information available when the decision is made

  • Domain is the set of possible actions
  • Drawn as a rectangle

Exactly one utility node

  • Parents: all random & decision variables on which the utility depends
  • Specifies a utility for each instantiation of its parents
  • Drawn as a diamond
slide-49
SLIDE 49

Examp mple D Deci cisi sion N Network rk

Decision nodes simply list the available decisions.

Which way Accident Wear Pads Utility long true true long true false long false true long false false short true true short true false short false true short false false 30 75 80 35 3 95 100 Which Way W Accident A P(A|W) long long short short true false true false 0.01 0.99 0.2 0.8

Explicitly shows dependencies. E.g., which variables affect the probability of an accident and the agent’s utility?

Which Way t f

slide-50
SLIDE 50

Examp mple D Deci cisi sion N Network rk

Decision nodes simply list the available decisions.

Which way Accident Wear Pads Utility long true true long true false long false true long false false short true true short true false short false true short false false 30 75 80 35 3 95 100 Which Way W Accident A P(A|W) long long short short true false true false 0.01 0.99 0.2 0.8

Explicitly shows dependencies. E.g., which variables affect the probability of an accident and the agent’s utility?

Which Way t f

slide-51
SLIDE 51

Slide 51

Appl pplet et f for

  • r B

Bay ayes esian an and and Dec ecision

  • n Net

etworks

The Belief and Decision Networks we have seen previously allows you to load predefined Decision networks for various domains and run queries on them. Select one of the available examples via “File -> Load Sample Problem For Deci cisi sion Netw tworks ks

  • Choose any of the examples below the blue line in the list that appears
  • Right click on a node to perform any of these operations
  • View

ew t the C he CPT/Decision t tabl able/Utility t tabl able f for

  • r a

a chanc hance/dec ecision/utility node node

  • Make an observation for a chance variable (i.e., set it to one of its values)
  • Query the current probability distribution for a chance node given the
  • bservations made
  • A dialogue box will appear the first time you do this. Select “Always brief”

at the bottom, and then click “Brief”.

  • To compute the optimal decision (policy) click on the “Optimize Decision” button in

the toolbar and select Brief in the dialogue box that will appear

  • To see the actual policy, view the decision table for each decision node in the

network

See available help pages and video tutorials for more details on how to use the Bayes applet (http://www.aispace.org/bayes/)

slide-52
SLIDE 52
  • Compare and contrast stochastic single-stage (one-off)

decisions vs. multistage (sequential) decisions

  • Define a Utility Function on possible worlds
  • Define and compute optimal one-off decisions
  • Represent one-off decisions as single stage decision

networks

  • Compute optimal decisions by Variable Elimination

Lear Learni ning ng Goal

  • als F

s For

  • r Dec

ecisi sion

  • n U

Under nder Unc ncertai aint nty s so

  • Far

ar