Computer Science CPSC 322
Lectur ture 2 e 23 Planni anning U ng Under er U Uncer ertai tainty nty a and Decision N
- n Networ
works
1
Lectur ture 2 e 23 Planni anning U ng Under er U Uncer ertai - - PowerPoint PPT Presentation
Computer Science CPSC 322 Lectur ture 2 e 23 Planni anning U ng Under er U Uncer ertai tainty nty a and Decision N on Networ works 1 Announ nouncem emen ents Final exam Mon, Dec. 18, 12noon Same general format as
1
Part short questions, part longer problems List from which I will draw the short questions is posted on Connect (“Final” folder) I will also post there some Practice problems
Covers material from the beginning of the course See list of posted learning goals for what you should know
3
and the evidence!
casting it as operations between factors - introduced next
Y
We need to compute this numerator for each value of Y, Y, yi We need to marginalize over all the variables Z1,…Zk not involved in the query
= = … = = =
k
Z i k Z i
e) ,E y ,Y ,Z , P(Z e E y Y P
1
1
... ) , (
To compute the denominator, marginalize over Y
constant ensuring that ∑
= =
Y i
E y P(Y 1 ) | Def of conditional probability
X Y Z val t t t 0.1 t t f 0.9 t f t 0.2 t f f 0.8 f t t 0.4 f t f 0.6 f f t 0.3 f f f 0.7
f(Z,X,Y)
Distribution Set of Distributions
One for each combination
Set of partial Distributions
If we assign variable A=a in factor f7(A,B), what is the correct form for the resulting factor?
When we assign variable A we remove it from the factor’s domain If we marginalize variable A out from factor f7(A,B), what is the correct form for the resulting factor?
When we marginalize out variable A we remove it from the factor’s domain If we multiply factors f4(X,Y) and f6(Z,Y), what is the correct form for the resulting factor?
1. Construct a factor for each conditional probability. 2. For each factor, assign the observed variables E to their observed values. 3. Given an elimination ordering, decompose sum of products 4. 4. Sum um out
5. Multiply the remaining factors (which only involve Y )
y
Y f ) (
To compute P(Y=yi| E1=e1, …, Ej=ej) = The JPD of a Bayesian network is Given: P(Y, E1…, Ej , Z1…,Zk )
) ) ( | ( ) , , P(
1 1
=
= …
n i i i n
X pa X P X X
)) ( , ( )) ( | (
i i i i i
X pa X f X pa X P =
= = =
= = =
1 1 1
, , 1 1 1
) ( ) , , , (
Z e E e E n i i Z j j
j j k
f e E e E Y P
Other variables not involved in the query
=
= … = = = … = =
y Y j j j j i
e , E , e E y Y P e , E , e E y Y P ) , ( ) , (
1 1 1 1
1. Construct a factor for each conditional probability. 2. For each factor, assign the observed variables E to their observed values. 3. Given an elimination ordering, decompose sum of products 4. Sum out all variables Zi not involved in the query (one a time)
5. Multiply the remaining factors (which only involve Y )
y
Y f ) (
To compute P(Y=yi| E1=e1, …, Ej=ej) = The JPD of a Bayesian network is Given: P(Y, E1…, Ej , Z1…,Zk )
) ) ( | ( ) , , P(
1 1
=
= …
n i i i n
X pa X P X X
)) ( , ( )) ( | (
i i i i i
X pa X f X pa X P =
= = =
= = =
1 1 1
, , 1 1 1
) ( ) , , , (
Z e E e E n i i Z j j
j j k
f e E e E Y P
Other variables not involved in the query
=
= … = = = … = =
y Y j j j j i
e , E , e E y Y P e , E , e E y Y P ) , ( ) , (
1 1 1 1
P(G,H) = ∑A,B,C,D,E,F,I P(A)P(B|A)P(C)P(D|B,C)P(E|C)P(F|D)P(G|F,E)P(H|G)P(I|G)
P(G,H) = ∑A,B,C,D,E,F,I f0(A) f1(B,A) f2(C) f3(D,B,C) f4(E,C) f5(F, D) f6(G,F,E) f7(H,G) f8(I,G)
9
Previous state: P(G,H) = ∑A,B,C,D,E,F,I f0(A) f1(B,A) f2(C) f3(D,B,C) f4(E,C) f5(F, D) f6(G,F,E) f7(H,G) f8(I,G) Observe H :
P(G,H=h1)=∑A,B,C,D,E,F,I f0(A) f1(B,A) f2(C) f3(D,B,C) f4(E,C) f5(F, D) f6(G,F,E) f9(G) f8(I,G)
10
Previous state: P(G,H=h1) = ∑A,B,C,D,E,F,I f0(A) f1(B,A) f2(C) f3(D,B,C) f4(E,C) f5(F, D) f6(G,F,E) f9(G) f8(I,G)
Elimination ordering A, C, E, I, B, D, F :
P(G,H=h1) = f9(G) ∑F ∑D f5(F, D) ∑B ∑I f8(I,G) ∑E f6(G,F,E) ∑C f2(C) f3(D,B,C) f4(E,C) ∑A f0(A) f1(B,A)
11
Previous state: P(G,H=h1) = f9(G) ∑F ∑D f5(F, D) ∑B ∑I f8(I,G) ∑E f6(G,F,E) ∑C f2(C) f3(D,B,C) f4(E,C) ∑A f0(A) f1(B,A) Eliminate A: perform product and sum out A in P(G,H=h1) = f9(G) ∑F ∑D f5(F, D) ∑B f10(B) ∑I f8(I,G) ∑E f6(G,F,E) ∑C f2(C) f3(D,B,C) f4(E,C)
f10(B) does not depend
push it outside of those sums.
12
Previous state: P(G,H=h1) = f9(G) ∑F ∑D f5(F, D) ∑B f10(B) ∑I f8(I,G) ∑E f6(G,F,E) ∑C f2(C) f3(D,B,C) f4(E,C) Eliminate C: perform product and sum out C in P(G,H=h1) = f9(G) ∑F ∑D f5(F, D) ∑B f10(B) ∑I f8(I,G) ∑E f6(G,F,E) f11(B,D,E)
13
Previous state: P(G,H=h1) = P(G,H=h1) = f9(G) ∑F ∑D f5(F, D) ∑B f10(B) ∑I f8(I,G) ∑E f6(G,F,E) f11(B,D,E) Eliminate E: perform product and sum out E in P(G,H=h1) = P(G,H=h1) = f9(G) ∑F ∑D f5(F, D) ∑B f10(B) f12(B,D,F,G) ∑I f8(I,G)
14
Previous state: P(G,H=h1) = P(G,H=h1) = f9(G) ∑F ∑D f5(F, D) ∑B f10(B) f12(B,D,F,G) ∑I f8(I,G) Eliminate I: perform product and sum out I in P(G,H=h1) = P(G,H=h1) = f9(G) f13(G)∑F ∑D f5(F, D) ∑B f10(B) f12(B,D,F,G)
15
Previous state: P(G,H=h1) = P(G,H=h1) = f9(G) f13(G)∑F ∑D f5(F, D) ∑B f10(B) f12(B,D,F,G) Eliminate B: perform product and sum out B in P(G,H=h1) = P(G,H=h1) = f9(G) f13(G)∑F ∑D f5(F, D) f14(D,F,G)
16
Previous state: P(G,H=h1) = P(G,H=h1) = f9(G) f13(G)∑F ∑D f5(F, D) f14(D,F,G) Eliminate D: perform product and sum out D in P(G,H=h1) = P(G,H=h1) = f9(G) f13(G)∑F f15(F,G)
Multiply remaining factors (all in G): P(G,H=h1) = f17(G)
17
Previous state: P(G,H=h1) = P(G,H=h1) = f9(G) f13(G)∑F f15(F,G) Eliminate F: perform product and sum out F in P(G,H=h1) = f9(G) f13(G)f16(G)
Multiply remaining factors (all in G): P(G,H=h1) = f17(G)
∈ ∈
= = = = = = = = = = = =
) ( ' 17 17 ) ( ' 1
) ' ( ) ( ) , ' ( ) , ( ) ( ) , ( ) | (
1 1 1 1
G dom g G dom g
g f g f h H g G P h H g G P h H P h H g G P h H g G P
Normalize
18
conditionally independent of the query Y given evidence E: Z ╨ Y | E – They cannot change the belief over Y given E!
– Since they are unobserved and not predecessors of the query nodes, they cannot influence the posterior probability of the query nodes
19
Thus, if the query is P(G=g| C=c1, F=f1, H=h1) we only need to consider this subnetwork
In those cases, we need approximate inference. See CS422 & CS540
Representation Reasoning Technique
This concludes the module
stochastic environments
Representation Reasoning Technique
Now we will look at acting in stochastic environments
uncertain beliefs
variables
richer notion: rating how happy the agent is in different situations.
new representation called Decision Networks
accidents that can damage the robot and prevent it from reaching the room
minimize the risk of damage?
consumption
resources and arriving fast
identify the action with the best expected outcome
Agent decides Chance decides
0.2 0.8
0.01 0.99 0.2 0.8
0.01 0.99 0.2 0.8 0.2 0.8
0.01 0.99 0.2 0.8 0.01 0.99 0.2 0.8
0.01 0.99 0.2 0.8 0.01 0.99 0.2 0.8
Utility probability
0.01 0.99 0.2 0.8 0.01 0.99 0.2 0.8
Utility probability
35 95 30 75 3 100 80
Which way Accident Wear Pads Utility long true true long true false long false true long false false short true true short true false short false true short false false 100 90
Which way Accident Wear Pads Utility long true true long true false long false true long false false short true true short true false short false true short false false 100 Which way Accident Wear Pads Utility long true true long true false long false true long false false short true true short true false short false true short false false 100 100 100 100
degree of preference for world w
Which way Accident Wear Pads Utility long true true long true false long false true long false false short true true short true false short false true short false false 100 100 100 100
35 35 95 95 0. 0.2 0. 0.8
E(U | D = d ) and it is computed as follows
probability of that world and its utility P(wi) × U(wi)
This notation indicates all the possible worlds in which d is true
0.01 0.99 0.2 0.8 0.01 0.99 0.2 0.8
Utility
35 35 95
Probability E[U|D]
35 30 75 35 3 100 35 80
E(U | D = d ) = ∑ w╞ (D = d )P(w) U(w) = P(w1)×U(w1) + ….+ P(wn)×U(wn)
0.01 0.99 0.2 0.8 0.01 0.99 0.2 0.8
Utility
35 35 95
Probability E[U|D]
35 30 75 35 3 100 35 80
E(U | D = d ) = ∑ w╞ (D = d )P(w) U(w) = P(w1)×U(w1) + ….+ P(wn)×U(wn)
0.01 0.99 0.2 0.8 0.01 0.99 0.2 0.8
Utility
35 35 95
Probability E[U|D]
83 35 30 75 35 3 100 35 80 74.55 80.6 79.2
E(U | D = d ) = ∑ w╞ (D = d )P(w) U(w) = P(w1)×U(w1) + ….+ P(wn)×U(wn)
decision to be made before acting
decision (WearPads, WhichWay) with domain {yes,no} × {long, short}
make observations decide on an action carry out the action
This is fundamentally different from everything we’ve seen so far Planning was sequential, but agent could still think first and then act
0.01 0.99 0.2 0.8 0.01 0.99 0.2 0.8
Utility
35 35 95
Conditional probability E[U|D]
83 35 30 75 35 3 100 35 80 74.55 80.6 79.2
its parents
represent information available when the decision is made
Which way Accident Wear Pads Utility long true true long true false long false true long false false short true true short true false short false true short false false 30 75 80 35 3 95 100 Which Way W Accident A P(A|W) long long short short true false true false 0.01 0.99 0.2 0.8
Explicitly shows dependencies. E.g., which variables affect the probability of an accident and the agent’s utility?
Which Way t f
Which way Accident Wear Pads Utility long true true long true false long false true long false false short true true short true false short false true short false false 30 75 80 35 3 95 100 Which Way W Accident A P(A|W) long long short short true false true false 0.01 0.99 0.2 0.8
Explicitly shows dependencies. E.g., which variables affect the probability of an accident and the agent’s utility?
Which Way t f
Slide 51
The Belief and Decision Networks we have seen previously allows you to load predefined Decision networks for various domains and run queries on them. Select one of the available examples via “File -> Load Sample Problem For Deci cisi sion Netw tworks ks
ew t the C he CPT/Decision t tabl able/Utility t tabl able f for
a chanc hance/dec ecision/utility node node
at the bottom, and then click “Brief”.
the toolbar and select Brief in the dialogue box that will appear
network
See available help pages and video tutorials for more details on how to use the Bayes applet (http://www.aispace.org/bayes/)