Slides Set 3: Building Bayesian Networks Rina Dechter Darwiche - - PowerPoint PPT Presentation

▶
slides set 3
SMART_READER_LITE
LIVE PREVIEW

Slides Set 3: Building Bayesian Networks Rina Dechter Darwiche - - PowerPoint PPT Presentation

Aug 28, 2022 514 likes •1.79k views

Reasoning with Graphical Models Slides Set 3: Building Bayesian Networks Rina Dechter Darwiche chapters 5 slides3 COMPSCI 2020 slides3 COMPSCI 2020 Queries: Different queries may be relevant for different scenarios

slide-1
SLIDE 1

Reasoning with Graphical Models

Slides Set 3:

Rina Dechter

slides3 COMPSCI 2020

Darwiche chapters 5

Building Bayesian Networks

slide-2
SLIDE 2

slides3 COMPSCI 2020

slide-3
SLIDE 3

Queries: Different queries may be relevant for different scenarios

slide-4
SLIDE 4

For other tools (e.g., GeNie/Smile) see class page http://reasoning.cs.ucla.edu/samiam

slide-5
SLIDE 5

Other type of evidence: We may want to know the probability that the patient has either a positive X-ray or dyspnoea, X =yes or D=yes.

slide-6
SLIDE 6
slide-7
SLIDE 7

slides3 COMPSCI 2020

slide-8
SLIDE 8

C= lung cancer

slides3 COMPSCI 2020

slide-9
SLIDE 9

slides3 COMPSCI 2020

slide-10
SLIDE 10

P(V=yes|E=yes) P(V=yes|E=no) =2 Define a CPT for V that satisfies this constraint

Soft evidence of Positive x-ray or Dyspnoea (X=yes or D = yes) with odds

  • f 2 to 1.

Modelling: Add E variable and Add V to model soft evidence. slides3 COMPSCI 2020

slide-11
SLIDE 11

MPE is also called MAP

slide-12
SLIDE 12

MPE is also called MAP

slide-13
SLIDE 13

slides3 COMPSCI 2020

slide-14
SLIDE 14

MAP is also called Marginal Map (MMAP)

slide-15
SLIDE 15

Is it correct?

slides3 COMPSCI 2020

slide-16
SLIDE 16

slides3 COMPSCI 2020

slide-17
SLIDE 17

Probabilistic Reasoning Problems

Tasks:

Combinatorial search / counting queries Exact reasoning NP-complete (or worse)

Sum-Inference (data likelihood, P(evidence)

Max-Inference (most likely config, MPE.)

Mixed-Inference (optimal prediction, MAP, Marginal Map) Harder

slide-18
SLIDE 18
slide-19
SLIDE 19

What about the boundary strata?

slides3 COMPSCI 2020

slide-20
SLIDE 20

Constructing a Bayesian Network for any Distribution P

Intuition: The causes of X can serve as the parents

slides3 COMPSCI 2020

slide-21
SLIDE 21

slides3 COMPSCI 2020

slide-22
SLIDE 22

Variables? Arcs? Try it.

slides3 COMPSCI 2020

slide-23
SLIDE 23

What about?

A naive Bayes structure has the following edges C -> A1, . . . , C -> Am, where C is called the class variable and A1; : : : ;Am are called the attributes.

slides3 COMPSCI 2020

slide-24
SLIDE 24

slides3 COMPSCI 2020

I(ST, Cond=cold,Fever)?

slide-25
SLIDE 25

Learn the model from data

slides3 COMPSCI 2020

slide-26
SLIDE 26

Learning the model

slides3 COMPSCI 2020

slide-27
SLIDE 27

Try it: Variables and values? Structure? CPTs?

slide-28
SLIDE 28

slides3 COMPSCI 2020

Try with GeNie

slide-29
SLIDE 29

slides3 COMPSCI 2020

slide-30
SLIDE 30

Read in the book. We will not cover this. Also about level of granularity

slide-31
SLIDE 31

Try it: Variables? Values? Structure?

slides3 COMPSCI 2020

slide-32
SLIDE 32

slides3 COMPSCI 2020

slide-33
SLIDE 33

slides3 COMPSCI 2020

slide-34
SLIDE 34

slides3 COMPSCI 2020

slide-35
SLIDE 35
slide-36
SLIDE 36

slides3 COMPSCI 2020

slide-37
SLIDE 37

slides3 COMPSCI 2020

slide-38
SLIDE 38

slides3 COMPSCI 2020

slide-39
SLIDE 39

slides3 COMPSCI 2020

slide-40
SLIDE 40

slides3 COMPSCI 2020

slide-41
SLIDE 41

slides3 COMPSCI 2020

slide-42
SLIDE 42

slides3 COMPSCI 2020

slide-43
SLIDE 43

slides3 COMPSCI 2020

slide-44
SLIDE 44

slides3 COMPSCI 2020

slide-45
SLIDE 45

Variables? Values? Structure?

slides3 COMPSCI 2020

slide-46
SLIDE 46

slides3 COMPSCI 2020

slide-47
SLIDE 47

slides3 COMPSCI 2020

slide-48
SLIDE 48

slides3 COMPSCI 2020

slide-49
SLIDE 49

slides3 COMPSCI 2020

slide-50
SLIDE 50

slides3 COMPSCI 2020

slide-51
SLIDE 51

Try it: Variables, values, structure?

slides3 COMPSCI 2020

slide-52
SLIDE 52

slides3 COMPSCI 2020

slide-53
SLIDE 53

slides3 COMPSCI 2020

slide-54
SLIDE 54

slides3 COMPSCI 2020

slide-55
SLIDE 55

slides3 COMPSCI 2020

slide-56
SLIDE 56
slide-57
SLIDE 57

slides3 COMPSCI 2020

slide-58
SLIDE 58

slides3 COMPSCI 2020

slide-59
SLIDE 59

What queries should we use here? P(Y not equal U) = 0.01

slide-60
SLIDE 60

slides3 COMPSCI 2020

slide-61
SLIDE 61

WER (word error rate), BER (bit error rate) MAP (MPE) minimizes WER, PM minimize BER… What do you think?

slide-62
SLIDE 62

Notice: Odds: o(x) = P(x)\P(bar(x)) K =Bayes factor = o’(x)\o(x) … the posterior odds after observing divided by prior odds For Gausian x: evidence on Y=y can be emulated with soft evidence on x with K =f(y|x) \f(y|bar(x)) = the expression above. Read chapter 5

slides3 COMPSCI 2020

slide-63
SLIDE 63

slides3 COMPSCI 2020

slide-64
SLIDE 64

slides3 COMPSCI 2020

slide-65
SLIDE 65

slides3 COMPSCI 2020

slide-66
SLIDE 66

slides3 COMPSCI 2020

slide-67
SLIDE 67

slides3 COMPSCI 2020

slide-68
SLIDE 68

slides3 COMPSCI 2020

slide-69
SLIDE 69

slides3 COMPSCI 2020

slide-70
SLIDE 70

slides3 COMPSCI 2020

slide-71
SLIDE 71

slides3 COMPSCI 2020

slide-72
SLIDE 72

The excitement about probabilistic decoding in the 90’s And the rise of belief propagation Task (PM for each bit)

slides3 COMPSCI 2020

slide-73
SLIDE 73

Commonsense reasoning

When SamBot goes home at night, he wants to know if his family is home before he tries the doors. Often when SamBot's wife leaves the house she turns on an outdoor light. However, she sometimes turns on this light if she is expecting a guest. Also, SamBot's family has a dog. When nobody is home, the dog is in the back yard. The same is true if the dog has bowel trouble. If the dog is in the back yard, SamBot will probablyhear her barking, but sometimes he can be confused by other dogs barking. SamBot is equipped with two sensors: a light-sensor for detecting outdoor lights and a sound-sensor for detecting the barking

  • f dogs. Both of these sensors are not completely reliable and can
  • break. Moreover, they both require SamBot's battery to be in good

condition.

slides3 COMPSCI 2020

Read on your own

slide-74
SLIDE 74

slides3 COMPSCI 2020

slide-75
SLIDE 75

slides3 COMPSCI 2020

slide-76
SLIDE 76

slides3 COMPSCI 2020

slide-77
SLIDE 77
slide-78
SLIDE 78

slides3 COMPSCI 2020

slide-79
SLIDE 79

slides3 COMPSCI 2020

slide-80
SLIDE 80

slides3 COMPSCI 2020

slide-81
SLIDE 81
slide-82
SLIDE 82

slides3 COMPSCI 2020

slide-83
SLIDE 83

If G1 and G2 are close then they are likely to pass down from the same haplotype (grandmother or grandfather)

slide-84
SLIDE 84

slides3 COMPSCI 2020

slide-85
SLIDE 85

slides3 COMPSCI 2020

slide-86
SLIDE 86

slides3 COMPSCI 2020

slide-87
SLIDE 87

slides3 COMPSCI 2020

slide-88
SLIDE 88

slides3 COMPSCI 2020

slide-89
SLIDE 89
slide-90
SLIDE 90

slides3 COMPSCI 2020

slide-91
SLIDE 91

slides3 COMPSCI 2020

slide-92
SLIDE 92

slides3 COMPSCI 2020

slide-93
SLIDE 93

slides3 COMPSCI 2020

slide-94
SLIDE 94

Two Loci Inheritance

Recombinant 2 1 A A B B a a b b A a B b 3 4 a a b b A a b b 5 6 A a B b

slides3 COMPSCI 2020

slide-95
SLIDE 95

151

Bayesian Network for Recombination

S23m L21f L21m L23m X21 S23f L22f L22m L23f X22 X23 S13m L11f L11m L13m X11 S13f L12f L12m L13f X12 X13 y3 y2 y1

{m,f} t s s P

t t

          where 1 1 ) , | (

13 23

    

Locus 1 Locus 2

P(e|Θ) ?

Deterministic relationships Probabilistic relationships

slides3 COMPSCI 2020

slide-96
SLIDE 96

L11m L11f X11 L12m L12f X12 L13m L13f X13 L14m L14f X14 L15m L15f X15 L16m L16f X16 S13m S15m S16m S15m S15m S15m L21m L21f X21 L22m L22f X22 L23m L23f X23 L24m L24f X24 L25m L25f X25 L26m L26f X26 S23m S25m S26m S25m S25m S25m L31m L31f X31 L32m L32f X32 L33m L33f X33 L34m L34f X34 L35m L35f X35 L36m L36f X36 S33m S35m S36m S35m S35m S35m

Linkage analysis: 6 people, 3 markers

slide-97
SLIDE 97

Outline

  • Bayesian networks and queries
  • Building Bayesian Networks
  • Special representations of CPTs
  • Causal Independence (e.g., Noisy OR)
  • Context Specific Independence
  • Determinism
  • Mixed Networks

slides3 COMPSCI 2020

slide-98
SLIDE 98

slides3 COMPSCI 2020

slide-99
SLIDE 99

A noisy-or circuit We wish to specify cpt with less parameters Think about headache and 10 different conditions that may cause it.

slides3 COMPSCI 2020

slide-100
SLIDE 100

slides3 COMPSCI 2020

Binary OR

A B X A B P(X=0|A,B) 0 0 1 P(X=1|A,B) 0 1 1 1 0 1 1 1 1

slide-101
SLIDE 101

slides3 COMPSCI 2020

slide-102
SLIDE 102

slides3 COMPSCI 2020

slide-103
SLIDE 103

slides3 COMPSCI 2020

slide-104
SLIDE 104

slides3 COMPSCI 2020

slide-105
SLIDE 105

slides3 COMPSCI 2020

slide-106
SLIDE 106

slides3 COMPSCI 2020

slide-107
SLIDE 107
slide-108
SLIDE 108

slides3 COMPSCI 2020

slide-109
SLIDE 109

slides3 COMPSCI 2020

slide-110
SLIDE 110

slides3 COMPSCI 2020

slide-111
SLIDE 111

slides3 COMPSCI 2020

slide-112
SLIDE 112

Noisy/OR CPDs

slides3 COMPSCI 2020

slide-113
SLIDE 113

slides3 COMPSCI 2020

slide-114
SLIDE 114

slides3 COMPSCI 2020

slide-115
SLIDE 115

slides3 COMPSCI 2020

slide-116
SLIDE 116

slides3 COMPSCI 2020

slide-117
SLIDE 117

Intelligence Difficulty Grade Letter SAT Job Apply

A student’s example

slides3 COMPSCI 2020

slide-118
SLIDE 118

slides3 COMPSCI 2020

A S L

(0.8,0.2) (0.9,0.1) (0.4,0.6) (0.1,0.9)

s1 a0 a1 s0 l1 l0

Tree CPD

If the student does not Apply, SAT and L are irrelevant Tree-CPD for job

slide-119
SLIDE 119

slides3 COMPSCI 2020

C L2

(0.1,0.9)

l21 c1 c2 l20

L1

(0.8,0.2) (0.3,0.7)

l11 l10

(0.9,0.1)

Letter1 Job Letter2 Choice

Captures irrelevant variables

slide-120
SLIDE 120

slides3 COMPSCI 2020

Multiplexer CPD

A CPD P(Y|A,Z1,Z2,…,Zk) is a multiplexer iff Val(A)=1,2,…k, and P(Y|A,Z1,…Zk)=Z_a

Letter1 Letter Letter2 Choice Job

slide-121
SLIDE 121

Mixture of trees

Meila and Jordan, 2000

slides3 COMPSCI 2020

slide-122
SLIDE 122

Mixture model with shared structure

Meila and Jordan, 2000

slides3 COMPSCI 2020

slide-123
SLIDE 123

Can we use hidden variables?

slides3 COMPSCI 2020

slide-124
SLIDE 124

Mixed Networks

(Dechter 2013) Augmenting Probabilistic networks with constraints because:

  • Some information in the world is deterministic and

undirected (X ≠ Y)

  • Some queries are complex or evidence are

complex (cnfs)

Queries are probabilistic queries

slides3 COMPSCI 2020

slide-125
SLIDE 125

Probabilistic Reasoning

Party example: the weather effect

Alex is-likely-to-go in bad weather Chris rarely-goes in bad weather Becky is indifferent but unpredictable Questions: Given bad weather, which group of individuals is most likely to show up at the party? What is the probability that Chris goes to the party but Becky does not?

P(W,A,C,B) = P(B|W) ꞏ P(C|W) ꞏ P(A|W) ꞏ P(W) P(A,C,B|W=bad) = 0.9 ꞏ 0.1 ꞏ 0.5

P(A|W=bad)=.9

W A

P(C|W=bad)=.1

W C

P(B|W=bad)=.5

W B W P(W) P(A|W) P(C|W) P(B|W) B C A

W A P(A|W) good .01 good 1 .99 bad .1 bad 1 .9

slides3 COMPSCI 2020

slide-126
SLIDE 126

Party Example Again

P(C|W) P(B|W) P(W) P(A|W) W B A C

Query: Is it likely that Chris goes to the party if Becky does not but the weather is bad?

Bayes Network Constraint Network

Semantics? Algorithms?

) , , | , ( A C B A bad w B C P   

  • A→B

C→A

B A C P(C|W) P(B|W) P(W) P(A|W) W B A C

A→B C→A

B A C

slides3 COMPSCI 2020