Bayesian Networks Anders Ringgaard Kristensen Department of - - PowerPoint PPT Presentation

Department of Veterinary and Animal Sciences

Introduction to

Bayesian Networks

Anders Ringgaard Kristensen

Outline

Causal networks Bayesian Networks

• Evidence
• Conditional Independence and d-separation

Compilation

• The moral graph
• The triangulated graph
• The junction tree

Department of Veterinary and Animal Sciences

Slide 2

A quiz You have signed up for a quiz in a TV-show The rules are as follows:

• The host of the show will show you 3 doors
• Behind one of the doors a treasure is hidden
• You just have to choose the right door and the

treasure is yours.

• You have two choices:
• Initially you choose a door and tell the host which
• ne you have chosen.
• The host will open one of the other doors. He

always opens a door where the treasure is not hidden.

• You can now choose
• Either to keep your initial choice and the host

will open the door you first mentioned.

• Or you can change your choice and the host

will open the new door you have chosen.

Department of Veterinary and Animal Sciences

Slide 3

A quiz – let’s try!

1 2 3

Can we model the quiz? Identify the variables:

• True placement, ”True” ∈ {1, 2, 3}
• First choice, ”Choice 1” ∈ {1, 2, 3}
• Door opened, ”Opened” ∈ {1, 2, 3}
• Second choice, ”Choice 2” ∈ {Keep, Change}
• Reward, ”Gain” ∈ {0, 1000}

Department of Veterinary and Animal Sciences

Slide 5

Identify relations

Opened True Choice 1 Choice 2 Gain

Chosen initially at random Chosen initially at random Causal Causal Causal Decided by the player

Department of Veterinary and Animal Sciences

Slide 6

Notation

C

Random variable, Chance node

Parent 1 Child Parent 2

Edges into a chance node (yellow circle) correspond to a set of conditional

• probabilities. They express

the influence of the values

• f the parents on the value
• f the child.

Department of Veterinary and Animal Sciences

Slide 7

Baysian networks

Basically a static method A static version of data filtering Like dynamic linear models we may:

• Model observed phenomena by underlying unobservable

variables.

• Combine with our knowledge on animal production.

Like Markov decision processes, there is a structure and a set of parameters. All parameters are probabilities.

Department of Veterinary and Animal Sciences

Slide 8

The textbook

A general textbook on Bayesian networks and decision graphs. Written by professor Finn Verner Jensen from Ålborg University – one of the leading research centers for Bayesian networks. Many agricultural examples due to close collaboration with KVL and DJF through the Dina network, Danish Informatics Network in the Agricultural Sciences.

Department of Veterinary and Animal Sciences

Slide 9

Probabilities What is the probability that a farmer observes a particular cow in heat during a 3-week period?

• P(Heat = ”yes”) = a
• P(Heat = ”no”) = b
• a + b = 1 (no other options)
• The value of Heat (”yes” or ”no”) is observable.

What is the probability that the cow is pregnant?

• P(Pregnant = ”yes”) = c
• P(Pregnant = ”no”) = d
• c + d = 1 (no other options)
• The value of Pregnant (”yes” or ”no”) is not observable.

Department of Veterinary and Animal Sciences

Slide 10

Conditional probabilities Now, assume that the cow is pregnant. What is the conditional probability that the farmer observes it in heat?

• P(Heat = ”yes” | Pregnant = ”yes”) = ap+
• P(Heat = ”no” | Pregnant = ”yes”) = bp+
• Again, ap+ + bp+ = 1

Now, assume that the cow is not pregnant. Accordingly:

• P(Heat = ”yes” | Pregnant = ”no”) = ap-
• P(Heat = ”no” | Pregnant = ”no”) = bp-
• Again, ap- + bp- = 1

Each value of Pregnant defines a full probability distribution for Heat. Such a distribution is called conditional

Department of Veterinary and Animal Sciences

Slide 11

A small Bayesian net

Heat = ”yes” Heat = ”no” Pregnant = ”yes”

ap+ = 0.02 bp+ = 0.98

Pregnant = ”no”

ap- = 0.60 bp- = 0.40

Pregnant Heat

Pregnant = ”yes” Pregnant = ”no”

c = 0.5 d = 0.5

Let us build the net!

Department of Veterinary and Animal Sciences

Slide 12

Experience with the net: Evidence

By entering information on an observed value of Heat we can revise our belief in the value of the unobservable variable Pregnant. The observed value of a variable is called evidence. The revision of beliefs is done by use of Baye’s Theorem:

Department of Veterinary and Animal Sciences

Slide 13

Baye’s Theorem for our net

Department of Veterinary and Animal Sciences

Slide 14

Let us extend the example

A sow model:

• Insemination
• Several heat observations
• Pregnancy test

Consistent combination of information from different sources

Why build a Bayesian network Because you wish to estimate certainties for the values of variables that are not observable (or only observable at an unacceptable cost). Such variables are called “hypothesis variables”. The estimates are obtained by observing “information variables” that either

• Influence the value of the hypothesis variable (“risk factors”),
• r
• Depend on the hypothesis variable (“symptoms”)

Diagnostics/Trouble shooting

Department of Veterinary and Animal Sciences

Slide 16

Diagnostics/troubleshooting Risk 1 Risk 2 Risk 3 State Symp 1 Symp 2 Symp 3 Symp 4

Department of Veterinary and Animal Sciences

Slide 17

The sow pregnancy model Insem. Pregn. Heat 1 Heat 2 Heat 3 Test

Risk factor Hypothesis variable Symptoms

Department of Veterinary and Animal Sciences

Slide 18

Transmission of evidence

Age of a heifer/cow influences the probability that it has calved. Information on the “Calved” variable influences the probability that the animal is lactating. Thus, information on “Age” will influence our belief in the state of “Lact.” If, however, “Calved” is observed, there will be no influence of “Age” on “Lact.”! Evidence may be transmitted through a serial connection, unless the state of the intermediate variable is known. “Age” and “Lact” are d-separated given “Calved”. They are conditionally independent given observation of “Calved”

Age Calved Lact.

Num. Yes/No Yes/No

Diverging connections

The breed of a sow influences litter size as well as color. Observing the value of “Color” will tell us something about the “Breed” and, thus, indirectly about the “Litter size”. If, however, “Breed” is observed, there will be no influence of “Color” on “Litter size”! Evidence may be transmitted through a diverging connection, unless the state of the intermediate variable is known. “Litter size” and “Color” are d-separated given “Breed”. They are conditionally independent given observation of “Breed”

Breed Litter size Color

Num. White/Black/Brown… Landrace/Yorkshire/Duroc…

Converging connections

If nothing is known about “Temp.”, the values of “Mastitis” and “Heat” are independent. If, however, “Temp.” is observed at a high level, the supplementary information that the cow is in heat will decrease

• ur believe in the state “Yes” for “Mastitis”.

“Explaining away” effect. Evidence may only be transmitted through a converging connection if the connecting variable (or a descendant) is

• bserved.

Temp. Mastitis Heat

Num. Yes/No Yes/No

Example: Mastitis detection

Previous case Milk yield Mastitis index Mastitis Heat Conductivity Temperature

Department of Veterinary and Animal Sciences

Slide 22

Compilation of Bayesian networks

Compilation:

• Create a moral graph
• Add edges between all pairs of nodes having a

common child.

• Remove all directions
• Triangulate the moral graph
• Add edges until all cycles of more than 3

nodes have a chord

• Identify the cliques of the triangulated graph and
• rganize them into a junction tree.

The software system does it automatically (and can show all intermediate stages).

Department of Veterinary and Animal Sciences

Slide 23

Why use Bayesian networks? A consistent framework for

• Representation and dealing with uncertainty
• Combination of information from different

sources.

• Combination of numerical knowledge with

structural expert knowledge.

Department of Veterinary and Animal Sciences

Slide 24