Bayesian Networks Anders Ringgaard Kristensen Advanced Herd - - PowerPoint PPT Presentation

Advanced Herd Management 2006 1

Introduction to

Bayesian Networks

Anders Ringgaard Kristensen

KVL

Advanced Herd Management 2006 2 Anders Ringgaard Kristensen, I PH Anders Ringgaard Kristensen, I PH

Outline

• Causal networks
• Bayesian Networks

Evidence Conditional Independence and d-separation

• Compilation

The moral graph The triangulated graph The junction tree

Advanced Herd Management 2006 3 Anders Ringgaard Kristensen, I PH Anders Ringgaard Kristensen, I PH

A quiz

• You have signed up for a quiz in a TV-show
• The rules are as follows:

The host of the show will show you 3 doors Behind one of the doors a treasure is hidden You just have to choose the right door and the treasure is yours. You have two choices:

Initially you choose a door and tell the host which one you have chosen. The host will open one of the other doors. He always opens a door where the treasure is not hidden. You can now choose

• Either to keep your initial choice and the host will open the door

you first mentioned.

• Or you can change your choice and the host will open the new

door you have chosen.

Causal networks

Advanced Herd Management 2006 4 Anders Ringgaard Kristensen, I PH Anders Ringgaard Kristensen, I PH

A quiz – let’s try!

1 2 3

Causal networks

Advanced Herd Management 2006 5 Anders Ringgaard Kristensen, I PH Anders Ringgaard Kristensen, I PH

Can w e m odel the quiz?

• Identify the variables:

True placement, ”True” ∈ { 1, 2, 3} First choice, ”Choice 1” ∈ { 1, 2, 3} Door opened, ”Opened” ∈ { 1, 2, 3} Second choice, ”Choice 2” ∈ { Keep, Change} Reward, ”Gain” ∈ { 0, 1000}

Causal networks

Advanced Herd Management 2006 6 Anders Ringgaard Kristensen, I PH Anders Ringgaard Kristensen, I PH

I dentify relations

Opened True Choice 1 Choice 2 Gain

Chosen initially at random Chosen initially at random Causal Causal Causal Decided by the player

Causal networks

Advanced Herd Management 2006 7 Anders Ringgaard Kristensen, I PH Anders Ringgaard Kristensen, I PH

Notation

C

Random variable, Chance node

Parent 1 Child Parent 2

Edges into a chance node (yellow circle) correspond to a set of conditional

• probabilities. They express

the influence of the values

• f the parents on the value
• f the child.

Bayesian networks

Advanced Herd Management 2006 8 Anders Ringgaard Kristensen, I PH Anders Ringgaard Kristensen, I PH

Baysian netw orks

• Basically a static method
• A static version of data filtering
• Like dynamic linear models we may:
• Model observed phenomena by underlying unobservable

variables.

• Combine with our knowledge on animal production.
• Like Markov decision processes (Wednesday), there is a

structure and a set of parameters.

• All parameters are probabilities.

Bayesian networks

Advanced Herd Management 2006 9 Anders Ringgaard Kristensen, I PH Anders Ringgaard Kristensen, I PH

The textbook

• A general textbook on Bayesian networks and decision graphs.
• Written by professor Finn Verner Jensen from Ålborg

University – one of the leading research centers for Bayesian networks.

• Many agricultural examples due to close collaboration with

KVL and DJF through the Dina network, Danish Informatics Network in the Agricultural Sciences.

Bayesian networks

Advanced Herd Management 2006 10 Anders Ringgaard Kristensen, I PH Anders Ringgaard Kristensen, I PH

Probabilities

• What is the probability that a farmer observes a particular

cow in heat during a 3-week period?

• P(Heat = ”yes”) = a
• P(Heat = ”no”) = b
• a + b = 1 (no other options)
• The value of Heat (”yes” or ”no”) is observable.
• What is the probability that the cow is pregnant?
• P(Pregnant = ”yes”) = c
• P(Pregnant = ”no”) = d
• c + d = 1 (no other options)
• The value of Pregnant (”yes” or ”no”) is not observable.

Bayesian networks

Advanced Herd Management 2006 11 Anders Ringgaard Kristensen, I PH Anders Ringgaard Kristensen, I PH

Conditional probabilities

• Now, assume that the cow is pregnant. What is the

conditional probability that the farmer observes it in heat?

P(Heat = ”yes” | Pregnant = ”yes”) = ap+ P(Heat = ”no” | Pregnant = ”yes”) = bp+ Again, ap+ + bp+ = 1

• Now, assume that the cow is not pregnant. Accordingly:

P(Heat = ”yes” | Pregnant = ”no”) = ap- P(Heat = ”no” | Pregnant = ”no”) = bp- Again, ap- + bp- = 1

• Each value of Pregnant defines a full probability

distribution for Heat. Such a distribution is called conditional

Bayesian networks

Advanced Herd Management 2006 12 Anders Ringgaard Kristensen, I PH Anders Ringgaard Kristensen, I PH

A sm all Bayesian net

bp- = 0.40 ap- = 0.60

Pregnant = ”no”

bp+ = 0.98 ap+ = 0.02

Pregnant = ”yes” Heat = ”no” Heat = ”yes”

Pregnant Heat d = 0.5 c = 0.5

Pregnant = ”no” Pregnant = ”yes”

• Let us build the net!

Bayesian networks

Advanced Herd Management 2006 13 Anders Ringgaard Kristensen, I PH Anders Ringgaard Kristensen, I PH

Experience w ith the net: Evindence By entering information on an observed value

• f Heat we can revise our belief in the value of

the unobservable variable Pregnant. The observed value of a variable is called evidence. The revision of beliefs is done by use of Baye’s Theorem :

Bayesian networks: Evidence

Advanced Herd Management 2006 14 Anders Ringgaard Kristensen, I PH Anders Ringgaard Kristensen, I PH

Baye’s Theorem for our net

Bayesian networks: Evidence

Advanced Herd Management 2006 15 Anders Ringgaard Kristensen, I PH Anders Ringgaard Kristensen, I PH

Let us extend the exam ple

• A sow model:
• Insemination
• Several heat observations
• Pregnancy test
• Consistent combination of information from different sources

Bayesian networks: Evidence

Advanced Herd Management 2006 16 Anders Ringgaard Kristensen, I PH Anders Ringgaard Kristensen, I PH

W hy build a Bayesian netw ork

• Because you wish to estimate certainties for the values of

variables that are not observable (or only observable at an unacceptable cost). Such variables are called “hypothesis variables”.

• The estimates are obtained by observing “information variables”

that either

• Influence the value of the hypothesis variable (“risk factors”), or
• Depend on the hypothesis variable (“symptoms”)
• Diagnostics/ Trouble shooting

Bayesian networks

Advanced Herd Management 2006 17 Anders Ringgaard Kristensen, I PH Anders Ringgaard Kristensen, I PH

Diagnostics/ troubleshooting

Risk 1 Risk 2 Risk 3 State Symp 1 Symp 2 Symp 3 Symp 4

Bayesian networks

Advanced Herd Management 2006 18 Anders Ringgaard Kristensen, I PH Anders Ringgaard Kristensen, I PH

The sow pregnancy m odel

Insem. Pregn. Heat 1 Heat 2 Heat 3 Test

Risk factor Hypothesis variable Symptoms

Bayesian networks

Advanced Herd Management 2006 19 Anders Ringgaard Kristensen, I PH Anders Ringgaard Kristensen, I PH

Transm ission of evidence

• Age of a heifer/ cow influences the probability that it has calved.
• Information on the “Calved” variable influences the probability that

the animal is lactating.

• Thus, information on “Age” will influence our belief in the state of

“Lact.”

• If, however, “Calved” is observed, there will be no influence of “Age”
• n “Lact.”!
• Evidence may be transmitted through a serial connection, unless the

state of the intermediate variable is known.

• “Age” and “Lact” are d-separated given “Calved”.
• They are conditionally independent given observation of “Calved”

Age Calved Lact.

Num. Yes/No Yes/No

Conditional independence and d-separation

Advanced Herd Management 2006 20 Anders Ringgaard Kristensen, I PH Anders Ringgaard Kristensen, I PH

Diverging connections

• The breed of a sow influences litter size as well as color.
• Observing the value of “Color” will tell us something about the

“Breed” and, thus, indirectly about the “Litter size”.

• If, however, “Breed” is observed, there will be no influence of “Color”
• n “Litter size”!
• Evidence may be transmitted through a diverging connection, unless

the state of the intermediate variable is known.

• “Litter size” and “Color” are d-separated given “Breed”.
• They are conditionally independent given observation of “Breed”

Breed Litter size Color

Num. White/Black/Brown… Landrace/Yorkshire/Duroc…

Conditional independence and d-separation

Advanced Herd Management 2006 21 Anders Ringgaard Kristensen, I PH Anders Ringgaard Kristensen, I PH

Converging connections

• If nothing is known about “Temp.”, the values of “Mastitis” and “Heat” are

independent.

• If, however, “Temp.” is observed at a high level, the supplementary information

that the cow is in heat will decrease our believe in the state “Yes” for “Mastitis”.

• “Explaining away” effect.
• Evidence may only be transmitted through a converging connection if the

connecting variable (or a descendant) is observed.

Temp. Mastitis Heat

Num. Yes/No Yes/No

Conditional independence and d-separation

Advanced Herd Management 2006 22 Anders Ringgaard Kristensen, I PH Anders Ringgaard Kristensen, I PH

Exam ple: Mastitis detection

Previous case Milk yield Mastitis index Mastitis Heat Conductivity Temperature

Conditional independence and d-separation

Advanced Herd Management 2006 23 Anders Ringgaard Kristensen, I PH Anders Ringgaard Kristensen, I PH

Com pilation of Bayesian netw orks

• Compilation:
• Create a moral graph
• Add edges between all pairs of nodes having a common child.
• Remove all directions
• Triangulate the moral graph
• Add edges until all cycles of more than 3 nodes have a chord
• Identify the cliques of the triangulated graph and organize them

into a junction tree.

• The software system does it automatically (and can show all

intermediate stages).

Compilation

Advanced Herd Management 2006 24 Anders Ringgaard Kristensen, I PH Anders Ringgaard Kristensen, I PH

W hy use Bayesian netw orks?

• A consistent framework for

Representation and dealing with uncertainty Combination of information from different sources. Combination of numerical knowledge with structural expert knowledge.