1 Optimization in decision graphs The repeated milk test problem - - PDF document

1

Decision graphs II (Limited Memory) Influence Diagrams

Advanced Herd Management Anders Ringgaard Kristensen

Outline

Decision trees Optimization methods

• Decision tree
• Strong junction tree
• Single Policy Updating

Markov property versus “no forgetting” Decision node ordering Advantages and disadvantages of decision graphs

Decision trees

A very common technique for evaluation of alternative decisions over time. In particular popular in the veterinary community. Example diseased calf:

• Treat: Yes/ no
• Cost of treatment: 100 DKK
• Value of surviving calve: 1650 DKK
• Cost of dead calf: 70 DKK
• Survival of animals:
• Treated: 0.88
• Untreated: 0.60

Decision tree for diseased calf

Value of decision “Yes”:

• 0.88 × 1650 + 0.12 × (-70) – 100 = 1343.60

Value of decision “No”:

• 0.60 × 1650 + 0.40 × (-70) = 962

The optimal decision is obviously to treat.

Treat Die Die

Y: -100 N: 0 N: 0.88 Y: 0.12

• 70

1650

• 70

1650

Untreated, dead Untreated, survived Treated, dead Treated, survived

N: 0.60 Y: 0.40

The treatment problem as a decision graph

Refer to file “Treatment.xbn” The answers are (fortunately) the same as with the decision tree. Any decision graph can be modeled as a decision tree.

A tiny part of the cow feeding problem

Number of leaves: 3 × 4 × 5 × 5 = 300 (only 5 shown) That’s why we need decision graphs! Hidden assumptions (e.g. true value for silage).

Me HS Me Me Obs Obs

Obs

Obs Mix Mix Mix R R

R

R R Mix Mix

2

Optimization in decision graphs Unfolding to decision tree

• Only option until Shachter (1986)

Influence diagram with “no forgetting” (like the decision tree):

• Famous article by Jensen, Jensen & Dittmer (1994):
• Strict ordering of nodes
• Creation of a “strong” junction tree
• Implemented in the Hugin software system

LImited Memory Influence Diagram (LIMID):

• Described by Lauritzen & Nilsson (2001):
• Decision nodes converted to chance nodes.
• Implemented in the Esthauge LIMID software system

The repeated milk test problem The reason for testing the milk from a particular cow is to decide whether or not to pour the milk into the bulk tank:

• If the milk from an infected cow is poured into the bulk tank, the dairy will

reduce the total paym ent by 10% .

• If the milk from the cow is not poured into the bulk tank, the value of that

milk is lost.

• The farmer has 50 cows.
• Under the action “Pour”:
• The value of the milk (if not infected) is 1000
• The value of the milk with reduction is 900
• Under the action “Don’t pour”:
• The value of the milk is 1000 × 49/ 50 = 980
• It doesn’t matter whether or not the milk is infected

As a decision graph

The result of the test is known when the decision is made. Is that enough? Let’s try!

Relevant past For a decision made at time t’ the values of all variables observed at time t ≤ t’ are in principle relevant. Moreover, all decisions made at previous time steps t ≤ t’ may be relevant. This observation is referred to, as a “no forgetting” assumption. Requires a strict ordering of the nodes!

Influence diagrams Jensen, Jensen & Dittmer (1994) “No forgetting” assumption:

• The value of any previously observed variable is remembered.
• Any decision made earlier is remembered.
• Graphically, this means that we must insert numerous implicit

edges into the net.

• Implem ented in the Hugin software system.

The decision graph with no forgetting

There are 13 edges into Pour7!

3

A comparison …

Without implicit “no forgetting” edges. Implicit “no forgetting” edges visible.

Consequences of “no forgetting” The decision strategy found is an optimal one. The optimal strategy gets very complex:

• The optimal decision for Pour7 depends on the value 13 other

variables.

Optimization becomes very demanding from a computational point of view:

• Even rather simple decision problems cannot be solved in

practice.

• The applicational experiences with influence diagrams have been

disappointing.

• Application to delivery policies in slaughter pigs failed.

LImited Memory Influence Diagrams

Working title: “Demented Influence Diagrams”. Due to the disappointments with influence diagrams in herd management, a research initiative was initiated:

• Dennis Nilsson as post doc at Aalborg University (later assistant

professor at LIFE)

• Michael Höhle as PhD student at LIFE

The goal was to come up with better optimization methods for decision graphs by relaxing the “no forgetting” assumption.

LIMIDs – the ideas behind Only one decision:

• Try the alternatives one by
• ne and select the best.

Extend the idea to larger nets.

Single Policy Updating in LIMIDs

Determine an optimization ordering (usually just backwards) Convert all decisions to chance nodes. Update the policy of each decision node one by one. Repeat until convergence.

Single policy updating in LIMIDs Lauritzen & Nilsson (2001) Usually only near-optimal solutions. Never more complex than a Bayesian network. Not so computationally demanding as influence diagrams. The algorithm may be applied to influence diagrams if all implicit edges are added. Rather efficient even for influence diagrams. Implemented in the Esthauge LIMID Software System.

4

Soluble LIMIDs For some LIMIDs, the Single Policy Updating algorithm will provide us with an exact solution. Such LIMIDs are called soluble. All influence diagrams are soluble (i.e. if all implicit edges are added):

• Some edges m ay be irrelevant.
• The software system can automatically remove irrelevant

information edges and find the so-called minimal reduction.

• The software system can check whether the minimal reduction is

soluble.

• If it is soluble, a unique decision node ordering is automatically

identified, and only one iteration is necessary.

Check for solubility

Advantages of decision graphs

State space representation:

• Variable by variable (as opposed to dynamic

programming).

• Allow unobservable variables.
• No forgetting – at least as an option (as opposed to

dynamic programming).

Disadvantages of decision graphs

No forgetting:

• Complexity – hard to solve (even though heavily

improved with LIMIDs). Only suited for static decision problems:

• Time steps must be explicitly modeled (as opposed to

dynamic programming). Only suited for strictly symmetric decision problems (cf irregular decision trees)

Herd constraints Optimization Biological variation Uncertainty Functional limitations Dynamics

Decision graphs

Properties of methods for decision support