1 Optimization in decision graphs Unfolding to decision tree - - PDF document

1

Slide 1

Decision graphs II Influence Diagrams

Advanced Herd Management Anders Ringgaard Kristensen

Slide 2

Outline

Optimization methods

• Decision tree
• Strong junction tree
• Single Policy Updating

Decision node ordering Advantages and disadvantages of decision graphs

2

Slide 3

Optimization in decision graphs

Unfolding to decision tree

• Only option until Shachter (1986)

Influence diagram with “no forgetting” (like the decision tree):

• Famous article by Jensen, Jensen & Dittmer (1994):
• Strict ordering of nodes
• Creation of a “strong” junction tree
• Implemented in the Hugin software system

LImited Memory Influence Diagram (LIMID):

• Described by Lauritzen & Nilsson (2001):
• Decision nodes converted to chance nodes.
• Implemented in the Esthauge LIMID software system

Slide 4

The repeated milk test problem The reason for testing the milk from a particular cow is to decide whether or not to pour the milk into the bulk tank:

• If the milk from an infected cow is poured into the bulk

tank, the dairy will reduce the total payment by 10%.

• If the milk from the cow is not poured into the bulk tank,

the value of that milk is lost.

• The farmer has 50 cows.
• Under the action “Pour”:
• The value of the milk (if not infected) is 1000
• The value of the milk with reduction is 900
• Under the action “Don’t pour”:
• The value of the milk is 1000 × 49/50 = 980
• It doesn’t matter whether or not the milk is infected

3

Slide 5

As a decision graph

The result of the test is known when the decision is made. Is that enough? Let’s try!

Slide 6

Relevant past For a decision made at time t’ the values of all variables observed at time t ≤ t’ are in principle relevant. Moreover, all decisions made at previous time steps t ≤ t’ may be relevant. This observation is referred to, as a “no forgetting” assumption. Requires a strict ordering of the nodes!

4

Slide 7

Influence diagrams Jensen, Jensen & Dittmer (1994) “No forgetting” assumption:

• The value of any previously observed variable is

remembered.

• Any decision made earlier is remembered.
• Graphically, this means that we must insert

numerous implicit edges into the net.

• Implemented in the Hugin software system.

Slide 8

The decision graph with no forgetting

There are 13 edges into Pour7!

5

Slide 9

A comparison …

Without implicit “no forgetting” edges. Implicit “no forgetting” edges visible.

Slide 10

Consequences of “no forgetting” The decision strategy found is an optimal one. The optimal strategy gets very complex:

• The optimal decision for Pour7 depends on the

value 13 other variables.

Optimization becomes very demanding from a computational point of view:

• Even rather simple decision problems cannot be

solved in practice.

• The applicational experiences with influence

diagrams have been disappointing.

• Application to delivery policies in slaughter pigs

failed.

6

Slide 11

LImited Memory Influence Diagrams Working title: “Demented Influence Diagrams”. Due to the disappointments with influence diagrams in herd management, a research initiative was initiated:

• Dennis Nilsson as post doc at Aalborg University (later assistant

professor at IHH)

• Michael Höhle as PhD student at IHH

The goal was to come up with better optimization methods for decision graphs by relaxing the “no forgetting” assumption.

Slide 12

LIMIDs – the ideas behind Only one decision:

• Try the alternatives
• ne by one and

select the best.

Extend the idea to larger nets.

Opened True Choice 1 Choice 2 Gain

7

Slide 13

Single Policy Updating in LIMIDs Determine an optimization ordering (usually just backwards) Convert all decisions to chance nodes. Update the policy of each decision node one by one. Repeat until convergence.

Pour7 Pour6 Pour5 Pour4 Pour3 Pour2 Pour1 Slide 14

Single policy updating in LIMIDs Lauritzen & Nilsson (2001) Usually only near-optimal solutions. Never more complex than a Bayesian network. Not so computationally demanding as influence diagrams. The algorithm may be applied to influence diagrams if all implicit edges are added. Rather efficient even for influence diagrams. Implemented in the Esthauge LIMID Software System.

8

Slide 15

Soluble LIMIDs For some LIMIDs, the Single Policy Updating algorithm will provide us with an exact solution. Such LIMIDs are called soluble. All influence diagrams are soluble (i.e. if all implicit edges are added):

• Some edges may be irrelevant.
• The software system can automatically remove

irrelevant information edges and find the so-called minimal reduction.

• The software system can check whether the

minimal reduction is soluble.

• If it is soluble, a unique decision node ordering is

automatically identified, and only one iteration is necessary.

Slide 16

Check for solubility

9

Slide 17

Advantages of decision graphs

State space representation:

• Variable by variable (as opposed to dynamic

programming).

• Allow unobservable variables.
• No forgetting – at least as an option (as opposed to

dynamic programming).

Slide 18

Disadvantages of decision graphs

No forgetting:

• Complexity – hard to solve (even though heavily

improved with LIMIDs).

Only suited for static decision problems:

• Time steps must be explicitly modeled (as opposed to

dynamic programming).

Only suited for strictly symmetric decision problems (cf irregular decision trees)

10

Slide 19

Herd constraints Optimization Biological variation Uncertainty Functional limitations Dynamics

Decision graphs

Properties of methods for decision support