12/7/2006 Massachusetts Institute of Technology Forward - - PDF document

12/7/2006 1

Massachusetts Institute of Technology

Generalized Conflict Learning for Hybrid Discrete/Linear Optimization

Hui Li and Brian Williams

Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology

• Oct. 5th, 2005

2

Forward Conflict-Directed Search

• Backward conflict-directed search uses conflicts to select

backtrack points and as a cache used to prune nodes.

– dependency-directed backtracking [Stallman-Sussman-77] – conflict-directed backjumping [Prosser-93] – dynamic backtracking [Ginsberg-93] – LPSAT [Wolfman-Weld-99].

• Forward conflict-directed search guides the forward step of search

away from regions of the state space that are ruled out by known conflicts

– Conflict-directed A* [Williams-Nayak-AAAI96, Williams-Ragno-JDAM]. – Assumption-based DDBT [deKleer-Williams AAAI86, IJCAI89], Factor Out Failure[Freuder-IJCAI-95 – Candidate Generation [deKleer-Williams-AIJ87, Reiter-AIJ87]

Introduce Generalized Forward Conflict-directed Search

• n Hybrid Discrete/Linear Optimization

– Experiments on cooperative vehicle plan execution problems demonstrates that the approach significantly outperforms branch and bound using conflicts on backtracking.

3

Outline

• Context
• Review of Conflict-directed A*
• The GCD-BB algorithm
• Empirical Evaluation
• Conclusion

4

90’s Self-Repairing Explorers Solve COPs Using Forward, Conflict-directed Best First Search

• Deep Space 1 Remote Agent

Experiment (May, 1999)

• Livingstone Model-based Execution

System [Williams & Nayak, AAAI96]

• Optimal Satisfiability Problem
• OpSat uses conflicts (nogoods) in

the forward direction, to substantially improve Best-first Search (CD A*)

[Williams-Nayak-AAAI96,Williams-Ragno-JDAM].

xij = vij

Commands Observations

Plant

State goals State estimates

Mode Estimation: Tracks likely States Mode Reconfiguration: Tracks least-cost state goals

Conflict 3 Increasing Cost Feasible Infeasible Conflict 1 Infeasible Conflict 2 Model: HMMs + CSPs

[¬]

• [Leaute & Williams, AAAI05]

To put out the Burbank wildfires, . . . UAV1 Starts at home; {Gets fuel & water; drops on fire-1} [1, 5]; {Gets fuel & water; drops on fire-2} [2, 6]; Returns home.

00’s Plan-driven Agile Systems Solve Hybrid Discrete/Linear Optimization Problems via Forward Conflict-directed Search

• Hybrid Discrete/Linear

Optimization Problems

– Disjunctive Linear Programs (DLPs) [Balas-ADM-79] – Binary Integer Programming – LCNF [Wolfman-IJCAI-99] – Mixed Logical Linear Programs (MLLPs) [Hooker-JDAM-99]

• How do we generalize forward

conflict-directed search to HDLOPs? Generalized Conflict-directed Branch and Bound (GCD-BB)

• [Hoffman & Williams, ICAPS05]
• Hybrid Discrete/Linear

Optimization Problems

– Disjunctive Linear Programs (DLPs) [Balas-ADM-79] – Binary Integer Programming – LCNF [Wolfman-IJCAI-99] – Mixed Logical Linear Programs (MLLPs) [Hooker-JDAM-99]

• How do we generalize forward

conflict-directed search to HDLOPs? Generalized Conflict-directed Branch and Bound (GCD-BB)

Gait Poses

l1 r1 l1 r2 r2 l1 l2 r2 r1

Fwd Lat

l1 r2 l2

Foot placement

00’s Plan-driven Agile Systems Solve Hybrid Discrete/Linear Optimization Problems via Forward Conflict-directed Search

12/7/2006 2

7

Disjunctive Linear Programs [Balas 79]

Definition:

i j

Example:

clause

8

Generalized Conflict-directed B&B

Extends traditional Branch & Bound:

• 1. Constructs conflicts from search tree nodes found

– Infeasible or – Sub-optimal

• 2. Uses conflicts to guide forward search away from

infeasible and sub-optimal states.

• 3. Performs induced unit clause relaxation

– Deduces (some) entailed unit clauses.

Demonstrates substantial performance improvement

– For both Best-first and depth-first Branch & Bound search, – In terms of speed and memory usage. Compared with – BIPs – BFS and B&B without conflicts – Backup on conflicts.

9

Outline

• Context
• Review of Conflict-directed A*
• The GCD-BB algorithm
• Empirical Evaluation
• Conclusion

10

Optimal Satisfiability Problems

• Diagnosis

.05 .95

And2

.06 .94

And1

.07 .93

Or3

.09 .91

Or2

.08 .92

Or1

U G fi

Or1 Or2 Or3

And1 And2

A B C D E F G X Y Z 1 1 1 1 1

xij = vij

1.0 1.0 A-G X-Z 1 fi

( ) ( )

i i i

f x f x =∏

[¬]

11

Conflict 3 Increasing Cost Feasible Infeasible Conflict 1

Conflict-directed A*

Infeasible Conflict 2

• Select optimal state outside conflicts at each step.

[Williams-Nayak-AAAI96, Williams-Ragno-JDAM0?, deKleer-Williams-IJCA89]

Increasing Cost Infeasible Conflict 3 Conflict 2 Conflict 1

Conflict-directed A*

• Select optimal state outside conflicts at each step.
• Feasible subregions described by kernel assignments.

Use conflicts to search for kernel assignment containing the best cost candidate.

Kernel 1 Kernel 2 Kernel 3 Feasible

12/7/2006 3

13

Generating candidate after two iterations…

Or1 Or2 Or3

And1 And2

A B C D E F G X Y Z 1 1 1 1 1

• Diagnosis

Conflicts: {O1=G, O2=G, A1=G} is inconsistent {O1=G, A1=G A2=G} is inconsistent Constituent Kernels: {O1=U, O2=U, A1=U} at least one holds {O1=U, A1=U A2=U} at least one holds

(Resolve one conflict - All states outside conflict) A2=U M3=U {A1=U, A2=U, M1=U, M3=U} {A1=U} {M1=U} {M2=U}

{A1=U} {M1=U} {M1=U, A2=U} {M2=U, M3=U}

• The kernel assignments are the minimal coverings of

the constituent kernels. [deKleer-Williams-AIJ87, Reiter-AIJ87]

• The best kernel is found through A* search of the

covering tree.

[Williams-Nayak-AAAI96, Williams-Ragno-JDAM]

{A1=U, M1=U , M2=U} Constituent Kernels

1st iteration 2nd iteration 3rd iteration

(Kernels resolve ALL conflicts - ALL states outside ALL conflicts)

15

Outline

• Context
• Review of Conflict-directed A*
• The GCD-BB algorithm
• 1. Branch and Bound for DLPs
• 2. Induced Unit Clause Relaxation
• 3. Generalized Conflict Learning
• 4. Forward Conflict-directed Search
• Empirical Evaluation
• Conclusion

16

• 1. Branch and Bound for DLPs

1. Branch on subproblems. 2. Maintain running best soln in incumbent. 3. Bound cost using relaxed problems. 4. Prune infeasible and suboptimal branches.

For BIPs

y≤20 x≤10 B1 B2 y≤0 x≥30 x≥80 C1 C2 C3 y≤30 x≤100 A1 A2

min -x-3y s.t. x≤200 y≤200 x≤100 min -x-3y s.t. x≤200 y≤200 y≤30 min -x-3y s.t. x≤200 y≤200 x≤100 x≤10 min -x-3y s.t. x≤200 y≤200 x≤100 y≤20

For DLPs

root minimize -x-3y s.t. x≤200 y≤200 y≤30 V x≤100 y≤20 V x≤10 x≥80 V x≥30 V y≤0 min -x-3y s.t. x≤200 y≤200

For DLPs

min -x-3y s.t. x≤200 y≤200 x≤100 x≤10 y≤0 min -x-3y s.t. x≤200 y≤200 x≤100 x≤10 x≥30 min -x-3y s.t. x≤200 y≤200 x≤100 x≤10 x≥80

Incumbent

• 160

an upper bound x≤10

min -x-3y s.t. x≤200 y≤200 y≤30 x≤10

Clauses Binary variables

17

• 2. Induced Unit Clause Relaxation
• Simple DLP relaxation:

Remove all non-unit clauses from the DLP Problem: ignoring all non-unit clauses forms a weak relaxation. Induced unit clause relaxation:

– Strengthen by adding entailed unit clauses – Approach: Relax DLP to a propositional theory + apply unit propagation …. (alternatively, failed literals)

• BIP relaxation:

Binary constraint x∈{0,1} → Continuous constraint 0≤x≤1 Problem: adding binary variables and constraints increases the dimensionality of the search problem.

18

• 2. Induced Unit Clause Relaxation

DP at a sub-problem before relaxation: max x +3y s.t. x ≤ 200 y ≤ 200 x ≤ 100 y ≤ 5 v x ≥ 100 x > 80 v x ≥ 30 v y ≤ 0

Example:

Relaxed Problem: max x +3y s.t. x ≤ 200 y ≤ 200 x ≤ 100 y ≤ 5

a1

max x +3y s.t. a b c d e v f v g Unit propagate Reintroduce the linear inequalities

a1

max x +3y s.t. a b c d Relax non- unit clauses max x +3y s.t. a b c d v ¬ c e v f v g

a1

Relax to propositional clauses

12/7/2006 4

19

• 3. Generalized Conflict Learning

Extracts conflicts whenever branch pruned:

• Infeasibility conflict
• Sub-optimality conflict

Set of constraints that are not satisfiable for any value of x. Set of constraints, all of whose feasible states are worse than the incumbent (-100 > f(x*)).

Incumbent: x*(200,0)

A minimal infeasibility conflict A minimal sub-optimality conflict

20

• 3. Minimal Conflict Extraction
• Extracted based on duality theory:
• Incurs 1 additional LP per conflict
• Minimal sub-optimality conflicts:

The active constraint set at the solution, assuming no

• degeneracy. The active constraint set is identified by the

non-zero terms of the optimal dual vector. [Bertsimas&Tsitsiklis97].

• Minimal infeasibility conflicts:

The extreme rays of the cone formed by the modified dual of the original LP [Gleeson&Ryan90].

21

• 4. Forward Conflict-directed Search

y≤20 x≤10 B1 B2 y≤0 x≥30 x≥80 C1 C2 C3 min -x-3y s.t. x≤200 y≤200 x≤100 min -x-3y s.t. x≤200 y≤200 y≤30

For DLPs

min -x-3y s.t. x≤200 y≤200 min -x-3y s.t. x≤200 y≤200 x≤100 x≤10 y≤0 min -x-3y s.t. x≤200 y≤200 x≤100 x≤10 x≥30 min -x-3y s.t. x≤200 y≤200 x≤100 x≤10 x≥80 Incumbent

• 160

root

y≤30 x≤100 A1 A2 {x≤10, x≥30} {x≤10, x≥80} Conflicts: {b2, c2} {b2, c1}

{ b2, c2 } { b2, c1 } Conflicts Constituent Kernels { {¬b2}, {¬c2} } { {¬b2}, {¬c1} } Kernels { ¬b2 } { ¬c1, ¬c2 }

minimize -x-3y s.t. (u1)x≤200 (u2)y≤200 (a1)y≤30 V (a2)x≤100 (b1)y≤20 V (b2)x≤10 (c1)x≥80 V (c2)x≥30 V (c3)y≤0 min -x-3y s.t. x≤200 y≤200 y≤30 y≤20 min -x-3y s.t. x≤200 y≤200 y≤30 y≤0 u1 u2 a1 ¬b2 b1 V b2 c1 V c2 V c3 u1 u2 a1 ¬c1 ¬c2 b1 V b2 c1 V c2 V c3

DLP Candidates

u1 u2 a1 b1 c1 V c2 V c3 u1 u2 a1 c3 b1 V b2

unit propagation If infeasible Then prune Else unit clause relax

22

Outline

• Context
• Review of Conflict-directed A*
• The GCD-BB algorithm
• Empirical Evaluation
• Conclusion

23

Empirical Evaluation

Test problems: Model-based temporal plan execution for cooperative vehicles [Léauté-Williams-AAAI05]. Comparisons: GCD-BB v.s. BIP-BB v.s. algorithmic variants of GCD-BB Measures: Computation time - number of LPs & average LP size Memory use - maximum queue length Model of Vehicle & Terrain

extract control sequence solve up to limited horizon encode as disjunctive LP

24

A Mars Plane Scouting Valle Marineris

12/7/2006 5

25

Generalized forward conflict-directed search

• n DLPs improves runtime over BIP-BB.

BI P- BB v.s. GCD- BB

1000 2000 3000 4000 5000 6000 7000 8000 9000 1 2 3 4 Clause / Variable Number of relaxed LPs solved Series1 Series2 Series3

DLP+DFS+Infeas+Sub BIP-BB DLP+BFS+Infeas

80/36 700/144 1492/300 2456/480 26

Conflicts significantly improve best-first search runtime.

W it hout conflict s v.s. w it h conflict s

500 1000 1500 2000 2500 3000 1 2 3 4 Clause / Variable Number of relaxed LPs s

• lved

80/36 700/144 1492/300 2456/480 DLP+BFS+Without Conflicts DLP+BFS+With Conflicts

27

Backtrack w ith conflicts v.s. Forw ard conflict- directed search

200 400 600 800 1000 1200 1400 1600 1 2 3 4 Clause / Variable Number of relaxed LPs solved

80/36 700/144 1492/300 2456/480 DLP+BFS+Backtrack DLP+BFS+Forward Search

Forward conflict-directed search significantly

• utperforms conflicts used on backtrack.

BFS v.s. DFS

50 100 150 200 250 300 350 400 450 1 2 3 4 Clause / Variable BFS+ Without Conflicts BFS+ Infeas DFS+ Infeas+ Sub 80/36 700/144 1492/300 2456/480

Conflicts significantly improve Best-first search memory use Memory performance similar to depth-first-search w conflicts

29

BFS v.s. DFS

100 200 300 400 500 600 700 800 1 2 3 4 Clause / Variable Number of relaxed L Ps solved BFS+Infeas DFS+Infeas DFS+Infeas+Sub DFS+Sub

80/36 700/144 1492/300 2456/480

But BFS is not an anytime algorithm. Use Depth-first B&B with conflicts for infeasibility and suboptimality.

• Runtime performance similar to conflict-directed BFS
• Significantly better than B&B with infeasible conflicts alone.

30

Future Work

• Study influence of cutting planes, such as Bender’s

cuts, on performance.

• Study empirically why sub-optimality conflicts do not

speed up search as much as infeasibility conflicts.

• Apply GCD-BB to non-clausal forms of HLLPs.
• Generalize to non-linear programs.

12/7/2006 6

Generalized-Conflict-directed Branch & Bound

• 1. Learns conflicts from search tree nodes found

– Infeasible or – Sub-optimal

• 2. Branches using conflicts to guide forward search away from

infeasible and sub-optimal states.

• 3. Bounds using unit clause relaxation

– Deduces (some) entailed unit clauses.

Demonstrates substantial performance improvement

– For both Best-first and depth-first Branch & Bound search, – In terms of speed and memory usage.

Compared with

– BIPs – Traditional BFS and B&B – Backup on conflicts.

32

References

• [Gleeson&Ryan90] J. Gleeson and J. Ryan, Identifying minimally inconsistent

subsystems of inequalities, ORSA J. Computing 2, 1990.

• [Bertsimas&Tsitsiklis97] D. Bertsimas and J. Tsitsiklis, Introduction to Linear
• Optimization. Athena Scientific, 1997.
• [Stallman77] Stallman, R. and Sussman, G.J., Forward Reasoning and

Dependency-Directed Backtracking in a System for Computer-Aided Circuit

• Analysis. J. of Artificial Intelligence. 9. 1977.
• [Prosser93] Prosser, P., Hybrid Algorithms for the Constraint Satisfaction
• Problem. J. of Computational Intelligence, 9(3),1993.
• [Ginsberg93] Ginsberg, M., Dynamic Backtracking. J. of Artificial Intelligence

Research, 1, 1993.

• [Wolfman99] Wolfman, S. and Weld, D., The LPSAT Engine & Its Application to

Resource Planning. IJCAI. 1999.

• [Williams05] Williams, B. and Ragno, R., Conflict-directed A* and its Role in

Model-based Embedded Systems. JDAM, to appear 2005.

• [Katsirelos03] Katsirelos, G. and Bacchus, F., Unrestricted Nogood Recording in

CSP Search. CP, 2003.

• [Léauté05] Léauté, T. and Williams, B., Coordinating Agile Systems Through The

Model-based Execution of Temporal Plans. AAAI. 2005.