Exhaustive Generation: Backtracking and Branch-and-bound Lucia - PowerPoint PPT Presentation

Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura Fall 2013 Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura

Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Knapsack Knapsack Problem Knapsack (Optimization) Problem Instance: Profits p 0 , p 1 , . . . , p n − 1 Weights w 0 , w 1 , . . . , w n − 1 Knapsack capacity M and n -tuple [ x 0 , x 1 , . . . , x n − 1 ] ∈ { 0 , 1 } n Find: such that P = � n − 1 i =0 p i x i is maximized, subject to � n − 1 i =0 w i x i ≤ M . Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura

Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Knapsack Example Objects: 1 2 3 4 weight (lb) 8 1 5 4 profit $500 $1,000 $ 300 $ 210 Knapsack capacity: M = 10 lb. Two feasible solutions and their profit: x 1 x 2 x 3 x 4 profit 1 1 0 0 $ 1,500 0 1 1 1 $ 1,510 This problem is NP-hard. Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura

Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Knapsack Naive Backtracking Algorithm for Knapsack Examine all 2 n tuples and keep the ones with maximum profit. Global Variables X, OptP, OptX . Algorithm Knapsack1 ( l ) if ( l = n ) then if � n − 1 i =0 w i x i ≤ M then CurP ← � n − 1 i =0 p i x i ; if ( CurP > OptP ) then OptP ← CurP ; OptX ← [ x 0 , x 1 , . . . , x n − 1 ] ; else x l ← 1 ; Knapsack1 ( l + 1) ; x l ← 0 ; Knapsack1 ( l + 1) ; First call: OptP ← − 1 ; Knapsack1 (0) . Running time: 2 n n -tuples are checked, and it takes Θ( n ) to check each solution. The total running time is Θ( n 2 n ) . Exhaustive Generation: Backtracking and Branch-and-bound Note: not all n -tuples are feasible but the algorithm will test all (the whole Lucia Moura

Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound A General Backtracking Algorithm A General Backtracking Algorithm Represent a solution as a list: X = [ x 0 , x 1 , x 2 , . . . ] . Each x i ∈ P i (possibility set) Given a partial solution: X = [ x 0 , x 1 , . . . , x l − 1 ] , we can use constraints of the problem to limit the choice of x l to C l ⊆ P l (choice set). By computing C l we prune the search tree, since for all y ∈ P l \ C l the subtree rooted on [ x 0 , x 1 , . . . , x l − 1 , y ] is not considered. Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura

Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound A General Backtracking Algorithm Part of the search tree for the previous Knapsack example: w i 8 1 5 4 M = 10 . p i $500 $1,000 $ 300 $ 210 [ ] C1={0,1} [1] [0] C2={0,1} ������� ������� ������� ������� ������� ������� [1,1] this part ������� ������� C3={0} [1,0] C3={0} not shown ������� ������� ������� ������� ������� ������� [1,1,0] C4={0} [1,0,1] [1,0,0] C4={0} ������� ������� ������� ������� ������� ������� ������� ������� [1,1,0,0] [1,0,0,0] ������� ������� profit=$1,500 profit= $500 : pruning Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura

Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Backtracking Algorithm with Pruning General Backtracking Algorithm with Pruning Global Variables X = [ x 0 , x 1 , . . . ] , C l , for l = 0 , 1 , . . . ) . Algorithm Backtrack ( l ) if ( X = [ x 0 , x 1 , . . . , x l − 1 ] is a feasible solution) then “Process it” Compute C l ; for each x ∈ C l do x l ← x ; Backtrack ( l + 1) ; Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura

Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Backtracking Algorithm with Pruning Backtracking with Pruning for Knapsack Global Variables X, OptP, OptX . Algorithm Knapsack2 ( l, CurW ) if ( l = n ) then if ( � n − 1 i =0 p i x i > OptP ) then OptP ← � n − 1 i =0 p i x i ; OptX ← [ x 0 , x 1 , . . . , x n − 1 ] ; if ( l = n ) then C l ← ∅ else if ( CurW + w l ≤ M ) then C l ← { 0 , 1 } ; else C l ← { 0 } ; for each x ∈ C l do x l ← x ; Knapsack2 ( l + 1 , CurW + w l x l ) ; First call: Knapsack2 (0 , 0) . Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura

Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Generating all cliques Backtracking: Generating all Cliques Problem : All Cliques Instance : a graph G = ( V, E ) . Find : all cliques of G without repetition 0 1 2 6 5 4 3 Cliques (and maximal cliques): ∅ , { 0 } , { 1 } , . . . , { 6 } , { 0 , 1 } , { 0 , 6 } , { 1 , 2 } , { 1 , 5 } , { 1 , 6 } , { 2 , 3 } , { 2 , 4 } , { 3 , 4 } , { 5 , 6 } , { 0 , 1 , 6 } , { 1 , 5 , 6 } , { 2 , 3 , 4 } . Definition Clique in G ( V, E ) : C ⊆ V such that for all x, y ∈ C , x � = y , { x, y } ∈ E . Maximal clique: a clique not properly contained into another clique. Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura

Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Generating all cliques Many combinatorial problems can be reduced to finding cliques (or the largest clique): Largest independent set in G (stable set): is the same as largest clique in G . Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura

Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Generating all cliques Many combinatorial problems can be reduced to finding cliques (or the largest clique): Largest independent set in G (stable set): is the same as largest clique in G . Exact cover of sets by subsets: find clique with special property. Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura

Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Generating all cliques Many combinatorial problems can be reduced to finding cliques (or the largest clique): Largest independent set in G (stable set): is the same as largest clique in G . Exact cover of sets by subsets: find clique with special property. Find a Steiner triple system of order v : find a largest clique in a special graph. Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura

Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Generating all cliques Many combinatorial problems can be reduced to finding cliques (or the largest clique): Largest independent set in G (stable set): is the same as largest clique in G . Exact cover of sets by subsets: find clique with special property. Find a Steiner triple system of order v : find a largest clique in a special graph. Find all intersecting set systems: find all cliques in a special graph. Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura

Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Generating all cliques Many combinatorial problems can be reduced to finding cliques (or the largest clique): Largest independent set in G (stable set): is the same as largest clique in G . Exact cover of sets by subsets: find clique with special property. Find a Steiner triple system of order v : find a largest clique in a special graph. Find all intersecting set systems: find all cliques in a special graph. Etc. Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura

Backtracking Intro Generating all cliques Estimating tree size Exact Cover Bounding Branch-and-Bound Generating all cliques In a Backtracking algorithm, X = [ x 0 , x 1 , . . . , x l − 1 ] is a partial solution ⇐ ⇒ { x 0 , x 1 , . . . , x l − 1 } is a clique. But we don’t want ot get the same k -clique k ! times: [0 , 1] extends to [0 , 1 , 6] [0 , 6] extends to [0 , 6 , 1] So we require partial solutions for be in sorted order: x 0 < x 1 < x 2 < . . . < x l − 1 . Let S l − 1 = { x 0 , x 1 , . . . , x l − 1 } for X = [ x 0 , x 1 , . . . , x l − 1 ] . The choice set of this point is: if l = 0 then C 0 = V if l > 0 then C l { v ∈ V \ S l − 1 : v > x l − 1 and { v, x } ∈ E for all x ∈ S l − 1 } = = { v ∈ C l − 1 \ { x l − 1 } : { v, x l − 1 } ∈ E and v > x l − 1 } Exhaustive Generation: Backtracking and Branch-and-bound Lucia Moura