MAX-2-SAT: How Good is Tabu Search in the Worst-Case? Monaldo - - PowerPoint PPT Presentation

MAX-2-SAT:

How Good is Tabu Search in the Worst-Case?

Monaldo Mastrolilli and Luca Maria Gambardella IDSIA, Lugano (Switzerland)

AAAI-04 2

Main result

(Max-2-Sat) we give the first theoretical evidence of the advantage of a tabu search strategy over the basic local search alone that critically depends on the tabu list length T.

AAAI-04 3

Max-2-Sat

Input: n binary variables: A,B,C,… m clauses Goal: find assignment to variables that satisfies the max number of clauses : A=1 ; B=0 ; C=0 A + C A + B ~C + ~B ~B + ~A

AAAI-04 4

Local Search: 1–flip Neighborhood

A=1 ; B=1 ; C=1

• A + C

A + B ~C + ~B ~B + ~A A=1 ; B=0 ; C=1

• A + C

A + B ~C + ~B ~B + ~A A basic local search LS starts with any given assignment, and then repeatedly changes (“flips”) the assignment of a variable that leads to the largest decrease in the total number of unsatisfied clauses.

AAAI-04 5

Local Search: worst-case

A local optimal solution is defined as a state whose local neighborhood does not include a state that is strictly better. Let Optloc denote the number of satisfied clauses at a local optimum of an instance of Max-2-Sat. A=0 ; B=1 ; C=1

• A + B

A + C ~B + ~C

Opt 3 2 Opt loc ≥

1990] Jaumard [Hansen,

AAAI-04 6

IDEA IDEA: keep track of the T most recent flipped variables (tabu variables unless improvement)

Tabu Search = LS + memory

TS(T)

• tabu search with tabu tenure at most T
• 1-flip
• choose the best non tabu + asp. criterion
• random choice for equivalent candidates
• if all moves tabu, choose the “less” tabu

1986] [Glover

AAAI-04 7

Literature

TSAT TSAT Mazure, Sais and Gregoire: Tabu Search for Sat, AAAI-97

• TSAT is extremely good and simple!
• In their empirical study they found that the length of the tabu list is

crucial!!

• They show (experimentally) that the optimal length of the tabu list is

linear with respect to the number n of variables.

AAAI-04 8

Aim of this work

The practical superiority of tabu search over local search alone has been already shown experimentally several times. A natural question addressed here is to understand if this superiority holds also from the worst-case point of view. Moreover, it is well known that a critical parameter of tabu techniques is the tabu list length. ISSUE ISSUE: is TS(T) better than LS even in the worst case?

AAAI-04 9

Let c=n/T be the ratio between the number of variables n and the maximum tabu tenure T. We study the approximation ratio R(c) of TS(T) as a function of c.

R(c) = BestF ound(TS(T))

Opt

.

Approximation Ratio

AAAI-04 10

100 80 60 40 20 0.775 0.75 0.725 0.7 0.675 c R c R

Approximation Ratio Upper Bound

Theorem 3 2

Starting from any arbitrary initial solution, the approximation ratio of TS(T) for MAX-2-SAT is bounded from above by R(c) = 2c2 3c2 − 2c + 2. (1)

c=n/T

AAAI-04 11

Corollary: For MAX-2SAT, the asymptotic ratio of TS(T) is 2/3 for any T = o(n) (=sub linear). Opt 3 2 ≥

loc

Opt : LS

• f

ratio e performanc the Recall Bad News:

Approximation Ratio Upper Bound

AAAI-04 12

Approximation Ratio Upper Bound

100 80 60 40 20 0.775 0.75 0.725 0.7 0.675 c R c R

Is there a value of T = Θ(n) such that TS(T) exhibits a worst-case superiority

• ver the basic local search alone? The Figure suggests that interesting values

can be found when the ratio c = n/T is “small”.

AAAI-04 13

Tabu Search vs Local Search

THEOREM: For MAX-2SAT, TS(n) achieves a performance ratio of 3/4 in O(nm) steps. Good News: strong separation !! Opt 3 2 ≥

loc

Opt : LS

• f

ratio e performanc the Recall

AAAI-04 14

Weighted Max-2-Sat

We note that the same analysis and approximation ratio can be obtained for the weighted case of Max-2-Sat, although the time complexity might be

• nly pseudopolynomially bounded in the input size

(simply take multiple copies of the clauses).

AAAI-04 15

Future Work

In this work we analyzed the worst-case behavior

• f tabu search as a function of the tabu list

length for Max-2-Sat. It would be interesting to understand if a similar analysis can be provided for the general MAX-SAT problem. An interesting starting case is Max-3-Sat.

AAAI-04 16

THEOREM: For MAX-2SAT, TS(n) achieves a performance ratio of 3/4 in O(nm) steps.

Theoretical Lens: proof

• S(j): solution at step j
• Ch(j): subset of clauses with exactly h literals satisfied

by S(j) (e.g. C0(j) is the set of unsat. clauses) Proof sketch Proof sketch NOTE: the presence of the aspiration criterion ensures that the best found solution by TS(n) is also a local optimum for LS

AAAI-04 17

Assume that at step k we reach a local optimum. Two possibilities: (a) (a) S(k) is a 3/4 approximate solution ==> done!! (b) (b) otherwise. If case (b) (b) we will prove that in at most n steps a better solution is found (= the number of satisfied clauses is increased by at least 1).

Theoretical Lens: proof (ctd)

NOTE: Opt ≤ m, hence in at most O(mn) steps we are in case (a) (a) !!

AAAI-04 18

Theoretical Lens: proof (ctd)

Assumption Assumption: At step k we reach a local optimum S(k) and we are in case (b). (For simplicity, assume tabu list empty at step k.) Goal Goal: prove that in at most n steps a better solution than S(k) is found. OBSERVE: there is h h ≤ n such that at step k k flip one not tabu variable that appears in C0(k k) k+1 k+1 flip one not tabu variable that appears in C0(k+1 k+1) … k+h k+h-

• 1

1 flip one not tabu variable that appears in C0(k+h k+h-

• 1

1) k+h k+h all variables in C0(k+h) have been flipped during the last h ≤ n (i.e. all tabu)

AAAI-04 19

Theoretical Lens: proof (ctd)

… at step k+h we reach a solution such that all the variables in the not satisfied clauses, i.e. C0(k+h), have been flipped during the last h steps. all unsatisfied clauses at step k are sat. at step k+h all clauses in C1(k) are still sat. at step k+h

C k C k h) ( ) ( ∩ + = ∅ C k C k h)

1

( ) ( ∩ + = ∅

AAAI-04 20

Theoretical Lens: proof (ctd)

C k C k h) ( ) ( ∩ + = ∅ C k C k h)

1

( ) ( ∩ + = ∅

| ( | ( )| | ( )| C k h)| m C k C k

1

+ ≤ − −

Two possibilities: (1) (1) |C0(k)| > |C0(k+h)| ==> done!! (2) (2) |C0(k)| ≤ |C0(k+h)|.

AAAI-04 21

Theoretical Lens: proof (ctd)

| ( | ( )| | ( )| C k h)| m C k C k

1

+ ≤ − −

(2) (2) |C0(k)| ≤ |C0(k+h)|

m C k h)| C k C k C k C k C k C k m O pt ≥ + + + ≥ + ≥ ⇒ ≤ ≤ | ( | ( )| | ( )| | ( )| | ( )| | ( )| | ( )|

1 1

2 4 1 4 1 4

Known: 2|C0(k)| ≤ |C1(k)| since S(k) loc. opt. ..a contradiction !!

AAAI-04 22

Tabu Search Origin

It was first suggested by

Glover, F. (1986) “Future paths for integer programming and links to artificial intelligence”, Computers & Operations Research, Vol. 13, pp. 533-549.

The basic ideas of TS have also been sketched by:

Hansen, P. “The steepest ascent mildest descent heuristic for combinatorial programming”, Congress on Numerical Methods in Combinatorial Optimization, Capri, Italy, 1986.