Plan for today 1st half NP-Complete Problems A sampling of some NP - - PDF document

1

NP-Complete Problems Plan for today

 1st half

 A sampling of some NP Complete Problems

 2nd half

 Final review  Course evaluations.

Let’s Review…

 The Turing Machine

 Build a theoretical a “human computer”  Likened to a human with a paper and pencil that can solve

problems in an algorithmic way

 The theoretical machine provides a means to determine:  If an algorithm or procedure exists for a given problem  What that algorithm or procedure looks like  How long would it take to run this algorithm or procedure.

The Church-Turing Thesis (1936)

 Any algorithmic procedure that can be

carried out by a human or group of humans can be carried out by some Turing Machine”

 Equating algorithm with running on a TM  Turing Machine is still a valid

computational model for most modern computers.

Are you a good witch?

 So what should be the cutoff between a “good”

algorithm and a “bad” algorithm?

 In the 1960s, Jack Edmonds proposed:  A “good” algorithm is one whose running time is a polynomial

function of the size of the input

 Other algorithms are “bad”  This definition was adopted:  A problem is called tractable if there exists a “good”

(polynomial time) algorithm that solves it.

 A problem is called intractable otherwise.

The class P

 The class P contains all decision problems that are

decidable by an “algorithm” that runs in polynomial time.

 Does this define a class of languages?  Yes…  The set of all problems whose encodings of “yes instances” (a

language) is recognized by a TM M

 M recognizes the above language in Polynomial Time.  Recall: These languages are recursive

2

Non-deterministic TM

 Let’s reconsider the NDTM

 Same as the ordinary TM except:  The transition function will return a set of triplets  (q, x, D)  For each state / symbol combination, 0 or more transitions can

be defined.

 The machine can “choose” which transition to take. Non-

deterministic TM



δ: Q x Γ → 2 Q x Γ x {R, L}

Non-deterministic TM The class NP

 The class NP contains all decision problems that are

decidable by a non-deterministic “algorithm” that runs in polynomial time.

 For each w accepted, there is at least one accepting

sequence that will run in polynomial time.

 Does this define a class of languages? -- Yes…  The set of all problems whose encodings of “yes instances” (a

language) is recognized by a NDTM M

 M recognizes the above language in Polynomial Time.

Reducing one language to another

 Worked well for decidability…Let’s use it here.  Basic idea

 Take an encoding of one problem  Convert it to another problem we know to be in P or NP  Conversion must be done in polynomial time!

Reducing one language to another

Conversion TM TM recognizing L2 Instance

• f P1

Corresponding Instance of P2 YES NO

Runs in Ptime Runs in Ptime

The class NP-complete

 The hardest of the problems in class NP are in the

class of NP-complete problems:

 A language L is in NP if

 L ∈ NP  For every other language L1 ∈ NP  L1 ≤p L

 All NP-complete problems are “equally” difficult.

3

The class NP-complete

 Implications

 Hardest problem  It’s probably not worth looking for a solution for

an NP-complete problem

 How do we show a problem to be NP-complete

 Reduction  We need a NP-complete problem to kick things off.

Satisfiability (SAT)

 INSTANCE

 A Logical expression containing

 variables xi  logical connectors &, |, and !

 PROBLEM

 Is there an assignment of truth values to

each of the variables such that the expression will evaluate to true.

Theory Hall of Fame

 Steven Cook

 A local boy  b. Buffalo, NY.  PhD – Harvard (1966)  At Berkley from 1966-1970  At University of Toronto since

1970

 Published this Theorem in 1971.

Cook’s Theorem

 SATISFIABILITY is NP-Complete

 Reduced any TM in NP to SAT  Shown by describing workings of a NDTM

as an expression involving boolean variables

 Now we have a problem to start the ball

rolling!

NP Complete Problems

 Why show that a problem is NP

complete.

 These problems are the hardest to solve.  Unlikely you will find an efficient algorithm

to solve them.

Lot’s of NP Complete Problems

 When confronted with trying to show a

problem is NP-Complete

 There are lots to chose from which to

perform a reduction.

Over 300 NP-Complete problems listed (and the book was published in 1979!) Wikipedia reports over 3000 known NP complete problems today

4

The big 7

 Basic core of known NP-complete

problems.

 Basis for beginner in chosing a problem for

reduction

 Note: I will not cover the actual reductions

 See Garey and Johnson for details.

The Big Daddy

 SATISFIABILITY (SAT)

 INSTANCE

 A set of U boolean variables and a collection C

• f boolean clauses over U

 PROBLEM

 Is there a satisfying truth assignment for C?

 The one that started it all!

Describing NP-Complete Problems

 NAME: Name of problem  INSTANCE or INPUT

 On what input is the problem defined.

 PROBLEM or OUTPUT

 Under what condition will the answer to the

problem be “yes”.

 REDUCED FROM:

 From what NP-Complete problem has this been

reduced

The other 6

 3-SATISFIABILITY (3SAT)

 INSTANCE:

 Collection C = {c1,c2, …, cm } of boolean

clauses on a finite set of boolean variables U such that each ci contains exactly 3 variables.

 PROBLEM:

 Is there a truth assignment for U that satisfies

all clauses of C

 REDUCED FROM: SAT

3SAT

Conversion TM Instance

• f SAT

Corresponding Instance of 3SAT

Runs in Ptime

Boolean expression of SAT is satisfiable iff corresponding boolean expression of 3SAT is satisfiable.

The other 6

 3-DIMENSIONAL MATCHING (3DM)

 Informally:

 A generalization of the “marriage” problem  Given:  n unmarried men  n unmarried women  a list of pairs of partners who would marry  Can you arrange n marriages that avoid

polygomy and makes everyone happy.

5

The other 6

 3-DIMENSIONAL MATCHING (3DM)

 Marriage problem with an extra dimension (add a

pet).

 Given:  n unmarried men  n unmarried women  n unowned pets  a list of triplets of partners who would marry and the pet

they want

 Can you arrange n marriages, with a pet as a wedding

gift, that avoid polygomy and makes everyone happy…including the pet!

The other 6

 3-DIMENSIONAL MATCHING (3DM)

 INSTANCE:

 A set M ⊆ (W x X x Y) where W, X, Y are

disjoint sets having the same number of elements

 PROBLEM

 Does M contain a matching  A subset M’ ⊆ M such that M’ has the same size as M

and no 2 elements in M’ agree in any coordinate.

 REDUCED FROM: 3SAT

The other 6

 The next 3 are all related to graph

theory

 Vertex Cover  Clique  Hamiltonian Circuit

The other 6: Graph Algorithms

 VERTEX COVER

 The vertex cover of a graph G is a set of

vertices V so that every edge of G is incident to at least one vertex in V.

Mathworld

The other 6: Graph Algorithms

 VERTEX COVER (VC)

 INSTANCE:

 A Graph G = (V,E) and integer k ≤ |V|

 PROBLEM:

 Is there a vertex cover of size k or less on G  Subset of vertices V’ such that for each edge {u,v},

at least one of u and v belongs to V’

 REDUCED FROM: 3SAT

The other 6: Graph Algorithms

 CLIQUE

 A clique of a graph G is any complete

subgraph (any subset of vertices in which each pair of graph vertices is connected by an edge)

Mathworld

6

The other 6: Graph Algorithms

 CLIQUE

 INSTANCE:

 A Graph G = (V,E) and integer k ≤ |V|

 PROBLEM:

 Does G contain a clique of size k or more.  A subset of vertices, V’ such that every 2 vertices in

V’ are joined by an edge

 REDUCED FROM: VC

The other 6: Graph Algorithms

 HAMILTONIAN CIRCUIT

 A Hamiltonian circuit is a cycle in an

undirected graph which visits each vertex exactly once and also returns to the starting vertex.

Mathworld

The other 6: Graph Algorithms

 HAMILTONIAN CIRCUIT (HC)

 INSTANCE:

 A Graph G = (V,E)

 PROBLEM:

 Does G contain a hamiltonian circuit  An ordering of all vertices <v1,v2, …, vn> such that

{vi, vj} is an edge and {vn, v1} is an edge.

 REDUCED FROM: VC

The other 6: one from set theory

 PARTITION

 given a multiset S of integers, is there a

way to partition S into two subsets S1 and S2 such that the sums of the numbers in each subset are equal?

 The subsets S1 and S2 must form a

partition in the sense that they are disjoint and they cover S.

The other 6: one from set theory

 PARTITION

 INSTANCE:

 A finite set A and a “size function” s(a) a ∈A

 PROBLEM:

 Is there a subset A’ ⊆ A such that

 REDUCED FROM: 3DM

• =

' '

) ( ) (

A a A A a

a s a s The big 7

SAT 3-SAT 3DM VC PARTITION HC CLIQUE

7

The big 7

 Listed in the original list of 21 NP

complete problems as published by Karp

 Landmark works:

 The Complexity of Theorem Proving

Procedures” by Steven Cook, 1971

 “Reducibility Among Combinatorial

Problems” by Richard Karp, 1972

Lots and Lots of NP Complete Problems

 Graph Theory  Network Design  Sets and Partitions,  Storage and Retrieval

 Database Related Problems, Data Storage, Sparse

Matrix Compression

 Sequence and Scheduling

 Multiprocessor Scheduling

 Mathematical Programming

 Linear Programming

Problem Categories

 Algebra and Number Theory  Games and Puzzles  Logic

 SAT, 3SAT

 Automata and Languages  Program Optimization

 Program Equivalences

 Other

Some Favorite NP-Complete Problems

 Traveling Salesman

 INSTANCE

 A finite set of “cities” C = { c1, c2, …, cn }  A distance function d: C2 →N  d( ci, cj ) = distance between ci and cj  Upper bounds B

 PROBLEM

 Is there a “tour” of all cities such that the distance

traveled will be less than B?

 REDUCED FROM: HAMILTONIAN CIRCUIT

Some NP-Complete Problems

 NDFA inequivalence

 INSTANCE

 Two nondeterministic finite automata A1 and A2

having the same input language Σ.

 PROBLEM

 Do A1 and A2 recognize different languages?

 NOTE

 Can be done in polynomial time if A1 and A2 are

deterministic.

Some NP-Complete Problems

 Crossword Puzzle Construction

 INSTANCE

 A finite set W ⊆ Σ* of words  An n by n matrix A consisting of 1’s and 0’s

 PROBLEM

 Can an n by n crossword puzzle be built up

including all words from W and blank squares consisting of the “0” cells in A?

8

Some NP-Complete Problems

 You want more?

 “Computers and Intractability” by Michael

• R. Garey and David S. Johnson.

 Questions?

Summary of Complexity Classes

 P

 set of decision problems solvable by

deterministic algorithms in polynomial time

 NP

 Set of decision problems solvable by non-

deterministic algorithms in polynomial time.

 Certainly P is a subset of NP

Summary of Complexity Classes

P NP

Summary of Complexity Classes

 NP-Hard

 Problems L such that any problem in NP reduces

to L

 NP-Complete

 Problems both in NP and NP-Hard  SAT

 Note that SAT is in P iff P = NP  No one has proven:  SAT is in P  SAT cannot be in P

NP-Complete and P

 The (sad) world of NP

P NP Is there something in here? After 40 years of research…nobody knows NP-complete

Quick road to Theory Fame

 Find a deterministic polynomial algorithm for

ANY problem in NP-complete…

 And you’ve found one for all  In fact, it will show that P = NP.

 They’ll be naming buildings after you before

you are dead!

9

Where to next?

 Language Design / Compiler

Construction

 4003-580 Language Processors  4003-711 Compiler Construction

 Complexity & Computation

 4003-481 Complexity and Computability  4003-515 Analysis of Algorithms

The Theory Hall of Fame

 Stephen Cole Kleene

 Regular expressions & FAs

 Noam Chomsky

 The grammar guy

 W. Ogden

 Pumping Lemma for CFLs

 Thompson, Schmidt, Lesk

 Bell Labs guys

 Kurt Godel

 Kicked off study of computability

The Theory Hall of Fame

 Alan Turing

 Turing Machines

 Alonzo Church

 Unlimited support for TMs

 Emil Post

 Post Correspondence Problem

 H.G. Rice

 Decision problems for TMs

 Stephen Cook

 NP-completeness

The Theory Hall of Fame

 YOU

 If you find a polynomial solution to any

problem in NP-complete!

So There You Have It!

 I bet you thought I forgot!

 What is a language?  What is a class of languages?

 Thanks for playing.

Break

 After break: Final exam review