An Overview of Computational Complexity - - PowerPoint PPT Presentation

رتويپماك‌و‌قرب‌يسدنهم‌ي‌هدكشناداه‌نيشام‌و‌اه‌نابز‌ي‌هيرظن

يتابساحم‌يگديچيپ‌رب‌يرورم

An Overview of Computational Complexity

يدلبوف‌مظاك kazim@fouladi.ir رتويپماك‌و‌قرب‌يسدنهم‌ي‌هدكشناد نارهت‌هاگشناد

مهدراهچ‌لصف

2

Computational Complexity

3

Time Complexity: Space Complexity: The number of steps during a computation Space used during a computation

4

Time Complexity

• We use a multitape Turing machine
• We count the number of steps until

a string is accepted

• We use the O(k) notation

5

Example:

} : {   n b a L

n n

Algorithm to accept a string :

w

• Use a two-tape Turing machine
• Copy the on the second tape
• Compare the and

a a b

6

|) (| w O

• Copy the on the second tape
• Compare the and

a a b

Time needed:

|) (| w O

Total time:

|) (| w O } : {   n b a L

n n

7

For string of length time needed for acceptance:

} : {   n b a L

n n

n ) (n O

8

Language class:

) (n DTIME

A Deterministic Turing Machine accepts each string of length in time

) (n DTIME

1

L

2

L

3

L ) (n O n

9

) (n DTIME } : {  n b a

n n

} {ww

10

In a similar way we define the class

)) ( ( n T DTIME ) (n T

for any time function: Examples:

),... ( ), (

3 2

n DTIME n DTIME

11

Example: The membership problem for context free languages

} grammar by generated is : { G w w L  ) ( 3 n DTIME L

(CYK - algorithm)

Polynomial time

12

Theorem:

) ( k n DTIME ) (

1  k

n DTIME

1

( ) ( )

k k

DTIME n DTIME n

13

Polynomial time algorithms:

) ( k n DTIME

Represent tractable algorithms: For small we can compute the result fast

k

14

) ( k n DTIME P  

for all k The class P

• All tractable problems
• Polynomial time

15

P

CYK-algorithm

} {

n nb

a



} {ww

16

Exponential time algorithms:

) 2 ( n DTIME

Represent intractable algorithms: Some problem instances may take centuries to solve

17

Example: the Traveling Salesperson Problem Question: what is the shortest route that connects all cities?

5 3 2 1 8 10 2 4 3 6

18

Question: what is the shortest route that connects all cities?

5 3 2 1 8 10 2 4 3 6

19

Example: the Hamiltonian Problem Question: is there a Hamiltonian path from s to t? s t

20

s t YES!

21

) 2 ( ) ! (

n

DTIME n DTIME L  

Exponential time Intractable problem A solution: search exhaustively all paths L = {<G,s,t>: there is a Hamiltonian path in G from s to t}

22

Example: The Satisfiability Problem Boolean expressions in Conjunctive Normal Form:

k

t t t t     

3 2 1 p i

x x x x t      

3 2 1

Variables Question: is expression satisfiable?

23

) ( ) (

3 1 2 1

x x x x   

Satisfiable:

1 , 1 ,

3 2 1

   x x x 1 ) ( ) (

3 1 2 1

    x x x x

Example:

24

2 1 2 1

) ( x x x x   

Not satisfiable Example:

25

e} satisfiabl is expression : { w w L  ) 2 ( n DTIME L

For variables:

n

Algorithm: search exhaustively all the possible binary values of the variables exponential

26

Non-Determinism

Language class:

) (n NTIME

A Non-Deterministic Turing Machine accepts each string of length in time

) (n NTIME

1

L

2

L

3

L ) (n O n

27

Example:

} {ww L 

Non-Deterministic Algorithm to accept a string :

ww

• Use a two-tape Turing machine
• Guess the middle of the string

and copy on the second tape

• Compare the two tapes

w

28

|) (| w O

Time needed:

|) (| w O

Total time:

|) (| w O } {ww L 

• Use a two-tape Turing machine
• Guess the middle of the string

and copy on the second tape

• Compare the two tapes

w

29

) (n NTIME } {ww L 

30

In a similar way we define the class

)) ( ( n T NTIME ) (n T

for any time function: Examples:

),... ( ), (

3 2

n NTIME n NTIME

31

Non-Deterministic Polynomial time algorithms:

) ( k n NTIME L

32

) ( k n NTIME P  

for all k The class NP Non-Deterministic Polynomial time

NP

33

Example: The satisfiability problem Non-Deterministic algorithm:

• Guess an assignment of the variables

e} satisfiabl is expression : { w w L 

• Check if this is a satisfying assignment

34

Time for variables:

n ) (n O

e} satisfiabl is expression : { w w L 

Total time:

• Guess an assignment of the variables
• Check if this is a satisfying assignment

) (n O ) (n O

35

e} satisfiabl is expression : { w w L  NP L

The satisfiability problem is an - Problem

NP

36

Observation:

NP P 

Deterministic Polynomial Non-Deterministic Polynomial

37

Open Problem:

? NP P 

WE DO NOT KNOW THE ANSWER

38

Example: Does the Satisfiability problem have a polynomial time deterministic algorithm? WE DO NOT KNOW THE ANSWER Open Problem:

? NP P 

39

NP-Completeness

A problem is NP-complete if:

• It is in NP
• Every NP problem is reduced to it

(in polynomial time)

40

Polynomial Time Reductions

Polynomial Computable function : f For any computes in polynomial time

) (w f

w

41

Language A is polynomial time reducible to language B if there is a polynomial computable function such that:

f B w f A w    ) (

42

Suppose that is polynomial reducible to . If then . Theorem:

P B A B P A

Proof: Machine to accept in polynomial time:

A

On input :

w

• 1. Compute

) (w f

Let be the machine to accept B

M

• 2. Run on input

) (w f

M

43

3CNF formula:

) ( ) ( ) ( ) (

6 5 4 4 6 3 6 5 3 3 2 1

x x x x x x x x x x x x           

Each clause has three literals 3SAT ={ : is a satisfiable 3CNF formula}

w w

Language:

44

Clique: A 5-clique CLIQUE = { : contains a -clique}

  k G,

G

k

45

Theorem: 3SAT is polynomial time reducible to CLIQUE Proof: give a polynomial time reduction

• f one problem to the other

46

) ( ) ( ) (

3 2 1 2 2 1 2 1 1

x x x x x x x x x        

1

x

1

x

2

x

1

x

2

x

2

x

1

x

2

x

3

x

47

) ( ) ( ) (

3 2 1 2 2 1 2 1 1

x x x x x x x x x        

1

x

1

x

2

x

1

x

2

x

2

x

1

x

2

x

3

x 1 1

3 2 1

   x x x

48

Cook’s Theorem: The satisfiability problem is NP-complete Proof: Convert a Non-Deterministic Turing Machine to a Boolean expression in conjunctive normal form

49

Other NP-Complete Problems:

• The Traveling Salesperson Problem
• Vertex cover
• Hamiltonian Path

All the above are reduced to the satisfiability problem

50

Observation: If we can solve any NP-complete problem in Deterministic Polynomial Time (P time) then we know:

NP P 

51

Observation: If we prove that we cannot solve an NP-complete problem in Deterministic Polynomial Time (P time) then we know:

NP P 

52

Observations: It is unlikely that NP-complete problems are in P The NP-complete problems have exponential time algorithms Approximations of these problems are in P