CSE 105 THEORY OF COMPUTATION Fall 2016 - - PowerPoint PPT Presentation

CSE 105

THEORY OF COMPUTATION

Fall 2016 http://cseweb.ucsd.edu/classes/fa16/cse105-abc/

This Week

• Beyond Theory of Computation:

–

Computational Complexity

–

Reading: Sipser Chapter 7

–

Not covered in final exam

• Revisit familiar topics

–

Diagonalization

–

Reduction

• NP-complenetess

What we studied so far

RE co-RE Context-free Regular Decidable

Theory of Computation

• Question: What computational problems admit an

algorithmic solution?

–

Decidable problems: Yes, they can be solved by a TM

–

Undecidable problems: No, no algorithmic solution

• Classifjcation of computational problems based on

restricted models of computation:

–

DFA, NFA: Regular languages

–

PDA, CFG: Context Free Languages

–

Decidable: TM

Computational Complexity

• Goal: better understanding of decidable problems
• Computational Model: T

uring Machine (or equivalent)

• Questions:

–

How much time is needed to solve a problem?

–

How much memory is needed to solve a problem?

• Are there problems that intrinsically require a lot
• f time or memory?

Measuring Time and Space

• Assume a TM M is run on input w
• Computation: Cinit → C1 → C2 → … → Chalt
• Time(M(w)):

– Number of steps in the computation

• Space(M(w)):

– Number of tape cells used during the computation.

Equivalently, size of largest Ck

Running time as a function

• Fix TM/program M
• Running time Time(M(w)) depends on input w

–

It is natural to expect computation to take longer as w gets larger

• We are not insterested in exact running time

–

We don’t care if the running time (number of steps) is even or odd

–

It is usually enough to know an upper bound on Time(M(w))

• Time(n) = max { Time(M(w)) | |w|=n}

–

Maximum running time over all inputs of size n

Asymptotic notation

• Time(n) = O(f(n))

–

There are constant a,b (independent of n) such that T < a*f(n) + b

• Signifjcance of a and b:

–

You can think of b as some fixed startup cost

–

You can think of a as using a slower or faster computer

• Theoretical goal

–

Get bounds on running time that are independent of details of computational model

–

Ignore low level details, for the sake of generality

Some example problems

• Addition: on input x and y, output x+y
• Multiplication: on input x and y, output x*y
• How can we model them as decision problems?

–

Addition: on input (x,y,i), determine if the ith (binary) digit of x+y is 1

–

Multiplication: on input (x,y,i), determine if the ith (binary) digit of x*y is 1.

• How fast can you add up or multiply numbers?

Some example problems

• Addition: on input x and y, output x+y
• Multiplication: on input x and y, output x*y
• How can we model them as decision problems?

–

Addition: on input (x,y,i), determine if the ith (binary) digit of x+y is 1

–

Multiplication: on input (x,y,i), determine if the ith (binary) digit of x*y is 1.

• How fast can you add up or multiply numbers?

What is the time complexity of Addition? A) T=O(n): You need to add all digits one by one B) T=O(3n): You need to readn 2n input digits and output an n-digit result C) T=1: Addition is a single machine instruction. D) It depends on how clock-speed, etc.

Some example problems

• Addition: on input x and y, output x+y
• Multiplication: on input x and y, output x*y
• How can we model them as decision problems?

–

Addition: on input (x,y,i), determine if the ith (binary) digit of x+y is 1

–

Multiplication: on input (x,y,i), determine if the ith (binary) digit of x*y is 1.

• How fast can you add up or multiply numbers?

What is the time complexity of Multiplication? A) T=O(n) because output is 2n digits long B) T=O(n2) as taught in elemntary school C) Somewhere between O(n) and O(n2), exact answer is not known D) It depends on how clock-speed, etc.

Complexity of Multiplication

• Schoolbook algorithm:

–

O(n2)

• 1960: Karatsuba algorithm

–

O(n1.585)

• 1971: Schonhage-Strassen algorithm

–

O(n * log(n) * log(log(n)))

• 2007: Furer algorithm

–

O(n * log(n) * exp(log*(n))) What is the complexity of multiplication? A) O(n / log n) B) O(n) C) O(n * log(n)) D) O(n * log(n) * exp(log*(n)))

How hard can it be?

• Even simple problems may require more than O(n)

computation

• Parsing: decide L(G) for a CFG G

–

Can be solved in Time = O(n3) for any G

–

Can it be solved in O(n2) for any G?

–

Can it be solved in O(n) for any G?

• Are there problems that require T=O(n100)?
• Are there problems that require T=O(exp(n))?

Computationally hard problems

• Is there a decidable problem that

–

Cannot be solved in time O(n100) ?

–

Cannot be solved in time O(exp(n)) ?

–

Cannot be solved in time O(exp(exp(exp( … exp(n)…))) ?

• How can we come up with such a problem?
• How do we show it cannot be solved in time T(n)?

An undecidable language

• We want a language L that is

–

difgerent from L(M1)

–

difgerent from L(M2)

–

difgerent from L(M3)

–

…..

–

difgerent from L(Mk)

–

…..

Diagonalization

• We want a language L that is

–

difgerent from L(M1) at <M1>

–

difgerent from L(M2) at <M2>

–

difgerent from L(M3) at <M3>

–

…..

–

difgerent from L(Mk) at <Mk>

–

…..

• Diag = { <M> | M is a TM s.t. <M> is not in L(M) }

Diagonalization

• We want a language L that is

–

difgerent from L(M1) at <M1>

–

difgerent from L(M2) at <M2>

–

difgerent from L(M3) at <M3>

–

…..

–

difgerent from L(Mk) at <Mk>

–

…..

• Diag = { <M> | M is a TM s.t. <M> is not in L(M) }

Question: What can you tell about Diag? A) Diag is in RE, but not in coRE B) Diag is coRE, but not in RE C) Diag is neither in RE nor in coRE D) Diag is decidable E) I don’t know

T wist on Diag

• Diag ={ <M> | M is a TM s.t. <M> is not in L(M) }
• Diag2={<M> | M is a TM s.t. M(<M>) does not

accept in less than 2|<M>| steps}

• <M> is in Diag2 if

–

M(<M>) rejects, or

–

M(<M>) takes more than 2n steps

T wist on Diag

• Diag ={ <M> | M is a TM s.t. <M> is not in L(M) }
• Diag2={<M> | M is a TM s.t. M(<M>) does not

accept in less than 2|<M>| steps}

• <M> is in Diag2 if
• 1. M(<M>) rejects, or
• 2. M(<M>) takes more than 2n steps

Question: What can you say about Diag2? A) Diag2 is in RE, but not in coRE B) Diag2 is coRE, but not in RE C) Diag2 is neither in RE nor in coRE D) Diag2 is decidable

T wist on Diag

• Diag ={ <M> | M is a TM s.t. <M> is not in L(M) }
• Diag2={<M> | M is a TM s.t. M(<M>) does not

accept in less than 2|<M>| steps}

• <M> is in Diag2 if
• 1. M(<M>) rejects, or
• 2. M(<M>) takes more than 2n steps

Question: How Diag2 compare to Diag? A) Diag is a subset of Diag2 B) Diag is a proper subset of Diag2 C) Diag is a superset of Diag2 D) Diag is a proper superset of Diag2

The complexity of Diag2

• Diag2={<M> | M is a TM s.t. M(<M>) does not

accept in less than 2|<M>| steps}

• So, Diag2 is decidable
• But, how fast can you solve Diag2?

–

Is there an algorithm deciding Diag2 in time O(n3)?

–

Is there an algorithm deciding Diag2 in time O(2n)?

Diag2 cannot be solved in T<2n

• Diag2={<M> | M is a TM s.t. M(<M>) does not

accept in less than 2|<M>| steps}

• Assume there is a TM P such that

–

L(P) = Diag2

–

Time(P(w))<2n for every input of length |w|=n

• Does P accept w=<P>?

✔

Yes → w is in Diag2 → either P(w) rejects of T(P(w))>2n Contradiction!

✗

No→ w is not in Diag2 → P(w) accepts (in time <2n). Contradiction!