CSE 105 THEORY OF COMPUTATION Fall 2016 - PowerPoint PPT Presentation

CSE 105 THEORY OF COMPUTATION Fall 2016 http://cseweb.ucsd.edu/classes/fa16/cse105-abc/

This Week ● Beyond Theory of Computation: Computational Complexity – Reading: Sipser Chapter 7 – Not covered in final exam – ● Revisit familiar topics Diagonalization – Reduction – ● NP-complenetess

What we studied so far Decidable Context-free RE co-RE Regular

Theory of Computation ● Question: What computational problems admit an algorithmic solution? Decidable problems: Yes, they can be solved by a TM – Undecidable problems: No, no algorithmic solution – ● Classifjcation of computational problems based on restricted models of computation: DFA, NFA: Regular languages – PDA, CFG: Context Free Languages – Decidable: TM –

Computational Complexity ● Goal: better understanding of decidable problems ● Computational Model: T uring Machine (or equivalent) ● Questions: How much time is needed to solve a problem? – How much memory is needed to solve a problem? – ● Are there problems that intrinsically require a lot of time or memory?

Measuring Time and Space ● Assume a TM M is run on input w ● Computation: C init → C 1 → C 2 → … → C halt ● Time(M(w)): – Number of steps in the computation ● Space(M(w)): – Number of tape cells used during the computation. Equivalently, size of largest C k

Running time as a function ● Fix TM/program M ● Running time Time(M(w)) depends on input w It is natural to expect computation to take longer as w gets larger – ● We are not insterested in exact running time We don’t care if the running time (number of steps) is even or odd – It is usually enough to know an upper bound on Time(M(w)) – ● Time(n) = max { Time(M(w)) | |w|=n} Maximum running time over all inputs of size n –

Asymptotic notation ● Time(n) = O(f(n)) There are constant a,b (independent of n) such that T < a*f(n) + b – ● Signifjcance of a and b: You can think of b as some fixed startup cost – – You can think of a as using a slower or faster computer ● Theoretical goal Get bounds on running time that are independent of details of computational – model Ignore low level details, for the sake of generality –

Some example problems ● Addition: on input x and y, output x+y ● Multiplication: on input x and y, output x*y ● How can we model them as decision problems? Addition: on input (x,y,i), determine if the ith (binary) digit of x+y is 1 – Multiplication: on input (x,y,i), determine if the ith (binary) digit of x*y is 1. – ● How fast can you add up or multiply numbers?

Some example problems What is the time complexity of Addition? ● Addition: on input x and y, output x+y A) T=O(n): You need to add all digits one by one ● Multiplication: on input x and y, output x*y B) T=O(3n): You need to readn 2n input digits and output an n-digit result ● How can we model them as decision problems? C) T=1: Addition is a single machine instruction. D) It depends on how clock-speed, etc. Addition: on input (x,y,i), determine if the ith (binary) digit of x+y is 1 – Multiplication: on input (x,y,i), determine if the ith (binary) digit of x*y is 1. – ● How fast can you add up or multiply numbers?

Some example problems What is the time complexity of Multiplication? ● Addition: on input x and y, output x+y A) T=O(n) because output is 2n digits long ● Multiplication: on input x and y, output x*y B) T=O(n 2 ) as taught in elemntary school C) Somewhere between O(n) and O(n 2 ), exact answer ● How can we model them as decision problems? is not known D) It depends on how clock-speed, etc. Addition: on input (x,y,i), determine if the ith (binary) digit of x+y is 1 – Multiplication: on input (x,y,i), determine if the ith (binary) digit of x*y is 1. – ● How fast can you add up or multiply numbers?

Complexity of Multiplication ● Schoolbook algorithm: What is the complexity of multiplication? – O(n 2 ) ● 1960: Karatsuba algorithm A) O(n / log n) B) O(n) O(n 1.585 ) – C) O(n * log(n)) ● 1971: Schonhage-Strassen algorithm D) O(n * log(n) * exp(log*(n))) O(n * log(n) * log(log(n))) – ● 2007: Furer algorithm O(n * log(n) * exp(log*(n))) –

How hard can it be? ● Even simple problems may require more than O(n) computation ● Parsing: decide L(G) for a CFG G Can be solved in Time = O(n 3 ) for any G – Can it be solved in O(n 2 ) for any G? – Can it be solved in O(n) for any G? – ● Are there problems that require T=O(n 100 )? ● Are there problems that require T=O(exp(n))?

Computationally hard problems ● Is there a decidable problem that Cannot be solved in time O(n 100 ) ? – Cannot be solved in time O(exp(n)) ? – Cannot be solved in time O(exp(exp(exp( … exp(n)…))) ? – ● How can we come up with such a problem? ● How do we show it cannot be solved in time T(n)?

An undecidable language ● We want a language L that is difgerent from L(M 1 ) – difgerent from L(M 2 ) – difgerent from L(M 3 ) – ….. – difgerent from L(M k ) – ….. –

Diagonalization ● We want a language L that is difgerent from L(M 1 ) at <M 1 > – difgerent from L(M 2 ) at <M 2 > – difgerent from L(M 3 ) at <M 3 > – ….. – difgerent from L(M k ) at <M k > – ….. – ● Diag = { <M> | M is a TM s.t. <M> is not in L(M) }

Diagonalization ● We want a language L that is Question: What can you tell about Diag? difgerent from L(M 1 ) at <M 1 > – A) Diag is in RE, but not in coRE difgerent from L(M 2 ) at <M 2 > – B) Diag is coRE, but not in RE difgerent from L(M 3 ) at <M 3 > C) Diag is neither in RE nor in coRE – D) Diag is decidable ….. – E) I don’t know difgerent from L(M k ) at <M k > – ….. – ● Diag = { <M> | M is a TM s.t. <M> is not in L(M) }

T wist on Diag ● Diag ={ <M> | M is a TM s.t. <M> is not in L(M) } ● Diag2={<M> | M is a TM s.t. M(<M>) does not accept in less than 2 |<M>| steps} ● <M> is in Diag2 if M(<M>) rejects, or – M(<M>) takes more than 2 n steps –

T wist on Diag ● Diag ={ <M> | M is a TM s.t. <M> is not in L(M) } ● Diag2={<M> | M is a TM s.t. M(<M>) does not accept in less than 2 |<M>| steps} ● <M> is in Diag2 if Question: What can you say about Diag2? 1. M(<M>) rejects, or A) Diag2 is in RE, but not in coRE 2. M(<M>) takes more than 2 n steps B) Diag2 is coRE, but not in RE C) Diag2 is neither in RE nor in coRE D) Diag2 is decidable

T wist on Diag ● Diag ={ <M> | M is a TM s.t. <M> is not in L(M) } ● Diag2={<M> | M is a TM s.t. M(<M>) does not accept in less than 2 |<M>| steps} ● <M> is in Diag2 if Question: How Diag2 compare to Diag? 1. M(<M>) rejects, or A) Diag is a subset of Diag2 2. M(<M>) takes more than 2 n steps B) Diag is a proper subset of Diag2 C) Diag is a superset of Diag2 D) Diag is a proper superset of Diag2

The complexity of Diag2 ● Diag2={<M> | M is a TM s.t. M(<M>) does not accept in less than 2 |<M>| steps} ● So, Diag2 is decidable ● But, how fast can you solve Diag2? Is there an algorithm deciding Diag2 in time O(n 3 )? – Is there an algorithm deciding Diag2 in time O(2 n )? –

Diag2 cannot be solved in T<2 n ● Diag2={<M> | M is a TM s.t. M(<M>) does not accept in less than 2 |<M>| steps} ● Assume there is a TM P such that L(P) = Diag2 – Time(P(w))<2 n for every input of length |w|=n – ● Does P accept w=<P>? Yes → w is in Diag2 → either P(w) rejects of T(P(w))>2 n ✔ Contradiction! No→ w is not in Diag2 → P(w) accepts (in time <2 n ). Contradiction! ✗