Input/Output Assumptions I/O Assumption Input: Finite string of - - PowerPoint PPT Presentation

Computation and Turing Machines

Nabil Mustafa Computability and Complexity

Input/Output Assumptions

I/O Assumption Input: Finite string of bits, i.e., sequence of 0’s and 1’s. Output: Finite string of bits. Claim Without loss of generality, one can always assume the above input. Set of all strings of length n denoted by {0, 1}n. Set of all strings: {0, 1}∗ =

Can represent Integers Alphabets Graphs, using adjacency matrix Use ⌊x⌋ to represent the encoding of x

Representing tuples

Use notation x, y to represent the pair x and y Can encode x, y as follows: First encode x, y as ⌊x⌋ # ⌊y⌋ Use mapping 0 → 00, 1 → 11, and # → 01, Can decode properly by Scanning left to right, Read two consecutive bits and decode

Representing Problems

Example Problem: Given a graph G = (V , E) and an integer k, does there exist an independent of size k in G? Answer is a ‘yes’ or a ‘no’ Can encode output ‘yes’ by 1 and ‘no’ by 0 Can assume that G is given by a string of bits. Then, can model this problem by a boolean function. Boolean function Given a boolean function f : {0, 1}∗ → {0, 1}, define Lf = {x : f (x) = 1} as the set of inputs on which f accepts x.

Languages

Boolean function Given a boolean function f : {0, 1}∗ → {0, 1}, define Lf = {x : f (x) = 1} as the set of inputs on which f accepts x. The function f defines the language Lf For example, Lf is equivalent to: Lf = {⌊G⌋◦⌊k⌋ : ∃S ⊂ V s.t. |S| = k and ∀u, v ∈ S, (u, v) / ∈ E} Computational Problem Given the input ⌊G⌋, compute f (⌊G⌋)?

Computation

Computation Given the input ⌊G⌋ how exactly does f compute f (⌊G⌋)? There is an algorithm for computing f (⌊G⌋) Enumerate over all k-tuples of vertices of G For each k-tuple, check if it is an independent set. Construct a computing machine that runs any algorithm Input and output storage. Code to represent the algorithm (e.g., in C++, Java) Computation machine Details of such a machine?

Storage

Use Tape for storage: infinite one-dimensional line of cells Each cell contains exactly one symbol from our alphabet Each tape has a tape head, positioned on one cell The tape head can only move to the cell to the left or right. Write a new symbol to the current cell only 1

Storage

Our machine uses: 1 input tape: binary strings written on it represent the input 1 output tape: once the machine computes the answer, it is written on the output tape 1 work tape: initially empty tape which the machine uses as a ‘scratch pad’. Code How is the code of the machine represented?

Turing machine

A Turing machine (TM ) is a tuple M = (Γ, Q, δ) where Γ: set of symbols that TM ’s tapes can contain. : symbols denoting a ‘blank’ ⊲: the designated start symbol {0, 1} Q: possible states TM can be in. A special register stores the current state. qstart: the TM starts in this state qhalt: the TM halts when this state is reached The code δ: Q × Γ2 → Q × Γ × {L, R, S}. δ is called the transition function.

Transition Function

At each step, TM is in some state, say q TM reads a symbol from input tape, and working tape, say s1 and s2. TM looks up the transition function: δ(q, (s1, s2)) → q′, t2, {L, S} TM then: changes state to q′

• verwrites the symbol s2 with t2 on work-tape

moves the input tape head to the left, the tape head of the working tape stays same

Example

Only one input and one output tape. No work tape. States Q = {qstart, q1, qhalt} Symbols Γ = {0, 1, ⊲, } Transition function δ: Q IT Γ Q OT Γ {L, S, R} qstart ⊲ q1 − R q1 q1 − R q1 1 qhalt 1 S q1

• qhalt

S What does this TM do?

Working of the example TM

Q IT Γ Q OT Γ {L, S, R} qstart ⊲ q1 − R q1 q1 − R q1 1 qhalt 1 S q1

• qhalt

S 1 1 Step 5: State = qhalt with Accept

Program size

Question For a TM = (Γ, Q, δ), what is the size of the table given by δ? δ: Q × Γ2 → Q × Γ × {L, R, S} Size of δ solely determined by |Q| and |Γ| Hence total size: |Q| · |Γ| · |Γ| This is a constant if |Q| and |Γ| are constant. Note that different δ’s might define the same behavior.

Running time

TM halts and outputs one of {0, 1} Running time of a TM is number of steps from start to end. TM does not halt (e.g., infinite loops) Running time is undefined. Different inputs take different steps Running time Given: A Turing machine M f : {0, 1}∗ → {0, 1} T : N → N we say that M computes f in T(n)-time if for every x ∈ {0, 1}∗, M computes f (x) in at most T(|x|) steps.

Palindrome testing

PAL (x) = 1 if x is a palindrome, 0 otherwise. This defines a language: LPAL = {x ∈ {0, 1}∗ : PAL (x) = 1} Examples of palindromes: 00001010000, AVIVA, 11111111111111, 1010101001010101 Problem Given a string x, construct a TM that computes PAL (x).

3-tape TM for PAL

Algorithm: Copy the input string to the work-tape Move the tape head of input-tape to the start At each step, move the tape head of input-tape forward, work-tape backward. And then check for same symbols. Each of these tasks done via a state. While the TM is in that state, its doing that task. Q qstart, qcopy, qmoveleft, qtest, qhalt

3-tape TM for PAL

qstart: Just go into the qcopy state. qcopy: Till the end of input tape is reached, keep copying input tape to work tape. Q Γ2 Q Γ2 {L, S, R}3 qstart ⊲, ∗ qcopy (⊲, −) (R, R, S) qcopy 0, ∗ qcopy (0, −) (R, R, S) qcopy 1, ∗ qcopy (1, −) (R, R, S) qcopy , ∗ qmoveleft (−, −) (L, L, S) *: perform for all possible symbols

3-tape TM for PAL

qmoveleft: Keep moving the input tape-head left till it hits the start

• symbol. Let the work-tape head stay at the right.

Q Γ2 Q Γ2 {L, S, R}3 qmoveleft 0, ∗ qmoveleft (−, −) (L, S, S) qmoveleft 1, ∗ qmoveleft (−, −) (L, S, S) qmoveleft ⊲, ∗ qtest (−, −) (R, S, S)

3-tape TM for PAL

qtest: At each step: advance input-tape head to the right work-tape head to the left, check if the symbols are the same. Q Γ2 Q Γ2 {L, S, R}3 qtest 1, 1 qtest (−, −) (R, L, S) qtest 0, 0 qtest (−, −) (R, L, S) qtest 0, 1 qhalt (−, 0) (S, S, S) qtest 1, 0 qhalt (−, 0) (S, S, S) qtest , ⊲, ∗ qhalt (−, 1) (S, S, S)

3-tape TM for PAL

Time taken: O(n) Question Assume only input tape. Then how fast can one compute PAL (x)? Answer: Ω(n2)!

Lower-bound for 1-tape TM

Claim Any 1-tape TM takes Ω(n2) steps to compute PAL (x), |x| = n. Specifically, for any TM and any n, there exists a string x, |x| = n, such that TM takes Ω(n2) to check if x is a palindrome. Proof by counting by averaging. Fix a Turing machine for computing PAL , say the TM M. Lets assume the worst-case running time (over all n-sized strings) is T(n) Goal: lower-bound T(n)

Lower-bound for 1-tape TM

Proof. Goal: From 2n strings, there is a string taking Ω(n2) time. Show stronger statement: ∃ from a set of 2n/3 strings Let m = n/3. Then, for any string |x| = m, define P(x, y) = x ◦ {0}m ◦ y r P(x, x) is a palindrome T(x): running time of P(x, x), T(n) = maxx,|x|=n T(x). We will count the following: d · n2 · 2m ≤

• x∈{0,1}m

T(x) ≤ 2m · T(n)

Lower-bound for 1-tape TM

To prove claim, count the sum of the running times T(x) Our first trivial claim is that

• x∈{0,1}m

T(x) ≤ 2m · T(n) This holds trivially because the number of strings of length m are 2m and T(n) is the worst case running time over all n sized strings which would obviously be greater than any T(m) The second harder claim holds the key to entire proof. d · n2 · 2m ≤

• x∈{0,1}m

T(x) Putting together the two claims = ⇒ ∃ a string of size n/3 taking Ω(n2) time for PAL (x) function.

Lower-bound for 1-tape TM

C(x, y, c ): the sequence of states of Q in which M crosses the cut c when executed on the string P(x, y)

Lower-bound for 1-tape TM

In the given example we have 1 cut in the middle So any particular C(x, y, c ) would resemble: Qi, Qj, Qk, . . . , Qm where the elements of the sequence are some states. Main Claim: For any cut c , C(x, x, c ) = C(y, y, c ) We assume for the sake of contradiction that C(x, x, c )=C(y, y, c ) holds for some cut c Note however that the TM might take different number of steps to reach c while computing P(x, x) and P(y, y), and the TM might be in different states in the two computations before reaching c

Lower-bound for 1-tape TM

C(x, c ) = C(x, x, c ) Claim For any cut c , C(x, x, c ) = C(y, y, c ) Proof. Assume that for x = y, C(x, x, c ) = C(y, y, c ). Then, C(x, x, c ) = C(x, y, c ) Imagine the computation for P(x, y) Since first input x is the same, C(x, y, c )[1] = C(x, x, c )[1] Since second input y is the same and C(x, y, c )[1] = C(x, x, c )[1] = C(y, y, c )[1] Once tape-head crosses c , computation same as P(y, y) Continuing on, entire computation same (induction). Then, P(x, x) has the same answer as P(x, y) Contradiction!

More Explanation

First – we can split the computation of P(x, y) into two portions, one in which we are in the x-portion and the other in which we are in the y-portion. Second – since we assume C(x, x, c ) = C(y, y, c ) to be true, that means whenever the TM crosses the cut c the TM is in the same state for both P(x, x) and P(y, y). So now when the TM would be computing P(x, y), the first input is x, so when it crosses the cut c from left to right it would be in the same state as P(x, x) would be when the TM would be computing it, and crossing the cut from left to right. Similarly the second input is y so when the TM crosses from right to left it would be in the same state as P(y, y).

More Explanation

Why do P(x, x) and P(x, y) have the same answer? If P(x, y) halts in the first portion, its computation will be the same as in the first portion of P(x, x) and since P(x, x)

• utputs a 1, P(x, y) should also output a 1.

If P(x, y) halts in the second portion its computation will be the same as in the second portion of P(y, y) and since P(y, y) outputs a 1, P(x, y) should also output a 1. P(x, y) cannot halt in middle: |C(x, x, c )| = |C(y, y, c )| TM gives a 1 for P(x, x) while P(x, y) should give a 0 Contradiction! Hence C(x, x, c ) = C(y, y, c )

Lower-bound for 1-tape TM

Lower-bound for 1-tape TM

Fix κ = (m + 1) log|Q| 2 − log|Q| m − 1 + log|Q|(|Q| − 1) Lower-bound:

≥

• x
• cut c

Number of times computation of x crosses c =

• cut c
• x

Number of times computation of x crosses c =

• cut c
• x

|C(x, c )| ≥ m ·

• x

|C(x, c )|, where c is the worst cut = m ·

• i=0

i · (# of strings x that cross c i times)

Lower-bound for 1-tape TM

Lower-bound:

= m ·

• i=0

i · (# of strings x that cross c i times) = m · (

• i≤κ

i · # of strings x that cross c i times +

• i>κ

i · # of strings x that cross c i times) ≥ m ·

• i>κ

i · (# of strings x that cross c i times) ≥ m · κ · (# of strings x that cross c > κ times) ≥ m · κ · (2m − 2m+1/m) ≥ m · Θ(m) · (2m m − 2 m ) ≥ d · m2 · 2m, where d is some constant

Claim # of strings x that cross any cut c > κ times ≥ 2m−2m+1/m # of strings x that cross any cut c ≤ κ times ≤ 2m+1/m Proof. Fix a string x that crosses c i times. This sequence of states are not repeated by any string. ∀i, # strings that cross c i times ≤ |Q|i Summing up over all i, strings crossing c ≤ κ times ≤

• i=0

|Q|i ≤ |Q|κ+1 |Q| − 1 = 2m+1 m