Lecture Slides for MAT-73006 Theoretical computer science PART IIb: - - PowerPoint PPT Presentation

Lecture Slides for MAT-73006 Theoretical computer science PART IIb: Decidability

Henri Hansen February 3, 2014

1

Decidability

• We introduced the TM as a general purpose model of com-

putation.

• An ”algorithm” was basically defined as a Turing machine

that decides a language.

• We shall now investigate the limitations of turing machines

– of algorithms, in so far as we believe the Church-Turing thesis.

• There are some surprising problems that simply cannot be

solved, and it is important to understand the limitations that this imposes on computation

2

• In order to understand what computation is, you must un-

derstand its limits!

Decidable languages

• We started with finite automata and it is only natural that

we state some computational problems related to these au- tomata.

• We shall engage a lot in a sort of ”self study” by computa-

tional models: We try to find out, to what extent are turing machines able to decide computational properties of other computational models (including turing machines themself!)

• Given a DFA B and a string w, we define the decision prob-

lem: Does B accept w.

3

• As a language, supposing that we can describe DFAs as

strings, define the language ADFA = {(B, w) | B is a DFA that accepts w}

• Theorem: ADFA is a decidable language. Proof is a con-

struction of a turing machine M that does the following – Follow the computation steps on input w – if the computation steps end in an accepting state, then accept, if not, reject

• Obviously the TM is going to be somewhat complicated, so

we simply look at some of the more important details

• The input is a representation of B and w.

When M re- ceives the input, it first determines whether it describes a turing machine. For instance, the representation of B could consist of ”lists” for states, alphabet, transitions and accept states.

• The simulation keeps track of which state is being simu-
• lated. It crosses the next letter from w and checks where it

should move next.

• When all letters have been crossed it checks whether the

simulated state is accepting. If it is, then it accepts.

More decidable languages

• Given an NFA and a string, does it accept?

ANFA = {(B, w) | B is an NFA that accepts w}

• Theorem: ANFA is a decidable language.
• Again, the proof is a TM that decides the language. On

input (B, w) where B is an NFA:

• 1. Convert the NFA B into a DFA D
• 2. Run the TM M of the previous theorem on (D, w)

4

• 3. accept accordingly
• It is very common to use already established turing ma-

chines as ”subroutines”; we shall do so whenever we can solve a problem easier using some problem that we have already solved

• The conversion of an NFA to a DFA is nontrivial, but we

already know how to do it. The states are subsets of the NFA states and transitions are defined as leading to the set

• f all possible states in the NFA (this set is a unique state).

Accepting states are those that contain an accepting state

• f the original, and initial state contains the inital state and

nothing else.

• Given a regular expression, does it match a given word?

AREX = {(R, w) | R is a regular expression that generates w}

• Theorem AREX is decidable: Convert R to an NFA, and

check with the TM given in the previous theorem

• EDFA = {(D) | L(D) = ∅} is the language of DFAs with

an empty language. It is decidable

• 1. Mark the start state of D
• 2. Repeat 3 until no state gets marked:
• 3. Mark a state that has a transition coming in from a marked

state

• 4. If no accept state is marked, accept. Otherwise, reject.
• EQDFA = {(A, B) | A and B are DFAs and L(A) = L(B)}

is a decidable language

• 1. Generate a DFA C that accepts strings that are accepted

by A or B but not both (question: How and why?)

• 2. Check that L(C) = ∅ using the previous theorem
• 3. Accept accordingly.
• Let ACFG = {(G, w) | G is a CFG that generates w}. This

language is decidable

• The proof needs another concept for CFG’s, that of Chom-

sky normal form, of ChNF. ChNF is a canonical form for CFG’s, and it guarantees that there are exactly 2n−1 steps in every derivation of a word w whose length is n.

• The TM that decides the language first converts G into ChNF,

then tries all derivations with the appropriate number of steps.

• ECFG = {(G) | G is a CFG and L(G) = ∅} is also de-

cidable

• Nonempty languages are ”easy” to recognize, simply try all

possible strings until one is generated by the grammar.

• Such a TM however runs forever if G generates the empty
• language. So we propose a different algorithm, which is, in

fact, easy:

• 1. Mark all terminal symbols in G
• 2. Repeat 3 until no variables get marked
• 3. If there is a rule A → U1, . . . , Uk such that each Ui is

marked and A is unmarked, mark A

• 4. If the start variable is not marked, accept, otherwise re-

ject.

• Theorem: Every context free language is decidable.

• Proof: Let G be a CFG for some language A. Let M be the

turing machine that decides ACFG. We generate MG that reads input w and does the following:

• 1. MG transforms the tape so that it contains (G, w) (re-

member, G is fixed, so we can do this easily)

• 2. Then MG behaves exactly as M does.

Undecidability

• We saw examples of languages that are decidable.
• Proving decidability is often straightforward: simply give the

Turing machine that decides the language.

• Proving that a language is not decidable, is much harder.
• Why would we want to explore languages that ar undecid-

able?

• Are undecidable languages merely theoretical curiosities

that are of no practical value?

5

An undecidable language

• Remember that the languages ADFA and ACFG were de-

cidable.

• In the same way, consider the following language

ATM = {(M, w) | M is a TM that accepts w}

• Theorem: ATM is undecidable.
• The proof of this theorem is not straightforward directly, so

we do it in steps.

6

• Before showing that ATM is undecidable, it is worth noting

that it is Turing-recognizeble, because one can simply run M on w, and behave accordingly.

• The trouble comes from infinite executions that M might

have.

• To be able to cover the theorem and understand it, we must

explore a little bit more theory.

Diagonalization

• Cantor used diagnolization in 1873, when he was studying

the infinite sets and their ”sizes”

• Consider two sets A and B, and a function f : A → B.

– f is one-to-one if f never maps two elements of A to the same value, i.e., a = b implies f(a) = f(b) – f is onto if for every x ∈ B there is some a ∈ A such that f(a) = x – A function that is both one-to-one and onto is called a bijection, or a correspondence

7

– We say that A and B have the same size if there is a bijection between them

• For example, the set of natural numbers without zero {1, 2, · · · }

and the set of even numbers without zero {2, 4, 6, . . .} have the same size

• In general, a set is countable if it has the same size as the

set of natural numbers.

• For example: The rational numbers have the same size as

natural numbers:

– Considerthe sequence of sequences L1, L2, . . . where Li is the sequence i

1, i 2, i 3, . . ., so that Lij = i

• j. Clearly

every rational number appears in one of these sequences. – Consider the sequence L11, L21, L12, L31, L22, L13, . . .. This sequence lists all the Lij at some point, and if f(k) is the kth element in this sequence, then f gives the correspondence between rational and natural numbers

• The set of real numbers is not countable

– Let us express real numbers as infinite decimal, such as 3.14159 · · · – Assume that f is some correspondence between real numbers and natural numbers

– We can construct a number that does not appear in the sequence f(1), f(2), . . . – Let x be defined as follows: it is of the form 0.d1d2 · · · , such that di is f(i)’s ith digit plus one or minus one, as long as it is never 0 or 9. – By construction x = f(i) for every i, because f(i) and x differn on the ith place – The assumption that f is a correspondence, results in contradiction, so there is no correspondence

• Corollary to this: There exists a language that is not decid-

able

• The proof is as follows: The set of all turing machines is

countable, whereas the set of all languages is not

• Turing machines are countable, because every turing ma-

chine has a finite descrption, and therefore is expressible by a string in Σ∗. This language is countable, because Σ∗ is countable: it does not contain infinite words

• The set L of all languages is uncountable, because we can

build a correspondence between L and real numbers: con- sider binary representations of real numbers. Then con- sider Σ∗. List all strings in Σ∗, and each infinite binary dec- imal denotes one language: the ith bit is 1 if the ith word in the list is in the language and 0 if it is not.

A proof for the undecidability of ATM

• Let us assume ATM is decidable, and that H decides it,

i.e, for every TM M, H(M, w) accepts if M accepts w and rejects if M does not accept w.

• Consider the Turing machine D whose input is a description
• f a turing machine M
• 1. Run H on input M, w where w is the description of M.
• 2. Output exactly the opposite of what H outputs
• What happens when D is given its own description as an

input?

8

• 1. If H reports that D accepts, then D rejects, so that H

does not answer correctly!

• 2. If H reports that D rejects, then D accepts, again H is

wrong!

• Obviously H does not decide ATM, i.e., there is no Turing

machine that decides ATM

• In fact, this proof is exactly like diagonalization.

Turing recognizable and unrecognizable languages

• We proved that ATM is undecidable.
• However, it is clearly Turing-recognizable: Simply run the

input machine on the input string, and accept accordingly

• So the problem comes from the negative answer, when a

word is not accepted by a machine (and out of those, the situations that lead into infinite number of steps); The com- plement of ATM is not Turing-recognizable.

• Languages whose complements are Turing-recogizable are

called co-Turing recognizable.

9

• Theorem: A language is decidable if and only if it is Turing-

recognizable and co-Turing-recognizable

• Proof: Let A be Turing-recognizable and co-Turing-recognizable.

Then there are Turing machines M1 and M2 that recognize A and ¯ A, resp.

• Let M function so that it runs both M1 and M2 in parallel,

and if M1 accepts, M accepts, if M2 accepts, M rejects.

• Clearly M is a decider for A.
• Corollary:

¯ ATM is not Turing-recognizable.