CS 301 Lecture 17 ChurchTuring thesis Stephen Checkoway March 19, - - PowerPoint PPT Presentation

CS 301

Lecture 17 – Church–Turing thesis Stephen Checkoway March 19, 2018

1 / 17

An abridged modern history of formalizing algorithms

An algorithm is a finite, unambiguous sequence of steps for solving a problem

1Polynomial equation with integer coefficients 2Statements with ∀ and ∃ 2 / 17

An abridged modern history of formalizing algorithms

An algorithm is a finite, unambiguous sequence of steps for solving a problem The name algorithm comes from 9th Century Iranian mathematician Muh .ammad ibn M¯ us¯ a al’Khw¯ arizm¯ ı

1Polynomial equation with integer coefficients 2Statements with ∀ and ∃ 2 / 17

An abridged modern history of formalizing algorithms

An algorithm is a finite, unambiguous sequence of steps for solving a problem The name algorithm comes from 9th Century Iranian mathematician Muh .ammad ibn M¯ us¯ a al’Khw¯ arizm¯ ı In the 17th century, German mathematician Gottfried Leibniz wanted to build a machine that could determine if mathematical statements were true or false

1Polynomial equation with integer coefficients 2Statements with ∀ and ∃ 2 / 17

An abridged modern history of formalizing algorithms

An algorithm is a finite, unambiguous sequence of steps for solving a problem The name algorithm comes from 9th Century Iranian mathematician Muh .ammad ibn M¯ us¯ a al’Khw¯ arizm¯ ı In the 17th century, German mathematician Gottfried Leibniz wanted to build a machine that could determine if mathematical statements were true or false In 1900, German mathematician David Hilbert posed a list of 23 open problems, the tenth asks for a general algorithm by which any Diophantine equation1 could be solved

1Polynomial equation with integer coefficients 2Statements with ∀ and ∃ 2 / 17

An abridged modern history of formalizing algorithms

An algorithm is a finite, unambiguous sequence of steps for solving a problem The name algorithm comes from 9th Century Iranian mathematician Muh .ammad ibn M¯ us¯ a al’Khw¯ arizm¯ ı In the 17th century, German mathematician Gottfried Leibniz wanted to build a machine that could determine if mathematical statements were true or false In 1900, German mathematician David Hilbert posed a list of 23 open problems, the tenth asks for a general algorithm by which any Diophantine equation1 could be solved In 1928, Hilbert posed 3 more problems, the third of which became known as Hilbert’s Entscheidungsproblem (decision problem). It asks for an algorithm which takes as input a statement in first-order logic2

1Polynomial equation with integer coefficients 2Statements with ∀ and ∃ 2 / 17

Formalizing algorithms

In 1933, Austrian-American mathematician Kurt Gödel and French mathematician Jacques Herbrand3 defined the class of general recursive functions; these were designed to be intuitively “computable”

3Herbrand had actually died at age 23 several years earlier 4Functional programming owes much to λ-calculus and lambdas and closures in many programming

languages derive from this

3 / 17

Formalizing algorithms

In 1933, Austrian-American mathematician Kurt Gödel and French mathematician Jacques Herbrand3 defined the class of general recursive functions; these were designed to be intuitively “computable” In 1936, American mathematician Alonzo Church defined the λ-calculus as a model of computable functions4

3Herbrand had actually died at age 23 several years earlier 4Functional programming owes much to λ-calculus and lambdas and closures in many programming

languages derive from this

3 / 17

Formalizing algorithms

In 1933, Austrian-American mathematician Kurt Gödel and French mathematician Jacques Herbrand3 defined the class of general recursive functions; these were designed to be intuitively “computable” In 1936, American mathematician Alonzo Church defined the λ-calculus as a model of computable functions4 Independently, in 1936, English mathematician Alan Turing defined his machines

3Herbrand had actually died at age 23 several years earlier 4Functional programming owes much to λ-calculus and lambdas and closures in many programming

languages derive from this

3 / 17

Formalizing algorithms

In 1933, Austrian-American mathematician Kurt Gödel and French mathematician Jacques Herbrand3 defined the class of general recursive functions; these were designed to be intuitively “computable” In 1936, American mathematician Alonzo Church defined the λ-calculus as a model of computable functions4 Independently, in 1936, English mathematician Alan Turing defined his machines Church and Turing proved that all three of these models of computation are equivalent: A function is λ-computable iff it is computable by a Turing machine iff it is general recursive

3Herbrand had actually died at age 23 several years earlier 4Functional programming owes much to λ-calculus and lambdas and closures in many programming

languages derive from this

3 / 17

Church–Turing thesis

The Church–Turing thesis is that the intuitive notion of an algorithm (called an “effectively calculable function”) is equivalent to a Turing machine

4 / 17

Hilbert’s problems

Hilbert framed his tenth problem in 1900 and his Entscheidungsproblem in 1928 as a positive: Give an algorithm to solve the problems As late as 1930, he didn’t seem to realize that there would be any unsolvable problems

5 / 17

Hilbert’s problems

Hilbert framed his tenth problem in 1900 and his Entscheidungsproblem in 1928 as a positive: Give an algorithm to solve the problems As late as 1930, he didn’t seem to realize that there would be any unsolvable problems Unfortunately (depending on your point of view), both of these problems don’t have general solutions

5 / 17

Types of problems

There are several types of problems we could consider, consider the problem of paths in a weighted, undirected graph G between two vertices u and v Decision Is there some path between u and v in G of distance at most k? Search What is the length of the shortest path between u and v in G? Counting How many paths between u and v in G have distance at most k? In this course, we’re concerned with the simplest of these: decision problems

6 / 17

Decision problems as languages

We can frame decision problems as membership in a language. What language corresponds to the decision problem: Is there some path between u and v in graph G of distance at most k?

7 / 17

Decision problems as languages

We can frame decision problems as membership in a language. What language corresponds to the decision problem: Is there some path between u and v in graph G of distance at most k? {⟨G, u, v, k⟩ ∣ G = (V, E) is an undirected graph, u, v ∈ V , and there is a path from u to v of distance at most k} ⟨G, u, v, k⟩ means some fixed representation of the graph G, vertices u and v, and natural number k

7 / 17

Representation

For the rest of the course, we’re going to be working with languages whose elements are representations of mathematical objects like

• integers
• graphs
• finite automata
• regular expressions
• CFGs
• PDAs
• Turing machines
• finite sets of mathematical objects
• finite, ordered lists of mathematical objects

We need some way to represent these as strings over some alphabet

8 / 17

Explicit representation

Most of the time, we won’t give an explicit representation We’ll mostly assume some representation exists and we can convert from one representation to another easily

9 / 17

Example: Graph representation

As a mathematical object, an undirected graph5 is a pair G = (V, E) where V a finite set of vertices E a finite set of pairs of vertices We can specify the graph in a number of ways and easily convert between them

• Vertex and edge lists G = (V, E) where V = {1, 2, 3, 4, 5} and

E = {(1, 2), (2, 3), (3, 4), (4, 5), (5, 1), (1, 3), (2, 4), (3, 5)} 1 2 3 4 5

5Review Sipser, chapter 0 as needed 10 / 17

Example: Graph representation

As a mathematical object, an undirected graph5 is a pair G = (V, E) where V a finite set of vertices E a finite set of pairs of vertices We can specify the graph in a number of ways and easily convert between them

• Vertex and edge lists G = (V, E) where V = {1, 2, 3, 4, 5} and

E = {(1, 2), (2, 3), (3, 4), (4, 5), (5, 1), (1, 3), (2, 4), (3, 5)}

• Adjacency matrix aij = 1 if (i, j) ∈ E

⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ 1 2 3 4 5

5Review Sipser, chapter 0 as needed 10 / 17

Example: Graph representation

As a mathematical object, an undirected graph5 is a pair G = (V, E) where V a finite set of vertices E a finite set of pairs of vertices We can specify the graph in a number of ways and easily convert between them

• Vertex and edge lists G = (V, E) where V = {1, 2, 3, 4, 5} and

E = {(1, 2), (2, 3), (3, 4), (4, 5), (5, 1), (1, 3), (2, 4), (3, 5)}

• Adjacency matrix aij = 1 if (i, j) ∈ E

⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦

• Adjacency list: [{2, 3, 5}, {1, 3, 4}, {1, 2, 4, 5}, {2, 3, 5}, {1, 3, 4}]

1 2 3 4 5

5Review Sipser, chapter 0 as needed 10 / 17

Explicit representation for a DFA

Consider M = (Q, Σ, δ, q0, F) where Q = {1, 2, . . . , m}, Σ = {1, 2, . . . , n}, and F = {f1, f2, . . . , fk} Build it up piece by piece using the alphabet Σ = {0, 1} ⟨k⟩ = 0k integers

11 / 17

Explicit representation for a DFA

Consider M = (Q, Σ, δ, q0, F) where Q = {1, 2, . . . , m}, Σ = {1, 2, . . . , n}, and F = {f1, f2, . . . , fk} Build it up piece by piece using the alphabet Σ = {0, 1} ⟨k⟩ = 0k integers ⟨Q⟩ = 1⟨1⟩1⟨2⟩1⋯1⟨m⟩1 integers separated by 1s = 101001⋯10m1

11 / 17

Explicit representation for a DFA

Consider M = (Q, Σ, δ, q0, F) where Q = {1, 2, . . . , m}, Σ = {1, 2, . . . , n}, and F = {f1, f2, . . . , fk} Build it up piece by piece using the alphabet Σ = {0, 1} ⟨k⟩ = 0k integers ⟨Q⟩ = 1⟨1⟩1⟨2⟩1⋯1⟨m⟩1 integers separated by 1s = 101001⋯10m1 ⟨Σ⟩ = 1⟨1⟩1⟨2⟩1⋯1⟨n⟩1 integers separated by 1s = 101001⋯10n1

11 / 17

Explicit representation for a DFA

Consider M = (Q, Σ, δ, q0, F) where Q = {1, 2, . . . , m}, Σ = {1, 2, . . . , n}, and F = {f1, f2, . . . , fk} Build it up piece by piece using the alphabet Σ = {0, 1} ⟨k⟩ = 0k integers ⟨Q⟩ = 1⟨1⟩1⟨2⟩1⋯1⟨m⟩1 integers separated by 1s = 101001⋯10m1 ⟨Σ⟩ = 1⟨1⟩1⟨2⟩1⋯1⟨n⟩1 integers separated by 1s = 101001⋯10n1 ⟨δ(q, t) = r⟩ = ⟨q⟩1⟨t⟩1⟨r⟩ = 0q10t10r

11 / 17

Explicit representation for a DFA

Consider M = (Q, Σ, δ, q0, F) where Q = {1, 2, . . . , m}, Σ = {1, 2, . . . , n}, and F = {f1, f2, . . . , fk} Build it up piece by piece using the alphabet Σ = {0, 1} ⟨k⟩ = 0k integers ⟨Q⟩ = 1⟨1⟩1⟨2⟩1⋯1⟨m⟩1 integers separated by 1s = 101001⋯10m1 ⟨Σ⟩ = 1⟨1⟩1⟨2⟩1⋯1⟨n⟩1 integers separated by 1s = 101001⋯10n1 ⟨δ(q, t) = r⟩ = ⟨q⟩1⟨t⟩1⟨r⟩ = 0q10t10r ⟨δ⟩ = 11⟨δ(1, 1) = r11⟩11⋯11⟨δ(q, s) = rqs⟩11

11 / 17

Explicit representation for a DFA

Consider M = (Q, Σ, δ, q0, F) where Q = {1, 2, . . . , m}, Σ = {1, 2, . . . , n}, and F = {f1, f2, . . . , fk} Build it up piece by piece using the alphabet Σ = {0, 1} ⟨k⟩ = 0k integers ⟨Q⟩ = 1⟨1⟩1⟨2⟩1⋯1⟨m⟩1 integers separated by 1s = 101001⋯10m1 ⟨Σ⟩ = 1⟨1⟩1⟨2⟩1⋯1⟨n⟩1 integers separated by 1s = 101001⋯10n1 ⟨δ(q, t) = r⟩ = ⟨q⟩1⟨t⟩1⟨r⟩ = 0q10t10r ⟨δ⟩ = 11⟨δ(1, 1) = r11⟩11⋯11⟨δ(q, s) = rqs⟩11 ⟨q0⟩ = 0q0 integer

11 / 17

Explicit representation for a DFA

Consider M = (Q, Σ, δ, q0, F) where Q = {1, 2, . . . , m}, Σ = {1, 2, . . . , n}, and F = {f1, f2, . . . , fk} Build it up piece by piece using the alphabet Σ = {0, 1} ⟨k⟩ = 0k integers ⟨Q⟩ = 1⟨1⟩1⟨2⟩1⋯1⟨m⟩1 integers separated by 1s = 101001⋯10m1 ⟨Σ⟩ = 1⟨1⟩1⟨2⟩1⋯1⟨n⟩1 integers separated by 1s = 101001⋯10n1 ⟨δ(q, t) = r⟩ = ⟨q⟩1⟨t⟩1⟨r⟩ = 0q10t10r ⟨δ⟩ = 11⟨δ(1, 1) = r11⟩11⋯11⟨δ(q, s) = rqs⟩11 ⟨q0⟩ = 0q0 integer ⟨F⟩ = 1⟨f1⟩1⟨f2⟩1⋯1⟨fk⟩1 integers separated by 1s

11 / 17

Explicit representation for a DFA

Consider M = (Q, Σ, δ, q0, F) where Q = {1, 2, . . . , m}, Σ = {1, 2, . . . , n}, and F = {f1, f2, . . . , fk} Build it up piece by piece using the alphabet Σ = {0, 1} ⟨k⟩ = 0k integers ⟨Q⟩ = 1⟨1⟩1⟨2⟩1⋯1⟨m⟩1 integers separated by 1s = 101001⋯10m1 ⟨Σ⟩ = 1⟨1⟩1⟨2⟩1⋯1⟨n⟩1 integers separated by 1s = 101001⋯10n1 ⟨δ(q, t) = r⟩ = ⟨q⟩1⟨t⟩1⟨r⟩ = 0q10t10r ⟨δ⟩ = 11⟨δ(1, 1) = r11⟩11⋯11⟨δ(q, s) = rqs⟩11 ⟨q0⟩ = 0q0 integer ⟨F⟩ = 1⟨f1⟩1⟨f2⟩1⋯1⟨fk⟩1 integers separated by 1s ⟨M⟩ = 111⟨Q⟩111⟨Σ⟩111⟨δ⟩111⟨q0⟩111⟨F⟩111

11 / 17

Explicit representation

Consider the simple DFA: 1 2 1 2 1 2 ⟨Q⟩ = 1 0 1 00 1 ⟨Σ⟩ = 1 0 1 00 1 ⟨δ⟩ = 11 01010 11 0100100 11 0010100 11 0010010 11 ⟨q0⟩ = 0 ⟨F⟩ = 1 00 1 ⟨M⟩ = 111 101001

• Q

111 101001

• Σ

111 110101011010010011001010011001001011

• δ

111 0

• q0

111 1001

• F

111

12 / 17

Representations of other objects

We’re not going to bother giving explicit representations of other objects But we could if we really needed to and that’s enough for our purposes

13 / 17

Taking representations as inputs to TMs

We’re going to be dealing with languages like ADFA = {⟨M, w⟩ ∣ M is a DFA, w is a string and w ∈ L(M)} This language is related to the acceptance problem for DFAs which involves testing if the DFA M accepts a string w

14 / 17

Taking representations as inputs to TMs

We’re going to be dealing with languages like ADFA = {⟨M, w⟩ ∣ M is a DFA, w is a string and w ∈ L(M)} This language is related to the acceptance problem for DFAs which involves testing if the DFA M accepts a string w

Theorem

The language ADFA is decidable.

14 / 17

Taking representations as inputs to TMs

We’re going to be dealing with languages like ADFA = {⟨M, w⟩ ∣ M is a DFA, w is a string and w ∈ L(M)} This language is related to the acceptance problem for DFAs which involves testing if the DFA M accepts a string w

Theorem

The language ADFA is decidable.

Proof.

Let’s build a TM to decide ADFA. M = “On input ⟨B, w⟩ where B is a DFA and w is a string,

1 Simulate B on input w. 2 If the simulation ends in an accept state, accept. If it ends in a nonaccepting

state, reject.”

14 / 17

Some notes about that construction

First, the form of the construction: M = “On input ⟨ ⟩,

1 First step. 2 Second step. 3 Last step.”

Your constructions should follow exactly this form

15 / 17

Some notes about that construction

First, the form of the construction: M = “On input ⟨ ⟩,

1 First step. 2 Second step. 3 Last step.”

Your constructions should follow exactly this form Second, note that the input to this TM is supposed to be a representation of a DFA and a string The representation is not the same thing as the mathematical object If M is a DFA, then ⟨M⟩ is a string TMs take strings as input, nothing else

15 / 17

How does this construction actually work?

First, the TM M is going to check that the input is a valid representation of a DFA and a string Which representation is that? Then, it needs to simulate the DFA by keeping track of the DFA’s initial state and how much of the input it has read so far Initially, the DFA starts in its initial state and it has read none of the input so far The simulation proceeds step by step where M needs to compare the current state and the next input symbol to the representation of δ to find the next state

16 / 17

Next time

We’ll discuss some more decidable languages related to

• the acceptance problem for other models of computation
• The emptiness problem for models of computation (E.g., is the language of a DFA

the empty language?)

• The equivalence problem for models of computation (E.g., do two DFAs have the

same language?) Later, we’ll discuss some related problems (and languages) that are not decidable!

17 / 17