Is there always an algorithm? An introduction to computability - - PowerPoint PPT Presentation

Is there always an algorithm? An introduction to computability theory.

Computer Science Department Seminar 9th October 2009 Rick Thomas

1

Equations

If we want to solve the quadratic equation ax2 +bx+c = 0, we know that the solutions are −b+√

b2−4ac 2a

and −b−√

b2−4ac 2a

. Question: Can we do the same for cubic equations? What about quartic equations? Higher degrees? For cubic equations the first advances were by Scipione dal Ferro (1465-1526): he could solve equations of the form x3 +mx = n (where m and n are positive). Just before his death he revealed his method to his student Antonio Fior.

2

Cubic Equations

Nicolo of Brescia (known as “Tartaglia”) managed to solve equations

• f the form x3 +mx2 = n.

Fior challenged Tartaglia to a public contest. Eight days before the problems were to be collected, Tartaglia found the general method for solving all cubics. For more information about the topics discussed here, see the MacTutor History of Mathematics archive: http://www-history.mcs.st-and.ac.uk/

3

Nicolo Tartaglia (1500 - 1557)

4

Tartaglia and Cardano

News of Tartaglia’s victory reached Girolamo Cardano in Milan. Cardano invited Tartaglia to visit him; Tartaglia eventually revealed his method, but made Cardano promise to keep it secret until Tartaglia had published it himself. In 1545 Cardano published Ars Magna, the first Latin treatise on algebra, containing Tartaglia’s method.

5

Girolamo Cardano (1501 - 1576)

6

Quartic Equations

Cardano then encouraged his own student, Lodovico Ferrari (1522 - 1565), to examine quartic equations. Ferrari managed to solve the quartic. Cardano published the method in Ars Magna. But . . . In 1823 Niels Abel proved that there is no such method using the given ingredients for the degree five case (the quintic).

7

Niels Abel (1802 - 1829)

8

The General Situation

Evariste Galois then gave a general method for deciding whether or not a given degree n equation admits a solution of this type: “On the condition that an equation be soluble by radicals.” This research essentially introduced the idea of a “group” into Mathematics.

9

Evariste Galois (1811 - 1832)

10

Algorithms

David Hilbert’s famous address “Mathematical Problems” was delivered to the Second International Congress of Mathematicians in Paris in 1900. One question was the issue as to whether Mathematics could be finitely “mechanized”. Can we create a machine into which one feeds a statement about Mathematics and which delivers an answer: true or false? Given a formal system, can we produce a machine that would eventually output all the “truths” of that system?

11

David Hilbert (1862 - 1943)

12

Hilbert’s Tenth Problem

A Diophantine Equation is a polynomial equation where we insist that the solutions be integers. “Determination of the solvability of a Diophantine equation. Given a Diophantine equation with any number of unknown quantities and with rational integral numerical coefficients: to devise a process according to which it can be determined by a finite number of operations whether the equation is solvable in rational integers.”

13

G¨

• del’s Incompleteness Theorem

A set of rules is complete if, for any statement in the relevant language, either that statement or its negation is provable from the rules. A set of rules is consistent if there is no statement so that both the statement and its negation are provable from the rules. Any sufficiently rich formal system cannot be both consistent and complete. In particular, for any sufficiently rich formal system, there is a statement that is true but not provable in the system.

14

Kurt G¨

• del (1906 - 1978)

15

A Mechanical Approach?

Around the same time, various people were working on formalizing what we might mean by a “mechanical method.” There were various models proposed, such as the lambda calculus (by Alonzo Church) . . .

• If x is a variable, then x ∈ Λ;
• If x is a variable and M ∈ Λ, then (λx.M) ∈ Λ;
• If M,N ∈ Λ, then (MN) ∈ Λ.

. . . and partial recursive functions (by Stephen Kleene) . . .

• Functions built up from certain elementary functions by

composition, primitive recursion and minimization.

16

Alonzo Church (1903 - 1995)

17

Stephen Kleene (1909 - 1994)

18

The Turing Machine

Alan Turing considered a very simple mechanized device. We have input, memory, a finite collection of states and transitions.

Input goes here # This is used as a work tape Finite state control

19

Alan Turing (1912 - 1954)

20

Church-Turing Thesis

Church-Turing Thesis. Any algorithmic problem for which there exists an algorithm that can be programmed in some programming language on some computer is also solvable by a Turing machine. Turing’s fundamental paper was part of the inspiration for the first computers, and strongly influenced other researchers such as John von Neumann.

21

John von Neumann (1903 - 1957)

22

Longevity of the Model

Where a calculator on the ENIAC is equipped with 18,000 vacuum tubes and weighs 30 tons, computers in the future may have only 1,000 vacuum tubes and perhaps weigh 1.5

• tons. Popular Mechanics, March 1949

However, the Turing Machine models the limits of the power of computers today (just as it did in the 1940’s). We can show that there is no algorithm to decide the following question. INPUT: A Turing machine (program) M and an input α. QUESTION: Does the machine (program) M halt on input α?

23

Halting Problem

Suppose we had a program H as in the figure below that would accept a (listing of a) program M and the input α for M, and decide whether

• r not M would halt with input α.

M, α

AAAAA AAAAA AAAAA

H Y M halts with input α N

M doesn't halt with input α

24

Halting Problem

We could then use H to construct another program J as in the figure below which would go into an infinite loop if M halted with input α and would terminate otherwise.

M, α

AAAAA AAAAA AAAAA

J ∞ loop M halts with input α N

M doesn't halt with input α

25

Halting Problem

If J existed, we could then construct a simpler program K as in the figure below which performs in the same way as J but with α equal to (the listing of) M.

M

AAAAA AAAAA AAAAA AAAAA

K ∞ loop M halts with input M N

M doesn't halt with input M

26

Halting Problem

We can now input (the listing of) K into K as in the figure below.

AAAAA AAAAA

K ∞ loop K halts with input K N

K doesn't halt with input K

K

Now comes the fun bit!

27

Halting Problem

AAAAA AAAAA

K ∞ loop K halts with input K N

K doesn't halt with input K

K

This gives a contradiction: we now have that K halts with input K if it doesn’t and K doesn’t halt with input K if it does! So the original machine H can’t exist, i.e. the halting problem is unsolvable.

28

Undecidability

It is important to appreciate what the term “undecidable” means. To say that a problem is undecidable means that there is no way, even given unlimited resources (in terms of time and space), that the problem can be decided by algorithmic means. Some more examples of undecidable problems:

• Given a program P, is there any input for which P terminates?
• Given two programs P and Q, are they equivalent (i.e. do they

produce the same output for any given input)?

29

Languages

We can think of the valid terms derived from some axiomatic system as being a language. If we have some finite set of symbols, then some strings of symbols will represent valid terms and some will not. A language that is decidable is often said to be “recursive”: A language L is recursive if there is a Turing machine that, given an input α, will output “yes” if α ∈ L and “no” if α ∈ L. It is important to distinguish this from the concept of a language being recursively enumerable: A language L is recursively enumerable if there is a Turing machine that, given an input α, will output “yes” if α ∈ L.

30

Diophantine Equations

If we have a finite axiomatic system then, in general, the set of valid terms will be recursively enumerable. However, it may well not be recursive. Let us reconsider Hilbert’s tenth problem: “Determination of the solvability of a Diophantine equation. Given a diophantine equation with any number of unknown quantities and with rational integral numerical coefficients: to devise a process according to which it can be determined by a finite number of operations whether the equation is solvable in rational integers.”

31

Diophantine Equations

It is not difficult to see that the set of solutions to a Diophantine equation is recursively enumerable. We call such a set Diophantine. Building on work of Julia Robinson, Martin Davis and Hilary Putnam, in 1970 Yuri Matiyasevich showed that the converse is also true (i.e. that every recursively enumerable set is Diophantine). As a result, we see that Hilbert’s tenth problem is unsolvable.

32

Julia Robinson (1919 - 1985) and Yuri Matiyasevich (1947 - )

33

Back to Program Termination

No one knows whether or not the following simple program always terminates (where x is a positive integer): read(x) : while x = 1 do if x is even then x := x/2 else x := 3x+1; end; Of course, someone might decide this at some point (though it has been an open question for a very long time). The set of values x for which the program terminates is recursively enumerable.

34

Back to Program Termination

We have defined a function: f(x) =

x 2

if x is even; f(x) = 3x+1 if x is odd. We can define other functions of this form - for example: g(x) = x

2

if x ≡ 0 mod 4; g(x) = 2x if x ≡ 1 mod 4; g(x) = x+2

2

if x ≡ 2 mod 4; g(x) = 3x+1 if x ≡ 3 mod 4. John Conway showed that there is no general algorithm to decide the following question: INPUT: A function of the above form and a positive integer x. QUESTION: Does x always get back to 1?

35

John Conway (1937 - )

36

Wang Tiles

INPUT: A set of square coloured tiles of the same size. QUESTION: Can we tile the plane with these tiles so that the colours

• f adjoining edges always match? (The tiles cannot be rotated or

reflected.) This problem is also undecidable.

37

Monoids

In many cases, we can describe aspects of computational systems by sequences of actions. Sometimes two sequences of actions will have the same effect on the system. We can describe this general situation as follows:

• We have a finite set of actions.
• We have equations equating sequences of actions.

A general question is whether, given such a system, are two sequences

• f actions equivalent?

The set of all such sequences (up to equivalence) forms a monoid. The equivalence question is known as the word problem for monoids

38

Monoids

Not surprisingly, the word problem for monoids is undecidable in general. Are there natural classes of monoids where we can decide the word problem? Groups form a (very special) subclass of the class of monoids. One very successful idea there is the notion of a hyperbolic group. The word problem for a hyperbolic group is not only decidable; it can be solved in time O (n).

39

Hyperbolic Groups

40

Hyperbolic Monoids

The notion of a hyperbolic group has now been generalized to monoids. In the original approach (proposed by Duncan and Gilman) the best known algorithm for the word problem had time complexity O (2n).

(Hoffmann, Kuske, Otto & Thomas, 2002)

A new approach gives a class of monoids which naturally extends the definition for groups and where the word problem can be solved in time

O (nlogn). (Hoffmann & Thomas, 2007)

41