The Halting Problem Theorem. There is no algorithm for deciding - PDF document

✬ ✩ ✬ ✩ 2001 1 2001 2 The Halting Problem Theorem. There is no algorithm for deciding whether a computer program ever halts. Information-theoretic proof. Without restricting the generality assume that all programs Notes on the Complexity incorporate inputs – which are coded as natural numbers. So, a program may run forever or may of Algorithms just eventually stop, in which case it prints a natural number. Professor Cristian S. Calude Email: cristian@cs.auckland.ac.nz ✫ ✪ ✫ ✪ ✬ ✩ ✬ ✩ 2001 3 2001 4 Assume that there exists a halting program deciding whether an arbitrary program will ever halt. Construct the following program: The above program halts for every natural N . How long is it? It is about log N bits. Reason: to know N we need log 2 N bits (in binary); the rest 1. read a natural N ; of the program is a constant, so our program is 2. generate all programs up to N bits in log N + O(1) bits. size; Observe that there is a big difference between the size – in bits – of our program and the size of the 3. use the halting program to check for output produced by this program . Indeed, for large each generated program whether it enough N , our program will belong to the set of halts; programs having less than N bits. Hint: 4. simulate the running of the above log N + O(1) < N . generated programs, and 5. output the double of the biggest value output by these programs. ✫ ✪ ✫ ✪

✬ ✩ ✬ ✩ 2001 5 2001 6 Instantaneous Coding Consider two alphabets Y = { y 1 , y 2 , ..., y N } , Accordingly, the program will be generated by itself – at some stage of the computation. In this A = { a 1 , a 2 , ..., a Q } , case we have got a contradiction since our having program will output a natural number two times 2 ≤ Q < N bigger than the output produced by itself! elements. Hence, the hypothesis regarding the existence of A (finite) code is an injective function the halting program is false. ϕ : Y → A ∗ . An instantaneous code is a code for which ϕ ( Y ) is prefix-free (a set is prefix-free if no string in the set is a proper prefix of a different string in the set). ✫ ✪ ✫ ✪ ✬ ✩ ✬ ✩ 2001 7 2001 8 Example: Kraft’s Theorem. Y = { y 1 , y 2 , y 3 , y 4 } , A = { 0 , 1 } Let ( n i ) i = 1 , 2 , . . . , N be positive integers. These numbers are the lengths of the code–strings of an instantaneous code ϕ : Y → A ∗ iff y 1 y 2 y 3 y 4 ϕ 1 : 00 01 10 11 N � Q − n i ≤ 1 . ϕ 2 : 10 110 1110 11110 i =1 ϕ 3 : 10 10 110 1110 ϕ 4 : 01 011 0111 01111 ✫ ✪ ✫ ✪

✬ ✩ ✬ ✩ 2001 9 2001 10 Proof. Let ϕ : Y → A ∗ be an instantaneous code with Dividing the inequality | ϕ ( y i ) | = n i , 1 ≤ i ≤ N. r m ≤ Q m − n 1 Q m − 1 − r 2 Q m − 2 − · · · − r m − 1 Q Let r i = the number of the code–strings of length by Q m , we get i . Clearly, r j = 0 for j > m = max { n 1 , . . . , n N } , so m � r i Q − i ≤ 1 . r 1 ≤ Q, i =1 ( Q − r 1 ) Q = Q 2 − r 1 Q, r 2 ≤ Finally, (( Q − r 1 ) Q − r 2 ) Q = Q 3 − r 1 Q 2 − r 2 Q, r 3 ≤ m N . . � r i Q − i � Q − n j . = ≤ 1 . Q m − r 1 Q m − 1 − r 2 Q m − 2 − · · · − r m − 1 Q i =1 j =1 r m ≤ ✫ ✪ ✫ ✪ ✬ ✩ ✬ ✩ 2001 11 2001 12 So, Conversely, r 1 ≤ Q, r 2 ≤ ( Q − r 1 ) Q, m r i Q − i ≤ 1 , . � r 1 Q − 1 ≤ . . i =1 Q m − r 1 Q m − 1 − · · · − r m − 1 Q, r m ≤ m r 1 Q − 1 + r 2 Q − 2 r i Q − i ≤ 1 , � ≤ showing that we have enough elements to i =1 . construct the instantaneous code. . . The inequality m N r 1 Q − 1 + r 2 Q − 2 + · · · + r m Q − m ≤ r i Q − i ≤ 1 . � � Q − n i ≤ 1 i =1 i =1 is called Kraft’s inequality . ✫ ✪ ✫ ✪

✬ ✩ ✬ ✩ 2001 13 2001 14 A Chaitin computer or simply a computer is a Theorem. There exists a p.c. function p.c. (partially computable) function → A ∗ having a prefix–free domain. o C : A ∗ o F : N + × A ∗ → A ∗ A universal (Chaitin) computer is a (Chaitin) computer such that for every p.c. function o U : A ∗ → A ∗ o ϕ : A ∗ → A ∗ o A ∗ → A ∗ such that for every computer C : there exists a constant c — depending upon U with prefix–free domain there exists an i ∈ N + and C — such that if C ( x ) = y, then there exists such that x ′ ∈ A ∗ such that F ( i, x ) = ϕ ( x ) , • U ( x ′ ) = y, • | x ′ | ≤ | x | + c. for all x ∈ A ∗ . ✫ ✪ ✫ ✪ ✬ ✩ ✬ ✩ 2001 15 2001 16 Program-size complexity induced by a computer o C : A ∗ → A ∗ Theorem. There exists a universal computer. is the function Proof: Let H C ( x ) = min { | u | | C ( u ) = x } , ( min ∅ = ∞ ) . o F : N + × A ∗ A ∗ → Fix a fixed universal computer U and put H ( x ) = H U ( x ). be universal p.c. function for all Chaitin computers, and define The canonical program corresponding to x is U ( a i 1 a 2 x ) = F ( i, x ) , x ∗ = min { u ∈ A ∗ | U ( u ) = x } , i ≥ 1 , x ∈ A ∗ . where min is taken according to the quasi-lexicographical order. ✫ ✪ ✫ ✪

✬ ✩ ✬ ✩ 2001 17 2001 18 Theorem (Invariance). For every computer C there exists a constant c such that Lemma. For all x ∈ A ∗ x ∗ � = λ. x ∗ 1) exists , H ( x ) ≤ H C ( x ) + c, x = U ( x ∗ ) , 2) H ( x ) = | x ∗ | . for all x ∈ A ∗ . 3) For all universal computers U, U ′ Corollary. For all x ∈ A ∗ , Theorem. there exists a constant c > 0 such that 0 < H ( x ) < ∞ . | H U ( x ) − H U ′ ( x ) | ≤ c, for all x ∈ A ∗ . ✫ ✪ ✫ ✪ ✬ ✩ ✬ ✩ 2001 19 2001 20 o Theorem (Kraft-Chaitin). Let ϕ : N + → N be a p.c. function having as domain an initial segment of N + . The following two statements are equivalent: (1) We can effectively construct an injective p.c. The set of canonical programs is function CP = { x ∗ | x ∈ A ∗ } . θ : dom ( ϕ ) → A ∗ such that: Theorem. The set of canonical programs is a) for every n ∈ dom ( ϕ ) , | θ ( n ) | = ϕ ( n ) , immune, i.e. it is infinite and has no infinite c.e. subset. b) range ( θ ) is prefix-free. (2) One has: Q − ϕ ( i ) ≤ 1 . � i ∈ dom ( ϕ ) ✫ ✪ ✫ ✪

✬ ✩ ✬ ✩ 2001 21 2001 22 Proof. The set CP is clearly infinite, as the function x → x ∗ is injective. We can construct a computer C such that for We proceed now by contradiction, starting with every i ≥ 1, there exists a string u such that the assumption that there exists an infinite c.e. C ( u ) = g ( i ) and set S ⊂ CP . Let S be enumerated by the | u | ≤ log i + 2 log log i + 1 ≤ 3 log i. injective computable function f : N → A ∗ . Define the function g : N → A ∗ by By the Invariance Theorem we get a constant c 1 such that for all i ∈ N , g (0) = f (0) , g ( n + 1) = f (min j [ | f ( j ) | > n + 1]) . H ( g ( i )) ≤ H C ( g ( i )) + c 1 ≤ 3 log i + c 1 . It is straightforward to check that g is (total) computable, S ′ = g ( N + ) is c.e., infinite, S ′ ⊂ S and | g ( i ) | > i , for all i > 0. ✫ ✪ ✫ ✪ ✬ ✩ ✬ ✩ 2001 23 2001 24 Take x = y ∗ , z = x ∗ . One has: We continue with a result which is interesting in D ( z ) = U ( U ( z )) = U ( U ( x ∗ )) = U ( x ) = U ( y ∗ ) = y, itself: Intermediate Step. There exists a constant so c 2 ≥ 0 such that for every x in CP , one has: H D ( y ) ≤ H ( x ) , H ( x ) ≥ | x | − c 2 . | x | = | y ∗ | = H ( y ) ≤ H D ( y ) + c 2 ≤ H ( x ) + c 2 . Construct a computer For i ≥ 1, if g ( i ) ∈ CP , then | g ( i ) | > i , so D ( u ) = U ( U ( u )) i − c 2 < | g ( i ) | − c 2 ≤ H ( g ( i )) ≤ 3 log i + c 1 , and pick the constant c 2 coming from the and consequently only a finite number of elements Invariance Theorem (applied to U and D ). in S ′ can be in CP . ✫ ✪ ✫ ✪

✬ ✩ ✬ ✩ 2001 25 2001 26 Proof. We have to prove that the “approximation from above” of the graph of H ( x ), i.e. the set { ( x, n ) | x ∈ A ∗ , n ∈ N , H ( x ) < n } is c.e. This is easy since H ( x ) < n iff there exist y ∈ A ∗ and Corollary. The function t ∈ N such that | y | < n and U ( y ) = x in at most t f : A ∗ → A ∗ , f ( x ) = x ∗ is not computable. steps. Proof. The function f is injective and its range is exactly CP . For the second part of the theorem we prove a bit more, namely: Theorem. The program-size complexity H ( x ) is semi-computable from above, but not computable. o There is no p.c. function ϕ : A ∗ Claim. → N with infinite domain and such that H ( x ) = ϕ ( x ), for all x ∈ dom ( ϕ ). ✫ ✪ ✫ ✪ ✬ ✩ ✬ ✩ 2001 27 2001 28 Assume, by absurdity, that H ( x ) = ϕ ( x ), for all x ∈ dom ( ϕ ), where ϕ : A ∗ o For infinitely many i > 0 , → N is a p.c. function with an infinite domain. Let B ⊂ dom ( ϕ ) be a H ( f ( a i 1 a 2 )) ≥ Q i . → A ∗ be the computable, infinite set and let f : A ∗ o partial function given by (Recall that C ( a i 1 a 2 ) = f ( a i 1 a 2 ) is a computer.) Accordingly, in view of the Invariance Theorem, f ( a i 1 a 2 ) = min { x ∈ B | H ( x ) ≥ Q i } , i ≥ 1 . for infinitely many i > 0, we have: Since ϕ ( x ) = H ( x ), for x ∈ B , it follows that f is Q i ≤ H ( f ( a i 1 a 2 )) ≤ H C ( f ( a i 1 a 2 )) + c ≤ i + 1 + c. a p.c. function. Moreover, f has a computable graph and f takes as values strings of arbitrarily This yields a contradiction. long length. ✫ ✪ ✫ ✪