XI. Computational Complexity Yuxi Fu BASICS, Shanghai Jiao Tong - PowerPoint PPT Presentation

XI. Computational Complexity Yuxi Fu BASICS, Shanghai Jiao Tong University

Mathematic proofs, like computations, are energy consuming business. There are mathematical theorems, and computational tasks, that require more resources than what are available to us. A mathematical proof, and a program as well, must be short enough to be practically relevant. Computability Theory, by Y. Fu XI. Computational Complexity 1 / 34

In real world we are not only interested in if a problem is solvable but also how costly it is to solve the problem. Computability Theory, by Y. Fu XI. Computational Complexity 2 / 34

Synopsis 1. Complexity Measure 2. Blum’s Speedup Theorem 3. Gap Theorem Computability Theory, by Y. Fu XI. Computational Complexity 3 / 34

1. Complexity Measure Computability Theory, by Y. Fu XI. Computational Complexity 4 / 34

Time appears to be the best criterion for the amount of energy necessary to execute a program. Computability Theory, by Y. Fu XI. Computational Complexity 5 / 34

Time Function Given a program P , the time function t ( n ) P ( � x ) is defined by t ( n ) P ( � x ) = µ t . ( P ( � x ) terminates in t steps ) . We write t ( n ) x ) for t ( n ) e ( � P e ( � x ). Remark: ◮ The time function is computable since‘ P ( � x ) ↓ in t steps’ is a primitive recursive predicate. ◮ We shall omit the superscript ( n ) when n = 1. Computability Theory, by Y. Fu XI. Computational Complexity 6 / 34

Time Function Fact . (i) dom ( t ( n ) x )) = dom ( φ ( n ) e ( � e ( � x )) for all n , e . (ii) The predicate “ t ( n ) e ( � x ) ≤ y ” is decidable for all n . Computability Theory, by Y. Fu XI. Computational Complexity 7 / 34

Blum Complexity Measure ( φ i , Φ i ) is a Blum complexity measure if the following hold: ◮ Φ i ( x ) is defined iff φ i ( x ) is defined. ◮ Φ i ( x ) ≤ n is decidable. Manuel Blum ◮ A Machine-Independent Theory of the Complexity of Recursive Functions. J. ACM 14:322-336, 1967. Computability Theory, by Y. Fu XI. Computational Complexity 8 / 34

We are mainly interested in asymptotic behaviour of time function. A predicate M ( n ) holds almost everywhere (a.e.) if M ( n ) holds for all but finitely many natural numbers n . Computability Theory, by Y. Fu XI. Computational Complexity 9 / 34

Theorem . Given a total computable function b ( x ), there is a total computable function f ( x ) with range { 0 , 1 } such that t e ( x ) > b ( x ) a.e. for every index e of f ( x ). There are arbitrarily complex time functions. Computability Theory, by Y. Fu XI. Computational Complexity 10 / 34

Define f so that it differs from every function in the sequence φ j 0 , φ j 1 , φ j 2 , . . . , φ j k , . . . . (1) A function φ i appears in (1) if φ i ( m ) ≤ b ( m ) for infinitely many m . Suppose f (0) , f (1) , . . . , f ( n − 1) have been defined. Let i n be µ i . ( i ≤ n , t i ( n ) ≤ b ( n ) , i is not yet defined ) and let f ( n ) be defined by � 1 , if i n is defined and φ i n ( n ) = 0 , f ( n ) = 0 , otherwise . 1. If φ e = f , then e � = i n whenever i n is defined. 2. If t i ( m ) ≤ b ( m ) for infinitely many m then i = i n for some n . Computability Theory, by Y. Fu XI. Computational Complexity 11 / 34

2. Blum’s Speedup Theorem Computability Theory, by Y. Fu XI. Computational Complexity 12 / 34

Is there always a best program that solves a problem? Blum’s Speedup Theorem says that the answer is negative. Computability Theory, by Y. Fu XI. Computational Complexity 13 / 34

Speedup Theorem Lemma . Let r be a total computable function. There is a total computable function f such that given any program P i for f we can construct effectively a program P j with the following properties: 1. φ j is total and φ j ( x ) = f ( x ) a.e.. 2. r ( t j ( x )) < t i ( x ) a.e.. Computability Theory, by Y. Fu XI. Computational Complexity 14 / 34

Proof of the Lemma By S-m-n Theorem, there is a total computable function s st φ s ( e , u ) ( x ) ≃ φ (2) e ( u , x ) . (2) We will construct some e using Recursion Theorem such that φ (2) e ( u , x ) ≃ g ( e , u , x ) , (3) where g ( e , u , x ) is obtained by the diagonalisation construction described on the next slide. Computability Theory, by Y. Fu XI. Computational Complexity 15 / 34

Proof of the Lemma Suppose some finite sets of canceled indices C e , u , 0 , . . . , C e , u , x − 1 have been defined. If t s ( e , i +1) ( x ) is defined for all i ∈ { u , . . . , x − 1 } , then let � � � C e , u , x = i | u ≤ i < x , t i ( x ) ≤ r ( t s ( e , i +1) ( x )) \ C e , u , y ; y < x otherwise let C e , u , x be undefined. Clearly C e , u , x is computable, and if i ∈ C e , u , x then φ i ( x ) ↓ . Now g ( e , u , x ) is defined by � 1 + max { φ i ( x ) | i ∈ C e , u , x } , if C e , u , x is defined , g ( e , u , x ) = ↑ , otherwise . Computability Theory, by Y. Fu XI. Computational Complexity 16 / 34

Proof of the Lemma C e , 0 , 0 , C e , 0 , 1 , C e , 0 , 2 , . . . , C e , 0 , x , C e , 0 , x +1 , . . . ∪ ∪ ∪ ∪ ∪ C e , 1 , 0 , C e , 1 , 1 , C e , 1 , 2 , . . . , C e , 1 , x , C e , 1 , x +1 , . . . ∪ ∪ ∪ ∪ ∪ . . . . . . . . . . . . . . . ∪ ∪ ∪ ∪ ∪ C e , x , 0 , C e , x , 1 , C e , x , 2 , . . . , C e , x , x , C e , x , x +1 , . . . ∪ ∪ ∪ ∪ ∪ . . . . . . . . . . . . . . . The unary function g ( e , u , ) is defined to differ from all functions eliminated in � x ∈ ω C e , u , x . Computability Theory, by Y. Fu XI. Computational Complexity 17 / 34

Proof of the Lemma For each pair of e , x we show that g ( e , u , x ) is total. ◮ If u ≥ x , then C e , u , x = ∅ and consequently g ( e , u , x ) = 1. ◮ Suppose u < x and g ( e , x , x ), . . . , g ( e , u + 1 , x ) are defined. ◮ φ s ( e , x ) ( x ), . . . , φ s ( e , u +1) ( x ) are defined according to (3). ◮ Hence t s ( e , x ) ( x ), . . . , t s ( e , u +1) ( x ) are defined. ◮ It follows that C e , u , x is defined. ◮ Consequently g ( e , u , x ) is also defined. This completes the downward induction. We conclude that g is a total function, which implies that t s ( e , u +1) ( x ) is always defined. Computability Theory, by Y. Fu XI. Computational Complexity 18 / 34

Proof of the Lemma Fact . Some v exists such that φ 2 e (0 , x ) = φ 2 e ( u , x ) for all x > v . Proof. By definition C e , u , x = C e , 0 , x ∩ { u , u + 1 , . . . , x − 1 } . Let v = max { x | C e , 0 , x contains an index i < u } . It is clear that C e , 0 , x ⊆ { u , u + 1 , . . . , x − 1 } for all x > v . Hence C e , 0 , x = C e , u , x for all x > v . Computability Theory, by Y. Fu XI. Computational Complexity 19 / 34

Proof of the Lemma Fact . If φ i = φ 2 e (0 , x ), then r ( t s ( e , i +1) ( x )) < t i ( x ) a.e.. Proof. Let i be an index for φ 2 e (0 , x ). It should be clear that for all x > i , the following holds: r ( t s ( e , i +1) ( x )) < t i ( x ) . If not, i would have been canceled at some stage, say i ∈ C e , 0 , w . But then φ i ( w ) � = g ( e , 0 , w ) by definition. That is φ i ( w ) � = φ 2 e (0 , w ), contradicting to the assumption. Computability Theory, by Y. Fu XI. Computational Complexity 20 / 34

Proof of the Lemma Let f ( x ) = φ (2) e (0 , x ). We have proved that f satisfies the property stated in the lemma. Computability Theory, by Y. Fu XI. Computational Complexity 21 / 34

Theorem (Blum, 1967). Let r be a total computable function. There is a total computable function f such that given any program P i for f there is another program P j for f satisfying r ( t j ( x )) < t i ( x ) a.e.. Computability Theory, by Y. Fu XI. Computational Complexity 22 / 34

W.l.o.g. assume that r is increasing. By a slight modification of the proof of the lemma, we get a total computable function f such that given any program P i for f there is a program P k for f satisfying the following: ◮ φ k ( x ) is total and φ k ( x ) = f ( x ) a.e.. ◮ r ( t k ( x ) + x ) < t i ( x ) a.e.. Some c exists such that φ k ( x ) = f ( x ) whenever x > c . We get a program P j from P k by short-cutting computations at inputs ≤ c . If c is large enough such that the additional computation cost is less than c , then the program P j satisfies r ( t j ( x )) < t i ( x ) a.e.. Computability Theory, by Y. Fu XI. Computational Complexity 23 / 34

We cannot define time complexity for problems. We can however define time complexity for solutions. Computability Theory, by Y. Fu XI. Computational Complexity 24 / 34

Hartmanis and Stearns’ Linear Speedup Theorem Theorem (Hartmanis and Stearns, 1965). If L is decidable in T ( n ) time, then for any ǫ > 0 it is decidable in ǫ T ( n ) + n + 2 time. Computability Theory, by Y. Fu XI. Computational Complexity 25 / 34