Complexity of Well-Quasi-Orderings and Well-Structured Transition - - PowerPoint PPT Presentation

Complexity of Well-Quasi-Orderings and Well-Structured Transition Systems

Part II: Fast-Growing Complexity

Philippe Schnoebelen LSV, CNRS & ENS Cachan + Oxford 1-year visitor Oxford Dept. Comp. Sci, Feb. 23rd, 2012

IF YOU MISSED PART I

(X,) is a well-quasi-ordering (a WQO) if any infinite sequence x0,x1,x2 ... over X contains an increasing pair xi xj (for some i < j) Examples.

• 1. (Nk,prod) is a WQO (Dickson’s Lemma)

where, e.g., 3,2,1 5,2,2 but 1,2,3 5,2,2

• 2. (Σ∗,⊑) is a WQO (Higman’s Lemma)

where, e.g., abc ⊑ bacbc but cba bacbc Intuition motivating this course: Analyzing the complexity of algorithms based on WQO-theory ≃ Bounding the index j (in the increasing pair above) as a function of some relevant parameters

2/17

IF YOU MISSED PART III

• Def. A sequence x0,x1,... over X is bad

def

⇔ there is no increasing pair

• NB. Over a WQO, a bad sequence is necessarily finite

Length Function Theorem. Any bad sequence x0,x1,...,xl over X has length l L(X,...) Next Talk: Definition of L(X,...) and proof of the Length Function Theorem Plan for this talk:

• 1. Some intuition of why L can be big, using examples of bad

sequences

• 2. The Fast-Growing Hierarchy (where L lives)
• 3. A bit of context: who else is lost up there?

3/17

MAKING LONG BAD SEQUENCES

Consider X = N and start with x0 = 5 (for example)

• Puzzle. Longest bad sequence?

5,4,3,2,1,0 has length 6 Now start with some other x0, say “m” Longest bad sequence is m,m − 1,m − 2,...,1,0 has length m + 1

• Conclusion. There exist arbitrarily long bad sequences (if one can

choose the starting point x0)

• Conclusion. x0 must be a parameter : l L(X,x0,...)

4/17

MAKING LONG BAD SEQUENCES —2

Consider now X = N2 and fix x0 = 5,2 Some possible bad sequences: 5,2, 6,1, 7,0, 6,0, 5,0, 4,0, 3,0, 2,0, 1,0, 0,0 5,2, 6,1, 7,0, 6,0, 4,999999, ··· 4,0, 3,99..9999999, ···

• Note. Sequence can be made arbitrarily long even with fixed x0.

Note also. Not only the length can be chosen arbitrarily large, but

• ne can choose several times, and arbitrarily late, for arbitrarily large

remaining sequence.

• Conclusion. Choosing arbitrarily large x0 (at start) or xk+1 (after

some xk along the sequence) is not effective. Such arbitrarily large choices cannot be performed by a nondeterministic algorithm (or said “algorithm” may fail to terminate).

• Conclusion. The (class of) processes that generate the long bad

sequence must be a parameter : l L(X,x0,C).

5/17

MAKING LONG BAD SEQUENCES —3

Consider X = Nk and assume that the sequence is generated by a counter machine M with k counters using simple instructions like c:=c+1, c:=c-1 and c:=c’ M can be programmed to generate, e.g., a decreasing (hence bad) sequence for the lexicographic ordering: 5,2, 5,1, 5,0, 4,5, 4,4, 4,3, 4,2, 4,1, 3,4, ... 1,0, 0,1, 0,0 has length 23. Starting from x0 = a,b, M generates a bad sequence of length

1 2a2 + 3 2a + b + 1, i.e., O(n2) when n = |x0|.

In general, length is O(nk) for k counters. One can do much worse

6/17

MAKING LONG BAD SEQUENCES —4

Add one extra counter that keeps increasing. Use it when restarting a counter. 5,2,0, 5,1,1, 5,0,2, 4,2,3, 4,1,4, 4,0,5, 3,5,6, 3,4,7, 3,3,8, ... 3,0,11, 2,11,12, ... 2,0,23, 1,23,24, ... 1,0,47, 0,47,48, ... 0,0,95 More generally n,a,0, ... n,0,a, n − 1,a,a + 1, ... n − 1,0,2a + 1, n − 1,2a + 1,2a + 2, ... n − 2,0,4a + 3, ... 0,0,2n(a + 1) − 1 And now with one more counter: n,a,a,0, ... n,a,0,a, n,a − 1,a,a + 1, ... n,a − 2,2a + 1,2a + 2, ... n,0,0,2a(a + 1) − 1, n − 1,2a(a + 1) − 1,2a(a + 1) − 1,2a(a + 1), ... n − 2,0,0,> 22a+1, ... 0,0,0,> 22 . . .

a

• With 3 + 1 counters, M takes nonelementary time!

7/17

THE FAST-GROWING HIERARCHY

A.k.a. The (Extended) Grzegorczyk Hierarchy For α = 0,1,2,... define Fα : N → N with: F0(n)

def

= n + 1 (D1) Fα+1(n)

def

= Fn+1

α

(n) =

n+1 times

• Fα(Fα(...Fα(n)...))

(D2) This yields: F1(n) = 2n + 1 F2(n) = (n + 1)2n+1 − 1 and F3(n) > 22 . . .

2

n times F4 is . . . impossible to grasp intuitively (at least for me). Though each Fα is a simple 2-line program. Back to M and lexicographic descent: With k + 1 counters, M halts after ≈ Fk(n) steps, where n

def

= |x0|.

8/17

LONG BAD SEQUENCES OVER Σ∗ WITH SUBWORD

ORDERING

Consider now another WQO: (Σ∗,⊑), with Σ = {a,b} (say). How long can a bad sequence be? Start with x0 = ab We can make arbitrarily long bad sequences: ab,

p

• bb...bb

q

• aa...aa, ..

Here, and as with X = Nk before, it is unreasonable to allow choosing an arbitrarily large xk+1 (irrespectively of the start x0,...,xk) when continuing a bad sequence In reality, the sequence is generated by a machine that can only perform computations of “limited” complexity. For example “xk+1 ∈ Σ∗ is obtained from x0,...,xk in polynomial-time”

9/17

LONG BAD SEQUENCES OVER (Σ∗,⊑)

Start with x0 = aaabbba If we retain the 3 b’s, never adding adding/removing a b, the a’s can be used to simulate a sequence over N4: aaabbba encodes 3,0,0,1 aaabbba, aabababaa, aababbaaa, aabbaaabaaaa, . . . aabbbaaaaaaa

• 2,0,0,7

, abaaaaaaabaaaaaaabaaaaaaaa

• 1,7,7,8

,

• Note. The simulated sequence is bad (over N4) iff the “3-b’s”

sequence is bad (over Σ∗) More generally, if we start with x0 = abb..bb, this process M generates a bad sequence of length F|x0|−1(1). This complexity dominates any Fk(|x0|). In fact n → Fn(1) is Ackermann’s function, or its twin brother.

10/17

THE FAST-GROWING HIERARCHY —CONTINUED

Define Fα : N → N for α = 0, 1, 2, . . . , ω, ω + 1, . . . , ω · 2, ω · 2 + 1, . . . , ω2, ω2 + 1, . . . , ωω, ωω + 1, . . . [S.Wainer et al.] Recall that ordinals up to ε0 can be written in Cantor Normal Form (i.e., with 0, sum, and ω-exponentiation) α ≡ ωα1 + ··· + ωαp with α > α1 α2 ··· αp (and the αi’s themselves in normal form) α above is — 0, “zero”, iff p = 0, — a “successor” ordinal, “β + 1”, iff αp = 0, — a “limit” ordinal, “λ”, iff αp > 0, Every limit λ is the limit of an increasing sequence λ0 < λ1 < λ2 < ···

11/17

LIMIT ORDINALS AND FUNDAMENTAL SEQUENCES

We fix an “assignment of fundamental sequences” (λ) : n → λn and such that λ = limn∈N λn For λ = ω, the fundamental sequence is 1, 2, 3, ... For λ = ω · 2, the fundamental sequence is ω + 1, ω + 2, ω + 3, ... For λ = ωω5, the fundamental sequence is ωω4, ωω4·2, ωω4·3, ... For λ = ωωω + ω · 3, the fundamental sequence is ωωω + ω · 2 + 1, ωωω + ω · 2 + 2, ωωω + ω · 2 + 3, ... More formally, we let

• γ + ωβ+1

n

def

= γ + ωβ · (n + 1)

• γ + ωλ

n

def

= γ + ωλn This is a standard choice (and it can be extended beyond ε0)

12/17

THE FAST-GROWING HIERARCHY —CONTINUED

F0(n)

def

= n + 1 (D1) Fα+1(n)

def

= Fn+1

α

(n) =

n+1 times

• Fα(Fα(...Fα(n)...))

(D2) Fλ(n)

def

= Fλn(n) (D3) This gives: — Fω(n) = Fn+1(n). Fω is Ackermannian! — Fω+1(n) =

n+1 times

• Fω(Fω(...Fω(n)...))

— Fω·2(n) = Fω+n+1(n) =

n+1 times

• Fω+n(Fω+n(...Fω+n(n)...))

— Fωω(n) = Fωn+1(n) = Fωn·(n+1)(n) = Fωn·n+ωn−1·(n+1)(n) = ··· = Fωn·n+ωn−1·n+···+ω·n+n+1(n)

• Prop. Fα dominates all Fα′ s.t. α′ < α.

13/17

THE FAST-GROWING HIERARCHY —CONTINUED

From a hierarchy (Fα)α of fast-growing functions, one defines a hierarchy of complexity classes: Fα

def

=

• f
• f is computable in time/space < Fk

α(n) for some k

• Equivalently (for α > 1):

Fα contains all functions definable with compositions, projections, Succ, +, −, ×, F0, F1, . . . , Fα (⇔ all functions “elementary in Fα”)

• Prop. This constitues a strict, cumulative hierarchy: Fα Fα′ when

α < α′

14/17

THE FAST-GROWING HIERARCHY —CONTINUED

F0 = F1

(linear)

F2

(elementary)

F3

• k Fk

(primitive-recursive)

Fω

• k Fωk

(multiply-recursive)

Fωω

• ···
• ···
• ···

Prop.

α<ǫ0 Fα contains exactly the functions provably total in

Peano Arithmetic

15/17

BACK TO LONG BAD SEQUENCES OVER (Σ∗,⊑)

• Puzzle. Are there “simple” bad sequences whose length is not

bounded by Fω or perhaps Fω+1? Sure! Especially when using a larger alphabet Σ = {a0,...,ap} Say xi = (wi,ni) with wi ∈ Σ∗ and ni is an extra “counter” Build x0,x1,x2,... with the following simple process: – if xi = (ǫ,n), then halt! – if xi = (u a0,n) then let xi+1

def

= (u,n + 1) – if xi = (u al+1,n) then let xi+1

def

= (ual ...al

• n+1 times

,n + 1)

• Example. (a4 a2,0), (a4 a1,1), (a4 a0 a0,2), (a4 a0,3), (a4,4),

(a3 a3 a3 a3,5), (a3 a3 a3 a2 a2 a2 a2 a2,6),

• Prop. The sequence w0,w1,w2,... is bad. Its length is bounded by

Fωl(|w0| + n0).

16/17

WHO ELSE CARES ABOUT FAST-GROWING

FUNCTIONS?

Fast-growing functions are used by logicians (starting in the sixties) to prove that a (true) statement is not provable in some logic, hence deduce the incompleteness of said logic This relies on proof-theoretical results of the form “if ⊢L ∀n∃mΦ, then m Fα(n) where α depends on L (and Φ)”

• Examples. Termination of Goodstein sequences,

Finite forms of Kruskal’s Theorem,

• f Roberston-Seymour Theorem, etc.

[See Friedman’s invited paper at LICS 1999 for many more examples]

17/17