Kolmogorov-Loveland stochasticity and Kolmogorov complexity Laurent - - PowerPoint PPT Presentation

Stochasticity Randomness and Kolmogorov complexity Our result

Kolmogorov-Loveland stochasticity and Kolmogorov complexity

Laurent Bienvenu

Laboratoire d’Informatique Fondamentale Marseille, France

Symposium on Theoretical Aspects of Computer Science, 2007

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

Outline

1 Stochasticity 2 Randomness and Kolmogorov complexity 3 Our result

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

Algorithmic randomness

The goal of algorithmic randomness is to define what it means for an individual object to be random. Here we consider infinite binary sequences. What makes a sequence random ? Look at the following two

• sequences. Which one is random?

00100101111010011010011110011010111000110001101111001010101100110... 00000000000000000100000001000000001000000000000000100000000001000...

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

Algorithmic randomness

The goal of algorithmic randomness is to define what it means for an individual object to be random. Here we consider infinite binary sequences. What makes a sequence random ? Look at the following two

• sequences. Which one is random?

00100101111010011010011110011010111000110001101111001010101100110... 00000000000000000100000001000000001000000000000000100000000001000...

The second one does not satisfy the Law of Large Numbers!

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

A naive definition

It is tempting to say that a sequence A ∈ 2ω is random if it satisfies the Law of Large Numbers. But this is too naive:

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

A naive definition

It is tempting to say that a sequence A ∈ 2ω is random if it satisfies the Law of Large Numbers. But this is too naive:

01010101010101010101010101010101010101010101010101010101010101010...

This sequence does not look random at all!

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

The first attempt to define randomness was made by R. von Mises. It is based on the Law of Large Numbers and avoids the problem we had with our naive definition: Definition (von Mises, 1919) A sequence A ∈ 2ω is random (kollektiv) if every sequence B extracted from A via a reasonable process satisfies the Law of Large Numbers.

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

? ? ? ? ? ? ? ?

selected subsequence:

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

? ? ? ? ? ? ? ?

selected subsequence:

read

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

? ? ? ? ? ? ?

selected subsequence:

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

? ? ? ? ? ? ?

selected subsequence:

read

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

? ? ? 1 ? ? ?

selected subsequence:

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

? ? ? 1 ? ? ?

selected subsequence:

select

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

? ? 1 1 ? ? ?

selected subsequence: 1

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

? ? 1 1 ? ? ?

selected subsequence: 1

read

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

? 1 1 1 ? ? ?

selected subsequence: 1

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

? 1 1 1 ? ? ?

selected subsequence: 1

select

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

? 1 1 1 ? ?

selected subsequence: 1 0

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

? 1 1 1 ? ?

selected subsequence: 1 0

select

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

? 1 1 1 ? 1

selected subsequence: 1 0 1

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

? 1 1 1 ? 1

selected subsequence: 1 0 1

select

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

? 1 1 1 1

selected subsequence: 1 0 1 0

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

? 1 1 1 1

selected subsequence: 1 0 1 0

read

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

1 1 1 1 1

selected subsequence: 1 0 1 0

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

Definition (von Mises, 1919) A sequence A ∈ 2ω is random (kollektiv) if every sequence B extracted from A via a reasonable process satisfies the Law of Large Numbers.

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

Definition (von Mises, 1919) A sequence A ∈ 2ω is random (kollektiv) if every sequence B extracted from A via a reasonable process satisfies the Law of Large Numbers. Definition (Church, 1940) A sequence A ∈ 2ω is random (we now say Church stochastic) if every sequence B extracted from A via a computable process satisfies the Law of Large Numbers.

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

• J. Ville (1939) claimed that stochasticity is not a good notion of

randomness: Theorem (Ville 1939) There exists a sequence A ∈ 2ω such that: A is Church stochastic Every prefix of A has more 0’s than 1’s

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

Ville’s argument is essentially a diagonalization. Kolmogorov and Loveland proposed the following improvement of the notion of stochasticity: Definition (Kolmogorov-Loveland 19) A sequence is Kolmogorov-Loveland stochastic if for every sequence B extracted from A via a computable non-monotonic process, B satisfies the Law of Large Numbers.

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

Martin-L¨

• f randomness

Definition (Martin-L¨

• f 1966)

A sequence A ∈ 2ω is Martin-L¨

• f random if it does not belong to

any effective Gδ set (i.e. Π0

2) which is effectively of measure 0.

Theorem (Levin-Schnorr ∼ 1970) A is Martin-L¨

• f random iff

∃c ∀n K(A ↾ n) ≥ n − c

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

KL stochasticity and ML randomness

It is not difficult to prove that ML randomness implies KL

• stochasticity. The converse has been a longstanding open question,

and was finally answered negatively: Theorem (van Lambalgen-Shen 1989) There exists a sequence A ∈ 2ω such that: A is Kolmogorov-Loveland stochastic Every prefix of A has more 0’s than 1’s

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

Kolmogorov complexity and stochasticity

How complex are stochastic sequences?

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

Kolmogorov complexity and stochasticity

How complex are stochastic sequences? Theorem (An. A. Muchnik 1998, Merkle 2003) There exists a Church stochastic sequence A such that K(A ↾ n) ≤ log log n

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

Kolmogorov complexity and stochasticity

How complex are stochastic sequences? Theorem (An. A. Muchnik 1998, Merkle 2003) There exists a Church stochastic sequence A such that K(A ↾ n) ≤ log log n Theorem (Merkle, Miller, Nies, Stephan, Reimann 2005) For every Kolmogorov-Loveland stochastic sequence A: dim1(A) = lim inf

n→+∞

K(A ↾ n) n = 1

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

A Kolmogorov complexity deficiency lim inf

n→+∞

K(A ↾ n) n = 1 − s with s > 0 implies that maxbias(A) = sup

B sel. from A

Bias(B) > 0 How are s and maxbias related?

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

A Kolmogorov complexity deficiency lim inf

n→+∞

K(A ↾ n) n = 1 − s with s > 0 implies that maxbias(A) = sup

B sel. from A

Bias(B) > 0 How are s and maxbias related? (studied by Asarin and Durand-Vereshchagin for finite strings)

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

Our result

We denote by H(p) the quantity −p log p − (1 − p) log(1 − p)

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

Our result

We denote by H(p) the quantity −p log p − (1 − p) log(1 − p) Theorem Let δ ∈ Q. If dim1(A) ≤ H(1/2 + δ), then one can extract a subsequence B such that Bias(B) ≥ δ (and this bound is optimal).

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

Sketch of the proof

First step: splitting A. We split A into three sequences A0, A1 and A2 such that dim(A0)(A1) ≤ H(1/2 + δ) dim(A0)(A2) ≤ H(1/2 + δ)

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

Sketch of the proof

Second step: competing strategies. We use a game-theoretic method. Instead of selecting, we bet money on the values of the bits of A1 and A2.

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

Sketch of the proof

Second step: competing strategies. We use a game-theoretic method. Instead of selecting, we bet money on the values of the bits of A1 and A2. At each step, a we bet a fraction ρ ∈ [0, 1] of our capital on the value of some bit. If the bet is correct we double our stake. If it is not, we loose our stake.

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

There exists an infinite A0-r.e. set of integers {nk : k ∈ N} such that K (A0)(A1 ↾ nk) ≤ H(1/2 + δ)nk + o(nk) Let τk be the computation time needed to find nk. There exists an infinite A0-r.e. set of integers {n′

k : k ∈ N} such

that K (A0)(A2 ↾ n′

k) ≤ H(1/2 + δ)n′ k + o(n′ k)

Let τ ′

k be the computation time needed to find n′ k.

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

Either (∃∞k τk ≥ τ ′

k) or (∃∞k τ ′ k ≥ τk).

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

Either (∃∞k τk ≥ τ ′

k) or (∃∞k τ ′ k ≥ τk).

We design two strategies: S1 reads A0 and A2, finds out the τ ′

k’s and bets on A1, trying

to use the τ ′

k’s as an upper bound for the τk’s.

S2 reads A0 and A1 , finds out the τk’s and bets on A2, trying to use the τk’s as an upper bound for the τ ′

k’s.

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

Either (∃∞k τk ≥ τ ′

k) or (∃∞k τ ′ k ≥ τk).

We design two strategies: S1 reads A0 and A2, finds out the τ ′

k’s and bets on A1, trying

to use the τ ′

k’s as an upper bound for the τk’s.

S2 reads A0 and A1 , finds out the τk’s and bets on A2, trying to use the τk’s as an upper bound for the τ ′

k’s.

One can design S1 and S2 so that at least one of them satisfies: (Capital after m bets) ≥ 2H(1/2+δ)m for infinitely many m’s.

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

Third step: turning a strategy into a selection rule. The standard conversion technique, due to Muchnik, does not work for

• ur purposes. An argument of Ambos-Spies, Mayordomo, Wang

and Zheng does work, but in the very particular case where we always bet the same fraction ρ0 ∈ [0, 1].

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

Third step: turning a strategy into a selection rule. The standard conversion technique, due to Muchnik, does not work for

• ur purposes. An argument of Ambos-Spies, Mayordomo, Wang

and Zheng does work, but in the very particular case where we always bet the same fraction ρ0 ∈ [0, 1]. We use a compactness argument: since [0, 1] is compact, there must be a condensation point ρ0 ∈ [0, 1] of the succesful bets. Applying the argument of Ambos-Spies et al. to ρ0, we get the desired result.

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

The case of finite sequences

Theorem Let 0 < δ′ < δ. For every finite sequence x of length N such that K(x) ≤ H(1/2 + δ)N, there exists a selection rule of complexity O(1) which selects from x a subsequence y of length Ω(N) such that Bias(y) ≥ δ′.

Laurent Bienvenu KL stochasticity and Kolmogorov complexity

Stochasticity Randomness and Kolmogorov complexity Our result

THANK YOU

Laurent Bienvenu KL stochasticity and Kolmogorov complexity