Von Neumanns biased coin revisited Benoit Monin - LIAFA - University - - PowerPoint PPT Presentation

▶
von neumann s biased coin revisited
SMART_READER_LITE
LIVE PREVIEW

Von Neumanns biased coin revisited Benoit Monin - LIAFA - University - - PowerPoint PPT Presentation

Feb 15, 2023 394 likes •1.17k views

Von Neumanns coin trick Algorithmic randomness Randomness extraction Von Neumanns biased coin revisited Benoit Monin - LIAFA - University of Paris VII Join work with Laurent Bienvenu - CNRS & University of Paris VII 29 June 2012 Von

slide-1
SLIDE 1

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Von Neumann’s biased coin revisited

Benoit Monin - LIAFA - University of Paris VII

Join work with Laurent Bienvenu - CNRS & University of Paris VII

29 June 2012

slide-2
SLIDE 2

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Von Neumann’s coin trick

Section 1

Von Neumann’s coin trick

slide-3
SLIDE 3

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Von Neumann’s coin trick

I want to play Head or Tail Suppose that you want to play a fair game of ”head or tail”, but all you have at your disposal is a biased coin, and you don’t know the bias. How to achieve this ? An easy but nice solution is to group the bits two by two, then you replace 01 by 0, replace 10 by 1 and you discard blocks 00 and 11.

slide-4
SLIDE 4

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Von Neumann’s coin trick

I want to play Head or Tail Suppose that you want to play a fair game of ”head or tail”, but all you have at your disposal is a biased coin, and you don’t know the bias. How to achieve this ? An easy but nice solution is to group the bits two by two, then you replace 01 by 0, replace 10 by 1 and you discard blocks 00 and 11.

slide-5
SLIDE 5

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Von Neumann’s coin trick example Example

The biased coin : P♣headq ✏ p and P♣tailq ✏ 1 ✁ p

slide-6
SLIDE 6

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Von Neumann’s coin trick example Example

The biased coin : P♣headq ✏ p and P♣tailq ✏ 1 ✁ p The first results : 110111100101101101111100

slide-7
SLIDE 7

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Von Neumann’s coin trick example Example

The biased coin : P♣headq ✏ p and P♣tailq ✏ 1 ✁ p The first results : 110111100101101101111100 The trick : 11 ❧♦ ♦♦ ♦♥

p2

❧♦ ♦♦ ♦♥

✁

01 ❧♦ ♦♦ ♦♥

p♣1✁pq

❧♦♦♦♦♦♦♥ 11 ❧♦ ♦♦ ♦♥

p2

❧♦ ♦♦ ♦♥

✁

10 ❧♦ ♦♦ ♦♥

p♣1✁pq

❧♦♦♦♦♦♦♥

1

01 ❧♦ ♦♦ ♦♥

p♣1✁pq

❧♦♦♦♦♦♦♥ 01 ❧♦ ♦♦ ♦♥

p♣1✁pq

❧♦♦♦♦♦♦♥ 10 ❧♦ ♦♦ ♦♥

p♣1✁pq

❧♦♦♦♦♦♦♥

1

11 ❧♦ ♦♦ ♦♥

p2

❧♦ ♦♦ ♦♥

✁

01 ❧♦ ♦♦ ♦♥

p♣1✁pq

❧♦♦♦♦♦♦♥ 11 ❧♦ ♦♦ ♦♥

p2

❧♦ ♦♦ ♦♥

✁

11 ❧♦ ♦♦ ♦♥

p2

❧♦ ♦♦ ♦♥

✁

00 ❧♦ ♦♦ ♦♥

♣1✁pq2

❧♦♦♦♦♥

✁

slide-8
SLIDE 8

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Von Neumann’s coin trick example Example

The biased coin : P♣headq ✏ p and P♣tailq ✏ 1 ✁ p The first results : 110111100101101101111100 The trick : 11 ❧♦ ♦♦ ♦♥

p2

❧♦ ♦♦ ♦♥

✁

01 ❧♦ ♦♦ ♦♥

p♣1✁pq

❧♦♦♦♦♦♦♥ 11 ❧♦ ♦♦ ♦♥

p2

❧♦ ♦♦ ♦♥

✁

10 ❧♦ ♦♦ ♦♥

p♣1✁pq

❧♦♦♦♦♦♦♥

1

01 ❧♦ ♦♦ ♦♥

p♣1✁pq

❧♦♦♦♦♦♦♥ 01 ❧♦ ♦♦ ♦♥

p♣1✁pq

❧♦♦♦♦♦♦♥ 10 ❧♦ ♦♦ ♦♥

p♣1✁pq

❧♦♦♦♦♦♦♥

1

11 ❧♦ ♦♦ ♦♥

p2

❧♦ ♦♦ ♦♥

✁

01 ❧♦ ♦♦ ♦♥

p♣1✁pq

❧♦♦♦♦♦♦♥ 11 ❧♦ ♦♦ ♦♥

p2

❧♦ ♦♦ ♦♥

✁

11 ❧♦ ♦♦ ♦♥

p2

❧♦ ♦♦ ♦♥

✁

00 ❧♦ ♦♦ ♦♥

♣1✁pq2

❧♦♦♦♦♥

✁

The fair coin tossing : 010010

slide-9
SLIDE 9

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Von Neumann’s coin trick example

. . Nice things about von Neumann’s trick : We have a computable extraction procedure. It works even if the measure is not computable. It is uniform for all Bernoulli measures (except trivial ones) and all of their random elements.

slide-10
SLIDE 10

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Von Neumann’s coin trick example

. . Nice things about von Neumann’s trick : We have a computable extraction procedure. It works even if the measure is not computable. It is uniform for all Bernoulli measures (except trivial ones) and all of their random elements.

slide-11
SLIDE 11

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Von Neumann’s coin trick example

. . Nice things about von Neumann’s trick : We have a computable extraction procedure. It works even if the measure is not computable. It is uniform for all Bernoulli measures (except trivial ones) and all of their random elements.

slide-12
SLIDE 12

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Von Neumann’s coin trick example

. . Nice things about von Neumann’s trick : We have a computable extraction procedure. It works even if the measure is not computable. It is uniform for all Bernoulli measures (except trivial ones) and all of their random elements.

slide-13
SLIDE 13

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

A more general framework

. . On a more abstract level, the situation is the following : We have access to a random sequence for a given measure µ which we do not know. However, we do know that µ belongs to some particular class C. Based on this information we are able to build a computable procedure which works for all µ P C.

slide-14
SLIDE 14

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

A more general framework

. . On a more abstract level, the situation is the following : We have access to a random sequence for a given measure µ which we do not know. However, we do know that µ belongs to some particular class C. Based on this information we are able to build a computable procedure which works for all µ P C.

slide-15
SLIDE 15

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

A more general framework

. . On a more abstract level, the situation is the following : We have access to a random sequence for a given measure µ which we do not know. However, we do know that µ belongs to some particular class C. Based on this information we are able to build a computable procedure which works for all µ P C.

slide-16
SLIDE 16

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

A more general framework

. . On a more abstract level, the situation is the following : We have access to a random sequence for a given measure µ which we do not know. However, we do know that µ belongs to some particular class C. Based on this information we are able to build a computable procedure which works for all µ P C.

slide-17
SLIDE 17

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

A more general framework

. . On a more abstract level, the situation is the following : We have access to a random sequence for a given measure µ which we do not know. However, we do know that µ belongs to some particular class C. Based on this information we are able to build a computable procedure which works for all µ P C. For which other class C can such an extraction procedure be built ?

slide-18
SLIDE 18

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness

Section 2

Algorithmic randomness

slide-19
SLIDE 19

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness

Algorithmic randomness :

What does it mean for a string to be random ?

Are c :00000000000000100000000010000000000100000000000001 . . .

  • r

π :00100100001111110110101010001000100001011010001100 . . . random ?

slide-20
SLIDE 20

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness

Algorithmic randomness :

What does it mean for a string to be random ?

Intuition A sequence of 2ω should be random if it belongs to the smallest set of measure 1. Definition (Martin-L¨

  • f)

A sequence of 2ω is Martin-L¨

  • f random if it belongs to the

smallest Σ0

2 set, effectively of measure 1.

slide-21
SLIDE 21

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness

Algorithmic randomness :

What does it mean for a string to be random ?

Intuition A sequence of 2ω should be random if it belongs to the smallest set of measure 1. Definition (Martin-L¨

  • f)

A sequence of 2ω is Martin-L¨

  • f random if it belongs to the

smallest Σ0

2 set, effectively of measure 1.

slide-22
SLIDE 22

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness

Definition (Martin-L¨

  • f test)

A Π0

2 subset of 2ω is a Martin-L¨

  • f test if it is effectively of measure

0, which means that the n-th open set of the intersection should be of measure less than 2✁n. Definition (Martin-L¨

  • f test)

There is a largest Martin-L¨

  • f test. A sequence is not Martin-L¨
  • f

random if it belongs to the largest Martin-L¨

  • f test.
slide-23
SLIDE 23

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness

Definition (Martin-L¨

  • f test)

A Π0

2 subset of 2ω is a Martin-L¨

  • f test if it is effectively of measure

0, which means that the n-th open set of the intersection should be of measure less than 2✁n. Definition (Martin-L¨

  • f test)

There is a largest Martin-L¨

  • f test. A sequence is not Martin-L¨
  • f

random if it belongs to the largest Martin-L¨

  • f test.
slide-24
SLIDE 24

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness : Martin-L¨

  • f test example

Illustration of a test : Measure f♣., 1q f♣., 2q f♣., 3q f♣., 4q . . . λ♣f ♣1, Nqq ↕ 1

2

σ1,1 σ1,2 σ1,3 σ1,4 . . . λ♣f ♣2, Nqq ↕ 1

4

σ2,1 σ2,2 σ2,3 σ2,4 . . . λ♣f ♣3, Nqq ↕ 1

8

σ3,1 σ3,2 σ3,3 σ3,4 . . . λ♣f ♣4, Nqq ↕ 1

16

σ4,1 σ4,2 σ4,3 σ4,4 . . . . . . . . . . . . . . . . . . . . .

slide-25
SLIDE 25

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness : universal Martin-L¨

  • f test

Universal test : Measure Test 1 Test 2 Test 3 Test 4 . . . ❅n λ ✄↕

iPN

σn

1,i

☛ ↕ 1 2 ♣σ1

1,iqiPN

♣σ2

1,iqiPN

♣σ3

1,iqiPN

♣σ4

1,iqiPN

. . . ❅n λ ✄↕

iPN

σn

2,i

☛ ↕ 1 4 ♣σ1

2,iqiPN

♣σ2

2,iqiPN

♣σ3

2,iqiPN

♣σ4

2,iqiPN

. . . ❅n λ ✄↕

iPN

σn

3,i

☛ ↕ 1 8 ♣σ1

3,iqiPN

♣σ2

3,iqiPN

♣σ3

3,iqiPN

♣σ4

3,iqiPN

. . . ❅n λ ✄↕

iPN

σn

4,i

☛ ↕ 1 16 ♣σ1

4,iqiPN

♣σ2

4,iqiPN

♣σ3

4,iqiPN

♣σ4

4,iqiPN

. . . . . . . . . . . . . . . . . . . . .

slide-26
SLIDE 26

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness : universal Martin-L¨

  • f test

Universal test : Measure Test 1 Test 2 Test 3 Test 4 . . . ❅n λ ✄↕

iPN

σn

1,i

☛ ↕ 1 2 ♣σ1

1,iqiPN

♣σ2

1,iqiPN

♣σ3

1,iqiPN

♣σ4

1,iqiPN

. . . ❅n λ ✄↕

iPN

σn

2,i

☛ ↕ 1 4 ♣σ1

2,iqiPN

♣σ2

2,iqiPN

♣σ3

2,iqiPN

♣σ4

2,iqiPN

. . . ❅n λ ✄↕

iPN

σn

3,i

☛ ↕ 1 8 ♣σ1

3,iqiPN

♣σ2

3,iqiPN

♣σ3

3,iqiPN

♣σ4

3,iqiPN

. . . ❅n λ ✄↕

iPN

σn

4,i

☛ ↕ 1 16 ♣σ1

4,iqiPN

♣σ2

4,iqiPN

♣σ3

4,iqiPN

♣σ4

4,iqiPN

. . . . . . . . . . . . . . . . . . . . .

slide-27
SLIDE 27

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness : universal Martin-L¨

  • f test

Universal test : Measure Test 1 Test 2 Test 3 Test 4 . . . ❅n λ ✄↕

iPN

σn

1,i

☛ ↕ 1 2 ♣σ1

1,iqiPN

♣σ2

1,iqiPN

♣σ3

1,iqiPN

♣σ4

1,iqiPN

. . . ❅n λ ✄↕

iPN

σn

2,i

☛ ↕ 1 4 ♣σ1

2,iqiPN

♣σ2

2,iqiPN

♣σ3

2,iqiPN

♣σ4

2,iqiPN

. . . ❅n λ ✄↕

iPN

σn

3,i

☛ ↕ 1 8 ♣σ1

3,iqiPN

♣σ2

3,iqiPN

♣σ3

3,iqiPN

♣σ4

3,iqiPN

. . . ❅n λ ✄↕

iPN

σn

4,i

☛ ↕ 1 16 ♣σ1

4,iqiPN

♣σ2

4,iqiPN

♣σ3

4,iqiPN

♣σ4

4,iqiPN

. . . . . . . . . . . . . . . . . . . . .

slide-28
SLIDE 28

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness : universal Martin-L¨

  • f test

Universal test : Measure Test 1 Test 2 Test 3 Test 4 . . . ❅n λ ✄↕

iPN

σn

1,i

☛ ↕ 1 2 ♣σ1

1,iqiPN

♣σ2

1,iqiPN

♣σ3

1,iqiPN

♣σ4

1,iqiPN

. . . ❅n λ ✄↕

iPN

σn

2,i

☛ ↕ 1 4 ♣σ1

2,iqiPN

♣σ2

2,iqiPN

♣σ3

2,iqiPN

♣σ4

2,iqiPN

. . . ❅n λ ✄↕

iPN

σn

3,i

☛ ↕ 1 8 ♣σ1

3,iqiPN

♣σ2

3,iqiPN

♣σ3

3,iqiPN

♣σ4

3,iqiPN

. . . ❅n λ ✄↕

iPN

σn

4,i

☛ ↕ 1 16 ♣σ1

4,iqiPN

♣σ2

4,iqiPN

♣σ3

4,iqiPN

♣σ4

4,iqiPN

. . . . . . . . . . . . . . . . . . . . .

slide-29
SLIDE 29

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness : universal Martin-L¨

  • f test

Universal test : Measure Test 1 Test 2 Test 3 Test 4 . . . ❅n λ ✄↕

iPN

σn

1,i

☛ ↕ 1 2 ♣σ1

1,iqiPN

♣σ2

1,iqiPN

♣σ3

1,iqiPN

♣σ4

1,iqiPN

. . . ❅n λ ✄↕

iPN

σn

2,i

☛ ↕ 1 4 ♣σ1

2,iqiPN

♣σ2

2,iqiPN

♣σ3

2,iqiPN

♣σ4

2,iqiPN

. . . ❅n λ ✄↕

iPN

σn

3,i

☛ ↕ 1 8 ♣σ1

3,iqiPN

♣σ2

3,iqiPN

♣σ3

3,iqiPN

♣σ4

3,iqiPN

. . . ❅n λ ✄↕

iPN

σn

4,i

☛ ↕ 1 16 ♣σ1

4,iqiPN

♣σ2

4,iqiPN

♣σ3

4,iqiPN

♣σ4

4,iqiPN

. . . . . . . . . . . . . . . . . . . . .

slide-30
SLIDE 30

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness : Integrable test

Switch to analysis To test the randomness of sequences, we can equivalently use a more analytical notion. Intuition We can define t : 2ω Ñ R to be on a string x : The smallest n such that x does not belong to the n-th open set of the universal test.

slide-31
SLIDE 31

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness : Integrable test

Switch to analysis To test the randomness of sequences, we can equivalently use a more analytical notion. Intuition We can define t : 2ω Ñ R to be on a string x : The smallest n such that x does not belong to the n-th open set of the universal test.

slide-32
SLIDE 32

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness : Integrable test

Illustration t♣xq ✏ 4 : Num 1 2 3 4 5 6 7 8 . . . 1 σ1,1 σ1,2 σ1,3 σ1,4 σ1,5 σ1,6 σ1,7 σ1,8 . . . 2 σ2,1 σ2,2 σ2,3 σ2,4 σ2,5 σ2,6 σ2,7 σ2,8 . . . 3 σ3,1 σ3,2 σ3,3 σ3,4 σ3,5 σ3,6 σ3,7 σ3,8 . . . 4 σ4,1 σ4,2 σ4,3 σ4,4 σ4,5 σ4,6 σ4,7 σ4,8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

slide-33
SLIDE 33

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness : Integrable test

Illustration t♣xq ✏ 4 : Num 1 2 3 4 5 6 7 8 . . . 1 σ1,1 σ1,2 σ1,3 σ1,4 σ1,5 σ1,6 σ1,7 σ1,8 . . . 2 σ2,1 σ2,2 σ2,3 σ2,4 σ2,5 σ2,6 σ2,7 σ2,8 . . . 3 σ3,1 σ3,2 σ3,3 σ3,4 σ3,5 σ3,6 σ3,7 σ3,8 . . . 4 σ4,1 σ4,2 σ4,3 σ4,4 σ4,5 σ4,6 σ4,7 σ4,8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

slide-34
SLIDE 34

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness : Integrable test

Illustration t♣xq ✏ 4 : Num 1 2 3 4 5 6 7 8 . . . 1 σ1,1 σ1,2 σ1,3 σ1,4 σ1,5 σ1,6 σ1,7 σ1,8 . . . 2 σ2,1 σ2,2 σ2,3 σ2,4 σ2,5 σ2,6 σ2,7 σ2,8 . . . 3 σ3,1 σ3,2 σ3,3 σ3,4 σ3,5 σ3,6 σ3,7 σ3,8 . . . 4 σ4,1 σ4,2 σ4,3 σ4,4 σ4,5 σ4,6 σ4,7 σ4,8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

slide-35
SLIDE 35

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness : Integrable test

Illustration t♣xq ✏ 4 : Num 1 2 3 4 5 6 7 8 . . . 1 σ1,1 σ1,2 σ1,3 σ1,4 σ1,5 σ1,6 σ1,7 σ1,8 . . . 2 σ2,1 σ2,2 σ2,3 σ2,4 σ2,5 σ2,6 σ2,7 σ2,8 . . . 3 σ3,1 σ3,2 σ3,3 σ3,4 σ3,5 σ3,6 σ3,7 σ3,8 . . . 4 σ4,1 σ4,2 σ4,3 σ4,4 σ4,5 σ4,6 σ4,7 σ4,8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

slide-36
SLIDE 36

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Properties of t We have that : t is computabily approximable from below, uniformily in x (t is lower semi-computable). ➩ t♣xqdx is finite (as the ➦

n♣n 1q2✁n is finite).

Definition (integrable test) Such a function is called an integrable test. There is a universal integrable test. Randomness We have that x is random iff t♣xq is finite for all integrable tests iff t♣xq is finite for the universal integrable test.

slide-37
SLIDE 37

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Properties of t We have that : t is computabily approximable from below, uniformily in x (t is lower semi-computable). ➩ t♣xqdx is finite (as the ➦

n♣n 1q2✁n is finite).

Definition (integrable test) Such a function is called an integrable test. There is a universal integrable test. Randomness We have that x is random iff t♣xq is finite for all integrable tests iff t♣xq is finite for the universal integrable test.

slide-38
SLIDE 38

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Properties of t We have that : t is computabily approximable from below, uniformily in x (t is lower semi-computable). ➩ t♣xqdx is finite (as the ➦

n♣n 1q2✁n is finite).

Definition (integrable test) Such a function is called an integrable test. There is a universal integrable test. Randomness We have that x is random iff t♣xq is finite for all integrable tests iff t♣xq is finite for the universal integrable test.

slide-39
SLIDE 39

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Properties of t We have that : t is computabily approximable from below, uniformily in x (t is lower semi-computable). ➩ t♣xqdx is finite (as the ➦

n♣n 1q2✁n is finite).

Definition (integrable test) Such a function is called an integrable test. There is a universal integrable test. Randomness We have that x is random iff t♣xq is finite for all integrable tests iff t♣xq is finite for the universal integrable test.

slide-40
SLIDE 40

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Properties of t We have that : t is computabily approximable from below, uniformily in x (t is lower semi-computable). ➩ t♣xqdx is finite (as the ➦

n♣n 1q2✁n is finite).

Definition (integrable test) Such a function is called an integrable test. There is a universal integrable test. Randomness We have that x is random iff t♣xq is finite for all integrable tests iff t♣xq is finite for the universal integrable test.

slide-41
SLIDE 41

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness for other measures

The space of probability measures on 2ω will be denoted by M♣2ωq. Question What if we want to define random sequences obtained by flipping a biased coin ? The definition generalizes itself pretty well as long as the measure is computable. Definition (Martin-L¨

  • f randomness for computable measure)

Let µ be a computable measure. A sequence of 2ω is Martin-L¨

  • f

random for the measure µ if it belongs to the smallest Σ0

2 set,

effectively of µ measure 1.

slide-42
SLIDE 42

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness for other measures

The space of probability measures on 2ω will be denoted by M♣2ωq. Question What if we want to define random sequences obtained by flipping a biased coin ? The definition generalizes itself pretty well as long as the measure is computable. Definition (Martin-L¨

  • f randomness for computable measure)

Let µ be a computable measure. A sequence of 2ω is Martin-L¨

  • f

random for the measure µ if it belongs to the smallest Σ0

2 set,

effectively of µ measure 1.

slide-43
SLIDE 43

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness for other measures

The space of probability measures on 2ω will be denoted by M♣2ωq. Question What if we want to define random sequences obtained by flipping a biased coin ? The definition generalizes itself pretty well as long as the measure is computable. Definition (Martin-L¨

  • f randomness for computable measure)

Let µ be a computable measure. A sequence of 2ω is Martin-L¨

  • f

random for the measure µ if it belongs to the smallest Σ0

2 set,

effectively of µ measure 1.

slide-44
SLIDE 44

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness for other measures

Problem When the measure is not computable, we cannot necessarily obtain universal Martin-L¨

  • f test for the measure...

Intuition A possibility is to add the measure as an oracle to create our test, but a measure can have many different binary representations having different Turing-degrees. So it is not clear which one to choose.

slide-45
SLIDE 45

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness for other measures

Problem When the measure is not computable, we cannot necessarily obtain universal Martin-L¨

  • f test for the measure...

Intuition A possibility is to add the measure as an oracle to create our test, but a measure can have many different binary representations having different Turing-degrees. So it is not clear which one to choose.

slide-46
SLIDE 46

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness for other measures

Idea You can take a representation such that any other representation can compute it. (the smallest one). Theorem (Day, Miller) Some measures does not have a smallest representation in the Turing degree ! Solution Instead of using representations, we should extend the notion of computability to the space of measures

slide-47
SLIDE 47

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness for other measures

Idea You can take a representation such that any other representation can compute it. (the smallest one). Theorem (Day, Miller) Some measures does not have a smallest representation in the Turing degree ! Solution Instead of using representations, we should extend the notion of computability to the space of measures

slide-48
SLIDE 48

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness for other measures

Idea You can take a representation such that any other representation can compute it. (the smallest one). Theorem (Day, Miller) Some measures does not have a smallest representation in the Turing degree ! Solution Instead of using representations, we should extend the notion of computability to the space of measures

slide-49
SLIDE 49

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness for other measures

Computability in M♣2ωq How can we extend the notion of integrable test t : M♣2ωq ✂ 2ω Ñ R ? What do we do in R ? In R we say that a function f is computable if from any fast cauchy sequence converging to x we can output a fast cauchy sequence converging to f ♣xq. What do we do in R ? Equivalently, f is computable if from any sequence of all intervals with rationals endpoints containing x, f can output the sequence

  • f intervals with rationals endpoints contaning f ♣xq.
slide-50
SLIDE 50

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness for other measures

Computability in M♣2ωq How can we extend the notion of integrable test t : M♣2ωq ✂ 2ω Ñ R ? What do we do in R ? In R we say that a function f is computable if from any fast cauchy sequence converging to x we can output a fast cauchy sequence converging to f ♣xq. What do we do in R ? Equivalently, f is computable if from any sequence of all intervals with rationals endpoints containing x, f can output the sequence

  • f intervals with rationals endpoints contaning f ♣xq.
slide-51
SLIDE 51

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness for other measures

Computability in M♣2ωq How can we extend the notion of integrable test t : M♣2ωq ✂ 2ω Ñ R ? What do we do in R ? In R we say that a function f is computable if from any fast cauchy sequence converging to x we can output a fast cauchy sequence converging to f ♣xq. What do we do in R ? Equivalently, f is computable if from any sequence of all intervals with rationals endpoints containing x, f can output the sequence

  • f intervals with rationals endpoints contaning f ♣xq.
slide-52
SLIDE 52

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness for different measures

A basic open set in the space of measure : ✈0✇ 1 ✂ ✈00✇ 1 ✂. . . ✂ ✈s102✇ 1 ✂. . . ✂ ✈s1000203✇ 1 ✂. . . M♣2ωq ❸ r0, 1sN

slide-53
SLIDE 53

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness for different measures

The space of measures The space of measure is a closed subset of r0, 1sN. ❅s P 2ω µ♣sq ✏ µ♣s0q µ♣s1q. The topology is the one induced by the product topology on r0, 1sN. A measure is computable iff the set of basic open sets containing it is effectively enumerable. Integrable tests To define what it means for a point x P 2ω to be µ-MLR, we extend the notion of integrable test.

slide-54
SLIDE 54

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness for different measures

The space of measures The space of measure is a closed subset of r0, 1sN. ❅s P 2ω µ♣sq ✏ µ♣s0q µ♣s1q. The topology is the one induced by the product topology on r0, 1sN. A measure is computable iff the set of basic open sets containing it is effectively enumerable. Integrable tests To define what it means for a point x P 2ω to be µ-MLR, we extend the notion of integrable test.

slide-55
SLIDE 55

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness for different measures

The space of measures The space of measure is a closed subset of r0, 1sN. ❅s P 2ω µ♣sq ✏ µ♣s0q µ♣s1q. The topology is the one induced by the product topology on r0, 1sN. A measure is computable iff the set of basic open sets containing it is effectively enumerable. Integrable tests To define what it means for a point x P 2ω to be µ-MLR, we extend the notion of integrable test.

slide-56
SLIDE 56

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness for different measures

The space of measures The space of measure is a closed subset of r0, 1sN. ❅s P 2ω µ♣sq ✏ µ♣s0q µ♣s1q. The topology is the one induced by the product topology on r0, 1sN. A measure is computable iff the set of basic open sets containing it is effectively enumerable. Integrable tests To define what it means for a point x P 2ω to be µ-MLR, we extend the notion of integrable test.

slide-57
SLIDE 57

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness for different measures

Definition (Uniform tests) A uniform integrable test is a lower semi-computable function t : 2ω ✂ M♣2ωq Ñ R such that : ➩ t♣x, µq dµ♣xq is finite for all µ Theorem (Levin-G´ acs-Hoyrup-Roj´ as) There exists a universal uniform integrable test u which dominates every other integrable tests up to a multiplicative constant. Randomness Intuitively u♣x, µq can represent the randomness deficiency of x with respect to the measure µ. We say that x is Martin-l¨

  • f random

for the measure µ iff u♣x, µq ➔ ✽. The notion matches the previous one for computable measures.

slide-58
SLIDE 58

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness for different measures

Definition (Uniform tests) A uniform integrable test is a lower semi-computable function t : 2ω ✂ M♣2ωq Ñ R such that : ➩ t♣x, µq dµ♣xq is finite for all µ Theorem (Levin-G´ acs-Hoyrup-Roj´ as) There exists a universal uniform integrable test u which dominates every other integrable tests up to a multiplicative constant. Randomness Intuitively u♣x, µq can represent the randomness deficiency of x with respect to the measure µ. We say that x is Martin-l¨

  • f random

for the measure µ iff u♣x, µq ➔ ✽. The notion matches the previous one for computable measures.

slide-59
SLIDE 59

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Algorithmic randomness for different measures

Definition (Uniform tests) A uniform integrable test is a lower semi-computable function t : 2ω ✂ M♣2ωq Ñ R such that : ➩ t♣x, µq dµ♣xq is finite for all µ Theorem (Levin-G´ acs-Hoyrup-Roj´ as) There exists a universal uniform integrable test u which dominates every other integrable tests up to a multiplicative constant. Randomness Intuitively u♣x, µq can represent the randomness deficiency of x with respect to the measure µ. We say that x is Martin-l¨

  • f random

for the measure µ iff u♣x, µq ➔ ✽. The notion matches the previous one for computable measures.

slide-60
SLIDE 60

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Randomness extraction

Section 3

Randomness extraction

slide-61
SLIDE 61

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Reformulation of the problem

Reformulation of the problem Given a class C ❸ M♣2ωq, is there a computable function f : 2ω Ñ 2ω such that : For all x such that u♣x, µq ➔ ✽ for some µ P C, f ♣xq is a binary sequence random for the uniform measure ? A first piece of the puzzle Randomness can be extracted when the measure is known.

slide-62
SLIDE 62

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

The Levin-Kautz conversion procedure

Theorem (Levin-Kautz) If µ is a computable measure on 2ω, then there is a computable f : 2ω Ñ 2ω such that f ♣xq is random for all µ-random x which are not atoms of µ (x is an atom of µ if µ♣tx✉q → 0, which implies that x is computable). It is not hard to see that Levin-Kautz theorem is uniform : Theorem (Levin-Kautz, extended) There is a computable f : 2ω ✂ M♣2ωq Ñ 2ω such that f ♣x, µq is random whenever x is µ-random without being an atom of µ.

slide-63
SLIDE 63

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

The Levin-Kautz conversion procedure

Theorem (Levin-Kautz) If µ is a computable measure on 2ω, then there is a computable f : 2ω Ñ 2ω such that f ♣xq is random for all µ-random x which are not atoms of µ (x is an atom of µ if µ♣tx✉q → 0, which implies that x is computable). It is not hard to see that Levin-Kautz theorem is uniform : Theorem (Levin-Kautz, extended) There is a computable f : 2ω ✂ M♣2ωq Ñ 2ω such that f ♣x, µq is random whenever x is µ-random without being an atom of µ.

slide-64
SLIDE 64

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Guessing the measure

Guessing the measure Suppose a measure µ was such that it could be guessed, in some uniform way from any of its random (non-atomic) elements. Then randomness extraction for such measures would be possible on that particular µ.

slide-65
SLIDE 65

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Guessing the measure

Example Suppose µ is represented with a Markov chain of type

1

p q 1-p 1-q

And we get a random x ✏ 0000000010000000110000000000 . . .. Can we deduce anything about p and q after reading finitely many bits ? No ! Maybe p is small and only the beginning of the sequence is atypical.

slide-66
SLIDE 66

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Guessing the measure

Example Suppose µ is represented with a Markov chain of type

1

p q 1-p 1-q

And we get a random x ✏ 0000000010000000110000000000 . . .. Can we deduce anything about p and q after reading finitely many bits ? No ! Maybe p is small and only the beginning of the sequence is atypical.

slide-67
SLIDE 67

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Guessing the measure

Example Suppose µ is represented with a Markov chain of type

1

p q 1-p 1-q

And we get a random x ✏ 0000000010000000110000000000 . . .. Can we deduce anything about p and q after reading finitely many bits ? No ! Maybe p is small and only the beginning of the sequence is atypical.

slide-68
SLIDE 68

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Guessing the measure

Example Suppose µ is represented with a Markov chain of type

1

p q 1-p 1-q

And we get a random x ✏ 0000000010000000110000000000 . . .. Can we deduce anything about p and q after reading finitely many bits ? No ! Maybe p is small and only the beginning of the sequence is atypical.

slide-69
SLIDE 69

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Layerwiseness

However... We could compute p and q if we knew a bound on the randomness deficiency of x with respect to µ ! Layerwise computability (Hoyrup and Roj´ as) A function F is µ-layerwise computable over a space x if it is defined on all µ-random reals and it can be uniformly computed modulo an ”advice” which is an upper bound on the randomness deficiency u♣x, µq. Definition (Bienvenu-Monin) A measure µ is (layerwise) learnable if it can be layerwise computed from its random elements.

slide-70
SLIDE 70

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Layerwiseness

However... We could compute p and q if we knew a bound on the randomness deficiency of x with respect to µ ! Layerwise computability (Hoyrup and Roj´ as) A function F is µ-layerwise computable over a space x if it is defined on all µ-random reals and it can be uniformly computed modulo an ”advice” which is an upper bound on the randomness deficiency u♣x, µq. Definition (Bienvenu-Monin) A measure µ is (layerwise) learnable if it can be layerwise computed from its random elements.

slide-71
SLIDE 71

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Layerwiseness

However... We could compute p and q if we knew a bound on the randomness deficiency of x with respect to µ ! Layerwise computability (Hoyrup and Roj´ as) A function F is µ-layerwise computable over a space x if it is defined on all µ-random reals and it can be uniformly computed modulo an ”advice” which is an upper bound on the randomness deficiency u♣x, µq. Definition (Bienvenu-Monin) A measure µ is (layerwise) learnable if it can be layerwise computed from its random elements.

slide-72
SLIDE 72

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

A criterion

A criterion for learnability : Theorem (Bienvenu-Monin) If a measure µ belongs to a class C of measures such that (i) C is Π0

1

(ii) no distinct ν1, ν2 have a random in common (✍) then µ is learnable. Surprisingly, the converse holds : Theorem (Bienvenu-Monin) If a measure µ is learnable, then it can be embedded into a Π0

1

class of measures with the (✍) property.

slide-73
SLIDE 73

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

A criterion

A criterion for learnability : Theorem (Bienvenu-Monin) If a measure µ belongs to a class C of measures such that (i) C is Π0

1

(ii) no distinct ν1, ν2 have a random in common (✍) then µ is learnable. Surprisingly, the converse holds : Theorem (Bienvenu-Monin) If a measure µ is learnable, then it can be embedded into a Π0

1

class of measures with the (✍) property.

slide-74
SLIDE 74

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Putting things together

We are ready to present a partial answer to the original question. Theorem (Bienvenu-Monin) Let C be a Π0

1 class of measures with the (✍) property. Then

uniform randomness extraction is possible, i.e., there exists a partial computable function f : 2ω Ñ 2ω such that : if x is µ-random for some µ P C and x is not an atom of µ, then f ♣xq is random for the uniform measure. (this even extends to Σ0

2 classes with the (✍) property).

slide-75
SLIDE 75

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Putting things together

We are ready to present a partial answer to the original question. Theorem (Bienvenu-Monin) Let C be a Π0

1 class of measures with the (✍) property. Then

uniform randomness extraction is possible, i.e., there exists a partial computable function f : 2ω Ñ 2ω such that : if x is µ-random for some µ P C and x is not an atom of µ, then f ♣xq is random for the uniform measure. (this even extends to Σ0

2 classes with the (✍) property).

slide-76
SLIDE 76

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

source x (x,µ)

Guess µ such that u(x,µ) < c Does u(x,µ) < c actually hold? if not, c := c+1

  • utput y

Levin-Kautz conversion (start with c=1)

slide-77
SLIDE 77

Von Neumann’s coin trick Algorithmic randomness Randomness extraction

Thank you. Questions ?