Shannon, Turing and Hats: Information Theory Incompleteness Francois Durand, Fabien Mathieu, Philippe Jacquet WITMSE 2017 September 13 th , 2017
Roadmap 1 Getting Rich without a DeLorean 2 The Shannon Approach 3 Days of infinite past 2 / 21
Roadmap 1 Getting Rich without a DeLorean 2 The Shannon Approach 3 Days of infinite past 3 / 21
A Time Machine An Almanac A rich guy What if no Time Machine? Back to back to the future Plot of episode 2 A poor guy 4 / 21
An Almanac A rich guy What if no Time Machine? Back to back to the future Plot of episode 2 A poor guy A Time Machine 4 / 21
A rich guy What if no Time Machine? Back to back to the future Plot of episode 2 A poor guy A Time Machine An Almanac 4 / 21
What if no Time Machine? Back to back to the future Plot of episode 2 A poor guy A Time Machine An Almanac A rich guy 4 / 21
Back to back to the future Plot of episode 2 A poor guy A Time Machine An Almanac A rich guy What if no Time Machine? 4 / 21
He wants to bet on one of his 4 favorite teams He gathers all statistics of last century Can he predict next year champion? YMCA LMCA problem Biff is a huge fan of English soccer 5 / 21
He gathers all statistics of last century Can he predict next year champion? YMCA LMCA problem Biff is a huge fan of English soccer He wants to bet on one of his 4 favorite teams 5 / 21
Can he predict next year champion? YMCA LMCA problem Biff is a huge fan of English soccer He wants to bet on one of his 4 favorite teams He gathers all statistics of last century 5 / 21
YMCA LMCA problem Biff is a huge fan of English soccer He wants to bet on one of his 4 favorite teams He gathers all statistics of last century Can he predict next year champion? 5 / 21
Roadmap 1 Getting Rich without a DeLorean 2 The Shannon Approach 3 Days of infinite past 6 / 21
2 3 7 10 7 13 Prediction of a LCMA sequence Shannon: the champion sequence is a signal With a long enough sequence, Biff can try to predict things. Statistics example: ( , , , ) = ( 18, 20, 6, 13 ) Premier league may be a Markov source... 7 / 21
2 3 7 10 100 100 0 7 13 Prediction of a LCMA sequence Shannon: the champion sequence is a signal With a long enough sequence, Biff can try to predict things. Statistics example: ( , , , ) = ( 18, 20, 6, 13 ) Premier league may be a Markov source... Fully deterministic 1 Biff Best success rate: X18 X20 Entropy: 1 1 1 X13 X6 7 / 21
2 3 7 10 7 13 Prediction of a LCMA sequence Shannon: the champion sequence is a signal With a long enough sequence, Biff can try to predict things. Statistics example: ( , , , ) = ( 18, 20, 6, 13 ) Premier league may be a Markov source... Fully deterministic 1 Biff Best success rate: 100 / 100 X18 X20 Entropy: 1 1 0 1 X13 X6 7 / 21
68.4 100 0.83 Prediction of a LCMA sequence Shannon: the champion sequence is a signal With a long enough sequence, Biff can try to predict things. Statistics example: ( , , , ) = ( 18, 20, 6, 13 ) Premier league may be a Markov source... With a one-year memory 1 / 3 2 / 3 7 / 10 Biff Best success rate: 6 / 13 3 / 10 Entropy: 1 7 / 13 7 / 21
Prediction of a LCMA sequence Shannon: the champion sequence is a signal With a long enough sequence, Biff can try to predict things. Statistics example: ( , , , ) = ( 18, 20, 6, 13 ) Premier league may be a Markov source... With a one-year memory 1 / 3 2 / 3 7 / 10 Biff Best success rate: 68.4 / 100 6 / 13 3 / 10 Entropy: 0.83 1 7 / 13 7 / 21
2 3 7 10 20 57 35 100 1.88 7 13 Prediction of a LCMA sequence Shannon: the champion sequence is a signal With a long enough sequence, Biff can try to predict things. Statistics example: ( , , , ) = ( 18, 20, 6, 13 ) Premier league may be a Markov source... Memoryless Biff Best success rate: , 18 / 57 , 20 / 57 Entropy: , 20 / 57 , 6 / 57 7 / 21
2 3 7 10 7 13 Prediction of a LCMA sequence Shannon: the champion sequence is a signal With a long enough sequence, Biff can try to predict things. Statistics example: ( , , , ) = ( 18, 20, 6, 13 ) Premier league may be a Markov source... Memoryless Biff Best success rate: , 18 / 57 , 20 / 57 20 / 57 ≈ 35 / 100 Entropy: 1.88 , 20 / 57 , 6 / 57 7 / 21
2 3 7 10 1 4 2 7 13 Prediction of a LCMA sequence Shannon: the champion sequence is a signal With a long enough sequence, Biff can try to predict things. Statistics example: ( , , , ) = ( 18, 20, 6, 13 ) Premier league may be a Markov source... Uniformly Memoryless Biff Best success rate: , 1 / 4 , 1 / 4 Entropy: , 1 / 4 , 1 / 4 7 / 21
2 3 7 10 7 13 Prediction of a LCMA sequence Shannon: the champion sequence is a signal With a long enough sequence, Biff can try to predict things. Statistics example: ( , , , ) = ( 18, 20, 6, 13 ) Premier league may be a Markov source... Uniformly Memoryless Biff Best success rate: , 1 / 4 , 1 / 4 1 / 4 Entropy: 2 , 1 / 4 , 1 / 4 7 / 21
Biff cannot go past 1 4: not enough to get rich. No matter how much data he gathers. What if past is infinite? If the source is uniformly memoryless... Biff problem 8 / 21
Biff cannot go past 1 4: not enough to get rich. No matter how much data he gathers. What if past is infinite? Biff problem If the source is uniformly memoryless... 8 / 21
No matter how much data he gathers. What if past is infinite? Biff problem If the source is uniformly memoryless... Biff cannot go past 1 / 4: not enough to get rich. 8 / 21
What if past is infinite? Biff problem If the source is uniformly memoryless... Biff cannot go past 1 / 4: not enough to get rich. No matter how much data he gathers. 8 / 21
Biff problem If the source is uniformly memoryless... Biff cannot go past 1 / 4: not enough to get rich. No matter how much data he gathers. What if past is infinite? 8 / 21
Roadmap 1 Getting Rich without a DeLorean 2 The Shannon Approach 3 Days of infinite past 9 / 21
Past is infinite: no big bang. Biff, Arsenal, Chelsea, Liverpool and Manchester are immortal. Year champion: uniform memoryless. Universe ends at year 576,000,000,000 AD. Infinite past model We'll make the following assumptions 10 / 21
Biff, Arsenal, Chelsea, Liverpool and Manchester are immortal. Year champion: uniform memoryless. Universe ends at year 576,000,000,000 AD. End 576,000,000,000 Infinite past model We'll make the following assumptions Past is infinite: no big bang. Infinite past Today -infinity 2017 10 / 21
Year champion: uniform memoryless. Universe ends at year 576,000,000,000 AD. Infinite past model We'll make the following assumptions Past is infinite: no big bang. Biff, Arsenal, Chelsea, Liverpool and Manchester are immortal. 10 / 21
Universe ends at year 576,000,000,000 AD. Infinite past model We'll make the following assumptions Past is infinite: no big bang. Biff, Arsenal, Chelsea, Liverpool and Manchester are immortal. Year champion: uniform memoryless. , 1 / 4 , 1 / 4 , 1 / 4 , 1 / 4 10 / 21
Infinite past model We'll make the following assumptions Past is infinite: no big bang. Biff, Arsenal, Chelsea, Liverpool and Manchester are immortal. Year champion: uniform memoryless. Universe ends at year 576,000,000,000 AD. Infinite past Today End -infinity 2017 576,000,000,000 10 / 21
With or without additional information. Finite or infinite population. A bunch of people with Hats. You do not know the color of your own Hat. Guess your color by looking at a subset of others' hats. Exist in multiple flavors Reminder: Hat puzzles Hat puzzles 11 / 21
With or without additional information. Finite or infinite population. You do not know the color of your own Hat. Guess your color by looking at a subset of others' hats. Exist in multiple flavors Reminder: Hat puzzles Hat puzzles A bunch of people with Hats. 11 / 21
With or without additional information. Finite or infinite population. Guess your color by looking at a subset of others' hats. Exist in multiple flavors Reminder: Hat puzzles Hat puzzles A bunch of people with Hats. You do not know the color of your own Hat. 11 / 21
With or without additional information. Finite or infinite population. Exist in multiple flavors Reminder: Hat puzzles Hat puzzles A bunch of people with Hats. You do not know the color of your own Hat. Guess your color by looking at a subset of others' hats. 11 / 21
With or without additional information. Finite or infinite population. Reminder: Hat puzzles Hat puzzles A bunch of people with Hats. You do not know the color of your own Hat. Guess your color by looking at a subset of others' hats. Exist in multiple flavors 11 / 21
Finite or infinite population. Reminder: Hat puzzles Hat puzzles A bunch of people with Hats. You do not know the color of your own Hat. Guess your color by looking at a subset of others' hats. Exist in multiple flavors With or without additional information. 11 / 21
Reminder: Hat puzzles Hat puzzles A bunch of people with Hats. You do not know the color of your own Hat. Guess your color by looking at a subset of others' hats. Exist in multiple flavors With or without additional information. Finite or infinite population. 11 / 21
Recommend
More recommend
