Analogy Between Gambling and Measurement-Based Work Extraction Dror - - PowerPoint PPT Presentation

Analogy Between Gambling and Measurement-Based Work Extraction

Dror Vinkler 1 Haim Permuter 1 Neri Merhav 2

1Ben Gurion University 2Technion - Israel Institute of Technology

ISIT 2014

1 / 17

Outline

Past results and brief physics background The analogy Gambling on continuous random variables Consequences

Universal engine Memory

Summary and future work

2 / 17

Past Results - Horse Race Gambling

Kelly 1956

A race of m horses.

3 / 17

Past Results - Horse Race Gambling

Kelly 1956

A race of m horses. Xi - winning horse. Yi - side information.

3 / 17

Past Results - Horse Race Gambling

Kelly 1956

A race of m horses. Xi - winning horse. Yi - side information. Xi, Yi are pairwise i.i.d.

3 / 17

Past Results - Horse Race Gambling

Kelly 1956

A race of m horses. Xi - winning horse. Yi - side information. Xi, Yi are pairwise i.i.d. Sn - gambler’s capital after n rounds, S1 = bX|Y (X1|Y1)oX(X1)S0, bX|Y (X|Y ) - betting strategy.

• X(X)
• odds.

3 / 17

Past Results - Horse Race Gambling

Kelly 1956

A race of m horses. Xi - winning horse. Yi - side information. Xi, Yi are pairwise i.i.d. Sn - gambler’s capital after n rounds, S1 = bX|Y (X1|Y1)oX(X1)S0, bX|Y (X|Y ) - betting strategy.

• X(X)
• odds.

Sn =

n

• i=1

bX|Y (Xi|Yi)oX(Xi)S0,

3 / 17

Past Results - Horse Race Gambling

Kelly 1956

The goal: finding the optimal betting strategy b∗

X|Y ,

b∗

X|Y = arg max bX|Y

E [log Sn] . log Sn - capital growth.

4 / 17

Past Results - Horse Race Gambling

Kelly 1956

The goal: finding the optimal betting strategy b∗

X|Y ,

b∗

X|Y = arg max bX|Y

E [log Sn] . log Sn - capital growth. This optimal betting strategy is b∗

X|Y = PX|Y .

4 / 17

Past Results - Horse Race Gambling

Kelly 1956

The goal: finding the optimal betting strategy b∗

X|Y ,

b∗

X|Y = arg max bX|Y

E [log Sn] . log Sn - capital growth. This optimal betting strategy is b∗

X|Y = PX|Y .

A bet is fair if oX(x) = 1/PX(x) ∀x.

4 / 17

Past Results - Horse Race Gambling

Kelly 1956

The goal: finding the optimal betting strategy b∗

X|Y ,

b∗

X|Y = arg max bX|Y

E [log Sn] . log Sn - capital growth. This optimal betting strategy is b∗

X|Y = PX|Y .

A bet is fair if oX(x) = 1/PX(x) ∀x. For a fair bet, the maximal growth is nI(X; Y ).

4 / 17

Brief Background on Statistical Mechanics

For large systems, classical analysis is not possible.

5 / 17

Brief Background on Statistical Mechanics

For large systems, classical analysis is not possible. Principle 1:

5 / 17

Brief Background on Statistical Mechanics

For large systems, classical analysis is not possible. Principle 1:

X - state of particle.

X = 0 X = 1 X = 2 X = 3 X = 4 E

5 / 17

Brief Background on Statistical Mechanics

For large systems, classical analysis is not possible. Principle 1:

X - state of particle. For a fixed temperature, PX(x) ∼ e−E(x)/kBT .

X = 0 X = 1 X = 2 X = 3 X = 4 E

PX(x) - the Boltzmann distribution.

5 / 17

Brief Background on Statistical Mechanics

For large systems, classical analysis is not possible. Principle 1:

X - state of particle. For a fixed temperature, PX(x) ∼ e−E(x)/kBT .

X = 0 X = 1 X = 2 X = 3 X = 4 E

PX(x) - the Boltzmann distribution.

5 / 17

Brief Background on Statistical Mechanics

For large systems, classical analysis is not possible. Principle 1:

X - state of particle. For a fixed temperature, PX(x) ∼ e−E(x)/kBT .

X = 0 X = 1 X = 2 X = 3 X = 4 E

PX(x) - the Boltzmann distribution. Principle 2:

5 / 17

Brief Background on Statistical Mechanics

For large systems, classical analysis is not possible. Principle 1:

X - state of particle. For a fixed temperature, PX(x) ∼ e−E(x)/kBT .

X = 0 X = 1 X = 2 X = 3 X = 4 E

PX(x) - the Boltzmann distribution. Principle 2:

The second law of thermodynamics: ∆S ≥ 0. S - the entropy of the system.

5 / 17

Brief Background on Statistical Mechanics

For large systems, classical analysis is not possible. Principle 1:

X - state of particle. For a fixed temperature, PX(x) ∼ e−E(x)/kBT .

X = 0 X = 1 X = 2 X = 3 X = 4 E

PX(x) - the Boltzmann distribution. Principle 2:

The second law of thermodynamics: ∆S ≥ 0. S - the entropy of the system. In a complete cycle: E[W] ≤ 0.

5 / 17

Past Results - Measurement-Based Work Extraction

Originated with Maxwell’s demon in the 19th century.

6 / 17

Past Results - Measurement-Based Work Extraction

Originated with Maxwell’s demon in the 19th century. Using fluctuation theorems, it was shown that [Sagawa and Ueda 2010] E[W] ≤ kBTI(X; Y ).

6 / 17

Past Results - Measurement-Based Work Extraction

Originated with Maxwell’s demon in the 19th century. Using fluctuation theorems, it was shown that [Sagawa and Ueda 2010] E[W] ≤ kBTI(X; Y ). Some systems achieve this, but no general achievability scheme.

6 / 17

Past Results - The Szilard Engine

Presented by Sagawa and Ueda as an example:

b c d

V = 1

7 / 17

Past Results - The Szilard Engine

Presented by Sagawa and Ueda as an example:

c d

V = 1 V0(X)

7 / 17

Past Results - The Szilard Engine

Presented by Sagawa and Ueda as an example:

d

V = 1 V0(X) Y

7 / 17

Past Results - The Szilard Engine

Presented by Sagawa and Ueda as an example: V = 1 V0(X) Y Vf(X)

7 / 17

Past Results - The Szilard Engine

Presented by Sagawa and Ueda as an example: V = 1 V0(X) Y Vf(X)

7 / 17

Past Results - The Szilard Engine

Presented by Sagawa and Ueda as an example: V = 1 V0(X) Y Vf(X) X - the particle’s location. Y - a noisy measurement.

7 / 17

Past Results - The Szilard Engine

Presented by Sagawa and Ueda as an example: V = 1 V0(X) Y Vf(X) X - the particle’s location. Y - a noisy measurement. Extracted work is W = kBT ln Vf(X|Y ) V0(X) .

7 / 17

Past Results - The Szilard Engine

Optimal value of Vf(X|Y ) is V ∗

f (X|Y ) = PX|Y (X|Y ).

8 / 17

Past Results - The Szilard Engine

Optimal value of Vf(X|Y ) is V ∗

f (X|Y ) = PX|Y (X|Y ).

Maximal extracted work is max E[W] = kBTI(X; Y ).

8 / 17

Past Results - The Szilard Engine

Optimal value of Vf(X|Y ) is V ∗

f (X|Y ) = PX|Y (X|Y ).

Maximal extracted work is max E[W] = kBTI(X; Y ). Upper bound is achieved.

8 / 17

Past Results - The Szilard Engine

Optimal value of Vf(X|Y ) is V ∗

f (X|Y ) = PX|Y (X|Y ).

Maximal extracted work is max E[W] = kBTI(X; Y ). Upper bound is achieved. Result is specific to one particle of ideal gas.

8 / 17

Main Results

Gambling Maxwell’s Demon Side information Measurements results

9 / 17

Main Results

Gambling Maxwell’s Demon Side information Measurements results

• X - odds

1/V0 - initial vol.

9 / 17

Main Results

Gambling Maxwell’s Demon Side information Measurements results

• X - odds

1/V0 - initial vol. bX|Y - betting strategy Vf - final vol.

9 / 17

Main Results

Gambling Maxwell’s Demon Side information Measurements results

• X - odds

1/V0 - initial vol. bX|Y - betting strategy Vf - final vol. log Sn - log of capital Wn - extracted work

9 / 17

Main Results

Gambling Maxwell’s Demon Side information Measurements results

• X - odds

1/V0 - initial vol. bX|Y - betting strategy Vf - final vol. log Sn - log of capital Wn - extracted work log Sn =

n

• i=1

log bX|Y (Xi|Yi)oX(Xi) , Wn =

n

• i=1

kBT ln Vf(Xi|Yi) V0(Xi) .

9 / 17

Some Immediate Consequences

• X ⇔ 1/V0 = 1/PX

10/ 17

Some Immediate Consequences

• X ⇔ 1/V0 = 1/PX ⇒ In physics, the bet is always fair.

10/ 17

Some Immediate Consequences

• X ⇔ 1/V0 = 1/PX ⇒ In physics, the bet is always fair.

Second law of thermo. - without measurements E[W] ≤ 0.

10/ 17

Some Immediate Consequences

• X ⇔ 1/V0 = 1/PX ⇒ In physics, the bet is always fair.

Second law of thermo. - without measurements E[W] ≤ 0. Enables straightforward generalization to m dividers V ∗

f (X|Y ) = PX|Y (X|Y ).

10/ 17

Recap

What we achieved so far:

11/ 17

Recap

What we achieved so far: One-to-one mapping.

11/ 17

Recap

What we achieved so far: One-to-one mapping. Specific to Szilard engine.

11/ 17

Continuous Random Variables - Physics

Horowitz and Parrondo 2011, Esposito and Van den Broeck 2011

A single particle in a potential field:

E0 fX x

12/ 17

Continuous Random Variables - Physics

Horowitz and Parrondo 2011, Esposito and Van den Broeck 2011

A single particle in a potential field:

E0 E0 fX x x fX|y Y

12/ 17

Continuous Random Variables - Physics

Horowitz and Parrondo 2011, Esposito and Van den Broeck 2011

A single particle in a potential field:

E0 E0 fX x x x fX|y fX|y Ef Y E0 → Ef

12/ 17

Continuous Random Variables - Physics

Horowitz and Parrondo 2011, Esposito and Van den Broeck 2011

A single particle in a potential field:

E0 E0 fX x x x fX|y fX|y Ef Y E0 → Ef

12/ 17

Continuous Random Variables - Physics

Horowitz and Parrondo 2011, Esposito and Van den Broeck 2011

A single particle in a potential field:

E0 E0 fX x x x fX|y fX|y Ef Y E0 → Ef

X - particle’s location. Y - noisy measurement.

12/ 17

Continuous Random Variables - Physics

Horowitz and Parrondo 2011, Esposito and Van den Broeck 2011

A single particle in a potential field:

E0 E0 fX x x x fX|y fX|y Ef Y E0 → Ef

X - particle’s location. Y - noisy measurement. QX|y - the Boltzmann distribution that stems from Ef.

12/ 17

Continuous Random Variables - Physics

Horowitz and Parrondo 2011, Esposito and Van den Broeck 2011

A single particle in a potential field:

E0 E0 fX x x x fX|y fX|y Ef Y E0 → Ef

X - particle’s location. Y - noisy measurement. QX|y - the Boltzmann distribution that stems from Ef. Theorem Q∗

X|y = arg min D(fX|y||QX|y) ∀y ∈ Y.

E[W] = kBT

• I(X; Y ) − D(fX|Y ||Q∗

X|Y |fY )

• .

12/ 17

Continuous Random Variables - Gambling

What’s the analogous case in gambling?

13/ 17

Continuous Random Variables - Gambling

What’s the analogous case in gambling? X is some continuous r.v., i.e., the price of a stock.

13/ 17

Continuous Random Variables - Gambling

What’s the analogous case in gambling? X is some continuous r.v., i.e., the price of a stock. Betting strategy:

1 2 3 4

x

Horse Race

bX|y

13/ 17

Continuous Random Variables - Gambling

What’s the analogous case in gambling? X is some continuous r.v., i.e., the price of a stock. Betting strategy:

1 2 3 4

x x

Horse Race Stocks

bX|y bX|y

13/ 17

Continuous Random Variables - Gambling

What’s the analogous case in gambling? X is some continuous r.v., i.e., the price of a stock. Betting strategy:

1 2 3 4

x x

Horse Race Stocks

bX|y bX|y Theorem b∗

X|y = arg min D(fX|y||bX|y) ∀y ∈ Y.

E[log S∗

n] = n

• I(X; Y ) − D(fX|Y ||b∗

X|Y |fY )

• .

13/ 17

Consequence 1 - Universal Engine

Gambling: X, Y discrete r.v. PX,Y is unknown.

14/ 17

Consequence 1 - Universal Engine

Gambling: X, Y discrete r.v. PX,Y is unknown.

• b - the universal portfolio [Cover and Ordentlich 1996].

14/ 17

Consequence 1 - Universal Engine

Gambling: X, Y discrete r.v. PX,Y is unknown.

• b - the universal portfolio [Cover and Ordentlich 1996].

Theorem lim

n→∞

1 nE[log Sn − log S∗

n] = 0.

• Sn - capital using
• b. S∗

n - capital using b∗ X|Y .

14/ 17

Consequence 1 - Universal Engine

Gambling: X, Y discrete r.v. PX,Y is unknown.

• b - the universal portfolio [Cover and Ordentlich 1996].

Theorem lim

n→∞

1 nE[log Sn − log S∗

n] = 0.

• Sn - capital using
• b. S∗

n - capital using b∗ X|Y .

Physics: X, Y discrete r.v. Unknown measurement error PY |X.

14/ 17

Consequence 1 - Universal Engine

Gambling: X, Y discrete r.v. PX,Y is unknown.

• b - the universal portfolio [Cover and Ordentlich 1996].

Theorem lim

n→∞

1 nE[log Sn − log S∗

n] = 0.

• Sn - capital using
• b. S∗

n - capital using b∗ X|Y .

Physics: X, Y discrete r.v. Unknown measurement error PY |X. We developed the analogous universal control protocol.

14/ 17

Consequence 1 - Universal Engine

Gambling: X, Y discrete r.v. PX,Y is unknown.

• b - the universal portfolio [Cover and Ordentlich 1996].

Theorem lim

n→∞

1 nE[log Sn − log S∗

n] = 0.

• Sn - capital using
• b. S∗

n - capital using b∗ X|Y .

Physics: X, Y discrete r.v. Unknown measurement error PY |X. We developed the analogous universal control protocol. Theorem lim

n→∞

1 nE[ Wn − W ∗

n] = 0.

14/ 17

Consequence 2 - Memory

Gambling: Dependent races. At round i, Xi−1, Y i are known.

15/ 17

Consequence 2 - Memory

Gambling: Dependent races. At round i, Xi−1, Y i are known. Theorem [Permuter, Kim and Weissman 2011] b∗

Xi|Xi−1,Y i = PXi|Xi−1,Y i.

E[log Sn(Xn||Y n)] − E[log Sn(Xn)] = I(Y n → Xn). Sn(Xn||Y n) - capital with causal side information. Sn(Xn)

• capital without side information.

15/ 17

Consequence 2 - Memory

Gambling: Dependent races. At round i, Xi−1, Y i are known. Theorem [Permuter, Kim and Weissman 2011] b∗

Xi|Xi−1,Y i = PXi|Xi−1,Y i.

E[log Sn(Xn||Y n)] − E[log Sn(Xn)] = I(Y n → Xn). Sn(Xn||Y n) - capital with causal side information. Sn(Xn)

• capital without side information.

Physics: Dependence through fast cycles, hysteresis, etc.

15/ 17

Consequence 2 - Memory

Gambling: Dependent races. At round i, Xi−1, Y i are known. Theorem [Permuter, Kim and Weissman 2011] b∗

Xi|Xi−1,Y i = PXi|Xi−1,Y i.

E[log Sn(Xn||Y n)] − E[log Sn(Xn)] = I(Y n → Xn). Sn(Xn||Y n) - capital with causal side information. Sn(Xn)

• capital without side information.

Physics: Dependence through fast cycles, hysteresis, etc. Theorem Q∗

Xi|Xi−1,Y i = PXi|Xi−1,Y i.

E[Wn(Xn||Y n) − Wn(Xn)] = kBTI(Y n → Xn).

15/ 17

Summary

One-to-one mapping between gambling and measurement-based work extraction.

16/ 17

Summary

One-to-one mapping between gambling and measurement-based work extraction. Extension of gambling to continuous random variables.

16/ 17

Summary

One-to-one mapping between gambling and measurement-based work extraction. Extension of gambling to continuous random variables. Universal work extraction.

16/ 17

Summary

One-to-one mapping between gambling and measurement-based work extraction. Extension of gambling to continuous random variables. Universal work extraction. Analysis of work extraction with cycle dependence.

16/ 17

Summary

One-to-one mapping between gambling and measurement-based work extraction. Extension of gambling to continuous random variables. Universal work extraction. Analysis of work extraction with cycle dependence. Full version available on arXiv: http://arxiv.org/abs/1404.6788

16/ 17

Future Work

Multiple particles.

17/ 17

Future Work

Multiple particles. Generalize universal portfolios for continuous r.v.

17/ 17

Future Work

Multiple particles. Generalize universal portfolios for continuous r.v.

Thank you !

17/ 17

Future Work

Multiple particles. Generalize universal portfolios for continuous r.v.

Thank you !

17/ 17

Future Work

Multiple particles. Generalize universal portfolios for continuous r.v.

Thank you !

17/ 17

Future Work

Multiple particles. Generalize universal portfolios for continuous r.v.

Thank you !

17/ 17

Maximization Over PX

In the Szilard engine: Control over PX through V0. Maximal extracted work is max E[W] = kBT max

PX I(X; Y ).

17/ 17

Maximization Over PX

In the Szilard engine: Control over PX through V0. Maximal extracted work is max E[W] = kBT max

PX I(X; Y ).

In gambling: Control over PX through choice of race track. Maximal capital growth is max E[log Sn] = n max

PX∈P I(X; Y ).

17/ 17