Stability of the Shannon-Stam Inequality Dan Mikulincer Students - PowerPoint PPT Presentation

Stability of the Shannon-Stam Inequality Dan Mikulincer Students Probability Day, 2019 Weizmann Institute of Science Joint work with Ronen Eldan 1

Relative Entropy The central quantity we will deal is relative entropy: Definition (Relative Entropy) Let X ∼ µ, Y ∼ ν be random vectors in R d , define the entropy of X , relative to Y as  � � d µ � ln d µ if µ ≪ ν  d ν  Ent ( X || Y ) = Ent ( µ || ν ) := . R d ∞ otherwise   2

The Shannon-Stam Inequality In 48 ′ Shannon noted the following inequality, which was later proved by Stam, in 56 ′ . Theorem (Shannon-Stam Inequality) Let X , Y be random vectors in R d and let G ∼ N (0 , I ) be a random vector with the law of the standard Gaussian. Then, for any λ ∈ [0 , 1] √ √ 1 − λ Y || G ) ≤ λ Ent ( X || G ) + (1 − λ ) Ent ( Y || G ) . Ent ( λ X + Moreover, equality holds if and only if X and Y are Gaussians with identical covariances. Remark: Shannon and Stam actually proved an equivalent form of the inequality, called the entropy power inequality. The equivalence was observed by Lieb in 78’. 3

The Shannon-Stam Inequality In 48 ′ Shannon noted the following inequality, which was later proved by Stam, in 56 ′ . Theorem (Shannon-Stam Inequality) Let X , Y be random vectors in R d and let G ∼ N (0 , I ) be a random vector with the law of the standard Gaussian. Then, for any λ ∈ [0 , 1] √ √ 1 − λ Y || G ) ≤ λ Ent ( X || G ) + (1 − λ ) Ent ( Y || G ) . Ent ( λ X + Moreover, equality holds if and only if X and Y are Gaussians with identical covariances. Remark: Shannon and Stam actually proved an equivalent form of the inequality, called the entropy power inequality. The equivalence was observed by Lieb in 78’. 3

Stability Define the deficit √ √ δ λ ( X , Y ) = λ Ent ( X || G )+(1 − λ ) Ent ( Y || G ) − Ent ( λ X + 1 − λ Y || G ) . The question of stability deals with approximate equality cases. Question Suppose that δ λ ( X , Y ) is small, must X and Y be ’close’ to Gaussian vectors, which are themselves ’close’ to each other? We will now show that the deficit can be bounded in terms of a stochastic process and that in certain cases this gives a positive answer to the above question. 4

Stability Define the deficit √ √ δ λ ( X , Y ) = λ Ent ( X || G )+(1 − λ ) Ent ( Y || G ) − Ent ( λ X + 1 − λ Y || G ) . The question of stability deals with approximate equality cases. Question Suppose that δ λ ( X , Y ) is small, must X and Y be ’close’ to Gaussian vectors, which are themselves ’close’ to each other? We will now show that the deficit can be bounded in terms of a stochastic process and that in certain cases this gives a positive answer to the above question. 4

Stability Define the deficit √ √ δ λ ( X , Y ) = λ Ent ( X || G )+(1 − λ ) Ent ( Y || G ) − Ent ( λ X + 1 − λ Y || G ) . The question of stability deals with approximate equality cases. Question Suppose that δ λ ( X , Y ) is small, must X and Y be ’close’ to Gaussian vectors, which are themselves ’close’ to each other? We will now show that the deficit can be bounded in terms of a stochastic process and that in certain cases this gives a positive answer to the above question. 4

F¨ ollmer Martingales We focus on the one dimensional case and λ = 1 2 . Let X be centered random variable, and let B t denote a standard Brownian motion. F¨ olmmer (1984) and then Lehec (2011) have shown that there exists a process Γ X t , such that 1 � Γ X • t dB t has the law of X . 0 � t ) 2 � (1 − Γ X E 1 • Ent ( X || G ) = 1 � dt . 2 1 − t 0 1 • If H X � H X t is another process such that t dB t has the law of X , 0 � t ) 2 � � t ) 2 � 1 1 (1 − H X (1 − Γ X E E � � dt ≥ dt . 1 − t 1 − t 0 0 5

F¨ ollmer Martingales We focus on the one dimensional case and λ = 1 2 . Let X be centered random variable, and let B t denote a standard Brownian motion. F¨ olmmer (1984) and then Lehec (2011) have shown that there exists a process Γ X t , such that 1 � Γ X • t dB t has the law of X . 0 � t ) 2 � (1 − Γ X E 1 • Ent ( X || G ) = 1 � dt . 2 1 − t 0 1 • If H X � H X t is another process such that t dB t has the law of X , 0 � t ) 2 � � t ) 2 � 1 1 (1 − H X (1 − Γ X E E � � dt ≥ dt . 1 − t 1 − t 0 0 5

F¨ ollmer Martingales We focus on the one dimensional case and λ = 1 2 . Let X be centered random variable, and let B t denote a standard Brownian motion. F¨ olmmer (1984) and then Lehec (2011) have shown that there exists a process Γ X t , such that 1 � Γ X • t dB t has the law of X . 0 � t ) 2 � (1 − Γ X E 1 • Ent ( X || G ) = 1 � dt . 2 1 − t 0 1 • If H X � H X t is another process such that t dB t has the law of X , 0 � t ) 2 � � t ) 2 � 1 1 (1 − H X (1 − Γ X E E � � dt ≥ dt . 1 − t 1 − t 0 0 5

F¨ ollmer Martingales We focus on the one dimensional case and λ = 1 2 . Let X be centered random variable, and let B t denote a standard Brownian motion. F¨ olmmer (1984) and then Lehec (2011) have shown that there exists a process Γ X t , such that 1 � Γ X • t dB t has the law of X . 0 � t ) 2 � (1 − Γ X E 1 • Ent ( X || G ) = 1 � dt . 2 1 − t 0 1 • If H X � H X t is another process such that t dB t has the law of X , 0 � t ) 2 � � t ) 2 � 1 1 (1 − H X (1 − Γ X E E � � dt ≥ dt . 1 − t 1 − t 0 0 5

Bounding the Deficit Now, for X , Y random variables, take two independent Brownian motions B X t , B Y and Γ X t , Γ Y t as above. Note that if G 1 and G 2 are t standard Gaussians, then for any a , b ∈ R law � a 2 + b 2 G , aG 1 + bG 2 = where G is another standard Gaussian. This implies  1 1  1 � t ) 2 + (Γ Y (Γ X t ) 2 X + Y 1 � � �  law Γ X t dB X Γ Y t dB Y √ √ = t + = dB t .  t 2 2 2 0 0 0 for some Brownian motion B t . 6

Bounding the Deficit Now, for X , Y random variables, take two independent Brownian motions B X t , B Y and Γ X t , Γ Y t as above. Note that if G 1 and G 2 are t standard Gaussians, then for any a , b ∈ R law � a 2 + b 2 G , aG 1 + bG 2 = where G is another standard Gaussian. This implies  1 1  1 � t ) 2 + (Γ Y (Γ X t ) 2 X + Y 1 � � �  law Γ X t dB X Γ Y t dB Y √ √ = t + = dB t .  t 2 2 2 0 0 0 for some Brownian motion B t . 6

Bounding the Deficit Now, for X , Y random variables, take two independent Brownian motions B X t , B Y and Γ X t , Γ Y t as above. Note that if G 1 and G 2 are t standard Gaussians, then for any a , b ∈ R law � a 2 + b 2 G , aG 1 + bG 2 = where G is another standard Gaussian. This implies  1 1  1 � t ) 2 + (Γ Y (Γ X t ) 2 X + Y 1 � � �  law Γ X t dB X Γ Y t dB Y √ √ = t + = dB t .  t 2 2 2 0 0 0 for some Brownian motion B t . 6

Bounding the Deficit (1 − H t ) 2 � 1 � � E � � (Γ X t ) 2 +(Γ Y t ) 2 X + Y ≤ 1 � If H t = , Ent 2 || G dt . √ 2 2 1 − t 0 Consequently, � t ) 2 � � t ) 2 � 1 (1 − Γ Y (1 − Γ X E E (1 − H t ) 2 � � � − E 2 δ 1 2 ( X , Y ) ≥ + dt 2(1 − t ) 2(1 − t ) 1 − t 0 1 2 E [ H t ] − E [Γ X t ] − E [Γ Y � t ] = . 1 − t 0 Using concavity of the square root then shows 1 (Γ X t − Γ Y t ) 2 � � � δ 1 2 ( X , Y ) � E dt . (1 − t )(Γ X t + Γ Y t ) 0 7

Bounding the Deficit (1 − H t ) 2 � 1 � � E � � (Γ X t ) 2 +(Γ Y t ) 2 X + Y ≤ 1 � If H t = , Ent 2 || G dt . √ 2 2 1 − t 0 Consequently, � t ) 2 � � t ) 2 � 1 (1 − Γ Y (1 − Γ X E E (1 − H t ) 2 � � � − E 2 δ 1 2 ( X , Y ) ≥ + dt 2(1 − t ) 2(1 − t ) 1 − t 0 1 2 E [ H t ] − E [Γ X t ] − E [Γ Y � t ] = . 1 − t 0 Using concavity of the square root then shows 1 (Γ X t − Γ Y t ) 2 � � � δ 1 2 ( X , Y ) � E dt . (1 − t )(Γ X t + Γ Y t ) 0 7

Bounding the Deficit (1 − H t ) 2 � 1 � � E � � (Γ X t ) 2 +(Γ Y t ) 2 X + Y ≤ 1 � If H t = , Ent 2 || G dt . √ 2 2 1 − t 0 Consequently, � t ) 2 � � t ) 2 � 1 (1 − Γ Y (1 − Γ X E E (1 − H t ) 2 � � � − E 2 δ 1 2 ( X , Y ) ≥ + dt 2(1 − t ) 2(1 − t ) 1 − t 0 1 2 E [ H t ] − E [Γ X t ] − E [Γ Y � t ] = . 1 − t 0 Using concavity of the square root then shows 1 (Γ X t − Γ Y t ) 2 � � � δ 1 2 ( X , Y ) � E dt . (1 − t )(Γ X t + Γ Y t ) 0 7

Bounding the Deficit (1 − H t ) 2 � 1 � � E � � (Γ X t ) 2 +(Γ Y t ) 2 X + Y ≤ 1 � If H t = , Ent 2 || G dt . √ 2 2 1 − t 0 Consequently, � t ) 2 � � t ) 2 � 1 (1 − Γ Y (1 − Γ X E E (1 − H t ) 2 � � � − E 2 δ 1 2 ( X , Y ) ≥ + dt 2(1 − t ) 2(1 − t ) 1 − t 0 1 2 E [ H t ] − E [Γ X t ] − E [Γ Y � t ] = . 1 − t 0 Using concavity of the square root then shows 1 (Γ X t − Γ Y t ) 2 � � � δ 1 2 ( X , Y ) � E dt . (1 − t )(Γ X t + Γ Y t ) 0 7