Regret bounds for online variational inference Pierre Alquier ACML - PowerPoint PPT Presentation

Regret bounds for online variational inference Pierre Alquier ACML – Nagoya, Nov. 18, 2019 Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

Co-authors Emtiyaz Khan Badr-Eddine Chérief-Abdellatif Approximate Bayesian Inference team https : // emtiyaz . github . io / Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

Motivation K. Osawa, S. Swaroop, A. Jain, R. Eschenhagen, R. E. Turner, R. Yokota, M. E. Khan (2019). Practical Deep Learning with Bayesian Principles . NeurIPS. Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

Motivation K. Osawa, S. Swaroop, A. Jain, R. Eschenhagen, R. E. Turner, R. Yokota, M. E. Khan (2019). Practical Deep Learning with Bayesian Principles . NeurIPS. 1 proposes a fast algorithm to approximate the posterior, 2 applies it to train Deep Neural Networks on CIFAR-10, ImageNet ... 3 observation : improved uncertainty quantification. Picture : Roman Bachmann. Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

Motivation K. Osawa, S. Swaroop, A. Jain, R. Eschenhagen, R. E. Turner, R. Yokota, M. E. Khan (2019). Practical Deep Learning with Bayesian Principles . NeurIPS. 1 proposes a fast algorithm to approximate the posterior, 2 applies it to train Deep Neural Networks on CIFAR-10, ImageNet ... 3 observation : improved uncertainty quantification. Picture : Roman Bachmann. Objective : provide a theoretical analysis of this algorithm. Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

Motivation K. Osawa, S. Swaroop, A. Jain, R. Eschenhagen, R. E. Turner, R. Yokota, M. E. Khan (2019). Practical Deep Learning with Bayesian Principles . NeurIPS. 1 proposes a fast algorithm to approximate the posterior, 2 applies it to train Deep Neural Networks on CIFAR-10, ImageNet ... 3 observation : improved uncertainty quantification. Picture : Roman Bachmann. Objective : provide a theoretical analysis of this algorithm. First step : simplified versions. Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

The sequential prediction problem Sequential prediction problem Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

The sequential prediction problem Sequential prediction problem x 1 given 1 1 Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

The sequential prediction problem Sequential prediction problem x 1 given 1 1 predict y 1 : ˆ 2 y 1 Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

The sequential prediction problem Sequential prediction problem x 1 given 1 1 predict y 1 : ˆ 2 y 1 y 1 is revealed 3 Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

The sequential prediction problem Sequential prediction problem x 1 given 1 1 predict y 1 : ˆ 2 y 1 y 1 is revealed 3 x 2 given 2 1 Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

The sequential prediction problem Sequential prediction problem x 1 given 1 1 predict y 1 : ˆ 2 y 1 y 1 is revealed 3 x 2 given 2 1 predict y 2 : ˆ 2 y 2 Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

The sequential prediction problem Sequential prediction problem x 1 given 1 1 predict y 1 : ˆ 2 y 1 y 1 is revealed 3 x 2 given 2 1 predict y 2 : ˆ 2 y 2 y 2 revealed 3 Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

The sequential prediction problem Sequential prediction problem x 1 given 1 1 predict y 1 : ˆ 2 y 1 y 1 is revealed 3 x 2 given 2 1 predict y 2 : ˆ 2 y 2 y 2 revealed 3 x 3 given 3 1 Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

The sequential prediction problem Sequential prediction problem x 1 given 1 1 predict y 1 : ˆ 2 y 1 y 1 is revealed 3 x 2 given 2 1 predict y 2 : ˆ 2 y 2 y 2 revealed 3 x 3 given 3 1 predict y 3 : ˆ 2 y 3 Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

The sequential prediction problem Sequential prediction problem x 1 given 1 1 predict y 1 : ˆ 2 y 1 y 1 is revealed 3 x 2 given 2 1 predict y 2 : ˆ 2 y 2 y 2 revealed 3 x 3 given 3 1 predict y 3 : ˆ 2 y 3 y 3 revealed 3 4 . . . Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

The sequential prediction problem Sequential prediction problem x 1 given 1 1 predict y 1 : ˆ Objective : 2 y 1 y 1 is revealed 3 x 2 given 2 1 predict y 2 : ˆ 2 y 2 y 2 revealed 3 x 3 given 3 1 predict y 3 : ˆ 2 y 3 y 3 revealed 3 4 . . . Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

The sequential prediction problem Sequential prediction problem x 1 given 1 1 predict y 1 : ˆ Objective : make sure that 2 y 1 y 1 is revealed we learn to predict well as 3 soon as possible . x 2 given 2 1 predict y 2 : ˆ 2 y 2 y 2 revealed 3 x 3 given 3 1 predict y 3 : ˆ 2 y 3 y 3 revealed 3 4 . . . Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

The sequential prediction problem Sequential prediction problem x 1 given 1 1 predict y 1 : ˆ Objective : make sure that 2 y 1 y 1 is revealed we learn to predict well as 3 soon as possible . Keep x 2 given 2 1 predict y 2 : ˆ 2 y 2 T y 2 revealed 3 � ℓ (ˆ y t , y t ) x 3 given 3 1 t = 1 predict y 3 : ˆ 2 y 3 y 3 revealed as small as possible. 3 4 . . . Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

Online gradient algorithm (OGA) Given a set of predictors { f θ , θ ∈ Θ ⊂ R d } , e.g f θ ( x ) = � θ, x � , an initial guess θ 1 , ˆ y t = f θ t ( x t ) and θ t + 1 = θ t − η ∇ θ ℓ ( f θ t ( x t ) , y t ) . Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

Online gradient algorithm (OGA) Given a set of predictors { f θ , θ ∈ Θ ⊂ R d } , e.g f θ ( x ) = � θ, x � , an initial guess θ 1 , y t = f θ t ( x t ) ˆ and θ t + 1 = θ t − η ∇ θ ℓ t ( θ t ) . Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

Online gradient algorithm (OGA) Given a set of predictors { f θ , θ ∈ Θ ⊂ R d } , e.g f θ ( x ) = � θ, x � , an initial guess θ 1 , y t = f θ t ( x t ) ˆ and θ t + 1 = θ t − η ∇ θ ℓ t ( θ t ) . Note that θ t + 1 can be obtained by : �� � � t + � θ − θ 1 � 2 � 1 min ∇ θ ℓ s ( θ s ) θ, , 2 η θ s = 1 �� � + � θ − θ t � 2 � 2 min θ, ∇ θ ℓ t ( θ t ) . 2 η θ Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

Bayesian learning and variational inference (VI) � � t � π t + 1 ( θ ) := π ( θ | x 1 , y 1 , . . . , x t , y t ) ∝ exp − η ℓ s ( θ ) π ( θ ) . s = 1 Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

Bayesian learning and variational inference (VI) � � t � π t + 1 ( θ ) := π ( θ | x 1 , y 1 , . . . , x t , y t ) ∝ exp − η ℓ s ( θ ) π ( θ ) . s = 1 Not tractable in general, leading to variational approximations : ˜ π t + 1 ( θ ) = arg min KL ( q , π t + 1 ) q ∈F � � � � t + KL ( q , π ) � = arg min ℓ s ( θ ) . E θ ∼ q η q ∈F s = 1 Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

Bayesian learning and variational inference (VI) � � t � π t + 1 ( θ ) := π ( θ | x 1 , y 1 , . . . , x t , y t ) ∝ exp − η ℓ s ( θ ) π ( θ ) . s = 1 Not tractable in general, leading to variational approximations : ˜ π t + 1 ( θ ) = arg min KL ( q , π t + 1 ) q ∈F � � � � t + KL ( q , π ) � = arg min ℓ s ( θ ) . E θ ∼ q η q ∈F s = 1 Formula for the online update of π t + 1 : π t + 1 ( θ ) ∝ exp ( − ηℓ t ( θ )) π t ( θ ) . Q1 : can we similarly define a sequential update for a variational approximation ? Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

Regret bounds for Bayesian inference Theorem (classical result) Under the assumption that the loss is bounded by B , the Bayesian update leads to T � E θ ∼ π t [ ℓ t ( θ )] t = 1 � T � E θ ∼ q [ ℓ t ( θ )] + η B 2 T + KL ( q , π ) � ≤ inf . 8 η q t = 1 Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

Regret bounds for Bayesian inference Theorem (classical result) Under the assumption that the loss is bounded by B , the Bayesian update leads to T � E θ ∼ π t [ ℓ t ( θ )] t = 1 � T � E θ ∼ q [ ℓ t ( θ )] + η B 2 T + KL ( q , π ) � ≤ inf . 8 η q t = 1 √ Derivation of the infimum and η ∼ T “usually” leads to T T � � � E θ ∼ π t [ ℓ t ( θ )] ≤ inf ℓ t ( θ ) + O ( dT log( T )) . θ t = 1 t = 1 Pierre Alquier, RIKEN AIP Regret bounds for online variational inference

Regret bounds for Bayesian inference Theorem (classical result) Under the assumption that the loss is bounded by B , the Bayesian update leads to T � E θ ∼ π t [ ℓ t ( θ )] t = 1 � T � E θ ∼ q [ ℓ t ( θ )] + η B 2 T + KL ( q , π ) � ≤ inf . 8 η q t = 1 √ Derivation of the infimum and η ∼ T “usually” leads to T T � � � E θ ∼ π t [ ℓ t ( θ )] ≤ inf ℓ t ( θ ) + O ( dT log( T )) . θ t = 1 t = 1 Q2 : can we derive similar results for online VI ? Pierre Alquier, RIKEN AIP Regret bounds for online variational inference