A Closer Look at Adaptive Regret Dmitry Adamskiy Joint work with - PowerPoint PPT Presentation

A Closer Look at Adaptive Regret Dmitry Adamskiy Joint work with Wouter Koolen, Volodya Vovk and Alexey Chernov Department of Computer Science Royal Holloway, University of London 15/11/2014 Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 1 / 21

Outline Why adaptive regret? 1 Setup 2 Results 3 Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 2 / 21

Why adaptive regret? 1 Setup 2 Results 3 Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 3 / 21

Weather forecasting: adaptivity Predictor Expert Expert Expert Nature Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

Weather forecasting: adaptivity Predictor Expert 30% Expert 90% Expert 20% Nature Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

Weather forecasting: adaptivity Predictor 55% Expert 30% Expert 90% Expert 20% Nature Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

Weather forecasting: adaptivity Predictor 55% Expert 30% Expert 90% Expert 20% Nature Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

Weather forecasting: adaptivity Predictor 55% Expert 30% 40% Expert 90% 70% Expert 20% 65% Nature Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

Weather forecasting: adaptivity Predictor 55% 65% Expert 30% 40% Expert 90% 70% Expert 20% 65% Nature Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

Weather forecasting: adaptivity Predictor 55% 65% Expert 30% 40% Expert 90% 70% Expert 20% 65% Nature Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

Weather forecasting: adaptivity Predictor 55% 65% Expert 30% 40% . . . Expert 90% 70% . . . Expert 20% 65% . . . Nature Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

Weather forecasting: adaptivity Predictor 55% 65% . . . Expert 30% 40% . . . Expert 90% 70% . . . Expert 20% 65% . . . Nature Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

Weather forecasting: adaptivity Predictor 55% 65% . . . Expert 30% 40% . . . 90% 70% . . . Expert Expert 20% 65% . . . Nature . . . Goal: close to the best expert overall (solution: AA) Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

Weather forecasting: adaptivity Predictor 55% 65% . . . Expert 30% 40% . . . is bad on foggy days! 90% 70% . . . Expert Expert 20% 65% . . . Nature . . . Goal: close to the best expert overall (solution: AA) Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

Weather forecasting: adaptivity Predictor 55% 65% . . . Expert 30% 40% . . . is bad on foggy days! 90% 70% . . . Expert Expert 20% 65% . . . drunk on weekends! Nature . . . Goal: close to the best expert overall (solution: AA) Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

Weather forecasting: adaptivity Predictor 55% 65% . . . Expert 30% 40% . . . is bad on foggy days! 90% 70% . . . goes on training! Expert Expert 20% 65% . . . drunk on weekends! Nature . . . Goal: close to the best expert overall (solution: AA) Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

Weather forecasting: adaptivity Predictor 55% 65% . . . Expert 30% 40% . . . is bad on foggy days! Expert 90% 70% . . . goes on training! Expert 20% 65% . . . drunk on weekends! Nature . . . Goal: close to the best expert overall (solution: AA) Adaptive goal : close to the best expert on every time interval Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

Example continued Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 5 / 21

Example continued Non-adaptive predictor would lose trust in the first guy. Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 5 / 21

Adaptive algorithms We studied several approaches to adaptivity: Blowing up the set of experts to compete with virtual sleeping experts [DA, Koolen, Chernov, Vovk, 2012] Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 6 / 21

Adaptive algorithms We studied several approaches to adaptivity: Blowing up the set of experts to compete with virtual sleeping experts [DA, Koolen, Chernov, Vovk, 2012] Turned out to be Fixed Share [Herbster, Warmuth 1998]! Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 6 / 21

Adaptive algorithms We studied several approaches to adaptivity: Blowing up the set of experts to compete with virtual sleeping experts [DA, Koolen, Chernov, Vovk, 2012] Turned out to be Fixed Share [Herbster, Warmuth 1998]! Restarting existing algorithms and combining their predictions [Hazan, Seshadhri, 2009] Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 6 / 21

Adaptive algorithms We studied several approaches to adaptivity: Blowing up the set of experts to compete with virtual sleeping experts [DA, Koolen, Chernov, Vovk, 2012] Turned out to be Fixed Share [Herbster, Warmuth 1998]! Restarting existing algorithms and combining their predictions [Hazan, Seshadhri, 2009] Also turned out to be Fixed Share! Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 6 / 21

Adaptive properties of Fixed Share: results Fixed Share is known for tracking . Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 7 / 21

Adaptive properties of Fixed Share: results Fixed Share is known for tracking . L FS [ 1 , T ] − L S [ 1 , T ] ≤ ln N +( m − 1 ) ln ( N − 1 ) − ( m − 1 ) ln α − ( T − m ) ln ( 1 − α ) , Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 7 / 21

Adaptive properties of Fixed Share: results Fixed Share is known for tracking . L FS [ 1 , T ] − L S [ 1 , T ] ≤ ln N +( m − 1 ) ln ( N − 1 ) − ( m − 1 ) ln α − ( T − m ) ln ( 1 − α ) , What about its adaptivity? Our results Figured out the Worst-Case adaptive regret of Fixed Share 1 Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 7 / 21

Adaptive properties of Fixed Share: results Fixed Share is known for tracking . L FS [ 1 , T ] − L S [ 1 , T ] ≤ ln N +( m − 1 ) ln ( N − 1 ) − ( m − 1 ) ln α − ( T − m ) ln ( 1 − α ) , What about its adaptivity? Our results Figured out the Worst-Case adaptive regret of Fixed Share 1 Proved the optimality of Fixed Share — “no algorithm could have 2 better guarantees on all time intervals” Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 7 / 21

Why adaptive regret? 1 Setup 2 Results 3 Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 8 / 21

Protocol: Mix loss for t = 1 , 2 , . . . do Learner announces probability vector � w t ∈ △ N Reality announces loss vector � ℓ t ∈ [ −∞ , ∞ ] N t e − ℓ n n w n Learner suffers loss ℓ t := − ln � t end for Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 9 / 21

Adaptive Regret Goal: On every time interval [ t 1 , t 2 ] we want to be not much worse than the best expert on that interval. We are interested in small adaptive regret Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 10 / 21

Adaptive Regret Goal: On every time interval [ t 1 , t 2 ] we want to be not much worse than the best expert on that interval. We are interested in small adaptive regret Definition The adaptive regret of the algorithm on the interval [ t 1 , t 2 ] is the loss of the algorithm there minus the lost of the best expert there: L j R [ t 1 , t 2 ] := L [ t 1 , t 2 ] − min [ t 1 , t 2 ] j Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 10 / 21

AA and Fixed Share Aggregating Algorithm [Vovk 1990] updates weights as: t e − ℓ n w n t w n t + 1 := t . t e − ℓ n n w n � Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 11 / 21

AA and Fixed Share Aggregating Algorithm [Vovk 1990] updates weights as: t e − ℓ n w n t w n t + 1 := t . t e − ℓ n n w n � Fixed Share family is defined by the sequence of “switching rates” α t . Then the weight update is t e − ℓ n w n α t + 1 � N � t w n t + 1 := N − 1 + 1 − N − 1 α t + 1 t . t e − ℓ n n w n � Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 11 / 21

AA and Fixed Share Aggregating Algorithm [Vovk 1990] updates weights as: t e − ℓ n w n t w n t + 1 := t . t e − ℓ n n w n � Fixed Share family is defined by the sequence of “switching rates” α t . Then the weight update is t e − ℓ n w n α t + 1 � N � t w n t + 1 := N − 1 + 1 − N − 1 α t + 1 t . t e − ℓ n n w n � Adaptivity hides in the first term. Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 11 / 21