A Closer Look at Adaptive Regret Dmitry Adamskiy Joint work with - - PowerPoint PPT Presentation

▶

a closer look at adaptive regret

A Closer Look at Adaptive Regret Dmitry Adamskiy Joint work with - - PowerPoint PPT Presentation

Oct 15, 2022 25 likes •531 views

A Closer Look at Adaptive Regret Dmitry Adamskiy Joint work with Wouter Koolen, Volodya Vovk and Alexey Chernov Department of Computer Science Royal Holloway, University of London 15/11/2014 Adamskiy (RHUL) A Closer Look at Adaptive Regret

slide-1

SLIDE 1

A Closer Look at Adaptive Regret

Dmitry Adamskiy

Joint work with Wouter Koolen, Volodya Vovk and Alexey Chernov Department of Computer Science Royal Holloway, University of London

15/11/2014

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 1 / 21

slide-2

SLIDE 2

Outline

1

Why adaptive regret?

2

Setup

3

Results

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 2 / 21

slide-3

SLIDE 3

1

Why adaptive regret?

2

Setup

3

Results

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 3 / 21

slide-4

SLIDE 4

Weather forecasting: adaptivity

Predictor Expert Expert Expert Nature

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

slide-5

SLIDE 5

Weather forecasting: adaptivity

Predictor Expert 30% Expert 90% Expert 20% Nature

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

slide-6

SLIDE 6

Weather forecasting: adaptivity

Predictor 55% Expert 30% Expert 90% Expert 20% Nature

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

slide-7

SLIDE 7

Weather forecasting: adaptivity

Predictor 55% Expert 30% Expert 90% Expert 20% Nature

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

slide-8

SLIDE 8

Weather forecasting: adaptivity

Predictor 55% Expert 30% 40% Expert 90% 70% Expert 20% 65% Nature

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

slide-9

SLIDE 9

Weather forecasting: adaptivity

Predictor 55% 65% Expert 30% 40% Expert 90% 70% Expert 20% 65% Nature

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

slide-10

SLIDE 10

Weather forecasting: adaptivity

Predictor 55% 65% Expert 30% 40% Expert 90% 70% Expert 20% 65% Nature

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

slide-11

SLIDE 11

Weather forecasting: adaptivity

Predictor 55% 65% Expert 30% 40% . . . Expert 90% 70% . . . Expert 20% 65% . . . Nature

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

slide-12

SLIDE 12

Weather forecasting: adaptivity

Predictor 55% 65% . . . Expert 30% 40% . . . Expert 90% 70% . . . Expert 20% 65% . . . Nature

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

slide-13

SLIDE 13

Weather forecasting: adaptivity

Predictor 55% 65% . . . Expert 30% 40% . . . Expert 90% 70% . . . Expert 20% 65% . . . Nature . . . Goal: close to the best expert overall (solution: AA)

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

slide-14

SLIDE 14

Weather forecasting: adaptivity

Predictor 55% 65% . . . Expert 30% 40% . . . is bad on foggy days! Expert 90% 70% . . . Expert 20% 65% . . . Nature . . . Goal: close to the best expert overall (solution: AA)

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

slide-15

SLIDE 15

Weather forecasting: adaptivity

Predictor 55% 65% . . . Expert 30% 40% . . . is bad on foggy days! Expert 90% 70% . . . Expert 20% 65% . . . drunk on weekends! Nature . . . Goal: close to the best expert overall (solution: AA)

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

slide-16

SLIDE 16

Weather forecasting: adaptivity

Predictor 55% 65% . . . Expert 30% 40% . . . is bad on foggy days! Expert 90% 70% . . . goes on training! Expert 20% 65% . . . drunk on weekends! Nature . . . Goal: close to the best expert overall (solution: AA)

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

slide-17

SLIDE 17

Weather forecasting: adaptivity

Predictor 55% 65% . . . Expert 30% 40% . . . is bad on foggy days! Expert 90% 70% . . . goes on training! Expert 20% 65% . . . drunk on weekends! Nature . . . Goal: close to the best expert overall (solution: AA) Adaptive goal: close to the best expert on every time interval

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

slide-18

SLIDE 18

Example continued

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 5 / 21

slide-19

SLIDE 19

Example continued

Non-adaptive predictor would lose trust in the first guy.

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 5 / 21

slide-20

SLIDE 20

Adaptive algorithms

We studied several approaches to adaptivity: Blowing up the set of experts to compete with virtual sleeping experts [DA, Koolen, Chernov, Vovk, 2012]

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 6 / 21

slide-21

SLIDE 21

Adaptive algorithms

We studied several approaches to adaptivity: Blowing up the set of experts to compete with virtual sleeping experts [DA, Koolen, Chernov, Vovk, 2012] Turned out to be Fixed Share [Herbster, Warmuth 1998]!

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 6 / 21

slide-22

SLIDE 22

Adaptive algorithms

We studied several approaches to adaptivity: Blowing up the set of experts to compete with virtual sleeping experts [DA, Koolen, Chernov, Vovk, 2012] Turned out to be Fixed Share [Herbster, Warmuth 1998]! Restarting existing algorithms and combining their predictions [Hazan, Seshadhri, 2009]

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 6 / 21

slide-23

SLIDE 23

Adaptive algorithms

We studied several approaches to adaptivity: Blowing up the set of experts to compete with virtual sleeping experts [DA, Koolen, Chernov, Vovk, 2012] Turned out to be Fixed Share [Herbster, Warmuth 1998]! Restarting existing algorithms and combining their predictions [Hazan, Seshadhri, 2009] Also turned out to be Fixed Share!

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 6 / 21

slide-24

SLIDE 24

Adaptive properties of Fixed Share: results

Fixed Share is known for tracking.

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 7 / 21

slide-25

SLIDE 25

Adaptive properties of Fixed Share: results

Fixed Share is known for tracking. LFS

[1,T]−LS [1,T] ≤ ln N +(m−1) ln(N −1)−(m−1) ln α−(T −m) ln(1−α) ,

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 7 / 21

slide-26

SLIDE 26

Adaptive properties of Fixed Share: results

Fixed Share is known for tracking. LFS

[1,T]−LS [1,T] ≤ ln N +(m−1) ln(N −1)−(m−1) ln α−(T −m) ln(1−α) ,

What about its adaptivity?

Our results

1

Figured out the Worst-Case adaptive regret of Fixed Share

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 7 / 21

slide-27

SLIDE 27

Adaptive properties of Fixed Share: results

Fixed Share is known for tracking. LFS

[1,T]−LS [1,T] ≤ ln N +(m−1) ln(N −1)−(m−1) ln α−(T −m) ln(1−α) ,

What about its adaptivity?

Our results

1

Figured out the Worst-Case adaptive regret of Fixed Share

2

Proved the optimality of Fixed Share — “no algorithm could have better guarantees on all time intervals”

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 7 / 21

slide-28

SLIDE 28

1

Why adaptive regret?

2

Setup

3

Results

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 8 / 21

slide-29

SLIDE 29

Protocol: Mix loss

for t = 1, 2, . . . do Learner announces probability vector wt ∈ △N Reality announces loss vector ℓt ∈ [−∞, ∞]N Learner suffers loss ℓt := − ln

n wn t e−ℓn

t

end for

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 9 / 21

slide-30

SLIDE 30

Adaptive Regret

Goal: On every time interval [t1, t2] we want to be not much worse than the best expert on that interval. We are interested in small adaptive regret

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 10 / 21

slide-31

SLIDE 31

Adaptive Regret

Goal: On every time interval [t1, t2] we want to be not much worse than the best expert on that interval. We are interested in small adaptive regret

Definition

The adaptive regret of the algorithm on the interval [t1, t2] is the loss of the algorithm there minus the lost of the best expert there: R[t1,t2] := L[t1,t2] − min

j

Lj

[t1,t2]

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 10 / 21

slide-32

SLIDE 32

AA and Fixed Share

Aggregating Algorithm [Vovk 1990] updates weights as: wn

t+1 :=

wn

t e−ℓn

t

n wn

t e−ℓn

t . Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 11 / 21

slide-33

SLIDE 33

AA and Fixed Share

Aggregating Algorithm [Vovk 1990] updates weights as: wn

t+1 :=

wn

t e−ℓn

t

n wn

t e−ℓn

t .

Fixed Share family is defined by the sequence of “switching rates” αt. Then the weight update is wn

t+1 :=

αt+1 N − 1 +

1 −

N N − 1αt+1

wn

t e−ℓn

t

n wn

t e−ℓn

t . Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 11 / 21

slide-34

SLIDE 34

AA and Fixed Share

Aggregating Algorithm [Vovk 1990] updates weights as: wn

t+1 :=

wn

t e−ℓn

t

n wn

t e−ℓn

t .

Fixed Share family is defined by the sequence of “switching rates” αt. Then the weight update is wn

t+1 :=

αt+1 N − 1 +

1 −

N N − 1αt+1

wn

t e−ℓn

t

n wn

t e−ℓn

t .

Adaptivity hides in the first term.

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 11 / 21

slide-35

SLIDE 35

1

Why adaptive regret?

2

Setup

3

Results

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 12 / 21

slide-36

SLIDE 36

Fixed Share Wort-Case Adaptive regret data

We proved that the worst case data for Fixed Share looks like this: 1 . . . t1 − 1 t1 . . . t2 Expert 1 ? ? Expert 2 ? ? ? . . . ? ? ? Expert N ? ? ? where denotes infinite loss, 0 – zero loss and ’?’ – losses that don’t matter.

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 13 / 21

slide-37

SLIDE 37

Fixed Share Worst-Case Adaptive regret formula

Knowing the worst-case data, we can plug it in and calculate the regret:

Theorem

The worst-case adaptive regret of Fixed Share with N experts on interval [t1, t2] equals − ln   αt1 N − 1

t2

t=t1+1

(1 − αt)   .

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 14 / 21

slide-38

SLIDE 38

Different α-s: examples

Classic Fixed Share (αt = const): ln(N − 1) − ln α − (t2 − t1) ln

1 − α
for t1 > 1, and

ln N − (t2 − 1) ln

1 − α
for t1 = 1.

Slowly decreasing αt = 1/t leads to regret of ln(N − 1) + ln t2 for t1 > 1, and ln N + ln t2 for t1 = 1.

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 15 / 21

slide-39

SLIDE 39

Different α-s: examples

Quickly decreasing switching rate. If we set αt = t−2 we have the upper bound for regret ln N + 2 ln t1 + ln 2. For t1 = 1 this is very close to classical AA regret!

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 16 / 21

slide-40

SLIDE 40

Combining adaptive stuff to get tracking bounds

The worst-case data we have just shown could be combined over the intervals, thus giving the overall worst-case data for partitions.

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 17 / 21

slide-41

SLIDE 41

Combining adaptive stuff to get tracking bounds

The worst-case data we have just shown could be combined over the intervals, thus giving the overall worst-case data for partitions.

Time Interval 1 Interval 2 Interval 3 Expert 1 . . . . . . . . . Expert 2 . . . . . . . . . Expert 3 . . . . . . . . .

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 17 / 21

slide-42

SLIDE 42

Combining adaptive stuff to get tracking bounds

The worst-case data we have just shown could be combined over the intervals, thus giving the overall worst-case data for partitions.

Time Interval 1 Interval 2 Interval 3 Expert 1 . . . . . . . . . Expert 2 . . . . . . . . . Expert 3 . . . . . . . . .

And the tracking bound can be recovered!

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 17 / 21

slide-43

SLIDE 43

Optimality – new stuff!

Ok, we have a regret of some algorithm. But is it optimal?

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 18 / 21

slide-44

SLIDE 44

Optimality – new stuff!

Ok, we have a regret of some algorithm. But is it optimal? It turns out to be in a very strong sense!

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 18 / 21

slide-45

SLIDE 45

Optimality – new stuff!

Ok, we have a regret of some algorithm. But is it optimal? It turns out to be in a very strong sense!

Theorem

1

(Any) Fixed Share is Pareto-optimal.

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 18 / 21

slide-46

SLIDE 46

Optimality – new stuff!

Ok, we have a regret of some algorithm. But is it optimal? It turns out to be in a very strong sense!

Theorem

1

(Any) Fixed Share is Pareto-optimal.

2

Any algorithm is dominated by an instance of Fixed Share.

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 18 / 21

slide-47

SLIDE 47

Proof sketch – key lemma

Let’s call φ(t1, t2) a candidate guarantee. If φ(t1, t2) is witnessed by some algorithm as its worst-case regret we can prove the following bounds: φ(t, t) ≥ ln N, φ(t1, t2) ≥ φ(t1, t1) +

t2

t=t1+1

− ln

1 − (N − 1)e−φ(t,t)

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 19 / 21

slide-48

SLIDE 48

Proof sketch – key lemma

Let’s call φ(t1, t2) a candidate guarantee. If φ(t1, t2) is witnessed by some algorithm as its worst-case regret we can prove the following bounds: φ(t, t) ≥ ln N, φ(t1, t2) ≥ φ(t1, t1) +

t2

t=t1+1

− ln

1 − (N − 1)e−φ(t,t)

Fixed Share with αt = (N − 1) exp−φ(t,t) satisfies the last one with equality.

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 19 / 21

slide-49

SLIDE 49

Summary

We studied two intuitive methods to obtain adaptive algorithms. They turned out to be Fixed Share. The worst-case Adaptive Regret of Fixed Share was studied and its optimality was established.

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 20 / 21

slide-50

SLIDE 50

Thank you!

Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 21 / 21