SLIDE 1
❖♣t✐♠❛❧ ▲♦ss❧❡ss ❙♦✉r❝❡ ❈♦❞❡s ❢♦r ❚✐♠❡❧② ❯♣❞❛t❡s
Pr❛t❤❛♠❡s❤ ▼❛②❡❦❛r
❏♦✐♥t ✇♦r❦ ✇✐t❤
P❛r✐♠❛❧ P❛r❛❣ ❛♥❞ ❍✐♠❛♥s❤✉ ❚②❛❣✐
❉❡♣❛rt♠❡♥t ♦❢ ❊❈❊✱ ■♥❞✐❛♥ ■♥st✐t✉t❡ ♦❢ ❙❝✐❡♥❝❡
▼♦t✐✈❛t✐♦♥
❙♦✉r❝❡ ✲ ❚❤❡ ❍✐♥❞✉
t ssss r s r - - PowerPoint PPT Presentation
t ssss r s r - - PowerPoint PPT Presentation
t ssss r s r ts Prts r t r t Pr Pr
SLIDE 2
SLIDE 3
▼♦t✐✈❛t✐♦♥
❙♦✉r❝❡ ✲ ❚❤❡ ❍✐♥❞✉
❙❡♥s♦r ❈❡♥t❡r ❚✐♠❡❧② ❯♣❞❛t❡s ❛r❡ ❝r✐t✐❝❛❧✳
✶
SLIDE 4
▼♦t✐✈❛t✐♦♥
❙♦✉r❝❡ ✲ ❚❤❡ ❍✐♥❞✉
❙❡♥s♦r ❈❡♥t❡r ❚✐♠❡❧② ❯♣❞❛t❡s ❛r❡ ❝r✐t✐❝❛❧✳
✶
SLIDE 5
▼♦t✐✈❛t✐♦♥
❙♦✉r❝❡ ✲ ❚❤❡ ❍✐♥❞✉
❙❡♥s♦r ❈❡♥t❡r ❚✐♠❡❧② ❯♣❞❛t❡s ❛r❡ ❝r✐t✐❝❛❧✳ ❆❣❡ ♦❢ ■♥❢♦r♠❛t✐♦♥✶ ✲ ♠❡tr✐❝ t♦ ❝❛♣t✉r❡ t✐♠❡❧✐♥❡ss✳
✶❑❛✉❧✱ ❙✳✱ ❨❛t❡s✱ ❘✳✱ ❛♥❞ ●r✉t❡s❡r✱ ▼✳ ✭✷✵✶✶✱ ❉❡❝❡♠❜❡r✮✳ ❖♥ ♣✐❣❣②❜❛❝❦✐♥❣
✐♥ ✈❡❤✐❝✉❧❛r ♥❡t✇♦r❦s✳ ■♥ ●❧♦❜❛❧ ❚❡❧❡❝♦♠♠✉♥✐❝❛t✐♦♥s ❈♦♥❢❡r❡♥❝❡ ✭●▲❖❇❊❈❖▼ ✷✵✶✶✮✱ ✷✵✶✶ ■❊❊❊ ✭♣♣✳ ✶✲✺✮✳ ■❊❊❊✳
✶
SLIDE 6
❆❣❡ ♦❢ ■♥❢♦r♠❛t✐♦♥ ✭❆❖■✮ ✲ ▼❡tr✐❝ ❢♦r ❚✐♠❡❧✐♥❡ss
◮ ❆❖■✿ ❚✐♠❡ ❧❛❣ ❜❡t✇❡❡♥ t❤❡ ❧❛t❡st ✐♥❢♦r♠❛t✐♦♥ ❛t t❤❡ ❘❳ ✇✳r✳t✳ t❤❛t ❛t ❚❳✳ ❙❡♥s♦r ❈❡♥t❡r ❲❡ ❛r❡ ✐♥t❡r❡st❡❞ ✐♥ ♠✐♥✐♠✐③✐♥❣ t❤❡ ❛✈❡r❛❣❡ ❛❣❡ ❲❡ r❡str✐❝t t♦ ▼❡♠♦r②❧❡ss ❯♣❞❛t❡ ❙❝❤❡♠❡s✳
✷
SLIDE 7
❆❣❡ ♦❢ ■♥❢♦r♠❛t✐♦♥ ✭❆❖■✮ ✲ ▼❡tr✐❝ ❢♦r ❚✐♠❡❧✐♥❡ss
◮ ❆❖■✿ ❚✐♠❡ ❧❛❣ ❜❡t✇❡❡♥ t❤❡ ❧❛t❡st ✐♥❢♦r♠❛t✐♦♥ ❛t t❤❡ ❘❳ ✇✳r✳t✳ t❤❛t ❛t ❚❳✳ ❙❡♥s♦r ❈❡♥t❡r ❲❡ ❛r❡ ✐♥t❡r❡st❡❞ ✐♥ ♠✐♥✐♠✐③✐♥❣ t❤❡ ❛✈❡r❛❣❡ ❛❣❡ ❲❡ r❡str✐❝t t♦ ▼❡♠♦r②❧❡ss ❯♣❞❛t❡ ❙❝❤❡♠❡s✳
✷
SLIDE 8
❆❣❡ ♦❢ ■♥❢♦r♠❛t✐♦♥ ✭❆❖■✮ ✲ ▼❡tr✐❝ ❢♦r ❚✐♠❡❧✐♥❡ss
◮ ❆❖■✿ ❚✐♠❡ ❧❛❣ ❜❡t✇❡❡♥ t❤❡ ❧❛t❡st ✐♥❢♦r♠❛t✐♦♥ ❛t t❤❡ ❘❳ ✇✳r✳t✳ t❤❛t ❛t ❚❳✳ ❙❡♥s♦r ❈❡♥t❡r Xt ❲❡ ❛r❡ ✐♥t❡r❡st❡❞ ✐♥ ♠✐♥✐♠✐③✐♥❣ t❤❡ ❛✈❡r❛❣❡ ❛❣❡ ❲❡ r❡str✐❝t t♦ ▼❡♠♦r②❧❡ss ❯♣❞❛t❡ ❙❝❤❡♠❡s✳
✷
SLIDE 9
❆❣❡ ♦❢ ■♥❢♦r♠❛t✐♦♥ ✭❆❖■✮ ✲ ▼❡tr✐❝ ❢♦r ❚✐♠❡❧✐♥❡ss
◮ ❆❖■✿ ❚✐♠❡ ❧❛❣ ❜❡t✇❡❡♥ t❤❡ ❧❛t❡st ✐♥❢♦r♠❛t✐♦♥ ❛t t❤❡ ❘❳ ✇✳r✳t✳ t❤❛t ❛t ❚❳✳ ❙❡♥s♦r ❈❡♥t❡r Xt XU(t) ❲❡ ❛r❡ ✐♥t❡r❡st❡❞ ✐♥ ♠✐♥✐♠✐③✐♥❣ t❤❡ ❛✈❡r❛❣❡ ❛❣❡ ❲❡ r❡str✐❝t t♦ ▼❡♠♦r②❧❡ss ❯♣❞❛t❡ ❙❝❤❡♠❡s✳
✷
SLIDE 10
❆❣❡ ♦❢ ■♥❢♦r♠❛t✐♦♥ ✭❆❖■✮ ✲ ▼❡tr✐❝ ❢♦r ❚✐♠❡❧✐♥❡ss
◮ ❆❖■✿ ❚✐♠❡ ❧❛❣ ❜❡t✇❡❡♥ t❤❡ ❧❛t❡st ✐♥❢♦r♠❛t✐♦♥ ❛t t❤❡ ❘❳ ✇✳r✳t✳ t❤❛t ❛t ❚❳✳ ❙❡♥s♦r ❈❡♥t❡r Xt XU(t) A(t) = t − U(t). ❲❡ ❛r❡ ✐♥t❡r❡st❡❞ ✐♥ ♠✐♥✐♠✐③✐♥❣ t❤❡ ❛✈❡r❛❣❡ ❛❣❡ ❲❡ r❡str✐❝t t♦ ▼❡♠♦r②❧❡ss ❯♣❞❛t❡ ❙❝❤❡♠❡s✳
✷
SLIDE 11
❆❣❡ ♦❢ ■♥❢♦r♠❛t✐♦♥ ✭❆❖■✮ ✲ ▼❡tr✐❝ ❢♦r ❚✐♠❡❧✐♥❡ss
◮ ❆❖■✿ ❚✐♠❡ ❧❛❣ ❜❡t✇❡❡♥ t❤❡ ❧❛t❡st ✐♥❢♦r♠❛t✐♦♥ ❛t t❤❡ ❘❳ ✇✳r✳t✳ t❤❛t ❛t ❚❳✳ ❙❡♥s♦r ❈❡♥t❡r Xt XU(t) A(t) = t − U(t). ◮ ❲❡ ❛r❡ ✐♥t❡r❡st❡❞ ✐♥ ♠✐♥✐♠✐③✐♥❣ t❤❡ ❛✈❡r❛❣❡ ❛❣❡ ¯ A lim sup
T→∞
1 T
T
- t=1
A(t). ❲❡ r❡str✐❝t t♦ ▼❡♠♦r②❧❡ss ❯♣❞❛t❡ ❙❝❤❡♠❡s✳
✷
SLIDE 12
❆❣❡ ♦❢ ■♥❢♦r♠❛t✐♦♥ ✭❆❖■✮ ✲ ▼❡tr✐❝ ❢♦r ❚✐♠❡❧✐♥❡ss
◮ ❆❖■✿ ❚✐♠❡ ❧❛❣ ❜❡t✇❡❡♥ t❤❡ ❧❛t❡st ✐♥❢♦r♠❛t✐♦♥ ❛t t❤❡ ❘❳ ✇✳r✳t✳ t❤❛t ❛t ❚❳✳ ❙❡♥s♦r ❈❡♥t❡r Xt XU(t) A(t) = t − U(t). ◮ ❲❡ ❛r❡ ✐♥t❡r❡st❡❞ ✐♥ ♠✐♥✐♠✐③✐♥❣ t❤❡ ❛✈❡r❛❣❡ ❛❣❡ ¯ A lim sup
T→∞
1 T
T
- t=1
A(t). ◮ ❲❡ r❡str✐❝t t♦ ▼❡♠♦r②❧❡ss ❯♣❞❛t❡ ❙❝❤❡♠❡s✳
✷
SLIDE 13
▼❡♠♦r②❧❡ss ❯♣❞❛t❡ ❙❝❤❡♠❡s
❙♦✉r❝❡ iid P ❊♥❝♦❞❡r Pr❡✜①✲❢r❡❡ ❈❤❛♥♥❡❧ ◆♦✐s❡❧❡ss ✶ ❇✐t✴❚✐♠❡ s❧♦t ❉❡❝♦❞❡r ❚✐♠❡
✸
SLIDE 14
▼❡♠♦r②❧❡ss ❯♣❞❛t❡ ❙❝❤❡♠❡s
❙♦✉r❝❡ iid P ❊♥❝♦❞❡r Pr❡✜①✲❢r❡❡ ❈❤❛♥♥❡❧ ◆♦✐s❡❧❡ss ✶ ❇✐t✴❚✐♠❡ s❧♦t ❉❡❝♦❞❡r ❚✐♠❡ X1 e(X1)
✸
SLIDE 15
▼❡♠♦r②❧❡ss ❯♣❞❛t❡ ❙❝❤❡♠❡s
❙♦✉r❝❡ iid P ❊♥❝♦❞❡r Pr❡✜①✲❢r❡❡ ❈❤❛♥♥❡❧ ◆♦✐s❡❧❡ss ✶ ❇✐t✴❚✐♠❡ s❧♦t ❉❡❝♦❞❡r ❚✐♠❡ X1 e(X1) X2
✸
SLIDE 16
▼❡♠♦r②❧❡ss ❯♣❞❛t❡ ❙❝❤❡♠❡s
❙♦✉r❝❡ iid P ❊♥❝♦❞❡r Pr❡✜①✲❢r❡❡ ❈❤❛♥♥❡❧ ◆♦✐s❡❧❡ss ✶ ❇✐t✴❚✐♠❡ s❧♦t ❉❡❝♦❞❡r ❚✐♠❡ X1 e(X1) X2 X3 e(X3) X1
✸
SLIDE 17
▼❡♠♦r②❧❡ss ❯♣❞❛t❡ ❙❝❤❡♠❡s
❙♦✉r❝❡ iid P ❊♥❝♦❞❡r Pr❡✜①✲❢r❡❡ ❈❤❛♥♥❡❧ ◆♦✐s❡❧❡ss ✶ ❇✐t✴❚✐♠❡ s❧♦t ❉❡❝♦❞❡r ❚✐♠❡ X1 e(X1) X2 X3 e(X3) X1 X4 X3 e(X4)
✸
SLIDE 18
▼❡♠♦r②❧❡ss ❯♣❞❛t❡ ❙❝❤❡♠❡s
❙♦✉r❝❡ iid P ❊♥❝♦❞❡r Pr❡✜①✲❢r❡❡ ❈❤❛♥♥❡❧ ◆♦✐s❡❧❡ss ✶ ❇✐t✴❚✐♠❡ s❧♦t ❉❡❝♦❞❡r ❚✐♠❡ X1 e(X1) X2 X3 e(X3) X1 X4 X3 e(X4) X5 X3
✸
SLIDE 19
▼❡♠♦r②❧❡ss ❯♣❞❛t❡ ❙❝❤❡♠❡s
❙♦✉r❝❡ iid P ❊♥❝♦❞❡r Pr❡✜①✲❢r❡❡ ❈❤❛♥♥❡❧ ◆♦✐s❡❧❡ss ✶ ❇✐t✴❚✐♠❡ s❧♦t ❉❡❝♦❞❡r ❚✐♠❡ X1 e(X1) X2 X3 e(X3) X1 X4 X3 e(X4) X5 X3 X6 X3
✸
SLIDE 20
■❧❧✉str❛t✐♦♥ ♦❢ ■♥st❛♥t❛♥❡♦✉s ❆❣❡
❙♦✉r❝❡ ❊♥❝♦❞❡r ❈❤❛♥♥❡❧ ❉❡❝♦❞❡r ❚✐♠❡ X1 e(X1) X2 X3 e(X3) X1 X4 X3 e(X4) X5 X3 X6 X3
A(t) = t − U(t) U(t) = ■♥❞❡① ♦❢ ❧❛t❡st ✐♥❢♦r♠❛t✐♦♥ ❛t t❤❡ ❞❡❝♦❞❡r
✵ ✶✰ ✷ A(t) ✰ ✸✰ ✹✰ ✶ ✷ ✸ ✹ t ✺ ✻ ✰ ✰ ✰ ✰ ✰
✹
SLIDE 21
❈❤❛r❛❝t❡r✐③❛t✐♦♥ ♦❢ ❆✈❡r❛❣❡ ❆❣❡
¯ A(e) lim sup
T→∞
1 T
T
- t=1
A(t) ℓ(x) ❝♦❞❡✲❧❡♥❣t❤ ❢♦r ❛ s②♠❜♦❧ x✱ L ℓ(X)✳
❚❤❡♦r❡♠
❋♦r ❛ ♣r❡✜①✲❢r❡❡ ❝♦❞❡ ✱
✺
SLIDE 22
❈❤❛r❛❝t❡r✐③❛t✐♦♥ ♦❢ ❆✈❡r❛❣❡ ❆❣❡
¯ A(e) lim sup
T→∞
1 T
T
- t=1
A(t) ℓ(x) ❝♦❞❡✲❧❡♥❣t❤ ❢♦r ❛ s②♠❜♦❧ x✱ L ℓ(X)✳
❚❤❡♦r❡♠
❋♦r ❛ ♣r❡✜①✲❢r❡❡ ❝♦❞❡ e✱ ¯ A(e) = E [L] +
E[L2] 2E[L] − 1 2
a.s..
✺
SLIDE 23
❈❤❛r❛❝t❡r✐③❛t✐♦♥ ♦❢ ❆✈❡r❛❣❡ ❆❣❡
¯ A(e) lim sup
T→∞
1 T
T
- t=1
A(t) ℓ(x) ❝♦❞❡✲❧❡♥❣t❤ ❢♦r ❛ s②♠❜♦❧ x✱ L ℓ(X)✳
❚❤❡♦r❡♠
❋♦r ❛ ♣r❡✜①✲❢r❡❡ ❝♦❞❡ e✱ ¯ A(e) = E [L] +
E[L2] 2E[L] − 1 2
a.s.. Pr♦♦❢ ■❞❡❛✿ ◮ Si ith r❡❝❡♣t✐♦♥
✵ ✶✰ ✷ A(t) ✰ ✸✰ ✹✰ S0 S1 S2 S3 S4 S5 ✰ ✰ ✰ ✰ ✰ ✰ ✰ ✰ ✰ ✰
✺
SLIDE 24
❈❤❛r❛❝t❡r✐③❛t✐♦♥ ♦❢ ❆✈❡r❛❣❡ ❆❣❡
¯ A(e) lim sup
T→∞
1 T
T
- t=1
A(t) ℓ(x) ❝♦❞❡✲❧❡♥❣t❤ ❢♦r ❛ s②♠❜♦❧ x✱ L ℓ(X)✳
❚❤❡♦r❡♠
❋♦r ❛ ♣r❡✜①✲❢r❡❡ ❝♦❞❡ e✱ ¯ A(e) = E [L] +
E[L2] 2E[L] − 1 2
a.s.. Pr♦♦❢ ■❞❡❛✿ ◮ Si ith r❡❝❡♣t✐♦♥
- Si+1 − Si
- i∈N ✐s iid
✵ ✶✰ ✷ A(t) ✰ ✸✰ ✹✰ S0 S1 S2 S3 S4 S5 ✰ ✰ ✰ ✰ ✰ ✰ ✰ ✰ ✰ ✰
✺
SLIDE 25
❈❤❛r❛❝t❡r✐③❛t✐♦♥ ♦❢ ❆✈❡r❛❣❡ ❆❣❡
¯ A(e) lim sup
T→∞
1 T
T
- t=1
A(t) ℓ(x) ❝♦❞❡✲❧❡♥❣t❤ ❢♦r ❛ s②♠❜♦❧ x✱ L ℓ(X)✳
❚❤❡♦r❡♠
❋♦r ❛ ♣r❡✜①✲❢r❡❡ ❝♦❞❡ e✱ ¯ A(e) = E [L] +
E[L2] 2E[L] − 1 2
a.s.. Pr♦♦❢ ■❞❡❛✿ ◮ Si ith r❡❝❡♣t✐♦♥
- Si+1 − Si
- i∈N ✐s iid
✵ ✶✰ ✷ A(t) ✰ ✸✰ ✹✰ S0 S1 S2 S3 S4 S5 ✰ ✰ ✰ ✰ ✰ ✰ ✰ ✰ ✰ ✰ R0 ✰ R1 R2 R3
Ri = ❙✉♠ ♦❢ ✐♥st❛♥t❛♥❡♦✉s ❛❣❡ ❛❢t❡r Si−1 t✐❧❧ Si
R4 R5
✺
SLIDE 26
❈❤❛r❛❝t❡r✐③❛t✐♦♥ ♦❢ ❆✈❡r❛❣❡ ❆❣❡
¯ A(e) lim sup
T→∞
1 T
T
- t=1
A(t) ℓ(x) ❝♦❞❡✲❧❡♥❣t❤ ❢♦r ❛ s②♠❜♦❧ x✱ L ℓ(X)✳
❚❤❡♦r❡♠
❋♦r ❛ ♣r❡✜①✲❢r❡❡ ❝♦❞❡ e✱ ¯ A(e) = E [L] +
E[L2] 2E[L] − 1 2
a.s.. Pr♦♦❢ ■❞❡❛✿ ◮ Si ith r❡❝❡♣t✐♦♥
- Si+1 − Si
- i∈N ✐s iid
◮ R2i+1
- i∈N ✐s iid✱
- R2i+2
- i∈N ✐s iid
✵ ✶✰ ✷ A(t) ✰ ✸✰ ✹✰ S0 S1 S2 S3 S4 S5 ✰ ✰ ✰ ✰ ✰ ✰ ✰ ✰ ✰ ✰ R0 ✰ R1 R2 R3 R4 R5
Ri = ❙✉♠ ♦❢ ✐♥st❛♥t❛♥❡♦✉s ❛❣❡ ❛❢t❡r Si−1 t✐❧❧ Si
✺
SLIDE 27
❈❤❛r❛❝t❡r✐③❛t✐♦♥ ♦❢ ❆✈❡r❛❣❡ ❆❣❡
¯ A(e) lim sup
T→∞
1 T
T
- t=1
A(t) ℓ(x) ❝♦❞❡✲❧❡♥❣t❤ ❢♦r ❛ s②♠❜♦❧ x✱ L ℓ(X)✳
❚❤❡♦r❡♠
❋♦r ❛ ♣r❡✜①✲❢r❡❡ ❝♦❞❡ e✱ ¯ A(e) = E [L] +
E[L2] 2E[L] − 1 2
a.s..
❲❤✐❝❤ s♦✉r❝❡ ❝♦❞✐♥❣ s❝❤❡♠❡ ✐s ♦♣t✐♠❛❧❄
✺
SLIDE 28
❈❤❛r❛❝t❡r✐③❛t✐♦♥ ♦❢ ❆✈❡r❛❣❡ ❆❣❡
¯ A(e) lim sup
T→∞
1 T
T
- t=1
A(t) ℓ(x) ❝♦❞❡✲❧❡♥❣t❤ ❢♦r ❛ s②♠❜♦❧ x✱ L ℓ(X)✳
❚❤❡♦r❡♠
❋♦r ❛ ♣r❡✜①✲❢r❡❡ ❝♦❞❡ e✱ ¯ A(e) = E [L] +
E[L2] 2E[L] − 1 2
a.s..
❲❤✐❝❤ s♦✉r❝❡ ❝♦❞✐♥❣ s❝❤❡♠❡ ✐s ♦♣t✐♠❛❧❄ ❆r❡ ❙❤❛♥♥♦♥ ❈♦❞❡s ❖♣t✐♠❛❧❄
✺
SLIDE 29
❙❤❛♥♥♦♥ ❝♦❞❡s ❝❛♥ ❜❡ ❢❛r ❢r♦♠ ♦♣t✐♠❛❧
❙❤❛♥♥♦♥ ❝♦❞❡ ❢♦r P✿ ℓ(x) = ⌈− log P(x)⌉ ∀x ❊①❛♠♣❧❡✿ ❈♦♥s✐❞❡r ❛♥❞ ❛ ♣♠❢ ♦♥ ❣✐✈❡♥ ❜② ❙❤❛♥♥♦♥ ❝♦❞❡s ❢♦r ❤❛✈❡ ❛♥ ❛✈❡r❛❣❡ ❛❣❡ ♦❢ ✳ ■♥st❡❛❞✱ ✉s❡ ❙❤❛♥♥♦♥ ❝♦❞❡s ❢♦r ♣♠❢ ✱ ✇❤❡r❡ ❙❤❛♥♥♦♥ ❝♦❞❡s ❢♦r ❤❛✈❡ ❛♥ ❛✈❡r❛❣❡ ❛❣❡ ♦❢ ✳ ❙❤❛♥♥♦♥ ❝♦❞❡s ❛r❡ ♦r❞❡r✲✇✐s❡ s✉❜♦♣t✐♠❛❧✦
✻
SLIDE 30
❙❤❛♥♥♦♥ ❝♦❞❡s ❝❛♥ ❜❡ ❢❛r ❢r♦♠ ♦♣t✐♠❛❧
❙❤❛♥♥♦♥ ❝♦❞❡ ❢♦r P✿ ℓ(x) = ⌈− log P(x)⌉ ∀x ❊①❛♠♣❧❡✿ ❈♦♥s✐❞❡r X = {0, ..., 2n} ❛♥❞ ❛ ♣♠❢ P ♦♥ X ❣✐✈❡♥ ❜② P(x) =
- 1 − 1
n,
x = 0
1 n2n ,
x ∈ {1, . . . , 2n}. ❙❤❛♥♥♦♥ ❝♦❞❡s ❢♦r ❤❛✈❡ ❛♥ ❛✈❡r❛❣❡ ❛❣❡ ♦❢ ✳ ■♥st❡❛❞✱ ✉s❡ ❙❤❛♥♥♦♥ ❝♦❞❡s ❢♦r ♣♠❢ ✱ ✇❤❡r❡ ❙❤❛♥♥♦♥ ❝♦❞❡s ❢♦r ❤❛✈❡ ❛♥ ❛✈❡r❛❣❡ ❛❣❡ ♦❢ ✳ ❙❤❛♥♥♦♥ ❝♦❞❡s ❛r❡ ♦r❞❡r✲✇✐s❡ s✉❜♦♣t✐♠❛❧✦
✻
SLIDE 31
❙❤❛♥♥♦♥ ❝♦❞❡s ❝❛♥ ❜❡ ❢❛r ❢r♦♠ ♦♣t✐♠❛❧
❙❤❛♥♥♦♥ ❝♦❞❡ ❢♦r P✿ ℓ(x) = ⌈− log P(x)⌉ ∀x ❊①❛♠♣❧❡✿ ❈♦♥s✐❞❡r X = {0, ..., 2n} ❛♥❞ ❛ ♣♠❢ P ♦♥ X ❣✐✈❡♥ ❜② P(x) =
- 1 − 1
n,
x = 0
1 n2n ,
x ∈ {1, . . . , 2n}. ❙❤❛♥♥♦♥ ❝♦❞❡s ❢♦r P ❤❛✈❡ ❛♥ ❛✈❡r❛❣❡ ❛❣❡ ♦❢ Ω(log |X|)✳ ■♥st❡❛❞✱ ✉s❡ ❙❤❛♥♥♦♥ ❝♦❞❡s ❢♦r ♣♠❢ ✱ ✇❤❡r❡ ❙❤❛♥♥♦♥ ❝♦❞❡s ❢♦r ❤❛✈❡ ❛♥ ❛✈❡r❛❣❡ ❛❣❡ ♦❢ ✳ ❙❤❛♥♥♦♥ ❝♦❞❡s ❛r❡ ♦r❞❡r✲✇✐s❡ s✉❜♦♣t✐♠❛❧✦
✻
SLIDE 32
❙❤❛♥♥♦♥ ❝♦❞❡s ❝❛♥ ❜❡ ❢❛r ❢r♦♠ ♦♣t✐♠❛❧
❙❤❛♥♥♦♥ ❝♦❞❡ ❢♦r P✿ ℓ(x) = ⌈− log P(x)⌉ ∀x ❊①❛♠♣❧❡✿ ❈♦♥s✐❞❡r X = {0, ..., 2n} ❛♥❞ ❛ ♣♠❢ P ♦♥ X ❣✐✈❡♥ ❜② P(x) =
- 1 − 1
n,
x = 0
1 n2n ,
x ∈ {1, . . . , 2n}. ❙❤❛♥♥♦♥ ❝♦❞❡s ❢♦r P ❤❛✈❡ ❛♥ ❛✈❡r❛❣❡ ❛❣❡ ♦❢ Ω(log |X|)✳ ■♥st❡❛❞✱ ✉s❡ ❙❤❛♥♥♦♥ ❝♦❞❡s ❢♦r ♣♠❢ P ′(x)✱ ✇❤❡r❡ P ′(x) =
- 1
2
√n ,
x = 0
1−2−√n 2n
, x ∈ {1, . . . , 2n}. ❙❤❛♥♥♦♥ ❝♦❞❡s ❢♦r ❤❛✈❡ ❛♥ ❛✈❡r❛❣❡ ❛❣❡ ♦❢ ✳ ❙❤❛♥♥♦♥ ❝♦❞❡s ❛r❡ ♦r❞❡r✲✇✐s❡ s✉❜♦♣t✐♠❛❧✦
✻
SLIDE 33
❙❤❛♥♥♦♥ ❝♦❞❡s ❝❛♥ ❜❡ ❢❛r ❢r♦♠ ♦♣t✐♠❛❧
❙❤❛♥♥♦♥ ❝♦❞❡ ❢♦r P✿ ℓ(x) = ⌈− log P(x)⌉ ∀x ❊①❛♠♣❧❡✿ ❈♦♥s✐❞❡r X = {0, ..., 2n} ❛♥❞ ❛ ♣♠❢ P ♦♥ X ❣✐✈❡♥ ❜② P(x) =
- 1 − 1
n,
x = 0
1 n2n ,
x ∈ {1, . . . , 2n}. ❙❤❛♥♥♦♥ ❝♦❞❡s ❢♦r P ❤❛✈❡ ❛♥ ❛✈❡r❛❣❡ ❛❣❡ ♦❢ Ω(log |X|)✳ ■♥st❡❛❞✱ ✉s❡ ❙❤❛♥♥♦♥ ❝♦❞❡s ❢♦r ♣♠❢ P ′(x)✱ ✇❤❡r❡ P ′(x) =
- 1
2
√n ,
x = 0
1−2−√n 2n
, x ∈ {1, . . . , 2n}. ❙❤❛♥♥♦♥ ❝♦❞❡s ❢♦r P ′ ❤❛✈❡ ❛♥ ❛✈❡r❛❣❡ ❛❣❡ ♦❢ O(
- log |X|)✳
❙❤❛♥♥♦♥ ❝♦❞❡s ❛r❡ ♦r❞❡r✲✇✐s❡ s✉❜♦♣t✐♠❛❧✦
✻
SLIDE 34
❙❤❛♥♥♦♥ ❝♦❞❡s ❝❛♥ ❜❡ ❢❛r ❢r♦♠ ♦♣t✐♠❛❧
❙❤❛♥♥♦♥ ❝♦❞❡ ❢♦r P✿ ℓ(x) = ⌈− log P(x)⌉ ∀x ❊①❛♠♣❧❡✿ ❈♦♥s✐❞❡r X = {0, ..., 2n} ❛♥❞ ❛ ♣♠❢ P ♦♥ X ❣✐✈❡♥ ❜② P(x) =
- 1 − 1
n,
x = 0
1 n2n ,
x ∈ {1, . . . , 2n}. ❙❤❛♥♥♦♥ ❝♦❞❡s ❢♦r P ❤❛✈❡ ❛♥ ❛✈❡r❛❣❡ ❛❣❡ ♦❢ Ω(log |X|)✳ ■♥st❡❛❞✱ ✉s❡ ❙❤❛♥♥♦♥ ❝♦❞❡s ❢♦r ♣♠❢ P ′(x)✱ ✇❤❡r❡ P ′(x) =
- 1
2
√n ,
x = 0
1−2−√n 2n
, x ∈ {1, . . . , 2n}. ❙❤❛♥♥♦♥ ❝♦❞❡s ❢♦r P ′ ❤❛✈❡ ❛♥ ❛✈❡r❛❣❡ ❛❣❡ ♦❢ O(
- log |X|)✳
❙❤❛♥♥♦♥ ❝♦❞❡s ❛r❡ ♦r❞❡r✲✇✐s❡ s✉❜♦♣t✐♠❛❧✦
✻
SLIDE 35
❖✉r ❆♣♣r♦❛❝❤
✼
SLIDE 36
❘❡❞✉❝t✐♦♥ t♦ ❛ s✐♠♣❧❡r ♣r♦❜❧❡♠
◆❡❡❞ t♦ s♦❧✈❡ ■P❀ min E [L] + E
- L2
2E [L] s✳t✳ ℓ ∈ Z|X|
+ ,
- x∈X
2−ℓ(x) ≤ 1 ■♥st❡❛❞ s♦❧✈❡ ❘P❀ s✳t✳ ❛♥❞ ✉s❡
Pr♦♣♦s✐t✐♦♥
❈♦st ✉s✐♥❣ t❤✐s ❛♣♣r♦❛❝❤ ✇✐❧❧ ❜❡ ❛t♠♦st ✷✳✺ ❜✐ts ❛✇❛② ❢r♦♠ t❤❡ ♦♣t✐♠❛❧ ❝♦st✳
✽
SLIDE 37
❘❡❞✉❝t✐♦♥ t♦ ❛ s✐♠♣❧❡r ♣r♦❜❧❡♠
◆❡❡❞ t♦ s♦❧✈❡ ■P❀ min E [L] + E
- L2
2E [L] s✳t✳ ℓ ∈ Z|X|
+ ,
- x∈X
2−ℓ(x) ≤ 1 ■♥st❡❛❞ s♦❧✈❡ ❘P❀ min E [L] + E
- L2
2E [L] s✳t✳ ℓ ∈ R|X|
+ ,
- x∈X
2−ℓ(x) ≤ 1 ❛♥❞ ✉s❡
Pr♦♣♦s✐t✐♦♥
❈♦st ✉s✐♥❣ t❤✐s ❛♣♣r♦❛❝❤ ✇✐❧❧ ❜❡ ❛t♠♦st ✷✳✺ ❜✐ts ❛✇❛② ❢r♦♠ t❤❡ ♦♣t✐♠❛❧ ❝♦st✳
✽
SLIDE 38
❘❡❞✉❝t✐♦♥ t♦ ❛ s✐♠♣❧❡r ♣r♦❜❧❡♠
◆❡❡❞ t♦ s♦❧✈❡ ■P❀ min E [L] + E
- L2
2E [L] s✳t✳ ℓ ∈ Z|X|
+ ,
- x∈X
2−ℓ(x) ≤ 1 ■♥st❡❛❞ s♦❧✈❡ ❘P❀ min E [L] + E
- L2
2E [L] s✳t✳ ℓ ∈ R|X|
+ ,
- x∈X
2−ℓ(x) ≤ 1 ❛♥❞ ✉s❡ ℓ(x) = ⌈ℓ∗(x)⌉ ∀x ∈ X
Pr♦♣♦s✐t✐♦♥
❈♦st ✉s✐♥❣ t❤✐s ❛♣♣r♦❛❝❤ ✇✐❧❧ ❜❡ ❛t♠♦st ✷✳✺ ❜✐ts ❛✇❛② ❢r♦♠ t❤❡ ♦♣t✐♠❛❧ ❝♦st✳
✽
SLIDE 39
❘❡❞✉❝t✐♦♥ t♦ ❛ s✐♠♣❧❡r ♣r♦❜❧❡♠
◆❡❡❞ t♦ s♦❧✈❡ ■P❀ min E [L] + E
- L2
2E [L] s✳t✳ ℓ ∈ Z|X|
+ ,
- x∈X
2−ℓ(x) ≤ 1 ■♥st❡❛❞ s♦❧✈❡ ❘P❀ min E [L] + E
- L2
2E [L] s✳t✳ ℓ ∈ R|X|
+ ,
- x∈X
2−ℓ(x) ≤ 1 ❛♥❞ ✉s❡ ℓ(x) = ⌈ℓ∗(x)⌉ ∀x ∈ X
Pr♦♣♦s✐t✐♦♥
❈♦st ✉s✐♥❣ t❤✐s ❛♣♣r♦❛❝❤ ✇✐❧❧ ❜❡ ❛t♠♦st ✷✳✺ ❜✐ts ❛✇❛② ❢r♦♠ t❤❡ ♦♣t✐♠❛❧ ❝♦st✳
✽
SLIDE 40
❙tr✉❝t✉r❛❧ ❘❡s✉❧t ❢♦r ❘P
❘❡❛❧ ✈❛❧✉❡❞ ❙❤❛♥♥♦♥ ❧❡♥❣t❤s ❢♦r P✿ ℓ(x) = − log P(x) ∀x
▼❛✐♥ ❚❤❡♦r❡♠
❖♣t✐♠❛❧ s♦❧✉t✐♦♥ ❢♦r ❘P ✐s ✉♥✐q✉❡ ❛♥❞ ✐s ❣✐✈❡♥ ❜② ℓ∗(x) = − log P ∗(x) ∀x ∈ X, ✇❤❡r❡ P ∗ ✐s ❛ t✐❧t✐♥❣ ♦❢ s♦✉r❝❡ ❞✐str✐❜✉t✐♦♥ P✳ ❝❛♥ ❜❡ ❢♦✉♥❞ ❜② ❛♥ ❊♥tr♦♣② ▼❛①✐♠✐③❛t✐♦♥ ♣r♦❝❡❞✉r❡✳
✾
SLIDE 41
❙tr✉❝t✉r❛❧ ❘❡s✉❧t ❢♦r ❘P
❘❡❛❧ ✈❛❧✉❡❞ ❙❤❛♥♥♦♥ ❧❡♥❣t❤s ❢♦r P✿ ℓ(x) = − log P(x) ∀x
▼❛✐♥ ❚❤❡♦r❡♠
❖♣t✐♠❛❧ s♦❧✉t✐♦♥ ❢♦r ❘P ✐s ✉♥✐q✉❡ ❛♥❞ ✐s ❣✐✈❡♥ ❜② ℓ∗(x) = − log P ∗(x) ∀x ∈ X, ✇❤❡r❡ P ∗ ✐s ❛ t✐❧t✐♥❣ ♦❢ s♦✉r❝❡ ❞✐str✐❜✉t✐♦♥ P✳ P ∗ ❝❛♥ ❜❡ ❢♦✉♥❞ ❜② ❛♥ ❊♥tr♦♣② ▼❛①✐♠✐③❛t✐♦♥ ♣r♦❝❡❞✉r❡✳
✾
SLIDE 42
❙tr✉❝t✉r❛❧ ❘❡s✉❧t ❢♦r ❘P
❘❡❛❧ ✈❛❧✉❡❞ ❙❤❛♥♥♦♥ ❧❡♥❣t❤s ❢♦r P✿ ℓ(x) = − log P(x) ∀x
▼❛✐♥ ❚❤❡♦r❡♠
❖♣t✐♠❛❧ s♦❧✉t✐♦♥ ❢♦r ❘P ✐s ✉♥✐q✉❡ ❛♥❞ ✐s ❣✐✈❡♥ ❜② ℓ∗(x) = − log P ∗(x) ∀x ∈ X, ✇❤❡r❡ P ∗ ✐s ❛ t✐❧t✐♥❣ ♦❢ s♦✉r❝❡ ❞✐str✐❜✉t✐♦♥ P✳ P ∗ ❝❛♥ ❜❡ ❢♦✉♥❞ ❜② ❛♥ ❊♥tr♦♣② ▼❛①✐♠✐③❛t✐♦♥ ♣r♦❝❡❞✉r❡✳ 1 2 3 4 5 6 7 8 0.1 0.2 0.3 0.4 P
✾
SLIDE 43
❙tr✉❝t✉r❛❧ ❘❡s✉❧t ❢♦r ❘P
❘❡❛❧ ✈❛❧✉❡❞ ❙❤❛♥♥♦♥ ❧❡♥❣t❤s ❢♦r P✿ ℓ(x) = − log P(x) ∀x
▼❛✐♥ ❚❤❡♦r❡♠
❖♣t✐♠❛❧ s♦❧✉t✐♦♥ ❢♦r ❘P ✐s ✉♥✐q✉❡ ❛♥❞ ✐s ❣✐✈❡♥ ❜② ℓ∗(x) = − log P ∗(x) ∀x ∈ X, ✇❤❡r❡ P ∗ ✐s ❛ t✐❧t✐♥❣ ♦❢ s♦✉r❝❡ ❞✐str✐❜✉t✐♦♥ P✳ P ∗ ❝❛♥ ❜❡ ❢♦✉♥❞ ❜② ❛♥ ❊♥tr♦♣② ▼❛①✐♠✐③❛t✐♦♥ ♣r♦❝❡❞✉r❡✳ 1 2 3 4 5 6 7 8 0.1 0.2 0.3 0.4 P P ∗
✾
SLIDE 44
Pr♦♦❢ s❦❡t❝❤ ♦❢ ▼❛✐♥ t❤❡♦r❡♠
◮ ▼❛✐♥ st❡♣✿ ▲✐♥❡❛r✐③✐♥❣ t❤❡ ❆✈❡r❛❣❡ ❆❣❡ ❈♦st E [L] + E
- L2
2E [L] = max
y∈Y
- x∈X
g(y, x)ℓ(x) ❚❤❡ ✇r❛♣✲✉♣ ✶✳ ▼✐♥✐♠❛① ❝❧❛✐♠ ✷✳ ✸✳ ❯s❡ ❊♥tr♦♣② ▼❛①✐♠✐③❛t✐♦♥ t♦ ✜♥❞ t❤❡ ❧❡❛st✲❢❛✈♦r❛❜❧❡ ▼✐♥✐♠✐③✐♥❣ ❧❡♥❣t❤s ❢♦r t❤❡ ❧❡❛st✲❢❛✈♦r❛❜❧❡ ❛r❡ ♦♣t✐♠❛❧ ✶✵
SLIDE 45
Pr♦♦❢ s❦❡t❝❤ ♦❢ ▼❛✐♥ t❤❡♦r❡♠
◮ ▼❛✐♥ st❡♣✿ ▲✐♥❡❛r✐③✐♥❣ t❤❡ ❆✈❡r❛❣❡ ❆❣❡ ❈♦st E [L] + E
- L2
2E [L] = max
y∈Y
- x∈X
g(y, x)ℓ(x) ◮ ❚❤❡ ✇r❛♣✲✉♣ ✶✳ ▼✐♥✐♠❛① ❝❧❛✐♠ ∆∗ = min
ℓ∈Λ max y∈Y
- x∈X
g(y, x)ℓ(x) ✷✳ ✸✳ ❯s❡ ❊♥tr♦♣② ▼❛①✐♠✐③❛t✐♦♥ t♦ ✜♥❞ t❤❡ ❧❡❛st✲❢❛✈♦r❛❜❧❡ ▼✐♥✐♠✐③✐♥❣ ❧❡♥❣t❤s ❢♦r t❤❡ ❧❡❛st✲❢❛✈♦r❛❜❧❡ ❛r❡ ♦♣t✐♠❛❧ ✶✵
SLIDE 46
Pr♦♦❢ s❦❡t❝❤ ♦❢ ▼❛✐♥ t❤❡♦r❡♠
◮ ▼❛✐♥ st❡♣✿ ▲✐♥❡❛r✐③✐♥❣ t❤❡ ❆✈❡r❛❣❡ ❆❣❡ ❈♦st E [L] + E
- L2
2E [L] = max
y∈Y
- x∈X
g(y, x)ℓ(x) ◮ ❚❤❡ ✇r❛♣✲✉♣ ✶✳ ▼✐♥✐♠❛① ❝❧❛✐♠ ∆∗ = min
ℓ∈Λ max y∈Y
- x∈X
g(y, x)ℓ(x) = max
y∈Y min ℓ∈Λ
- x∈X
g(y, x)ℓ(x) ✷✳ ✸✳ ❯s❡ ❊♥tr♦♣② ▼❛①✐♠✐③❛t✐♦♥ t♦ ✜♥❞ t❤❡ ❧❡❛st✲❢❛✈♦r❛❜❧❡ ▼✐♥✐♠✐③✐♥❣ ❧❡♥❣t❤s ❢♦r t❤❡ ❧❡❛st✲❢❛✈♦r❛❜❧❡ ❛r❡ ♦♣t✐♠❛❧ ✶✵
SLIDE 47
Pr♦♦❢ s❦❡t❝❤ ♦❢ ▼❛✐♥ t❤❡♦r❡♠
◮ ▼❛✐♥ st❡♣✿ ▲✐♥❡❛r✐③✐♥❣ t❤❡ ❆✈❡r❛❣❡ ❆❣❡ ❈♦st E [L] + E
- L2
2E [L] = max
y∈Y
- x∈X
g(y, x)ℓ(x) ◮ ❚❤❡ ✇r❛♣✲✉♣ ✶✳ ▼✐♥✐♠❛① ❝❧❛✐♠ ∆∗ = min
ℓ∈Λ max y∈Y
- x∈X
g(y, x)ℓ(x) = max
y∈Y, g(y,·)≥0
min
ℓ∈Λ
- x∈X
g(y, x)ℓ(x) ✷✳ ✸✳ ❯s❡ ❊♥tr♦♣② ▼❛①✐♠✐③❛t✐♦♥ t♦ ✜♥❞ t❤❡ ❧❡❛st✲❢❛✈♦r❛❜❧❡ ▼✐♥✐♠✐③✐♥❣ ❧❡♥❣t❤s ❢♦r t❤❡ ❧❡❛st✲❢❛✈♦r❛❜❧❡ ❛r❡ ♦♣t✐♠❛❧ ✶✵
SLIDE 48
Pr♦♦❢ s❦❡t❝❤ ♦❢ ▼❛✐♥ t❤❡♦r❡♠
◮ ▼❛✐♥ st❡♣✿ ▲✐♥❡❛r✐③✐♥❣ t❤❡ ❆✈❡r❛❣❡ ❆❣❡ ❈♦st E [L] + E
- L2
2E [L] = max
y∈Y
- x∈X
g(y, x)ℓ(x) ◮ ❚❤❡ ✇r❛♣✲✉♣ ✶✳ ▼✐♥✐♠❛① ❝❧❛✐♠ ∆∗ = min
ℓ∈Λ max y∈Y
- x∈X
g(y, x)ℓ(x) = max
y∈Y, g(y,·)≥0
min
ℓ∈Λ
- x∈X
g(y, x)ℓ(x) ✷✳ ■♥♥❡r min ✐s ❛tt❛✐♥❡❞ ❜② ℓ′(x) = − log P ′(x) ❢♦r P ′(x) ∝ g(y, x) ✸✳ ❯s❡ ❊♥tr♦♣② ▼❛①✐♠✐③❛t✐♦♥ t♦ ✜♥❞ t❤❡ ❧❡❛st✲❢❛✈♦r❛❜❧❡ ▼✐♥✐♠✐③✐♥❣ ❧❡♥❣t❤s ❢♦r t❤❡ ❧❡❛st✲❢❛✈♦r❛❜❧❡ ❛r❡ ♦♣t✐♠❛❧ ✶✵
SLIDE 49
Pr♦♦❢ s❦❡t❝❤ ♦❢ ▼❛✐♥ t❤❡♦r❡♠
◮ ▼❛✐♥ st❡♣✿ ▲✐♥❡❛r✐③✐♥❣ t❤❡ ❆✈❡r❛❣❡ ❆❣❡ ❈♦st E [L] + E
- L2
2E [L] = max
y∈Y
- x∈X
g(y, x)ℓ(x) ◮ ❚❤❡ ✇r❛♣✲✉♣ ✶✳ ▼✐♥✐♠❛① ❝❧❛✐♠ ∆∗ = min
ℓ∈Λ max y∈Y
- x∈X
g(y, x)ℓ(x) = max
y∈Y, g(y,·)≥0
min
ℓ∈Λ
- x∈X
g(y, x)ℓ(x) ✷✳ ■♥♥❡r min ✐s ❛tt❛✐♥❡❞ ❜② ℓ′(x) = − log P ′(x) ❢♦r P ′(x) ∝ g(y, x) ✸✳ ❯s❡ ❊♥tr♦♣② ▼❛①✐♠✐③❛t✐♦♥ t♦ ✜♥❞ t❤❡ ❧❡❛st✲❢❛✈♦r❛❜❧❡ y ∆∗ = max
y∈Y, g(y,·)≥0
- x∈X
g(y, x) log
- x∈X g(y, x)
g(y, x) ▼✐♥✐♠✐③✐♥❣ ❧❡♥❣t❤s ❢♦r t❤❡ ❧❡❛st✲❢❛✈♦r❛❜❧❡ ❛r❡ ♦♣t✐♠❛❧ ✶✵
SLIDE 50
Pr♦♦❢ s❦❡t❝❤ ♦❢ ▼❛✐♥ t❤❡♦r❡♠
◮ ▼❛✐♥ st❡♣✿ ▲✐♥❡❛r✐③✐♥❣ t❤❡ ❆✈❡r❛❣❡ ❆❣❡ ❈♦st E [L] + E
- L2
2E [L] = max
y∈Y
- x∈X
g(y, x)ℓ(x) ◮ ❚❤❡ ✇r❛♣✲✉♣ ✶✳ ▼✐♥✐♠❛① ❝❧❛✐♠ ∆∗ = min
ℓ∈Λ max y∈Y
- x∈X
g(y, x)ℓ(x) = max
y∈Y, g(y,·)≥0
min
ℓ∈Λ
- x∈X
g(y, x)ℓ(x) ✷✳ ■♥♥❡r min ✐s ❛tt❛✐♥❡❞ ❜② ℓ′(x) = − log P ′(x) ❢♦r P ′(x) ∝ g(y, x) ✸✳ ❯s❡ ❊♥tr♦♣② ▼❛①✐♠✐③❛t✐♦♥ t♦ ✜♥❞ t❤❡ ❧❡❛st✲❢❛✈♦r❛❜❧❡ y ∆∗ = max
y∈Y, g(y,·)≥0
- x∈X
g(y, x) log
- x∈X g(y, x)
g(y, x) ▼✐♥✐♠✐③✐♥❣ ❧❡♥❣t❤s ❢♦r t❤❡ ❧❡❛st✲❢❛✈♦r❛❜❧❡ y ❛r❡ ♦♣t✐♠❛❧ ✶✵
SLIDE 51
▼❛✐♥ ❙t❡♣✿ ▲✐♥❡❛r✐③✐♥❣ t❤❡ ❆✈❡r❛❣❡ ❆❣❡ ❈♦st
◮ ▲✐♥❡❛r✐③✐♥❣ t❤❡ r❛t✐♦♥❛❧ ❢♦r♠ ✭❡❛s②✮✿ E [L] + E
- L2
2E [L] = max
z≥0
- 1 − z2
2
- E [L] + z
- E [L2]
▲✐♥❡❛r✐③✐♥❣ t❤❡ ✲♥♦r♠ t❡r♠❄
❆ ♥❡✇ ✈❛r✐❛t✐♦♥❛❧ ❢♦r♠✉❧❛ ❢♦r ✲♥♦r♠ ♦❢ ❛ r❛♥❞♦♠ ✈❛r✐❛❜❧❡
✶✶
SLIDE 52
▼❛✐♥ ❙t❡♣✿ ▲✐♥❡❛r✐③✐♥❣ t❤❡ ❆✈❡r❛❣❡ ❆❣❡ ❈♦st
◮ ▲✐♥❡❛r✐③✐♥❣ t❤❡ r❛t✐♦♥❛❧ ❢♦r♠ ✭❡❛s②✮✿ E [L] + E
- L2
2E [L] = max
z≥0
- 1 − z2
2
- E [L] + z
- E [L2]
◮ ▲✐♥❡❛r✐③✐♥❣ t❤❡ 2✲♥♦r♠ t❡r♠❄
❆ ♥❡✇ ✈❛r✐❛t✐♦♥❛❧ ❢♦r♠✉❧❛ ❢♦r ✲♥♦r♠ ♦❢ ❛ r❛♥❞♦♠ ✈❛r✐❛❜❧❡
✶✶
SLIDE 53
▼❛✐♥ ❙t❡♣✿ ▲✐♥❡❛r✐③✐♥❣ t❤❡ ❆✈❡r❛❣❡ ❆❣❡ ❈♦st
◮ ▲✐♥❡❛r✐③✐♥❣ t❤❡ r❛t✐♦♥❛❧ ❢♦r♠ ✭❡❛s②✮✿ E [L] + E
- L2
2E [L] = max
z≥0
- 1 − z2
2
- E [L] + z
- E [L2]
◮ ▲✐♥❡❛r✐③✐♥❣ t❤❡ 2✲♥♦r♠ t❡r♠❄
❆ ♥❡✇ ✈❛r✐❛t✐♦♥❛❧ ❢♦r♠✉❧❛ ❢♦r p✲♥♦r♠ ♦❢ ❛ r❛♥❞♦♠ ✈❛r✐❛❜❧❡
Xp = max
Q≪P E
dQ dP p−1
p
|X|
- ❆ ♥❡✇ ✈❛r✐❛t✐♦♥❛❧ ❢♦r♠✉❧❛ ❢♦r
✲♥♦r♠ ♦❢ ❛ r❛♥❞♦♠ ✈❛r✐❛❜❧❡
✶✶
SLIDE 54
▼❛✐♥ ❙t❡♣✿ ▲✐♥❡❛r✐③✐♥❣ t❤❡ ❆✈❡r❛❣❡ ❆❣❡ ❈♦st
◮ ▲✐♥❡❛r✐③✐♥❣ t❤❡ r❛t✐♦♥❛❧ ❢♦r♠ ✭❡❛s②✮✿ E [L] + E
- L2
2E [L] = max
z≥0
- 1 − z2
2
- E [L] + z
- E [L2]
◮ ▲✐♥❡❛r✐③✐♥❣ t❤❡ 2✲♥♦r♠ t❡r♠❄
❆ ♥❡✇ ✈❛r✐❛t✐♦♥❛❧ ❢♦r♠✉❧❛ ❢♦r 2✲♥♦r♠ ♦❢ ❛ r❛♥❞♦♠ ✈❛r✐❛❜❧❡
- E [L2] = max
Q≪P
- x∈X
- Q(x)P(x)ℓ(x)
✶✶
SLIDE 55
▼❛✐♥ ❙t❡♣✿ ▲✐♥❡❛r✐③✐♥❣ t❤❡ ❆✈❡r❛❣❡ ❆❣❡ ❈♦st
◮ ▲✐♥❡❛r✐③✐♥❣ t❤❡ r❛t✐♦♥❛❧ ❢♦r♠ ✭❡❛s②✮✿ E [L] + E
- L2
2E [L] = max
z≥0
- 1 − z2
2
- E [L] + z
- E [L2]
◮ ▲✐♥❡❛r✐③✐♥❣ t❤❡ 2✲♥♦r♠ t❡r♠❄
❆ ♥❡✇ ✈❛r✐❛t✐♦♥❛❧ ❢♦r♠✉❧❛ ❢♦r 2✲♥♦r♠ ♦❢ ❛ r❛♥❞♦♠ ✈❛r✐❛❜❧❡
- E [L2] = max
Q≪P
- x∈X
- Q(x)P(x)ℓ(x)
E [L] + E
- L2
2E [L] = max
z≥0, Q≪P
- 1 − z2
2
- E [L] + z
- x∈X
- Q(x)P(x)ℓ(x)
✶✶
SLIDE 56
❙✐♠✉❧❛t✐♦♥ ❘❡s✉❧ts
Zipf(s, N) ✐s ❣✐✈❡♥ ❜② P(i) =
i−s N
j=1 j−s ,
1 ≤ i ≤ N✳ 1 2 3 4 5 2.5 5 7.5 10 12.5 s ❆✈❡r❛❣❡✲❛❣❡
❙❤❛♥♥♦♥ ❝♦❞❡s ❢♦r P ✭✐♥t❡❣❡r ❧❡♥❣t❤s✮ ❙❤❛♥♥♦♥ ❈♦❞❡s ❢♦r P ∗ ✭✐♥t❡❣❡r ❧❡♥❣t❤s✮ ❙❤❛♥♥♦♥ ❈♦❞❡s ❢♦r P ∗ ✭r❡❛❧ ❧❡♥❣t❤s✮
❈♦♠♣❛r✐s♦♥ ♦❢ ♣r♦♣♦s❡❞ ❝♦❞❡s ❛♥❞ ❙❤❛♥♥♦♥ ❝♦❞❡s ❢♦r Zipf(s, 256) ✇✳r✳t✳ s✳
✶✷
SLIDE 57
❆ r❡❧❛t❡❞ ♣r♦❜❧❡♠
◮ ❍♦✇ t♦ ❞❡s✐❣♥ s♦✉r❝❡✲❝♦❞❡s ❢♦r ▼✐♥✐♠✉♠ ◗✉❡✉✐♥❣ ❉❡❧❛②❄✷ ❈♦st ❋✉♥❝t✐♦♥✿ ❖❜s❡r✈❛t✐♦♥ ✐♥ ❍✉♠❜❧❡t ✭✶✾✼✽✮✿ ❈♦❞❡s ✇❤✐❝❤ ♠✐♥✐♠✐③❡ t❤❡ ✜rst ♠♦♠❡♥t ❛r❡ ✧r♦❜✉st✧✳ ❲❡ ❢♦r♠❛❧❧② ♣r♦✈❡ t❤✐s ❡♠♣✐r✐❝❛❧ ♦❜s❡r✈❛t✐♦♥ ✉s✐♥❣ ♦✉r r❡❝✐♣❡✳
❙tr✉❝t✉r❛❧ s♦❧✉t✐♦♥ ❢♦r t❤❡ r❡❧❛①❡❞ ♣r♦❜❧❡♠
✱ ✇❤❡r❡ s❛t✐s✜❡s
✷❍✉♠❜❧❡t✱ P✳ ❆✳ ✭✶✾✼✽✮✳ ❙♦✉r❝❡ ❝♦❞✐♥❣ ❢♦r ❝♦♠♠✉♥✐❝❛t✐♦♥ ❝♦♥❝❡♥tr❛t♦rs✳
✶✸
SLIDE 58
❆ r❡❧❛t❡❞ ♣r♦❜❧❡♠
◮ ❍♦✇ t♦ ❞❡s✐❣♥ s♦✉r❝❡✲❝♦❞❡s ❢♦r ▼✐♥✐♠✉♠ ◗✉❡✉✐♥❣ ❉❡❧❛②❄✷ ◮ ❈♦st ❋✉♥❝t✐♦♥✿ ¯ D(e) = E [L] +
λE[L2] 2(1−λE[L]),
λE [L] < 1, ∞, λE [L] ≥ 1. ❖❜s❡r✈❛t✐♦♥ ✐♥ ❍✉♠❜❧❡t ✭✶✾✼✽✮✿ ❈♦❞❡s ✇❤✐❝❤ ♠✐♥✐♠✐③❡ t❤❡ ✜rst ♠♦♠❡♥t ❛r❡ ✧r♦❜✉st✧✳ ❲❡ ❢♦r♠❛❧❧② ♣r♦✈❡ t❤✐s ❡♠♣✐r✐❝❛❧ ♦❜s❡r✈❛t✐♦♥ ✉s✐♥❣ ♦✉r r❡❝✐♣❡✳
❙tr✉❝t✉r❛❧ s♦❧✉t✐♦♥ ❢♦r t❤❡ r❡❧❛①❡❞ ♣r♦❜❧❡♠
✱ ✇❤❡r❡ s❛t✐s✜❡s
✷❍✉♠❜❧❡t✱ P✳ ❆✳ ✭✶✾✼✽✮✳ ❙♦✉r❝❡ ❝♦❞✐♥❣ ❢♦r ❝♦♠♠✉♥✐❝❛t✐♦♥ ❝♦♥❝❡♥tr❛t♦rs✳
✶✸
SLIDE 59
❆ r❡❧❛t❡❞ ♣r♦❜❧❡♠
◮ ❍♦✇ t♦ ❞❡s✐❣♥ s♦✉r❝❡✲❝♦❞❡s ❢♦r ▼✐♥✐♠✉♠ ◗✉❡✉✐♥❣ ❉❡❧❛②❄✷ ◮ ❈♦st ❋✉♥❝t✐♦♥✿ ¯ D(e) = E [L] +
λE[L2] 2(1−λE[L]),
λE [L] < 1, ∞, λE [L] ≥ 1. ◮ ❖❜s❡r✈❛t✐♦♥ ✐♥ ❍✉♠❜❧❡t ✭✶✾✼✽✮✿ ❈♦❞❡s ✇❤✐❝❤ ♠✐♥✐♠✐③❡ t❤❡ ✜rst ♠♦♠❡♥t ❛r❡ ✧r♦❜✉st✧✳ ❲❡ ❢♦r♠❛❧❧② ♣r♦✈❡ t❤✐s ❡♠♣✐r✐❝❛❧ ♦❜s❡r✈❛t✐♦♥ ✉s✐♥❣ ♦✉r r❡❝✐♣❡✳
❙tr✉❝t✉r❛❧ s♦❧✉t✐♦♥ ❢♦r t❤❡ r❡❧❛①❡❞ ♣r♦❜❧❡♠
✱ ✇❤❡r❡ s❛t✐s✜❡s
✷❍✉♠❜❧❡t✱ P✳ ❆✳ ✭✶✾✼✽✮✳ ❙♦✉r❝❡ ❝♦❞✐♥❣ ❢♦r ❝♦♠♠✉♥✐❝❛t✐♦♥ ❝♦♥❝❡♥tr❛t♦rs✳
✶✸
SLIDE 60
❆ r❡❧❛t❡❞ ♣r♦❜❧❡♠
◮ ❍♦✇ t♦ ❞❡s✐❣♥ s♦✉r❝❡✲❝♦❞❡s ❢♦r ▼✐♥✐♠✉♠ ◗✉❡✉✐♥❣ ❉❡❧❛②❄✷ ◮ ❈♦st ❋✉♥❝t✐♦♥✿ ¯ D(e) = E [L] +
λE[L2] 2(1−λE[L]),
λE [L] < 1, ∞, λE [L] ≥ 1. ◮ ❖❜s❡r✈❛t✐♦♥ ✐♥ ❍✉♠❜❧❡t ✭✶✾✼✽✮✿ ❈♦❞❡s ✇❤✐❝❤ ♠✐♥✐♠✐③❡ t❤❡ ✜rst ♠♦♠❡♥t ❛r❡ ✧r♦❜✉st✧✳ ◮ ❲❡ ❢♦r♠❛❧❧② ♣r♦✈❡ t❤✐s ❡♠♣✐r✐❝❛❧ ♦❜s❡r✈❛t✐♦♥ ✉s✐♥❣ ♦✉r r❡❝✐♣❡✳
❙tr✉❝t✉r❛❧ s♦❧✉t✐♦♥ ❢♦r t❤❡ r❡❧❛①❡❞ ♣r♦❜❧❡♠
ℓ∗(x) = − log P ∗(x)✱ ✇❤❡r❡ P ∗ s❛t✐s✜❡s D(P||P ∗) ≤ log
- 1 + 1
√ 2
- .
✷❍✉♠❜❧❡t✱ P✳ ❆✳ ✭✶✾✼✽✮✳ ❙♦✉r❝❡ ❝♦❞✐♥❣ ❢♦r ❝♦♠♠✉♥✐❝❛t✐♦♥ ❝♦♥❝❡♥tr❛t♦rs✳
✶✸
SLIDE 61
■♥ s✉♠♠❛r② ✳✳✳
◮ ◆❡✇ ✈❛r✐❛t✐♦♥❛❧ ❢♦r♠✉❧❛ ❢♦r pth ♥♦r♠ ♦❢ r❛♥❞♦♠ ✈❛r✐❛❜❧❡ ◮ ❘❡❝✐♣❡ ❢♦r ♠✐♥✐♠✐③✐♥❣ ❛✈❡r❛❣❡ ❛❣❡ ❜❛s❡❞ ♦♥ ❊♥tr♦♣② ▼❛①✐♠✐③❛t✐♦♥ ◮ ●❡♥❡r❛❧ ❘❡❝✐♣❡✿ ❝❛♥ ❜❡ ✉s❡❞ t♦ ♦♣t✐♠✐③❡ ♦t❤❡r ♥♦♥✲❧✐♥❡❛r ❝♦sts
✶✹
SLIDE 62
❇❛❝❦✉♣ ❙❧✐❞❡s
SLIDE 63
❙✐♠✐❧❛r ❈♦st ❋✉♥❝t✐♦♥
▼✐♥✐♠✉♠ ❉❡❧❛② Pr♦❜❧❡♠✸ ▼✐♥✐♠✉♠ ❆❣❡ Pr♦❜❧❡♠
¯ D(e) =
- E [L] +
λE[L2] 2(1−λE[L]),
λE [L] < 1, ∞, λE [L] ≥ 1. ¯ A(e) = E [L] + E
- L2
2E [L] − 1 2
❈♦♥✈❡① ❍✉❧❧ ❆❧❣♦r✐t❤♠✹ 1 2 3 4 5 4 8 12 E [L] E
- L2
✸❍✉♠❜❧❡t✱ P✳ ❆✳ ✭✶✾✼✽✮✳ ❙♦✉r❝❡ ❝♦❞✐♥❣ ❢♦r ❝♦♠♠✉♥✐❝❛t✐♦♥ ❝♦♥❝❡♥tr❛t♦rs✳ ✹▲❛r♠♦r❡✱ ▲✳ ▲✳ ✭✶✾✽✾✮✳ ▼✐♥✐♠✉♠ ❞❡❧❛② ❝♦❞❡s✳ ❙■❆▼ ❏♦✉r♥❛❧ ♦♥
❈♦♠♣✉t✐♥❣✱ ✶✽✭✶✮✱ ✽✷✲✾✹✳
✶
SLIDE 64
P❡r❢♦r♠❛♥❝❡ ♦❢ ❙❤❛♥♥♦♥ ❈♦❞❡s
❙❤❛♥♥♦♥ ❝♦❞❡ ❢♦r P✿ ℓ(x) = ⌈− log P(x)⌉ ∀x✳
▲❡♠♠❛
- ✐✈❡♥ ❛ ♣♠❢ P ♦♥ X✱ ❛ ❙❤❛♥♥♦♥ ❝♦❞❡ e ❢♦r P ❤❛s ❛✈❡r❛❣❡ ❛❣❡ ❛t