t r tt t - PowerPoint PPT Presentation

❲❤❡♥ ▼♦♥t❡ ❈❛r❧♦ ❛♥❞ ❖♣t✐♠✐③❛t✐♦♥ ♠❡t ✐♥ ❛ ▼❛r❦♦✈✐❛♥ ❞❛♥❝❡ ●❡rs❡♥❞❡ ❋♦rt ❈◆❘❙ ■♥st✐t✉t ❞❡ ▼❛t❤é♠❛t✐q✉❡s ❞❡ ❚♦✉❧♦✉s❡✱ ❋r❛♥❝❡ ■❈❚❙ ✧❆❞✈❛♥❝❡s ✐♥ ❆♣♣❧✐❡❞ Pr♦❜❛❜✐❧✐t②✧✱ ❇❡♥❣❛❧✉r✉✱ ❆✉❣✉st ✷✵✶✾ ✳

✳ ❆ ❞❛♥❝❡✱ ✇❤② ❄ ✳

❚♦ ✐♠♣r♦✈❡ ▼♦♥t❡ ❈❛r❧♦ ♠❡t❤♦❞s t❛r❣❡tt✐♥❣✿ d π = π d µ • ❚❤❡ ✧♥❛✐✈❡✧ ▼❈ s❛♠♣❧❡r ❞❡♣❡♥❞s ♦♥ ❞❡s✐❣♥ ♣❛r❛♠❡t❡rs ✐♥ R p ♦r ✐♥ ✐♥✜♥✐t❡ ❞✐♠❡♥s✐♦♥ θ • ❚❤❡♦r❡t✐❝❛❧ st✉❞✐❡s ❝❛r❛❝t❡r✐③❡ ❛♥ ♦♣t✐♠❛❧ ❝❤♦✐❝❡ ♦❢ t❤❡s❡s ♣❛r❛♠❡t❡rs θ ⋆ ❜② � θ ⋆ ∈ Θ s✳t✳ H ( θ, x ) d π ( x ) = 0 ♦r � θ ⋆ ∈ argmin θ ∈ Θ C ( θ, x ) d π ( x ) = 0 . • ❙tr❛t❡❣✐❡s✿ ✲ ❙tr❛t❡❣② ✶✿ ❛ ♣r❡❧✐♠✐♥❛r② ✧♠❛❝❤✐♥❡r②✧ ❢♦r t❤❡ ❛♣♣r♦①✐♠❛t✐♦♥ ♦❢ θ ⋆ ❀ t❤❡♥ r✉♥ t❤❡ ▼❈ s❛♠♣❧❡r ✇✐t❤ θ ← θ ⋆ ✲ ❙tr❛t❡❣② ✷✿ ❧❡❛r♥ θ ❛♥❞ s❛♠♣❧❡ ❝♦♥❝♦♠✐t❛♥t❧②

❚♦ ♠❛❦❡ ♦♣t✐♠✐③❛t✐♦♥ ♠❡t❤♦❞s tr❛❝t❛❜❧❡ • ■♥tr❛❝t❛❜❧❡ ♦❜❥❡❝t✐✈❡ ❢✉♥❝t✐♦♥ � θ s✳t✳ h ( θ ) = 0 ✇❤❡♥ h ✐s ♥♦t ❡①♣❧✐❝✐t h ( θ ) = X H ( θ, x ) d π θ ( x ) ♦r � argmin θ ∈ Θ X C ( θ, x ) d π θ ( x ) • ■♥tr❛❝t❛❜❧❡ ❛✉①✐❧✐❛r② q✉❛♥t✐t✐❡s ❊①✲✶ ●r❛❞✐❡♥t✲❜❛s❡❞ ♠❡t❤♦❞s � ∇ f ( θ ) = X H ( θ, x ) d π θ ( x ) ❊①✲✷ ▼❛❥♦r✐③❡✲▼✐♥✐♠✐③❛t✐♦♥ ♠❡t❤♦❞s � f ( θ ) ≤ F t ( θ ) = ❛t ✐t❡r❛t✐♦♥ t ✱ X H t ( θ, x ) d π t,θ ( x ) • ❙tr❛t❡❣✐❡s✿ ❯s❡ ▼♦♥t❡ ❈❛r❧♦ t❡❝❤♥✐q✉❡s t♦ ❛♣♣r♦①✐♠❛t❡ t❤❡ ✉♥❦♥♦✇♥ q✉❛♥t✐t✐❡s

■♥ t❤✐s t❛❧❦✱ ▼❛r❦♦✈ ✦ • ❢r♦♠ t❤❡ ▼♦♥t❡ ❈❛r❧♦ ♣♦✐♥t ♦❢ ✈✐❡✇✿ ✇❤✐❝❤ ❝♦♥❞✐t✐♦♥s ♦♥ t❤❡ ✉♣❞❛t✐♥❣ s❝❤❡♠❡ ❢♦r ❝♦♥✈❡r❣❡♥❝❡ ♦❢ t❤❡ s❛♠♣❧❡r ❄ ❈❛s❡✿ ▼❛r❦♦✈ ❝❤❛✐♥ ▼♦♥t❡ ❈❛r❧♦ s❛♠♣❧❡r • ❢r♦♠ t❤❡ ♦♣t✐♠✐③❛t✐♦♥ ♣♦✐♥t ♦❢ ✈✐❡✇✿ ✇❤✐❝❤ ❝♦♥❞✐t✐♦♥s ♦♥ t❤❡ ▼♦♥t❡ ❈❛r❧♦ ❛♣♣r♦①✐♠❛t✐♦♥ ❢♦r ❝♦♥✈❡r❣❡♥❝❡ ♦❢ t❤❡ st♦❝❤❛st✐❝ ♦♣t✐♠✐③❛t✐♦♥ ❄ ❈❛s❡✿ ❙t♦❝❤❛st✐❝ ❆♣♣r♦①✐♠❛t✐♦♥ ♠❡t❤♦❞s ✇✐t❤ ▼❛r❦♦✈✐❛♥ ✐♥♣✉ts • ✭❚❛❧❦✮ ❆♣♣❧✐❝❛t✐♦♥ t♦ ❛ ❈♦♠♣✉t❛t✐♦♥❛❧ ▼❛❝❤✐♥❡ ▲❡❛r♥✐♥❣ ♣❜♠✿ ♣❡♥❛❧✐③❡❞ ▼❛①✐♠✉♠ ▲✐❦❡❧✐❤♦♦❞ t❤r♦✉❣❤ ❙t♦❝❤❛st✐❝ Pr♦①✐♠❛❧✲●r❛❞✐❡♥t ❜❛s❡❞ ♠❡t❤♦❞s

✳ P❛rt ■✿ ▼♦t✐✈❛t✐♥❣ ❡①❛♠♣❧❡s ✳

✶st ❊①✳ ❆❞❛♣t✐✈❡ ■♠♣♦rt❛♥❝❡ s❛♠♣❧✐♥❣ ❜② ❲❛♥❣✲▲❛♥❞❛✉ ❛♣♣r♦❛❝❤❡s ✭✶✴✻✮ ❚❤❡ ♣r♦❜❧❡♠ • ❆ ❤✐❣❤❧② ♠✉❧t✐♠♦❞❛❧ t❛r❣❡t ❞❡♥s✐t② d π ♦♥ X ⊆ R d ✳ 8 8 7 7 2.5 6 6 2 5 5 1.5 4 4 3 1 3 2 2 0.5 1 1 0 0 0 −4 3 −0.5 2 −2 −1 1 0 0 −1.5 2 −1 −2 −2.5 −1.5 −2 −0.5 −1 −1 −1.5 −2 −2 0 4 −0.5 1 0.5 0.5 0 1.5 1 2 2 1.5 −2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2.5 2.5 2.5 • ❚✇♦ s❛♠♣❧❡rs ✇✐t❤ ❞✐✛❡r❡♥t ❜❡❤❛✈✐♦rs ✭♣❧♦t✿ t❤❡ x ✲♣❛t❤ ♦❢ ❛ ❝❤❛✐♥ ✐♥ R 2 ✮ beta=4 beta=4 2 2 1.5 1.5 1 1 0.5 0.5 0 0 −0.5 −0.5 −1 −1 −1.5 −1.5 −2 −2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 0 2 4 6 8 10 12 6 4 x 10 x 10

✶st ❊①✳ ✭✷✴✻✮ ❚❤❡ str❛t❡❣② ❢♦r ❝❤♦♦s✐♥❣ t❤❡ ♣r♦♣♦s❛❧ ♠❡❝❛♥✐s♠ • ❆ ❢❛♠✐❧② ♦❢ ♣r♦♣♦s❛❧ ♠❡❝❛♥✐s♠s ♦❜t❛✐♥❡❞ ❜② ❜✐❛s✐♥❣ ❧♦❝❛❧❧② t❤❡ t❛r❣❡t✿ ✲ ❣✐✈❡♥ ❛ ♣❛rt✐t✐♦♥ X 1 , · · · , X I ♦❢ X ✱ ✲ ❢♦r ❛♥② ✇❡✐❣❤t ✈❡❝t♦r θ = ( θ (1) , · · · , θ ( I )) I � 1 1 X i ( x ) d π ( x ) � d π θ ( x ) = θ ( i ) , ✇✐t❤ θ ⋆ ( i ) := d π ( u ) . θ ⋆ ( i ) � I X i i =1 i =1 θ ( i ) • ❖♣t✐♠❛❧ ♣r♦♣♦s❛❧✿ d π θ ⋆ ❁♣r♦♦❢❃ • ❯♥❢♦rt✉♥❛t❡❧②✱ θ ⋆ ✉♥❛✈❛✐❧❛❜❧❡✳

✶st ❊①✳ ✭✸✴✻✮ ■❢ π θ ⋆ ✇❡r❡ ❛✈❛✐❧❛❜❧❡ • ❚❤❡ ❛❧❣♦r✐t❤♠ ✇♦✉❧❞ ❜❡✿ ✲ ❙❛♠♣❧❡ X 1 , · · · , X n , · · · ✐✳✐✳❞✳ ✇✐t❤ ❞✐str✐❜✉t✐♦♥ d π θ ⋆ ✭♦r ❛ ▼❈▼❈ ✇✐t❤ t❛r❣❡t d π θ ⋆ ✮ ✲ ❈♦♠♣✉t❡ t❤❡ ✐♠♣♦rt❛♥❝❡ r❛t✐♦ I d π � ( X k ) = I 1 X i ( X k ) θ ⋆ ( i ) d π θ ⋆ i =1 • ❲❤❡♥ ❛♣♣r♦①✐♠❛t✐♥❣ ❛♥ ❡①♣❡❝t❛t✐♦♥✱ s❡t   T I � φ d π ≈ I � �  φ ( X t ) . 1 X i ( X t ) θ ⋆ ( i )  T t =1 i =1

✶st ❊①✳ ✭✹✴✻✮ θ ⋆ ❛♥❞ t❤❡r❡❢♦r❡ d π θ ⋆ ❛r❡ ✉♥❦♥♦✇♥✱ s♦ ❄ • θ ⋆ ∈ R I ❝♦❧❧❡❝ts � X i d π ❢♦r ❛❧❧ i ∈ { 1 , · · · , I } ✱ X H ( θ, x ) d π θ ( x ) ∈ R I ✇❤❡r❡ ❢♦r ❛❧❧ i ∈ { 1 , · · · , I } � • θ ⋆ t❤❡ ✉♥✐q✉❡ r♦♦t ♦❢ θ �→ I � H i ( θ, x ) := θ ( i )1 X ( i ) ( x ) − θ ( i ) 1 X j ( x ) θ ( j ) . j =1 t❤✉s s✉❣❣❡st✐♥❣ t❤❡ ✉s❡ ♦❢ ❛ ❙t♦❝❤❛st✐❝ ❆♣♣r♦①✐♠❛t✐♦♥ ♣r♦❝❡❞✉r❡✿ θ ⋆ ≈ lim t θ t θ t +1 = θ t + γ t +1 H ( θ t , X t +1 ) X t +1 ∼ d π θ t • ❚❤✐s ✉♣❞❛t❡ s❝❤❡♠❡ ✐s ❛ ♥♦r♠❛❧✐③❡❞ ❝♦✉♥t❡r ♦❢ t❤❡ ♥✉♠❜❡r ♦❢ ✈✐s✐ts t♦ X i ❁♣r♦♦❢❃

✶st ❊①✳ ✭✺✴✻✮ ❚❤❡ ❛❧❣♦r✐t❤♠✿ ❲❛♥❣✲▲❛♥❞❛✉ ❜❛s❡❞ ♣r♦❝❡❞✉r❡s • ■♥✐t✐❛❧✐s❛t✐♦♥✿ ❛ ✇❡✐❣❤t ✈❡❝t♦r θ 0 ❘❡♣❡❛t ❢♦r t = 1 , · · · , T ✲ s❛♠♣❧❡ ❛ ♣♦✐♥t X t +1 ∼ d π θ t ✲ ✉♣❞❛t❡ t❤❡ ❡st✐♠❛t❡ ♦❢ θ ⋆ θ t +1 = θ t + γ t +1 H ( θ t , X t +1 ) . ✇❤❡r❡ X t +1 ∼ P θ t ( X t , · ) ❛♥❞ P θ ✐♥✈✳ ✇rt d π θ ✳ • ❊①♣❡❝t❡❞✿ ✲ t❤❡ ❝♦♥✈❡r❣❡♥❝❡ ♦❢ θ t t♦ θ ⋆ ✿ ❙❆ s❝❤❡♠❡✱ ❢❡❞ ✇✐t❤ ❛❞❛♣t✐✈❡ ✭❝♦♥tr♦❧❧❡❞✮ ▼❈▼❈ s❛♠♣❧❡r✱ ✲ t❤❡ ❝♦♥✈❡r❣❡♥❝❡ ♦❢ t❤❡ ❞✐str✐❜✉t✐♦♥ ♦❢ X t t♦ d π θ ⋆

✶st ❊①✳ ✭✻✴✻✮ ❉♦❡s ✐t ✇♦r❦ ❄ P❧♦t✿ ❝♦♥✈❡r❣❡♥❝❡ ♦❢ θ t ❛♥❞ ✜rst ❡①✐t t✐♠❡s ❢r♦♠ ♦♥❡ ♠♦❞❡ 0.14 0.12 2.5 0.12 2 1.5 0.1 1 0.5 0.1 0 −0.5 0.08 −1 0.08 −1.5 −2 −2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2.5 0.06 0.06 0.04 0.04 ◮ s❡❡ ❋✱ ❑✉❤♥✱ ❏♦✉r❞❛✐♥✱ ▲❡❧✐è✈r❡✱ 0.02 0.02 ❙t♦❧t③ ✭✷✵✶✹✮❀ ❋✱ ❏♦✉r❞❛✐♥✱ 0 0 0 0.5 e6 1 e6 1.5 e6 2 e6 2.5 e6 3 e6 −2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2.5 ▲❡❧✐è✈r❡✱ ❙t♦❧t③ ✭✷✵✶✺✱✷✵✶✼✱✷✵✶✽✮ d = 96 1e+09 d = 48 1 d = 24 d = 12 1e+08 ❢♦r st✉❞✐❡s ♦❢ t❤❡s❡ ❲❛♥❣✲▲❛♥❞❛✉ d = 6 d = 3 0.5 1e+07 slope 1.25 Exit time ❜❛s❡s ❛❧❣♦r✐t❤♠s❀ ✐♥❝❧✉❞✐♥❣ 1e+06 X n,1 0 100000 s❡❧❢✲t✉♥❡❞ ❙❆ ✉♣❞❛t❡ r✉❧❡s ✭ γ t ✐s -0.5 10000 1000 r❛♥❞♦♠✮ ✳ -1 100 0 2e+08 4e+08 6e+08 8e+08 1e+09 1.2e+09 1.4e+09 2 4 6 8 10 12 14 Iterations β