t r ts tt - PowerPoint PPT Presentation

▼♦♥t❡ ❈❛r❧♦ ♠❡t❤♦❞s ❛♥❞ ❖♣t✐♠✐③❛t✐♦♥✿ ■♥t❡rt✇✐♥✐♥❣s ●❡rs❡♥❞❡ ❋♦rt ❈◆❘❙ ■♥st✐t✉t ❞❡ ▼❛t❤é♠❛t✐q✉❡s ❞❡ ❚♦✉❧♦✉s❡✱ ❋r❛♥❝❡ ▼♦♥t❡ ❈❛r❧♦ ▼❡t❤♦❞s✱ ❙②❞♥❡②✱ ❏✉❧② ✷✵✶✾✳

✳ ■♥t❡rt✇✐♥❡❞✱ ✇❤② ❄ ✳

❚♦ ✐♠♣r♦✈❡ ▼♦♥t❡ ❈❛r❧♦ ♠❡t❤♦❞s t❛r❣❡tt✐♥❣✿ d π = π d µ • ❚❤❡ ✧♥❛✐✈❡✧ ▼❈ s❛♠♣❧❡r ❞❡♣❡♥❞s ♦♥ ❞❡s✐❣♥ ♣❛r❛♠❡t❡rs ✐♥ R p ♦r ✐♥ ✐♥✜♥✐t❡ ❞✐♠❡♥s✐♦♥ θ • ❚❤❡♦r❡t✐❝❛❧ st✉❞✐❡s ❝❛r❛❝t❡r✐③❡ ❛♥ ♦♣t✐♠❛❧ ❝❤♦✐❝❡ ♦❢ t❤❡s❡s ♣❛r❛♠❡t❡rs θ ⋆ ❜② � θ ⋆ ∈ Θ s✳t✳ H ( θ, x ) d π ( x ) = 0 ♦r � θ ⋆ ∈ argmin θ ∈ Θ C ( θ, x ) d π ( x ) = 0 . • ❙tr❛t❡❣✐❡s✿ ✲ ❙tr❛t❡❣② ✶✿ ❛ ♣r❡❧✐♠✐♥❛r② ✧♠❛❝❤✐♥❡r②✧ ❢♦r t❤❡ ❛♣♣r♦①✐♠❛t✐♦♥ ♦❢ θ ⋆ ❀ t❤❡♥ r✉♥ t❤❡ ▼❈ s❛♠♣❧❡r ✇✐t❤ θ ← θ ⋆ ✲ ❙tr❛t❡❣② ✷✿ ❧❡❛r♥ θ ❛♥❞ s❛♠♣❧❡ ❝♦♥❝♦♠✐t❛♥t❧②

❚♦ ♠❛❦❡ ♦♣t✐♠✐③❛t✐♦♥ ♠❡t❤♦❞s tr❛❝t❛❜❧❡ • ■♥tr❛❝t❛❜❧❡ ♦❜❥❡❝t✐✈❡ ❢✉♥❝t✐♦♥ � θ s✳t✳ h ( θ ) = 0 ✇❤❡♥ h ✐s ♥♦t ❡①♣❧✐❝✐t h ( θ ) = X H ( θ, x ) d π θ ( x ) ♦r � argmin θ ∈ Θ X C ( θ, x ) d π θ ( x ) • ■♥tr❛❝t❛❜❧❡ ❛✉①✐❧✐❛r② q✉❛♥t✐t✐❡s ❊①✲✶ ●r❛❞✐❡♥t✲❜❛s❡❞ ♠❡t❤♦❞s � ∇ f ( θ ) = X H ( θ, x ) d π θ ( x ) ❊①✲✷ ▼❛❥♦r✐③❡✲▼✐♥✐♠✐③❛t✐♦♥ ♠❡t❤♦❞s � f ( θ ) ≤ F t ( θ ) = ❛t ✐t❡r❛t✐♦♥ t ✱ X H t ( θ, x ) d π t,θ ( x ) • ❙tr❛t❡❣✐❡s✿ ❯s❡ ▼♦♥t❡ ❈❛r❧♦ t❡❝❤♥✐q✉❡s t♦ ❛♣♣r♦①✐♠❛t❡ t❤❡ ✉♥❦♥♦✇♥ q✉❛♥t✐t✐❡s

■♥ t❤✐s t❛❧❦✱ ▼❛r❦♦✈ ✦ • ❢r♦♠ t❤❡ ▼♦♥t❡ ❈❛r❧♦ ♣♦✐♥t ♦❢ ✈✐❡✇✿ ✇❤✐❝❤ ❝♦♥❞✐t✐♦♥s ♦♥ t❤❡ ✉♣❞❛t✐♥❣ s❝❤❡♠❡ ❢♦r ❝♦♥✈❡r❣❡♥❝❡ ♦❢ t❤❡ s❛♠♣❧❡r ❄ ❈❛s❡✿ ▼❛r❦♦✈ ❝❤❛✐♥ ▼♦♥t❡ ❈❛r❧♦ s❛♠♣❧❡r • ❢r♦♠ t❤❡ ♦♣t✐♠✐③❛t✐♦♥ ♣♦✐♥t ♦❢ ✈✐❡✇✿ ✇❤✐❝❤ ❝♦♥❞✐t✐♦♥s ♦♥ t❤❡ ▼♦♥t❡ ❈❛r❧♦ ❛♣♣r♦①✐♠❛t✐♦♥ ❢♦r ❝♦♥✈❡r❣❡♥❝❡ ♦❢ t❤❡ st♦❝❤❛st✐❝ ♦♣t✐♠✐③❛t✐♦♥ ❄ ❈❛s❡✿ ❙t♦❝❤❛st✐❝ ❆♣♣r♦①✐♠❛t✐♦♥ ♠❡t❤♦❞s ✇✐t❤ ▼❛r❦♦✈✐❛♥ ✐♥♣✉ts • ❆♣♣❧✐❝❛t✐♦♥ t♦ ❛ ❈♦♠♣✉t❛t✐♦♥❛❧ ▼❛❝❤✐♥❡ ▲❡❛r♥✐♥❣ ♣❜♠✿ ♣❡♥❛❧✐③❡❞ ▼❛①✐♠✉♠ ▲✐❦❡❧✐❤♦♦❞ t❤r♦✉❣❤ ❙t♦❝❤❛st✐❝ Pr♦①✐♠❛❧✲●r❛❞✐❡♥t ♠❡t❤♦❞s

✳ P❛rt ■✿ ❚❤❡♦r② ♦❢ ❝♦♥tr♦❧❧❡❞ ✭♦r ❛❞❛♣t✐✈❡✮ ▼❛r❦♦✈ ❝❤❛✐♥s ✳

❊①❛♠♣❧❡ ✶✴ ❆❞❛♣t❡❞ ▼❛r❦♦✈ ❝❤❛✐♥ ▼♦♥t❡ ❈❛r❧♦ s❛♠♣❧❡rs • ❍❛st✐♥❣s✲▼❡tr♦♣♦❧✐s ❛❧❣♦r✐t❤♠✱ ✇✐t❤ ●❛✉ss✐❛♥ ♣r♦♣♦s❛❧ ❛♥❞ t❛r❣❡t d π ♦♥ X ⊆ R d Y t +1 ∼ N d ( X t , θ ) Pr♦♣♦s❛❧✿ � ✇✐t❤ ♣r♦❜❛❜✐❧✐t② α ( X t , Y t +1 ) Y t +1 ❆❝❝❡♣t✲❘❡❥❡❝t X t +1 = X t ♦t❤❡r✇✐s❡ s✉♠♠❛r✐③❡❞✿ X t +1 ∼ P θ ( X t , · ) • ✧❖♣t✐♠❛❧✧ ❝❤♦✐❝❡ ♦❢ t❤❡ ❝♦✈❛r✐❛♥❝❡ ♠❛tr✐① θ θ opt = (2 . 38) 2 Cov π ( X ) = (2 . 38) 2 Γ opt d d

❊①❛♠♣❧❡ ✶ ✭t♦ ❢♦❧❧♦✇✮✴ ❆❞❛♣t❡❞ ▼❛r❦♦✈ ❝❤❛✐♥ ▼♦♥t❡ ❈❛r❧♦ s❛♠♣❧❡rs • ❚❤❡ ❛❧❣♦r✐t❤♠ X t +1 ∼ P θ t ( X t , · ) ❙❛♠♣❧❡ ❙❆ s❝❤❡♠❡✿ Γ t +1 = ❡♠♣✐r✐❝❛❧ ❝♦✈ ♠❛tr✐① ♦❢ X 1: t +1 ❝♦♠♣✉t❡❞ ❢r♦♠ Γ t , X t +1 θ t +1 = (2 . 38) 2 d − 1 Γ t +1 • ■♥ t❤✐s ❡①❛♠♣❧❡✱ ❛ ❢❛♠✐❧② ♦❢ tr❛♥s✐t✐♦♥ ❦❡r♥❡❧s { P θ , θ ∈ Θ } ❛♥❞ ∀ θ, P θ ✐♥✈❛r✐❛♥t ✇✳r✳t✳ π • ❈♦♥✈❡r❣❡♥❝❡ r❡s✉❧ts✿ ✭❙❛❦s♠❛♥✲❱✐❤♦❧❛✱ ✷✵✶✵❀ ❋✳✲▼♦✉❧✐♥❡s✲Pr✐♦✉r❡t✱ ✷✵✶✷✮ ✲ lim t θ t = θ opt ✲ t❤❡ ❞✐str✐❜✉t✐♦♥ ♦❢ ( X t ) t ❝♦♥✈❡r❣❡s t♦ π ✭❝♦♥❞✐t✐♦♥s ♦♥ t❤❡ t❛✐❧s ♦❢ π ✮ ✲ str♦♥❣ ▲▲◆✱ ❈▲❚ ❢♦r t❤❡ s❛♠♣❧❡s { X t } t

❊①❛♠♣❧❡ ✷✴ ❆❞❛♣t❡❞ ■♠♣♦rt❛♥❝❡ s❛♠♣❧✐♥❣ ❜② ❲❛♥❣✲▲❛♥❞❛✉ ❛♣♣r♦❛❝❤❡s • ❆ ❤✐❣❤❧② ♠✉❧t✐♠♦❞❛❧ t❛r❣❡t ❞❡♥s✐t② d π ♦♥ X ⊆ R d ✳ • ❆ ❢❛♠✐❧② ♦❢ ♣r♦♣♦s❛❧ ♠❡❝❛♥✐s♠s✿ ●✐✈❡♥ ❛ ♣❛rt✐t✐♦♥ X 1 , · · · , X I ♦❢ X ✱ I 1 X i ( x ) d π ( x ) � d π θ ( x ) ∝ θ ( i ) , θ = ( θ (1) , · · · , θ ( I )) ❛ ✇❡✐❣❤t ✈❡❝t♦r i =1 � • ❖♣t✐♠❛❧ ♣r♦♣♦s❛❧✿ d π θ ⋆ ✇✐t❤ θ ⋆ ( i ) = X i d π ( u ) ✱ • θ ⋆ ✱ ✉♥✐q✉❡ ❧✐♠✐t✐♥❣ ✈❛❧✉❡ ♦❢ ❛ ❙t♦❝❤❛st✐❝ ❆♣♣r♦①✐♠❛t✐♦♥ s❝❤❡♠❡   I � �  . H ( θ, X ) d π θ ( x ) ❛♥❞ H i ( θ, x ) = θ ( i )  1 X i ( x ) − θ ( j )1 X j ( x ) ✇✐t❤ ♠❡❛♥ ✜❡❧❞ X j =1 0.14 0.12 8 7 0.12 0.1 6 2.5 5 0.1 2 4 0.08 1.5 3 0.08 1 2 0.06 0.5 0.06 1 0 0 0.04 3 0.04 −0.5 2 1 0.02 −1 0.02 0 −1.5 −1 −2.5 −1.5 −2 0 0 −0.5 −1 −2 0.5 0 −2 0 0.5 e6 1 e6 1.5 e6 2 e6 2.5 e6 3 e6 1.5 1 −2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2.5 2.5 2 −2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2.5

❊①❛♠♣❧❡ ✷ ✭t♦ ❢♦❧❧♦✇✮✴ ❆❞❛♣t❡❞ ■♠♣♦rt❛♥❝❡ s❛♠♣❧✐♥❣ ❜② ❲❛♥❣✲▲❛♥❞❛✉ ❛♣♣r♦❛❝❤❡s • ❚❤❡ ❛❧❣♦r✐t❤♠ X t +1 ∼ P θ t ( X t , · ) , ❙❛♠♣❧❡✿ ✇❤❡r❡ π θ P θ = π θ ❙❆ s❝❤❡♠❡✿ θ t +1 = θ t + γ t +1 H ( θ t , X t +1 ) • ■♥ t❤✐s ❡①❛♠♣❧❡✱ ❛ ❢❛♠✐❧② ♦❢ tr❛♥s✐t✐♦♥ ❦❡r♥❡❧s { P θ , θ ∈ Θ } s✉❝❤ t❤❛t ∀ θ, P θ ✐♥✈❛r✐❛♥t ✇✳r✳t✳ π θ • ❈♦♥✈❡r❣❡♥❝❡ r❡s✉❧ts✿ ✭❋✳✲❏♦✉r❞❛✐♥✲▲❡❧✐❡✈r❡✲❙t♦❧t③✲✷✵✶✺✱✷✵✶✼✱✷✵✶✽✮ ✲ θ t ❝♦♥✈❡r❣❡s t♦ θ ⋆ ❛✳s✳❀ ✲ t❤❡ ❞✐str✐❜✉t✐♦♥ ♦❢ X t ❝♦♥✈❡r❣❡s t♦ d π θ ⋆ ❀ ✲ θ t ✐s ❛♥ ❡st✐♠❛t❡ ♦❢ t❤❡ ✐♠♣♦rt❛♥❝❡ r❛t✐♦ [d π/ d π θ ⋆ ]( x ) ✱ ❝♦♥st❛♥t ❛❧♦♥❣ ❡❛❝❤ X i ✳

■s ❛ ✏t❤❡♦r②✑ r❡q✉✐r❡❞ ❄ ❨❊❙ ✦ ❝♦♥✈❡r❣❡♥❝❡ ❝❛♥ ❜❡ ❧♦st ❜② t❤❡ ❛❞❛♣t✐♦♥ ♠❡❝❛♥✐s♠ ❊✈❡♥ ✐♥ ❛ s✐♠♣❧❡ ❝❛s❡ ✇❤❡♥ ∀ θ ∈ Θ , P θ ✐♥✈❛r✐❛♥t ✇rt d π, ♦♥❡ ❝❛♥ ❞❡✜♥❡ ❛ s✐♠♣❧❡ ❛❞❛♣t✐♦♥ ♠❡❝❛♥✐s♠ X t +1 | ♣❛st 1: t ∼ P θ t ( X t , · ) θ t ∈ σ ( X 1: t ) s✉❝❤ t❤❛t � lim t E [ f ( X t )] � = f d π. ❆ { 0 , 1 } ✲✈❛❧✉❡❞ ❝❤❛✐♥ { X t } t ❞❡✜♥❡❞ ❜② X t +1 ∼ P X t ( X t , · ) ✇❤❡r❡ t❤❡ tr❛♥s✐t✐♦♥ ♠❛tr✐❝❡s ❛r❡ � � � � t 0 (1 − t 0 ) t 1 (1 − t 1 ) P 0 = P 1 = (1 − t 0 ) (1 − t 1 ) t 0 t 1 ❚❤❡♥ P 0 ❛♥❞ P 1 ❛r❡ ✐♥✈❛r✐❛♥t ✇✳r✳t [1 / 2 , 1 / 2] ❜✉t { X t } ✐s ❛ ▼❛r❦♦✈ ❝❤❛✐♥ ✐♥✈❛r✐❛♥t ✇✳r✳t✳ [ t 1 , t 0 ]