t t t r r t r
play

tt tr - PowerPoint PPT Presentation

Oct 03, 2022 •104 likes •1.33k views

tt tr rt r r r s rst

  1. ◆♦t❡ ✿ ❈♦♠♣✉t✐♥❣ ✐s r❡❧❛t✐✈❡❧② ❡❛s②✱ s✐♥❝❡ ✐s ❛ ❝♦♥❝❛✈❡ ❢✉♥❝t✐♦♥ ♦❢ ✳ ❚❡st✐♥❣ ♦♣t✐♠❛❧✐t② ■t ✐s ❡❛s② ✭✇❤②❄✮ t♦ s❡❡ t❤❛t R ( W ) ✐s ♦♣t✐♠❛❧ ✐❢ ❛♥❞ ♦♥❧② ✐❢ R ( W ⊗ W ) = 2 R ( W ) ∀ W. ❚❤❡ ❛❜♦✈❡ ❡q✉❛❧✐t② ✭❛❞❞✐t✐✈✐t②✮ ❢♦❧❧♦✇s ✐❢ t❤❡ ❢♦❧❧♦✇✐♥❣ s✉❜✲❛❞❞✐t✐✈✐t② ❤♦❧❞s✿ I ( X 1 , X 2 ; Y 1 , Y 2 ) ≤ I ( X 1 ; Y 1 ) + I ( X 2 ; Y 2 ) . I ( X 1 , X 2 ; Y 1 , Y 2 ) = I ( X 1 , X 2 ; Y 1 ) + I ( X 1 , X 2 ; Y 2 | Y 1 ) = I ( X 1 , X 2 ; Y 1 ) + I ( Y 1 , X 1 , X 2 ; Y 2 ) − I ( Y 1 ; Y 2 ) = I ( X 1 ; Y 1 ) + I ( X 2 ; Y 2 ) − I ( Y 1 ; Y 2 ) ≤ I ( X 1 ; Y 1 ) + I ( X 2 ; Y 2 ) . ✹

  2. ❚❡st✐♥❣ ♦♣t✐♠❛❧✐t② ■t ✐s ❡❛s② ✭✇❤②❄✮ t♦ s❡❡ t❤❛t R ( W ) ✐s ♦♣t✐♠❛❧ ✐❢ ❛♥❞ ♦♥❧② ✐❢ R ( W ⊗ W ) = 2 R ( W ) ∀ W. ❚❤❡ ❛❜♦✈❡ ❡q✉❛❧✐t② ✭❛❞❞✐t✐✈✐t②✮ ❢♦❧❧♦✇s ✐❢ t❤❡ ❢♦❧❧♦✇✐♥❣ s✉❜✲❛❞❞✐t✐✈✐t② ❤♦❧❞s✿ I ( X 1 , X 2 ; Y 1 , Y 2 ) ≤ I ( X 1 ; Y 1 ) + I ( X 2 ; Y 2 ) . I ( X 1 , X 2 ; Y 1 , Y 2 ) = I ( X 1 , X 2 ; Y 1 ) + I ( X 1 , X 2 ; Y 2 | Y 1 ) = I ( X 1 , X 2 ; Y 1 ) + I ( Y 1 , X 1 , X 2 ; Y 2 ) − I ( Y 1 ; Y 2 ) = I ( X 1 ; Y 1 ) + I ( X 2 ; Y 2 ) − I ( Y 1 ; Y 2 ) ≤ I ( X 1 ; Y 1 ) + I ( X 2 ; Y 2 ) . ◆♦t❡ ✿ ❈♦♠♣✉t✐♥❣ R ( W ) = sup p ( x ) I ( X ; Y ) ✐s r❡❧❛t✐✈❡❧② ❡❛s②✱ s✐♥❝❡ I ( X ; Y ) ✐s ❛ ❝♦♥❝❛✈❡ ❢✉♥❝t✐♦♥ ♦❢ p ( x ) ✳ ✹

  3. ❲❡ ❛r❡ ♥♦✇ ✭❢✉❧❧② ✐♠♠❡rs❡❞✮ ✐♥ ❛ ✇✐r❡❧❡ss ✇♦r❧❞ ◆❡t✇♦r❦ ♦❢ ✉s❡rs s❤❛r✐♥❣ s❛♠❡ ♠❡❞✐✉♠ ❈❧❡❛r ♥❡❡❞ t♦ ♠❛①✐♠❛❧❧② ✉t✐❧✐③❡ t❤❡ ❧✐♠✐t❡❞ r❡s♦✉r❝❡s ✭♣♦✇❡r✱ ❜❛♥❞✇✐❞t❤✱ ❡♥❡r❣②✮ ❉❡✈❡❧♦♣ ❛ s✐♠✐❧❛r ✉♥❞❡rst❛♥❞✐♥❣ ✐♥ ♥❡t✇♦r❦ s❡tt✐♥❣s ❇✉t ✇❡ ✜rst ♥❡❡❞ t♦ ❢✉❧❧② ✉♥❞❡rst❛♥❞ t❤❡ ❜❛s✐❝ ❜✉✐❧❞✐♥❣ ❜❧♦❝❦s ❙✉❝❝❡ss❡s ❚❤❡ ✈❛r✐♦✉s ✐❞❡❛s ✐♥tr♦❞✉❝❡❞ ❜② ❙❤❛♥♥♦♥ ❤❛✈❡ ❧❡❞ t♦ ❛♥ ✐♥❢♦r♠❛t✐♦♥ r❡✈♦❧✉t✐♦♥ ❘❛♥❞♦♠ ❝♦❞✐♥❣ ❛♥❞ ✐ts ♦♣t✐♠❛❧✐t② ❤❛✈❡ ❞✐r❡❝t❧② ✐♥s♣✐r❡❞ • ▲♦✇ ❞❡♥s✐t② ♣❛r✐t② ❝❤❡❝❦ ❝♦❞❡s ✭▲❉P❈✮ • P♦❧❛r ❝♦❞❡s ⋆ ♣r♦♦❢ ♦❢ s✉❜✲❛❞❞✐t✐✈✐t② ✺

  4. ❙✉❝❝❡ss❡s ❚❤❡ ✈❛r✐♦✉s ✐❞❡❛s ✐♥tr♦❞✉❝❡❞ ❜② ❙❤❛♥♥♦♥ ❤❛✈❡ ❧❡❞ t♦ ❛♥ ✐♥❢♦r♠❛t✐♦♥ r❡✈♦❧✉t✐♦♥ ❘❛♥❞♦♠ ❝♦❞✐♥❣ ❛♥❞ ✐ts ♦♣t✐♠❛❧✐t② ❤❛✈❡ ❞✐r❡❝t❧② ✐♥s♣✐r❡❞ • ▲♦✇ ❞❡♥s✐t② ♣❛r✐t② ❝❤❡❝❦ ❝♦❞❡s ✭▲❉P❈✮ • P♦❧❛r ❝♦❞❡s ⋆ ♣r♦♦❢ ♦❢ s✉❜✲❛❞❞✐t✐✈✐t② ❲❡ ❛r❡ ♥♦✇ ✭❢✉❧❧② ✐♠♠❡rs❡❞✮ ✐♥ ❛ ✇✐r❡❧❡ss ✇♦r❧❞ • ◆❡t✇♦r❦ ♦❢ ✉s❡rs s❤❛r✐♥❣ s❛♠❡ ♠❡❞✐✉♠ • ❈❧❡❛r ♥❡❡❞ t♦ ♠❛①✐♠❛❧❧② ✉t✐❧✐③❡ t❤❡ ❧✐♠✐t❡❞ r❡s♦✉r❝❡s ✭♣♦✇❡r✱ ❜❛♥❞✇✐❞t❤✱ ❡♥❡r❣②✮ • ❉❡✈❡❧♦♣ ❛ s✐♠✐❧❛r ✉♥❞❡rst❛♥❞✐♥❣ ✐♥ ♥❡t✇♦r❦ s❡tt✐♥❣s ⋆ ❇✉t ✇❡ ✜rst ♥❡❡❞ t♦ ❢✉❧❧② ✉♥❞❡rst❛♥❞ t❤❡ ❜❛s✐❝ ❜✉✐❧❞✐♥❣ ❜❧♦❝❦s ✺

  5. ❘❛♥❞♦♠ ❝♦❞✐♥❣ ❝❛♥ ❜❡ ✉s❡❞ t♦ ❛❝❤✐❡✈❡ r❛t❡ ♣❛✐rs t❤❛t s❛t✐s❢② ❢♦r s♦♠❡ ❀ ✐t s✉✣❝❡s t♦ ❝♦♥s✐❞❡r ✳ ❈❛❧❧ t❤✐s r❡❣✐♦♥ ✳ ❆❤❧s✇❡❞❡ ◗✉❡st✐♦♥ ✿ ■s t❤✐s t❤❡ ❝❛♣❛❝✐t② ✭♦♣t✐♠❛❧✮ r❡❣✐♦♥❄ ✭❨❊❙✮ ✭❆❤❧s✇❡❞❡ ✬✼✷✮ ✶✳ ▼✉❧t✐♣❧❡ ❆❝❝❡ss ❈❤❛♥♥❡❧ ✭✉♣❧✐♥❦✮ ✭❙❤❛♥♥♦♥ ✬✻✶✮ X n 1 M 1 ❊♥❝♦❞❡r ✶ Y n ( ˆ M 1 , ˆ W ( y | x 1 , x 2 ) ❉❡❝♦❞❡r M 2 ) M 2 ❊♥❝♦❞❡r ✷ X n r❢✇✐r❡❧❡ss✲✇♦r❧❞ 2 ✻

  6. ◗✉❡st✐♦♥ ✿ ■s t❤✐s t❤❡ ❝❛♣❛❝✐t② ✭♦♣t✐♠❛❧✮ r❡❣✐♦♥❄ ✭❨❊❙✮ ✭❆❤❧s✇❡❞❡ ✬✼✷✮ ✶✳ ▼✉❧t✐♣❧❡ ❆❝❝❡ss ❈❤❛♥♥❡❧ ✭✉♣❧✐♥❦✮ ✭❙❤❛♥♥♦♥ ✬✻✶✮ X n 1 M 1 ❊♥❝♦❞❡r ✶ Y n ( ˆ M 1 , ˆ W ( y | x 1 , x 2 ) ❉❡❝♦❞❡r M 2 ) M 2 ❊♥❝♦❞❡r ✷ X n r❢✇✐r❡❧❡ss✲✇♦r❧❞ 2 ❘❛♥❞♦♠ ❝♦❞✐♥❣ ❝❛♥ ❜❡ ✉s❡❞ t♦ ❛❝❤✐❡✈❡ r❛t❡ ♣❛✐rs ( R 1 , R 2 ) t❤❛t s❛t✐s❢② R 1 ≤ I ( X 1 ; Y | X 2 , Q ) R 2 ≤ I ( X 2 ; Y | X 1 , Q ) R 1 + R 2 ≤ I ( X 1 , X 2 ; Y | Q ) ❢♦r s♦♠❡ p ( q ) p ( x 1 | q ) p ( x 2 | q ) ❀ ✐t s✉✣❝❡s t♦ ❝♦♥s✐❞❡r |Q| ≤ 2 ✳ ❈❛❧❧ t❤✐s r❡❣✐♦♥ R ( W ) ✳ ❆❤❧s✇❡❞❡ ✻

  7. ✭❨❊❙✮ ✭❆❤❧s✇❡❞❡ ✬✼✷✮ ✶✳ ▼✉❧t✐♣❧❡ ❆❝❝❡ss ❈❤❛♥♥❡❧ ✭✉♣❧✐♥❦✮ ✭❙❤❛♥♥♦♥ ✬✻✶✮ X n 1 M 1 ❊♥❝♦❞❡r ✶ Y n ( ˆ M 1 , ˆ W ( y | x 1 , x 2 ) ❉❡❝♦❞❡r M 2 ) M 2 ❊♥❝♦❞❡r ✷ X n r❢✇✐r❡❧❡ss✲✇♦r❧❞ 2 ❘❛♥❞♦♠ ❝♦❞✐♥❣ ❝❛♥ ❜❡ ✉s❡❞ t♦ ❛❝❤✐❡✈❡ r❛t❡ ♣❛✐rs ( R 1 , R 2 ) t❤❛t s❛t✐s❢② R 1 ≤ I ( X 1 ; Y | X 2 , Q ) R 2 ≤ I ( X 2 ; Y | X 1 , Q ) R 1 + R 2 ≤ I ( X 1 , X 2 ; Y | Q ) ❢♦r s♦♠❡ p ( q ) p ( x 1 | q ) p ( x 2 | q ) ❀ ✐t s✉✣❝❡s t♦ ❝♦♥s✐❞❡r |Q| ≤ 2 ✳ ❈❛❧❧ t❤✐s r❡❣✐♦♥ R ( W ) ✳ ❆❤❧s✇❡❞❡ ◗✉❡st✐♦♥ ✿ ■s t❤✐s t❤❡ ❝❛♣❛❝✐t② ✭♦♣t✐♠❛❧✮ r❡❣✐♦♥❄ ✻

  8. ✶✳ ▼✉❧t✐♣❧❡ ❆❝❝❡ss ❈❤❛♥♥❡❧ ✭✉♣❧✐♥❦✮ ✭❙❤❛♥♥♦♥ ✬✻✶✮ X n 1 M 1 ❊♥❝♦❞❡r ✶ Y n ( ˆ M 1 , ˆ W ( y | x 1 , x 2 ) ❉❡❝♦❞❡r M 2 ) M 2 ❊♥❝♦❞❡r ✷ X n r❢✇✐r❡❧❡ss✲✇♦r❧❞ 2 ❘❛♥❞♦♠ ❝♦❞✐♥❣ ❝❛♥ ❜❡ ✉s❡❞ t♦ ❛❝❤✐❡✈❡ r❛t❡ ♣❛✐rs ( R 1 , R 2 ) t❤❛t s❛t✐s❢② R 1 ≤ I ( X 1 ; Y | X 2 , Q ) R 2 ≤ I ( X 2 ; Y | X 1 , Q ) R 1 + R 2 ≤ I ( X 1 , X 2 ; Y | Q ) ❢♦r s♦♠❡ p ( q ) p ( x 1 | q ) p ( x 2 | q ) ❀ ✐t s✉✣❝❡s t♦ ❝♦♥s✐❞❡r |Q| ≤ 2 ✳ ❈❛❧❧ t❤✐s r❡❣✐♦♥ R ( W ) ✳ ❆❤❧s✇❡❞❡ ◗✉❡st✐♦♥ ✿ ■s t❤✐s t❤❡ ❝❛♣❛❝✐t② ✭♦♣t✐♠❛❧✮ r❡❣✐♦♥❄ ✭❨❊❙✮ ✭❆❤❧s✇❡❞❡ ✬✼✷✮ ✻

  9. ❚❤❡ ❛❜♦✈❡ ❡q✉❛❧✐t② ✭❛❞❞✐t✐✈✐t②✮ ❢♦❧❧♦✇s ✐❢ t❤❡ ❢♦❧❧♦✇✐♥❣ s✉❜✲❛❞❞✐t✐✈✐t② ❤♦❧❞s✿ ❚❡st✐♥❣ ♦♣t✐♠❛❧✐t② ❉❡✜♥❡✱ ❢♦r λ ≥ 1 ✱ � S λ ( W ) = max λR 1 + R 2 } ( R 1 ,R 2 ) ∈R ( W ) � � ( λ − 1) I ( X 1 ; Y | X 2 ) + I ( X 1 , X 2 ; Y ) = max p 1 ( x 1 ) p 2 ( x 2 ) ✼

  10. ❚❤❡ ❛❜♦✈❡ ❡q✉❛❧✐t② ✭❛❞❞✐t✐✈✐t②✮ ❢♦❧❧♦✇s ✐❢ t❤❡ ❢♦❧❧♦✇✐♥❣ s✉❜✲❛❞❞✐t✐✈✐t② ❤♦❧❞s✿ ❚❡st✐♥❣ ♦♣t✐♠❛❧✐t② ❉❡✜♥❡✱ ❢♦r λ ≥ 1 ✱ � S λ ( W ) = max λR 1 + R 2 } ( R 1 ,R 2 ) ∈R ( W ) � � ( λ − 1) I ( X 1 ; Y | X 2 ) + I ( X 1 , X 2 ; Y ) = max p 1 ( x 1 ) p 2 ( x 2 ) 3 R 1 + R 2 R 1 2 R 1 + R 2 R 1 + R 2 R 2 ❙✉♣♣♦rt✐♥❣ ❤②♣❡r♣❧❛♥❡s ✼

  11. ❚❡st✐♥❣ ♦♣t✐♠❛❧✐t② ❉❡✜♥❡✱ ❢♦r λ ≥ 1 ✱ � S λ ( W ) = max λR 1 + R 2 } ( R 1 ,R 2 ) ∈R ( W ) � � ( λ − 1) I ( X 1 ; Y | X 2 ) + I ( X 1 , X 2 ; Y ) = max p 1 ( x 1 ) p 2 ( x 2 ) ❆s ❜❡❢♦r❡✱ R ( W ) ✐s ♦♣t✐♠❛❧ ✐❢ ❛♥❞ ♦♥❧② ✐❢ S λ ( W ⊗ W ) = 2 S λ ( W ) ∀ W, λ ≥ 1 . ❚❤❡ ❛❜♦✈❡ ❡q✉❛❧✐t② ✭❛❞❞✐t✐✈✐t②✮ ❢♦❧❧♦✇s ✐❢ t❤❡ ❢♦❧❧♦✇✐♥❣ s✉❜✲❛❞❞✐t✐✈✐t② ❤♦❧❞s✿ ( λ − 1) I ( X 11 , X 12 ; Y 1 , Y 2 | X 21 , X 22 ) + I ( X 11 , X 12 , X 21 , X 22 ; Y 1 , Y 2 ) ≤ ( λ − 1) I ( X 11 ; Y 1 | X 21 ) + I ( X 11 , X 21 ; Y 1 ) + ( λ − 1) I ( X 12 ; Y 2 | X 22 ) + I ( X 12 , X 22 ; Y 2 ) ✼

  12. ❚❡st✐♥❣ ♦♣t✐♠❛❧✐t② ❉❡✜♥❡✱ ❢♦r λ ≥ 1 ✱ � S λ ( W ) = max λR 1 + R 2 } ( R 1 ,R 2 ) ∈R ( W ) � � ( λ − 1) I ( X 1 ; Y | X 2 ) + I ( X 1 , X 2 ; Y ) = max p 1 ( x 1 ) p 2 ( x 2 ) ❆s ❜❡❢♦r❡✱ R ( W ) ✐s ♦♣t✐♠❛❧ ✐❢ ❛♥❞ ♦♥❧② ✐❢ S λ ( W ⊗ W ) = 2 S λ ( W ) ∀ W, λ ≥ 1 . ❚❤❡ ❛❜♦✈❡ ❡q✉❛❧✐t② ✭❛❞❞✐t✐✈✐t②✮ ❢♦❧❧♦✇s ✐❢ t❤❡ ❢♦❧❧♦✇✐♥❣ s✉❜✲❛❞❞✐t✐✈✐t② ❤♦❧❞s✿ ( λ − 1) I ( X 11 , X 12 ; Y 1 , Y 2 | X 21 , X 22 ) + I ( X 11 , X 12 , X 21 , X 22 ; Y 1 , Y 2 ) ≤ ( λ − 1) I ( X 11 ; Y 1 | X 21 ) + I ( X 11 , X 21 ; Y 1 ) + ( λ − 1) I ( X 12 ; Y 2 | X 22 ) + I ( X 12 , X 22 ; Y 2 ) ❖♥❡ ❝❛♥ ❡st❛❜❧✐s❤ t❤✐s ✐♥ s❛♠❡ ✇❛② ❛s ♣♦✐♥t✲t♦✲♣♦✐♥t s❡tt✐♥❣✳ ✼

  13. ❚❡st✐♥❣ ♦♣t✐♠❛❧✐t② ❉❡✜♥❡✱ ❢♦r λ ≥ 1 ✱ � S λ ( W ) = max λR 1 + R 2 } ( R 1 ,R 2 ) ∈R ( W ) � � ( λ − 1) I ( X 1 ; Y | X 2 ) + I ( X 1 , X 2 ; Y ) = max p 1 ( x 1 ) p 2 ( x 2 ) ❆s ❜❡❢♦r❡✱ R ( W ) ✐s ♦♣t✐♠❛❧ ✐❢ ❛♥❞ ♦♥❧② ✐❢ S λ ( W ⊗ W ) = 2 S λ ( W ) ∀ W, λ ≥ 1 . ❚❤❡ ❛❜♦✈❡ ❡q✉❛❧✐t② ✭❛❞❞✐t✐✈✐t②✮ ❢♦❧❧♦✇s ✐❢ t❤❡ ❢♦❧❧♦✇✐♥❣ s✉❜✲❛❞❞✐t✐✈✐t② ❤♦❧❞s✿ ( λ − 1) I ( X 11 , X 12 ; Y 1 , Y 2 | X 21 , X 22 ) + I ( X 11 , X 12 , X 21 , X 22 ; Y 1 , Y 2 ) ≤ ( λ − 1) I ( X 11 ; Y 1 | X 21 ) + I ( X 11 , X 21 ; Y 1 ) + ( λ − 1) I ( X 12 ; Y 2 | X 22 ) + I ( X 12 , X 22 ; Y 2 ) ◆♦t❡ ✿ ❈♦♠♣✉t✐♥❣ S λ ( W ) ✐s r❡❧❛t✐✈❡❧② ❡❛s② s✐♥❝❡ � � ( λ − 1) I ( X 1 ; Y | X 2 ) + I ( X 1 , X 2 ; Y ) ✐s ❝♦♥❝❛✈❡ ✐♥ p 1 ( x 1 ) , p 2 ( x 2 ) ✳ ✼

  14. ❙✉♣❡r♣♦s✐t✐♦♥ ❝♦❞✐♥❣ ❛♥❞ r❛♥❞♦♠ ❤❛s❤✐♥❣ ❝❛♥ ❜❡ ✉s❡❞ t♦ ❛❝❤✐❡✈❡ r❛t❡ tr✐♣❧❡s t❤❛t s❛t✐s❢② ▼❛rt♦♥ ❢♦r s♦♠❡ ✳ ❈❛❧❧ t❤✐s r❡❣✐♦♥ ✳ ◗✉❡st✐♦♥ ✿ ■s t❤✐s t❤❡ ❝❛♣❛❝✐t② ✭♦♣t✐♠❛❧✮ r❡❣✐♦♥❄ ✭❖♣❡♥✮ ✭s✐♥❝❡ ▼❛rt♦♥ ✬✼✾✮ ✷✳ ❇r♦❛❞❝❛st ❝❤❛♥♥❡❧ ✭❞♦✇♥❧✐♥❦✮ ✭❈♦✈❡r ✬✼✷✮ Y n 1 M 0 , ˆ ˆ W a ( y 1 | x ) ❉❡❝♦❞❡r ✶ M 1 X n ( M 0 , M 1 , M 2 ) ❊♥❝♦❞❡r Y n 2 M 0 , ˜ ˜ W b ( y 2 | x ) ❉❡❝♦❞❡r ✷ M 2 r❢✇✐r❡❧❡ss✲✇♦r❧❞ ✽

  15. ◗✉❡st✐♦♥ ✿ ■s t❤✐s t❤❡ ❝❛♣❛❝✐t② ✭♦♣t✐♠❛❧✮ r❡❣✐♦♥❄ ✭❖♣❡♥✮ ✭s✐♥❝❡ ▼❛rt♦♥ ✬✼✾✮ ✷✳ ❇r♦❛❞❝❛st ❝❤❛♥♥❡❧ ✭❞♦✇♥❧✐♥❦✮ ✭❈♦✈❡r ✬✼✷✮ Y n 1 M 0 , ˆ ˆ W a ( y 1 | x ) ❉❡❝♦❞❡r ✶ M 1 X n ( M 0 , M 1 , M 2 ) ❊♥❝♦❞❡r Y n 2 M 0 , ˜ ˜ W b ( y 2 | x ) ❉❡❝♦❞❡r ✷ M 2 r❢✇✐r❡❧❡ss✲✇♦r❧❞ ❙✉♣❡r♣♦s✐t✐♦♥ ❝♦❞✐♥❣ ❛♥❞ r❛♥❞♦♠ ❤❛s❤✐♥❣ ❝❛♥ ❜❡ ✉s❡❞ t♦ ❛❝❤✐❡✈❡ r❛t❡ tr✐♣❧❡s ( R 0 , R 1 , R 2 ) t❤❛t s❛t✐s❢② R 0 ≤ min { I ( Q ; Y 1 ) , I ( Q ; Y 2 ) } R 0 + R 1 ≤ I ( U, Q ; Y 1 ) R 0 + R 2 ≤ I ( V, Q ; Y 2 ) R 0 + R 1 + R 2 ≤ min { I ( Q ; Y 1 ) , I ( Q ; Y 2 ) } + I ( U ; Y 1 | Q ) + I ( V ; Y 2 | Q ) − I ( U ; V | Q ) ▼❛rt♦♥ ❢♦r s♦♠❡ p ( q, u, v, x ) ✳ ❈❛❧❧ t❤✐s r❡❣✐♦♥ R ( W a , W b ) ✳ ✽

  16. ✭❖♣❡♥✮ ✭s✐♥❝❡ ▼❛rt♦♥ ✬✼✾✮ ✷✳ ❇r♦❛❞❝❛st ❝❤❛♥♥❡❧ ✭❞♦✇♥❧✐♥❦✮ ✭❈♦✈❡r ✬✼✷✮ Y n 1 M 0 , ˆ ˆ W a ( y 1 | x ) ❉❡❝♦❞❡r ✶ M 1 X n ( M 0 , M 1 , M 2 ) ❊♥❝♦❞❡r Y n 2 M 0 , ˜ ˜ W b ( y 2 | x ) ❉❡❝♦❞❡r ✷ M 2 r❢✇✐r❡❧❡ss✲✇♦r❧❞ ❙✉♣❡r♣♦s✐t✐♦♥ ❝♦❞✐♥❣ ❛♥❞ r❛♥❞♦♠ ❤❛s❤✐♥❣ ❝❛♥ ❜❡ ✉s❡❞ t♦ ❛❝❤✐❡✈❡ r❛t❡ tr✐♣❧❡s ( R 0 , R 1 , R 2 ) t❤❛t s❛t✐s❢② R 0 ≤ min { I ( Q ; Y 1 ) , I ( Q ; Y 2 ) } R 0 + R 1 ≤ I ( U, Q ; Y 1 ) R 0 + R 2 ≤ I ( V, Q ; Y 2 ) R 0 + R 1 + R 2 ≤ min { I ( Q ; Y 1 ) , I ( Q ; Y 2 ) } + I ( U ; Y 1 | Q ) + I ( V ; Y 2 | Q ) − I ( U ; V | Q ) ▼❛rt♦♥ ❢♦r s♦♠❡ p ( q, u, v, x ) ✳ ❈❛❧❧ t❤✐s r❡❣✐♦♥ R ( W a , W b ) ✳ ◗✉❡st✐♦♥ ✿ ■s t❤✐s t❤❡ ❝❛♣❛❝✐t② ✭♦♣t✐♠❛❧✮ r❡❣✐♦♥❄ ✽

  17. ✷✳ ❇r♦❛❞❝❛st ❝❤❛♥♥❡❧ ✭❞♦✇♥❧✐♥❦✮ ✭❈♦✈❡r ✬✼✷✮ Y n 1 M 0 , ˆ ˆ W a ( y 1 | x ) ❉❡❝♦❞❡r ✶ M 1 X n ( M 0 , M 1 , M 2 ) ❊♥❝♦❞❡r Y n 2 M 0 , ˜ ˜ W b ( y 2 | x ) ❉❡❝♦❞❡r ✷ M 2 r❢✇✐r❡❧❡ss✲✇♦r❧❞ ❙✉♣❡r♣♦s✐t✐♦♥ ❝♦❞✐♥❣ ❛♥❞ r❛♥❞♦♠ ❤❛s❤✐♥❣ ❝❛♥ ❜❡ ✉s❡❞ t♦ ❛❝❤✐❡✈❡ r❛t❡ tr✐♣❧❡s ( R 0 , R 1 , R 2 ) t❤❛t s❛t✐s❢② R 0 ≤ min { I ( Q ; Y 1 ) , I ( Q ; Y 2 ) } R 0 + R 1 ≤ I ( U, Q ; Y 1 ) R 0 + R 2 ≤ I ( V, Q ; Y 2 ) R 0 + R 1 + R 2 ≤ min { I ( Q ; Y 1 ) , I ( Q ; Y 2 ) } + I ( U ; Y 1 | Q ) + I ( V ; Y 2 | Q ) − I ( U ; V | Q ) ▼❛rt♦♥ ❢♦r s♦♠❡ p ( q, u, v, x ) ✳ ❈❛❧❧ t❤✐s r❡❣✐♦♥ R ( W a , W b ) ✳ ◗✉❡st✐♦♥ ✿ ■s t❤✐s t❤❡ ❝❛♣❛❝✐t② ✭♦♣t✐♠❛❧✮ r❡❣✐♦♥❄ ✭❖♣❡♥✮ ✭s✐♥❝❡ ▼❛rt♦♥ ✬✼✾✮ ✽

  18. ◆♦t❡ ✿ ❈♦♠♣✉t✐♥❣ ✐s ❛ ♥♦♥✲❝♦♥✈❡① ♦♣t✐♠✐③❛t✐♦♥ ♣r♦❜❧❡♠✳ ❚❡st✐♥❣ ♦♣t✐♠❛❧✐t② ✭ R 0 = 0 ✮ ❉❡✜♥❡✱ ❢♦r λ ≥ 1 ✱ S λ ( W ) = ( R 1 ,R 2 ) ∈R ( W a ,W b ) { λR 1 + R 2 } max � = max ( λ − 1) I ( U, Q ; Y 1 ) + min { I ( Q ; Y 1 ) , I ( Q ; Y 2 ) } + I ( U ; Y 1 | Q ) p ( u,v,w,x ) � + I ( V ; Y 2 | Q ) − I ( U ; V | Q ) � = min max ( λ − α ) I ( Q ; Y 1 ) + αI ( Q ; Y 2 ) + λI ( U ; Y 1 | Q ) α ∈ [0 , 1] p ( u,v,w,x ) � + I ( V ; Y 2 | Q ) − I ( U ; V | Q ) ❆s ❜❡❢♦r❡✱ R ( W a , W b ) ✐s ♦♣t✐♠❛❧ ✐❢ ❛♥❞ ♦♥❧② ✐❢ S λ ( W a ⊗ W a , W b ⊗ W b ) = 2 S λ ( W a , W b ) ∀ W a , W b , λ ≥ 1 . ✾

  19. ❚❡st✐♥❣ ♦♣t✐♠❛❧✐t② ✭ R 0 = 0 ✮ ❉❡✜♥❡✱ ❢♦r λ ≥ 1 ✱ S λ ( W ) = ( R 1 ,R 2 ) ∈R ( W a ,W b ) { λR 1 + R 2 } max � = max ( λ − 1) I ( U, Q ; Y 1 ) + min { I ( Q ; Y 1 ) , I ( Q ; Y 2 ) } + I ( U ; Y 1 | Q ) p ( u,v,w,x ) � + I ( V ; Y 2 | Q ) − I ( U ; V | Q ) � = min max ( λ − α ) I ( Q ; Y 1 ) + αI ( Q ; Y 2 ) + λI ( U ; Y 1 | Q ) α ∈ [0 , 1] p ( u,v,w,x ) � + I ( V ; Y 2 | Q ) − I ( U ; V | Q ) ❆s ❜❡❢♦r❡✱ R ( W a , W b ) ✐s ♦♣t✐♠❛❧ ✐❢ ❛♥❞ ♦♥❧② ✐❢ S λ ( W a ⊗ W a , W b ⊗ W b ) = 2 S λ ( W a , W b ) ∀ W a , W b , λ ≥ 1 . ◆♦t❡ ✿ ❈♦♠♣✉t✐♥❣ S λ ( W a , W b ) ✐s ❛ ♥♦♥✲❝♦♥✈❡① ♦♣t✐♠✐③❛t✐♦♥ ♣r♦❜❧❡♠✳ ✾

  20. ❈♦✈❡r ✬✼✷✿ ❞❡✈❡❧♦♣♠❡♥t ♦❢ s✉♣❡r♣♦s✐t✐♦♥ ❝♦❞✐♥❣ str❛t❡❣② ●❛❧❧❛❣❡r ✬✼✹✿ ❝♦♥✈❡rs❡ t♦ t❤❡ ❞❡❣r❛❞❡❞ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧ ✭ s✉❜✲❛❞❞✐t✐✈✐t② ✮ ❲❡✐♥❣❛rt❡♥✲❙t❡✐♥❜❡r❣✲❙❤❛♠❛✐ ✬✵✻✿ ❖♣t✐♠❛❧✐t② ♦❢ ✭♦♥ ✮ ❢♦r ●❛✉ss✐❛♥ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧❀ ❞❡✈❡❧♦♣✐♥❣ ❛ ❢❛♠✐❧② ♦❢ t✐❣❤t ❝♦♥✈❡① r❡❧❛①❛t✐♦♥s t♦ ❝♦♠♣✉t❡ t❤❡ ♦♣t✐♠❛❧ ✈❛❧✉❡ ♦❢ ❛ ♥♦♥✲❝♦♥✈❡① ♦♣t✐♠✐③❛t✐♦♥ ♣r♦❜❧❡♠ ●❡♥❣✲◆❛✐r ✬✶✹✿ ❖♣t✐♠❛❧✐t② ♦❢ ❢♦r ●❛✉ss✐❛♥ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧✿ ❚❡❝❤♥✐q✉❡ ❢♦r ❡st❛❜❧✐s❤✐♥❣ ❡①tr❡♠❛❧✐t② ♦❢ ●❛✉ss✐❛♥ ❞✐str✐❜✉t✐♦♥s ✉s✐♥❣ s✉❜✲❛❞❞✐t✐✈✐t② ♦❢ ❢✉♥❝t✐♦♥❛❧s ❙✉❝❝❡ss❡s ■♥ s♣✐t❡ ♦❢ t❤❡ ✉♥❞❡r❧②✐♥❣ ♣r♦❜❧❡♠ ❜❡✐♥❣ ✐♥tr✐♥s✐❝❛❧❧② ♥♦♥✲❝♦♥✈❡① • R ( W a , W b ) ✐s ♦♣t✐♠❛❧ ♦♥ R 1 = 0 ✭♦r R 2 = 0 ✮ ⋆ ❉❡❣r❛❞❡❞ ♠❡ss❛❣❡ s❡ts✿ ❑♦r♥❡r ❛♥❞ ▼❛rt♦♥ ✭✬✼✼✮ • R ( W a , W b ) ✐s ♦♣t✐♠❛❧ ❢♦r s♦♠❡ ❝❧❛ss❡s ♦❢ ❝❤❛♥♥❡❧s ⋆ ●❛❧❧❛❣❡r ✬✼✹✱ ❑♦r♥❡r ❛♥❞ ▼❛rt♦♥ ✭✬✼✺✱ ✬✼✼✱ ✬✼✾✮✱ ●❡❧❢❛♥❞ ❛♥❞ P✐♥s❦❡r ✭✬✼✽✮✱ P♦❧t②r❡✈ ✭✬✼✽✮✱ ❊❧ ●❛♠❛❧ ✭✬✼✾✱ ✬✽✵✮ ⋆ ❲❡✐♥❣❛rt❡♥ ❛♥❞ ❙t❡✐♥❜❡r❣ ❛♥❞ ❙❤❛♠❛✐ ✬✵✻✱ ◆❛✐r ✬✶✵✱ ●❡♥❣ ❛♥❞ ●♦❤❛r✐ ❛♥❞ ◆❛✐r ❛♥❞ ❨✉ ✬✶✹✱ ●❡♥❣ ❛♥❞ ◆❛✐r ✬✶✹ • ◆♦✈❡❧ ✐❞❡❛s ❛♥❞ t❡❝❤♥✐q✉❡s ✇❡r❡ ♥❡❡❞❡❞ t♦ ❡st❛❜❧✐s❤ t❤❡s❡ ❝❛♣❛❝✐t② r❡❣✐♦♥s ✶✵

  21. ●❛❧❧❛❣❡r ✬✼✹✿ ❝♦♥✈❡rs❡ t♦ t❤❡ ❞❡❣r❛❞❡❞ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧ ✭ s✉❜✲❛❞❞✐t✐✈✐t② ✮ ❲❡✐♥❣❛rt❡♥✲❙t❡✐♥❜❡r❣✲❙❤❛♠❛✐ ✬✵✻✿ ❖♣t✐♠❛❧✐t② ♦❢ ✭♦♥ ✮ ❢♦r ●❛✉ss✐❛♥ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧❀ ❞❡✈❡❧♦♣✐♥❣ ❛ ❢❛♠✐❧② ♦❢ t✐❣❤t ❝♦♥✈❡① r❡❧❛①❛t✐♦♥s t♦ ❝♦♠♣✉t❡ t❤❡ ♦♣t✐♠❛❧ ✈❛❧✉❡ ♦❢ ❛ ♥♦♥✲❝♦♥✈❡① ♦♣t✐♠✐③❛t✐♦♥ ♣r♦❜❧❡♠ ●❡♥❣✲◆❛✐r ✬✶✹✿ ❖♣t✐♠❛❧✐t② ♦❢ ❢♦r ●❛✉ss✐❛♥ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧✿ ❚❡❝❤♥✐q✉❡ ❢♦r ❡st❛❜❧✐s❤✐♥❣ ❡①tr❡♠❛❧✐t② ♦❢ ●❛✉ss✐❛♥ ❞✐str✐❜✉t✐♦♥s ✉s✐♥❣ s✉❜✲❛❞❞✐t✐✈✐t② ♦❢ ❢✉♥❝t✐♦♥❛❧s ❙✉❝❝❡ss❡s ■♥ s♣✐t❡ ♦❢ t❤❡ ✉♥❞❡r❧②✐♥❣ ♣r♦❜❧❡♠ ❜❡✐♥❣ ✐♥tr✐♥s✐❝❛❧❧② ♥♦♥✲❝♦♥✈❡① • R ( W a , W b ) ✐s ♦♣t✐♠❛❧ ♦♥ R 1 = 0 ✭♦r R 2 = 0 ✮ ⋆ ❉❡❣r❛❞❡❞ ♠❡ss❛❣❡ s❡ts✿ ❑♦r♥❡r ❛♥❞ ▼❛rt♦♥ ✭✬✼✼✮ • R ( W a , W b ) ✐s ♦♣t✐♠❛❧ ❢♦r s♦♠❡ ❝❧❛ss❡s ♦❢ ❝❤❛♥♥❡❧s ⋆ ●❛❧❧❛❣❡r ✬✼✹✱ ❑♦r♥❡r ❛♥❞ ▼❛rt♦♥ ✭✬✼✺✱ ✬✼✼✱ ✬✼✾✮✱ ●❡❧❢❛♥❞ ❛♥❞ P✐♥s❦❡r ✭✬✼✽✮✱ P♦❧t②r❡✈ ✭✬✼✽✮✱ ❊❧ ●❛♠❛❧ ✭✬✼✾✱ ✬✽✵✮ ⋆ ❲❡✐♥❣❛rt❡♥ ❛♥❞ ❙t❡✐♥❜❡r❣ ❛♥❞ ❙❤❛♠❛✐ ✬✵✻✱ ◆❛✐r ✬✶✵✱ ●❡♥❣ ❛♥❞ ●♦❤❛r✐ ❛♥❞ ◆❛✐r ❛♥❞ ❨✉ ✬✶✹✱ ●❡♥❣ ❛♥❞ ◆❛✐r ✬✶✹ • ◆♦✈❡❧ ✐❞❡❛s ❛♥❞ t❡❝❤♥✐q✉❡s ✇❡r❡ ♥❡❡❞❡❞ t♦ ❡st❛❜❧✐s❤ t❤❡s❡ ❝❛♣❛❝✐t② r❡❣✐♦♥s ⋆ ❈♦✈❡r ✬✼✷✿ ❞❡✈❡❧♦♣♠❡♥t ♦❢ s✉♣❡r♣♦s✐t✐♦♥ ❝♦❞✐♥❣ str❛t❡❣② ✶✵

  22. ❲❡✐♥❣❛rt❡♥✲❙t❡✐♥❜❡r❣✲❙❤❛♠❛✐ ✬✵✻✿ ❖♣t✐♠❛❧✐t② ♦❢ ✭♦♥ ✮ ❢♦r ●❛✉ss✐❛♥ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧❀ ❞❡✈❡❧♦♣✐♥❣ ❛ ❢❛♠✐❧② ♦❢ t✐❣❤t ❝♦♥✈❡① r❡❧❛①❛t✐♦♥s t♦ ❝♦♠♣✉t❡ t❤❡ ♦♣t✐♠❛❧ ✈❛❧✉❡ ♦❢ ❛ ♥♦♥✲❝♦♥✈❡① ♦♣t✐♠✐③❛t✐♦♥ ♣r♦❜❧❡♠ ●❡♥❣✲◆❛✐r ✬✶✹✿ ❖♣t✐♠❛❧✐t② ♦❢ ❢♦r ●❛✉ss✐❛♥ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧✿ ❚❡❝❤♥✐q✉❡ ❢♦r ❡st❛❜❧✐s❤✐♥❣ ❡①tr❡♠❛❧✐t② ♦❢ ●❛✉ss✐❛♥ ❞✐str✐❜✉t✐♦♥s ✉s✐♥❣ s✉❜✲❛❞❞✐t✐✈✐t② ♦❢ ❢✉♥❝t✐♦♥❛❧s ❙✉❝❝❡ss❡s ■♥ s♣✐t❡ ♦❢ t❤❡ ✉♥❞❡r❧②✐♥❣ ♣r♦❜❧❡♠ ❜❡✐♥❣ ✐♥tr✐♥s✐❝❛❧❧② ♥♦♥✲❝♦♥✈❡① • R ( W a , W b ) ✐s ♦♣t✐♠❛❧ ♦♥ R 1 = 0 ✭♦r R 2 = 0 ✮ ⋆ ❉❡❣r❛❞❡❞ ♠❡ss❛❣❡ s❡ts✿ ❑♦r♥❡r ❛♥❞ ▼❛rt♦♥ ✭✬✼✼✮ • R ( W a , W b ) ✐s ♦♣t✐♠❛❧ ❢♦r s♦♠❡ ❝❧❛ss❡s ♦❢ ❝❤❛♥♥❡❧s ⋆ ●❛❧❧❛❣❡r ✬✼✹✱ ❑♦r♥❡r ❛♥❞ ▼❛rt♦♥ ✭✬✼✺✱ ✬✼✼✱ ✬✼✾✮✱ ●❡❧❢❛♥❞ ❛♥❞ P✐♥s❦❡r ✭✬✼✽✮✱ P♦❧t②r❡✈ ✭✬✼✽✮✱ ❊❧ ●❛♠❛❧ ✭✬✼✾✱ ✬✽✵✮ ⋆ ❲❡✐♥❣❛rt❡♥ ❛♥❞ ❙t❡✐♥❜❡r❣ ❛♥❞ ❙❤❛♠❛✐ ✬✵✻✱ ◆❛✐r ✬✶✵✱ ●❡♥❣ ❛♥❞ ●♦❤❛r✐ ❛♥❞ ◆❛✐r ❛♥❞ ❨✉ ✬✶✹✱ ●❡♥❣ ❛♥❞ ◆❛✐r ✬✶✹ • ◆♦✈❡❧ ✐❞❡❛s ❛♥❞ t❡❝❤♥✐q✉❡s ✇❡r❡ ♥❡❡❞❡❞ t♦ ❡st❛❜❧✐s❤ t❤❡s❡ ❝❛♣❛❝✐t② r❡❣✐♦♥s ⋆ ❈♦✈❡r ✬✼✷✿ ❞❡✈❡❧♦♣♠❡♥t ♦❢ s✉♣❡r♣♦s✐t✐♦♥ ❝♦❞✐♥❣ str❛t❡❣② ⋆ ●❛❧❧❛❣❡r ✬✼✹✿ ❝♦♥✈❡rs❡ t♦ t❤❡ ❞❡❣r❛❞❡❞ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧ ✭ s✉❜✲❛❞❞✐t✐✈✐t② ✮ ✶✵

  23. ●❡♥❣✲◆❛✐r ✬✶✹✿ ❖♣t✐♠❛❧✐t② ♦❢ ❢♦r ●❛✉ss✐❛♥ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧✿ ❚❡❝❤♥✐q✉❡ ❢♦r ❡st❛❜❧✐s❤✐♥❣ ❡①tr❡♠❛❧✐t② ♦❢ ●❛✉ss✐❛♥ ❞✐str✐❜✉t✐♦♥s ✉s✐♥❣ s✉❜✲❛❞❞✐t✐✈✐t② ♦❢ ❢✉♥❝t✐♦♥❛❧s ❙✉❝❝❡ss❡s ■♥ s♣✐t❡ ♦❢ t❤❡ ✉♥❞❡r❧②✐♥❣ ♣r♦❜❧❡♠ ❜❡✐♥❣ ✐♥tr✐♥s✐❝❛❧❧② ♥♦♥✲❝♦♥✈❡① • R ( W a , W b ) ✐s ♦♣t✐♠❛❧ ♦♥ R 1 = 0 ✭♦r R 2 = 0 ✮ ⋆ ❉❡❣r❛❞❡❞ ♠❡ss❛❣❡ s❡ts✿ ❑♦r♥❡r ❛♥❞ ▼❛rt♦♥ ✭✬✼✼✮ • R ( W a , W b ) ✐s ♦♣t✐♠❛❧ ❢♦r s♦♠❡ ❝❧❛ss❡s ♦❢ ❝❤❛♥♥❡❧s ⋆ ●❛❧❧❛❣❡r ✬✼✹✱ ❑♦r♥❡r ❛♥❞ ▼❛rt♦♥ ✭✬✼✺✱ ✬✼✼✱ ✬✼✾✮✱ ●❡❧❢❛♥❞ ❛♥❞ P✐♥s❦❡r ✭✬✼✽✮✱ P♦❧t②r❡✈ ✭✬✼✽✮✱ ❊❧ ●❛♠❛❧ ✭✬✼✾✱ ✬✽✵✮ ⋆ ❲❡✐♥❣❛rt❡♥ ❛♥❞ ❙t❡✐♥❜❡r❣ ❛♥❞ ❙❤❛♠❛✐ ✬✵✻✱ ◆❛✐r ✬✶✵✱ ●❡♥❣ ❛♥❞ ●♦❤❛r✐ ❛♥❞ ◆❛✐r ❛♥❞ ❨✉ ✬✶✹✱ ●❡♥❣ ❛♥❞ ◆❛✐r ✬✶✹ • ◆♦✈❡❧ ✐❞❡❛s ❛♥❞ t❡❝❤♥✐q✉❡s ✇❡r❡ ♥❡❡❞❡❞ t♦ ❡st❛❜❧✐s❤ t❤❡s❡ ❝❛♣❛❝✐t② r❡❣✐♦♥s ⋆ ❈♦✈❡r ✬✼✷✿ ❞❡✈❡❧♦♣♠❡♥t ♦❢ s✉♣❡r♣♦s✐t✐♦♥ ❝♦❞✐♥❣ str❛t❡❣② ⋆ ●❛❧❧❛❣❡r ✬✼✹✿ ❝♦♥✈❡rs❡ t♦ t❤❡ ❞❡❣r❛❞❡❞ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧ ✭ s✉❜✲❛❞❞✐t✐✈✐t② ✮ ⋆ ❲❡✐♥❣❛rt❡♥✲❙t❡✐♥❜❡r❣✲❙❤❛♠❛✐ ✬✵✻✿ ❖♣t✐♠❛❧✐t② ♦❢ R ( W a , W b ) ✭♦♥ R 0 = 0 ✮ ❢♦r ●❛✉ss✐❛♥ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧❀ ❞❡✈❡❧♦♣✐♥❣ ❛ ❢❛♠✐❧② ♦❢ t✐❣❤t ❝♦♥✈❡① r❡❧❛①❛t✐♦♥s t♦ ❝♦♠♣✉t❡ t❤❡ ♦♣t✐♠❛❧ ✈❛❧✉❡ ♦❢ ❛ ♥♦♥✲❝♦♥✈❡① ♦♣t✐♠✐③❛t✐♦♥ ♣r♦❜❧❡♠ ✶✵

  24. ❙✉❝❝❡ss❡s ■♥ s♣✐t❡ ♦❢ t❤❡ ✉♥❞❡r❧②✐♥❣ ♣r♦❜❧❡♠ ❜❡✐♥❣ ✐♥tr✐♥s✐❝❛❧❧② ♥♦♥✲❝♦♥✈❡① • R ( W a , W b ) ✐s ♦♣t✐♠❛❧ ♦♥ R 1 = 0 ✭♦r R 2 = 0 ✮ ⋆ ❉❡❣r❛❞❡❞ ♠❡ss❛❣❡ s❡ts✿ ❑♦r♥❡r ❛♥❞ ▼❛rt♦♥ ✭✬✼✼✮ • R ( W a , W b ) ✐s ♦♣t✐♠❛❧ ❢♦r s♦♠❡ ❝❧❛ss❡s ♦❢ ❝❤❛♥♥❡❧s ⋆ ●❛❧❧❛❣❡r ✬✼✹✱ ❑♦r♥❡r ❛♥❞ ▼❛rt♦♥ ✭✬✼✺✱ ✬✼✼✱ ✬✼✾✮✱ ●❡❧❢❛♥❞ ❛♥❞ P✐♥s❦❡r ✭✬✼✽✮✱ P♦❧t②r❡✈ ✭✬✼✽✮✱ ❊❧ ●❛♠❛❧ ✭✬✼✾✱ ✬✽✵✮ ⋆ ❲❡✐♥❣❛rt❡♥ ❛♥❞ ❙t❡✐♥❜❡r❣ ❛♥❞ ❙❤❛♠❛✐ ✬✵✻✱ ◆❛✐r ✬✶✵✱ ●❡♥❣ ❛♥❞ ●♦❤❛r✐ ❛♥❞ ◆❛✐r ❛♥❞ ❨✉ ✬✶✹✱ ●❡♥❣ ❛♥❞ ◆❛✐r ✬✶✹ • ◆♦✈❡❧ ✐❞❡❛s ❛♥❞ t❡❝❤♥✐q✉❡s ✇❡r❡ ♥❡❡❞❡❞ t♦ ❡st❛❜❧✐s❤ t❤❡s❡ ❝❛♣❛❝✐t② r❡❣✐♦♥s ⋆ ❈♦✈❡r ✬✼✷✿ ❞❡✈❡❧♦♣♠❡♥t ♦❢ s✉♣❡r♣♦s✐t✐♦♥ ❝♦❞✐♥❣ str❛t❡❣② ⋆ ●❛❧❧❛❣❡r ✬✼✹✿ ❝♦♥✈❡rs❡ t♦ t❤❡ ❞❡❣r❛❞❡❞ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧ ✭ s✉❜✲❛❞❞✐t✐✈✐t② ✮ ⋆ ❲❡✐♥❣❛rt❡♥✲❙t❡✐♥❜❡r❣✲❙❤❛♠❛✐ ✬✵✻✿ ❖♣t✐♠❛❧✐t② ♦❢ R ( W a , W b ) ✭♦♥ R 0 = 0 ✮ ❢♦r ●❛✉ss✐❛♥ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧❀ ❞❡✈❡❧♦♣✐♥❣ ❛ ❢❛♠✐❧② ♦❢ t✐❣❤t ❝♦♥✈❡① r❡❧❛①❛t✐♦♥s t♦ ❝♦♠♣✉t❡ t❤❡ ♦♣t✐♠❛❧ ✈❛❧✉❡ ♦❢ ❛ ♥♦♥✲❝♦♥✈❡① ♦♣t✐♠✐③❛t✐♦♥ ♣r♦❜❧❡♠ ⋆ ●❡♥❣✲◆❛✐r ✬✶✹✿ ❖♣t✐♠❛❧✐t② ♦❢ R ( W a , W b ) ❢♦r ●❛✉ss✐❛♥ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧✿ ❚❡❝❤♥✐q✉❡ ❢♦r ❡st❛❜❧✐s❤✐♥❣ ❡①tr❡♠❛❧✐t② ♦❢ ●❛✉ss✐❛♥ ❞✐str✐❜✉t✐♦♥s ✉s✐♥❣ s✉❜✲❛❞❞✐t✐✈✐t② ♦❢ ❢✉♥❝t✐♦♥❛❧s ✶✵

  25. ✸✳ ■♥t❡r❢❡r❡♥❝❡ ❈❤❛♥♥❡❧ ✭❆❤❧s✇❡❞❡ ✬✼✹✮ ❈r❡❞✐t✿ ✇✇✇✳♣❡rs♦♥❛❧✳♣s✉✳❡❞✉✴❜①❣✷✶✺✴r❡s❡❛r❝❤✳❤t♠❧ X n Y n 1 1 ˆ M 1 W a ( y 1 | x 1 , x 2 ) ❊♥❝♦❞❡r ✶ ❉❡❝♦❞❡r ✶ M 1 Y n 2 ˆ W b ( y 2 | x 1 , x 2 ) M 2 ❊♥❝♦❞❡r ✷ ❉❡❝♦❞❡r ✷ M 2 X n 2 ✶✶

  26. ◗✉❡st✐♦♥ ✿ ■s t❤✐s t❤❡ ❝❛♣❛❝✐t② r❡❣✐♦♥❄ ✸✳ ■♥t❡r❢❡r❡♥❝❡ ❈❤❛♥♥❡❧ ✭❆❤❧s✇❡❞❡ ✬✼✹✮ ❙✉♣❡r♣♦s✐t✐♦♥ ❝♦❞✐♥❣✱ ♠❡ss❛❣❡ s♣❧✐tt✐♥❣✱ ❝♦❞❡❞ t✐♠❡✲s❤❛r✐♥❣ ❝❛♥ ❜❡ ✉s❡❞ t♦ ❛❝❤✐❡✈❡ r❛t❡ ♣❛✐rs ( R 1 , R 2 ) t❤❛t s❛t✐s❢② R 1 < I ( X 1 ; Y 1 | U 2 , Q ) , R 2 < I ( X 2 ; Y 2 | U 1 , Q ) , R 1 + R 2 < I ( X 1 , U 2 ; Y 1 | Q ) + I ( X 2 ; Y 2 | U 1 , U 2 , Q ) , R 1 + R 2 < I ( X 2 , U 1 ; Y 2 | Q ) + I ( X 1 ; Y 1 | U 1 , U 2 , Q ) , R 1 + R 2 < I ( X 1 , U 2 ; Y 1 | U 1 , Q ) + I ( X 2 , U 1 ; Y 2 | U 2 , Q ) , ❍❛♥ 2 R 1 + R 2 < I ( X 1 , U 2 ; Y 1 | Q ) + I ( X 1 ; Y 1 | U 1 , U 2 , Q ) + I ( X 2 , U 1 ; Y 2 | U 2 , Q ) , R 1 + 2 R 2 < I ( X 2 , U 1 ; Y 2 | Q ) + I ( X 2 ; Y 2 | U 1 , U 2 , Q ) + I ( X 1 , U 2 ; Y 1 | U 1 , Q ) ❢♦r s♦♠❡ ♣♠❢ p ( q ) p ( u 1 , x 1 | q ) p ( u 2 , x 2 | q ) ✱ ✇❤❡r❡ | U 1 | ≤ | X 1 | + 4 ✱ | U 2 | ≤ | X 2 | + 4 ✱ ❛♥❞ | Q | ≤ 7 ✳ ❈❛❧❧ t❤✐s r❡❣✐♦♥ R ( W a , W b ) ✳ ❑♦❜❛②❛s❤✐ ✶✶

  27. ✸✳ ■♥t❡r❢❡r❡♥❝❡ ❈❤❛♥♥❡❧ ✭❆❤❧s✇❡❞❡ ✬✼✹✮ ❙✉♣❡r♣♦s✐t✐♦♥ ❝♦❞✐♥❣✱ ♠❡ss❛❣❡ s♣❧✐tt✐♥❣✱ ❝♦❞❡❞ t✐♠❡✲s❤❛r✐♥❣ ❝❛♥ ❜❡ ✉s❡❞ t♦ ❛❝❤✐❡✈❡ r❛t❡ ♣❛✐rs ( R 1 , R 2 ) t❤❛t s❛t✐s❢② R 1 < I ( X 1 ; Y 1 | U 2 , Q ) , R 2 < I ( X 2 ; Y 2 | U 1 , Q ) , R 1 + R 2 < I ( X 1 , U 2 ; Y 1 | Q ) + I ( X 2 ; Y 2 | U 1 , U 2 , Q ) , R 1 + R 2 < I ( X 2 , U 1 ; Y 2 | Q ) + I ( X 1 ; Y 1 | U 1 , U 2 , Q ) , R 1 + R 2 < I ( X 1 , U 2 ; Y 1 | U 1 , Q ) + I ( X 2 , U 1 ; Y 2 | U 2 , Q ) , ❍❛♥ 2 R 1 + R 2 < I ( X 1 , U 2 ; Y 1 | Q ) + I ( X 1 ; Y 1 | U 1 , U 2 , Q ) + I ( X 2 , U 1 ; Y 2 | U 2 , Q ) , R 1 + 2 R 2 < I ( X 2 , U 1 ; Y 2 | Q ) + I ( X 2 ; Y 2 | U 1 , U 2 , Q ) + I ( X 1 , U 2 ; Y 1 | U 1 , Q ) ❢♦r s♦♠❡ ♣♠❢ p ( q ) p ( u 1 , x 1 | q ) p ( u 2 , x 2 | q ) ✱ ✇❤❡r❡ | U 1 | ≤ | X 1 | + 4 ✱ | U 2 | ≤ | X 2 | + 4 ✱ ❛♥❞ | Q | ≤ 7 ✳ ❈❛❧❧ t❤✐s r❡❣✐♦♥ R ( W a , W b ) ✳ ◗✉❡st✐♦♥ ✿ ■s t❤✐s t❤❡ ❝❛♣❛❝✐t② r❡❣✐♦♥❄ ❑♦❜❛②❛s❤✐ ✶✶

  28. ✸✳ ■♥t❡r❢❡r❡♥❝❡ ❈❤❛♥♥❡❧ ✭❆❤❧s✇❡❞❡ ✬✼✹✮ ❙✉♣❡r♣♦s✐t✐♦♥ ❝♦❞✐♥❣✱ ♠❡ss❛❣❡ s♣❧✐tt✐♥❣✱ ❝♦❞❡❞ t✐♠❡✲s❤❛r✐♥❣ ❝❛♥ ❜❡ ✉s❡❞ t♦ ❛❝❤✐❡✈❡ r❛t❡ ♣❛✐rs ( R 1 , R 2 ) t❤❛t s❛t✐s❢② R 1 < I ( X 1 ; Y 1 | U 2 , Q ) , R 2 < I ( X 2 ; Y 2 | U 1 , Q ) , R 1 + R 2 < I ( X 1 , U 2 ; Y 1 | Q ) + I ( X 2 ; Y 2 | U 1 , U 2 , Q ) , R 1 + R 2 < I ( X 2 , U 1 ; Y 2 | Q ) + I ( X 1 ; Y 1 | U 1 , U 2 , Q ) , R 1 + R 2 < I ( X 1 , U 2 ; Y 1 | U 1 , Q ) + I ( X 2 , U 1 ; Y 2 | U 2 , Q ) , ❍❛♥ 2 R 1 + R 2 < I ( X 1 , U 2 ; Y 1 | Q ) + I ( X 1 ; Y 1 | U 1 , U 2 , Q ) + I ( X 2 , U 1 ; Y 2 | U 2 , Q ) , R 1 + 2 R 2 < I ( X 2 , U 1 ; Y 2 | Q ) + I ( X 2 ; Y 2 | U 1 , U 2 , Q ) + I ( X 1 , U 2 ; Y 1 | U 1 , Q ) ❢♦r s♦♠❡ ♣♠❢ p ( q ) p ( u 1 , x 1 | q ) p ( u 2 , x 2 | q ) ✱ ✇❤❡r❡ | U 1 | ≤ | X 1 | + 4 ✱ | U 2 | ≤ | X 2 | + 4 ✱ ❛♥❞ | Q | ≤ 7 ✳ ❈❛❧❧ t❤✐s r❡❣✐♦♥ R ( W a , W b ) ✳ ◗✉❡st✐♦♥ ✿ ■s t❤✐s t❤❡ ❝❛♣❛❝✐t② r❡❣✐♦♥❄ ❍❛❞ ❜❡❡♥ ♦♣❡♥ ✭s✐♥❝❡ ❍❛♥ ❛♥❞ ❑♦❜❛②❛s❤✐ ✬✽✶✮ ❑♦❜❛②❛s❤✐ ✶✶

  29. ✐s ♥♦t ♦♣t✐♠❛❧ ✐♥ ❣❡♥❡r❛❧ ✭◆❛✐r✱ ❳✐❛✱ ❨❛③❞❛♥♣❛♥❛❤ ✬✶✺✮ ❇r♦❛❞❝❛st ❛♥❞ ✐♥t❡r❢❡r❡♥❝❡ ❝❤❛♥♥❡❧s ❛r❡ ❢❛r t♦♦ ✐♠♣♦rt❛♥t ❚♦ ❧❡t ♥♦♥✲❝♦♥✈❡①✐t② ❞✐ss✉❛❞❡ ✉s ❚♦ ♥♦t ✐♥✈❡st✐❣❛t❡ s✐♠♣❧❡ ❝❧❛ss❡s t❤❛t r❡q✉✐r❡ ♥❡✇ ✐❞❡❛s ❙✉❝❝❡ss❡s ■♥ s♣✐t❡ ♦❢ t❤❡ ✉♥❞❡r❧②✐♥❣ ♣r♦❜❧❡♠ ❜❡✐♥❣ ✐♥tr✐♥s✐❝❛❧❧② ♥♦♥✲❝♦♥✈❡① • R ( W a , W b ) ✐s ♦♣t✐♠❛❧ ❢♦r s♦♠❡ ❝❧❛ss❡s ♦❢ ❝❤❛♥♥❡❧s ⋆ ❈❛r❧❡✐❛❧ ✬✼✺✱ ❙❛t♦ ✬✽✶✱ ❊❧ ●❛♠❛❧ ❛♥❞ ❈♦st❛ ✭✬✽✶ ❛♥❞ ✬✽✻✮ • R ( W a , W b ) ✐s ❝❧♦s❡ t♦ ♦♣t✐♠❛❧ ❢♦r ●❛✉ss✐❛♥ ■♥t❡r❢❡r❡♥❝❡ ❝❤❛♥♥❡❧ ⋆ ❊t❦✐♥ ❛♥❞ ❚s❡ ❛♥❞ ❲❛♥❣ ✭✬✵✾✮ • ◆♦✈❡❧ ✐❞❡❛s ❛♥❞ ♠❛t❤❡♠❛t✐❝❛❧ r❡s✉❧ts ❝❛♠❡ ♦✉t ❢r♦♠ t❤❡ ✐♥✈❡st✐❣❛t✐♦♥ ♦❢ ♦♣t✐♠❛❧✐t② ⋆ ❈♦♥❝❛✈✐t② ♦❢ ❡♥tr♦♣② ♣♦✇❡r ✭❈♦st❛ ✬✽✺✮ ⋆ ●❡♥✐❡ ❜❛s❡❞ ❛♣♣r♦❛❝❤ t♦ ♣r♦✈❡ s✉❜✲❛❞❞✐t✐✈✐t② ✭❊❧ ●❛♠❛❧ ❛♥❞ ❈♦st❛ ✬✽✶✱ ❑r❛♠❡r ✬✵✷✮ ✶✷

  30. ❙✉❝❝❡ss❡s ■♥ s♣✐t❡ ♦❢ t❤❡ ✉♥❞❡r❧②✐♥❣ ♣r♦❜❧❡♠ ❜❡✐♥❣ ✐♥tr✐♥s✐❝❛❧❧② ♥♦♥✲❝♦♥✈❡① • R ( W a , W b ) ✐s ♦♣t✐♠❛❧ ❢♦r s♦♠❡ ❝❧❛ss❡s ♦❢ ❝❤❛♥♥❡❧s ⋆ ❈❛r❧❡✐❛❧ ✬✼✺✱ ❙❛t♦ ✬✽✶✱ ❊❧ ●❛♠❛❧ ❛♥❞ ❈♦st❛ ✭✬✽✶ ❛♥❞ ✬✽✻✮ • R ( W a , W b ) ✐s ❝❧♦s❡ t♦ ♦♣t✐♠❛❧ ❢♦r ●❛✉ss✐❛♥ ■♥t❡r❢❡r❡♥❝❡ ❝❤❛♥♥❡❧ ⋆ ❊t❦✐♥ ❛♥❞ ❚s❡ ❛♥❞ ❲❛♥❣ ✭✬✵✾✮ • ◆♦✈❡❧ ✐❞❡❛s ❛♥❞ ♠❛t❤❡♠❛t✐❝❛❧ r❡s✉❧ts ❝❛♠❡ ♦✉t ❢r♦♠ t❤❡ ✐♥✈❡st✐❣❛t✐♦♥ ♦❢ ♦♣t✐♠❛❧✐t② ⋆ ❈♦♥❝❛✈✐t② ♦❢ ❡♥tr♦♣② ♣♦✇❡r ✭❈♦st❛ ✬✽✺✮ ⋆ ●❡♥✐❡ ❜❛s❡❞ ❛♣♣r♦❛❝❤ t♦ ♣r♦✈❡ s✉❜✲❛❞❞✐t✐✈✐t② ✭❊❧ ●❛♠❛❧ ❛♥❞ ❈♦st❛ ✬✽✶✱ ❑r❛♠❡r ✬✵✷✮ • R ( W a , W b ) ✐s ♥♦t ♦♣t✐♠❛❧ ✐♥ ❣❡♥❡r❛❧ ✭◆❛✐r✱ ❳✐❛✱ ❨❛③❞❛♥♣❛♥❛❤ ✬✶✺✮ ❇r♦❛❞❝❛st ❛♥❞ ✐♥t❡r❢❡r❡♥❝❡ ❝❤❛♥♥❡❧s ❛r❡ ❢❛r t♦♦ ✐♠♣♦rt❛♥t • ❚♦ ❧❡t ♥♦♥✲❝♦♥✈❡①✐t② ❞✐ss✉❛❞❡ ✉s • ❚♦ ♥♦t ✐♥✈❡st✐❣❛t❡ s✐♠♣❧❡ ❝❧❛ss❡s t❤❛t r❡q✉✐r❡ ♥❡✇ ✐❞❡❛s ✶✷

  31. ❆ ❝❧❛ss ♦❢ ♦♣❡♥ ♣r♦❜❧❡♠s ❆ s✉❜✲❝♦❧❧❡❝t✐♦♥ ♦❢ t❤❡ ✶✺ ♦♣❡♥ ♣r♦❜❧❡♠s ❧✐st❡❞ ✐♥ ❈❤❛♣s✳ ✺✲✾✳ ✺✳✶ ❲❤❛t ✐s t❤❡ ❝❛♣❛❝✐t② r❡❣✐♦♥ ♦❢ ❧❡ss ♥♦✐s② ❜r♦❛❞❝❛st✲❝❤❛♥♥❡❧s ✇✐t❤ ❢♦✉r ♦r ♠♦r❡ r❡❝❡✐✈❡rs❄ ✭t✇♦✲r❡❝❡✐✈❡r✿ ❑♦r♥❡r✲▼❛rt♦♥ ✬✼✻✱ t❤r❡❡✲r❡❝❡✐✈❡r✿ ◆❛✐r✲❲❛♥❣ ✬✶✵✮ ✺✳✷ ❲❤❛t ✐s t❤❡ ❝❛♣❛❝✐t② r❡❣✐♦♥ ♦❢ ♠♦r❡ ❝❛♣❛❜❧❡ ❜r♦❛❞❝❛st✲❝❤❛♥♥❡❧s ✇✐t❤ t❤r❡❡ ♦r ♠♦r❡ r❡❝❡✐✈❡rs❄ ✭t✇♦✲r❡❝❡✐✈❡r✿ ❊❧ ●❛♠❛❧ ✬✼✾✮ ✻✳✶ ❲❤❛t ✐s t❤❡ ❝❛♣❛❝✐t② r❡❣✐♦♥ ♦❢ t❤❡ ●❛✉ss✐❛♥ ■♥t❡r❢❡r❡♥❝❡ ❝❤❛♥♥❡❧ ✇✐t❤ ✇❡❛❦ ✐♥t❡r❢❡r❡♥❝❡❄ ✭str♦♥❣✲✐♥t❡r❢❡r❡♥❝❡✿ ❙❛t♦ ✬✼✾❀ ♠✐①❡❞✲✐♥t❡r❢❡r❡♥❝❡ ❝♦r♥❡r✲♣♦✐♥ts✿ ❙❛t♦✬ ✽✶✱ ❈♦st❛✬✽✺❀ ✇❡❛❦✲✐♥t❡r❢❡r❡♥❝❡ ❝♦r♥❡r✲♣♦✐♥ts✿ r❛t❡✲s✉♠ ✭♣❛rt✐❛❧✮✿ t❤r❡❡✲❣r♦✉♣s ✬✵✾ ✮ ✻✳✹ ■s t❤❡ ❍❛♥✲❑♦❜❛②❛s❤✐ ✐♥♥❡r ❜♦✉♥❞ t✐❣❤t ✐♥ ❣❡♥❡r❛❧ ❢♦r ✐♥t❡r❢❡r❡♥❝❡ ❝❤❛♥♥❡❧s❄ ✽✳✷ ■s s✉♣❡r♣♦s✐t✐♦♥ ❝♦❞✐♥❣ ♦♣t✐♠❛❧ ❢♦r t❤❡ ❣❡♥❡r❛❧ ✸✲r❡❝❡✐✈❡r ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧ ✇✐t❤ ♦♥❡ ♠❡ss❛❣❡ t♦ ❛❧❧ t❤r❡❡ r❡❝❡✐✈❡rs ❛♥❞ ❛♥♦t❤❡r ♠❡ss❛❣❡ t♦ t✇♦ r❡❝❡✐✈❡rs❄ ✽✳✸ ❲❤❛t ✐s t❤❡ s✉♠✲❝❛♣❛❝✐t② ♦❢ t❤❡ ❜✐♥❛r② s❦❡✇✲s②♠♠❡tr✐❝ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧❄ ✽✳✹ ■s t❤❡ ▼❛rt♦♥ ✐♥♥❡r ❜♦✉♥❞ t✐❣❤t ✐♥ ❣❡♥❡r❛❧ ❢♦r ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧s❄ ✾✳✷ ❈❛♥ t❤❡ ❝♦♥✈❡rs❡ ❢♦r t❤❡ ●❛✉ss✐❛♥ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧ ❜❡ ♣r♦✈❡❞ ❞✐r❡❝t❧② ❜② ♦♣t✐♠✐③✐♥❣ t❤❡ ◆❛✐r✲❊❧ ●❛♠❛❧ ♦✉t❡r ❜♦✉♥❞❄ ✾✳✸ ❲❤❛t ✐s t❤❡ ❝❛♣❛❝✐t② r❡❣✐♦♥ ♦❢ t❤❡ ✷✲r❡❝❡✐✈❡r ●❛✉ss✐❛♥ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧ ✇✐t❤ ❝♦♠♠♦♥ ♠❡ss❛❣❡❄ ✶✸

  32. ❆ ❝❧❛ss ♦❢ ♦♣❡♥ ♣r♦❜❧❡♠s ▼② r❡❢♦r♠✉❧❛t✐♦♥s ♦❢ ❛ ❢❡✇ ♦❢ t❤❡♠✳ ✺✳✶ ■s s✉♣❡r♣♦s✐t✐♦♥ ❝♦❞✐♥❣ ♦♣t✐♠❛❧ ❢♦r ❧❡ss✲♥♦✐s② ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧s ✇✐t❤ ❢♦✉r ♦r ♠♦r❡ r❡❝❡✐✈❡rs❄ ✺✳✷ ■s s✉♣❡r♣♦s✐t✐♦♥ ❝♦❞✐♥❣ ♦♣t✐♠❛❧ ❢♦r ♠♦r❡✲❝❛♣❛❜❧❡ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧s ✇✐t❤ t❤r❡❡ ♦r ♠♦r❡ r❡❝❡✐✈❡rs❄ ✻✳✶ ■s t❤❡ ❍❛♥✕❑♦❜❛②❛s❤✐ s❝❤❡♠❡ ✇✐t❤ ●❛✉ss✐❛♥ s✐❣♥❛❧✐♥❣ t✐❣❤t ❢♦r t❤❡ ●❛✉ss✐❛♥ ■♥t❡r❢❡r❡♥❝❡ ❝❤❛♥♥❡❧ ✇✐t❤ ✇❡❛❦ ✐♥t❡r❢❡r❡♥❝❡❄ ✻✳✹ ■s t❤❡ ❍❛♥✲❑♦❜❛②❛s❤✐ ✐♥♥❡r ❜♦✉♥❞ t✐❣❤t ✐♥ ❣❡♥❡r❛❧ ❢♦r ✐♥t❡r❢❡r❡♥❝❡ ❝❤❛♥♥❡❧s❄ ✽✳✷ ■s s✉♣❡r♣♦s✐t✐♦♥ ❝♦❞✐♥❣ ♦♣t✐♠❛❧ ❢♦r t❤❡ ❣❡♥❡r❛❧ ✸✲r❡❝❡✐✈❡r ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧ ✇✐t❤ ♦♥❡ ♠❡ss❛❣❡ t♦ ❛❧❧ t❤r❡❡ r❡❝❡✐✈❡rs ❛♥❞ ❛♥♦t❤❡r ♠❡ss❛❣❡ t♦ t✇♦ r❡❝❡✐✈❡rs❄ ✽✳✸ ❉♦❡s t❤❡ ▼❛rt♦♥ ✐♥♥❡r ❜♦✉♥❞ ❛❝❤✐❡✈❡ t❤❡ s✉♠✲❝❛♣❛❝✐t② ♦❢ t❤❡ ❜✐♥❛r② s❦❡✇✲s②♠♠❡tr✐❝ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧❄ ✽✳✹ ■s t❤❡ ▼❛rt♦♥ ✐♥♥❡r ❜♦✉♥❞ t✐❣❤t ✐♥ ❣❡♥❡r❛❧ ❢♦r ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧s❄ ✾✳✷ ❈❛♥ t❤❡ ❝♦♥✈❡rs❡ ❢♦r t❤❡ ●❛✉ss✐❛♥ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧ ❜❡ ♣r♦✈❡❞ ❞✐r❡❝t❧② ❜② ♦♣t✐♠✐③✐♥❣ t❤❡ ◆❛✐r✲❊❧ ●❛♠❛❧ ♦✉t❡r ❜♦✉♥❞❄ ✾✳✸ ❉♦❡s t❤❡ ▼❛rt♦♥ ✐♥♥❡r ❜♦✉♥❞ ❛❝❤✐❡✈❡ t❤❡ ❝❛♣❛❝✐t② r❡❣✐♦♥ ♦❢ t❤❡ ✷✲r❡❝❡✐✈❡r ●❛✉ss✐❛♥ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧ ✇✐t❤ ❝♦♠♠♦♥ ♠❡ss❛❣❡❄ ✶✸

  33. ❚❤❡ ❝♦♠♠♦♥ t❤❡♠❡ t♦ t❤❡s❡ ✭r❡❢♦r♠✉❧❛t❡❞✮ q✉❡st✐♦♥s ❈♦♠♠♦♥ t❤❡♠❡ ■s ❛ ❝❛♥❞✐❞❛t❡ r❛t❡ r❡❣✐♦♥ ♦♣t✐♠❛❧❄ ■❞❡❛ ❢♦r t❡st✐♥❣ ♦♣t✐♠❛❧✐t②✿ • S λ ( W ⊗ W ) ? = 2 S λ ( W ) • ❉❡t❡r♠✐♥❡ s✉❜✲❛❞❞✐t✐✈✐t② ♦❢ ❛♥ ❛ss♦❝✐❛t❡❞ ♥♦♥✲❝♦♥✈❡① ❢✉♥❝t✐♦♥❛❧ ✶✹

  34. ❙t❛t✉s ♦❢ t❤❡ ♦♣❡♥ ♣r♦❜❧❡♠s ✺✳✶ ■s s✉♣❡r♣♦s✐t✐♦♥ ❝♦❞✐♥❣ ♦♣t✐♠❛❧ ❢♦r ❧❡ss✲♥♦✐s② ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧s ✇✐t❤ ❢♦✉r ♦r ♠♦r❡ r❡❝❡✐✈❡rs❄ ✭❖P❊◆✮ ✺✳✷ ■s s✉♣❡r♣♦s✐t✐♦♥ ❝♦❞✐♥❣ ♦♣t✐♠❛❧ ❢♦r ♠♦r❡✲❝❛♣❛❜❧❡ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧s ✇✐t❤ t❤r❡❡ ♦r ♠♦r❡ r❡❝❡✐✈❡rs❄ ✭ ◆❖ ✿ ◆❛✐r✲❳✐❛ ✬✶✷✮ ✻✳✶ ■s t❤❡ ❍❛♥✕❑♦❜❛②❛s❤✐ s❝❤❡♠❡ ✇✐t❤ ●❛✉ss✐❛♥ s✐❣♥❛❧✐♥❣ t✐❣❤t ❢♦r t❤❡ ●❛✉ss✐❛♥ ■♥t❡r❢❡r❡♥❝❡ ❝❤❛♥♥❡❧ ✇✐t❤ ✇❡❛❦ ✐♥t❡r❢❡r❡♥❝❡❄✭❖P❊◆✮ ✭ ❨❊❙ ✿ ❝♦r♥❡r ♣♦✐♥ts ✉s✐♥❣ ✐❞❡❛s ✐♥ ♠❡❛s✉r❡ tr❛♥s♣♦rt❛t✐♦♥ ❜② P♦❧②❛♥s❦✐②✲❲✉ ✬✶✺✮ ✻✳✹ ■s t❤❡ ❍❛♥✲❑♦❜❛②❛s❤✐ ✐♥♥❡r ❜♦✉♥❞ t✐❣❤t ✐♥ ❣❡♥❡r❛❧ ❢♦r ✐♥t❡r❢❡r❡♥❝❡ ❝❤❛♥♥❡❧s❄ ✭ ◆❖ ✿ ◆❛✐r✲❳✐❛✲❨❛③❞❛♥♣❛♥❛❤ ✬✶✺✮ ✽✳✷ ■s s✉♣❡r♣♦s✐t✐♦♥ ❝♦❞✐♥❣ ♦♣t✐♠❛❧ ❢♦r t❤❡ ❣❡♥❡r❛❧ ✸✲r❡❝❡✐✈❡r ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧ ✇✐t❤ ♦♥❡ ♠❡ss❛❣❡ t♦ ❛❧❧ t❤r❡❡ r❡❝❡✐✈❡rs ❛♥❞ ❛♥♦t❤❡r ♠❡ss❛❣❡ t♦ t✇♦ r❡❝❡✐✈❡rs❄ ✭ ◆❖ ✿ ◆❛✐r✲❨❛③❞❛♥♣❛♥❛❤ ✬✶✼✮ ✽✳✸ ❉♦❡s t❤❡ ▼❛rt♦♥ ✐♥♥❡r ❜♦✉♥❞ ❛❝❤✐❡✈❡ t❤❡ s✉♠✲❝❛♣❛❝✐t② ♦❢ t❤❡ ❜✐♥❛r② s❦❡✇✲s②♠♠❡tr✐❝ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧❄ ✭❖P❊◆✮ ✽✳✹ ■s t❤❡ ▼❛rt♦♥ ✐♥♥❡r ❜♦✉♥❞ t✐❣❤t ✐♥ ❣❡♥❡r❛❧ ❢♦r ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧s❄ ✭❖P❊◆✮ ✾✳✷ ❈❛♥ t❤❡ ❝♦♥✈❡rs❡ ❢♦r t❤❡ ●❛✉ss✐❛♥ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧ ❜❡ ♣r♦✈❡❞ ❞✐r❡❝t❧② ❜② ♦♣t✐♠✐③✐♥❣ t❤❡ ◆❛✐r✲❊❧ ●❛♠❛❧ ♦✉t❡r ❜♦✉♥❞❄ ✭ ❨❊❙ ✿ ●❡♥❣✲◆❛✐r ✬✶✹✮ ✾✳✸ ❉♦❡s t❤❡ ▼❛rt♦♥ ✐♥♥❡r ❜♦✉♥❞ ❛❝❤✐❡✈❡ t❤❡ ❝❛♣❛❝✐t② r❡❣✐♦♥ ♦❢ t❤❡ ✷✲r❡❝❡✐✈❡r ●❛✉ss✐❛♥ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧ ✇✐t❤ ❝♦♠♠♦♥ ♠❡ss❛❣❡❄ ✭ ❨❊❙ ✿ ●❡♥❣✲◆❛✐r ✬✶✹✮ ✶✺

  35. ❖✉t❧✐♥❡ • ❇r♦❛❞❝❛st ❝❤❛♥♥❡❧✿ ❊st❛❜❧✐s❤✐♥❣ ♦♣t✐♠❛❧✐t② ♦❢ ▼❛rt♦♥✬s r❡❣✐♦♥ ❢♦r ▼■▼❖ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧ • ■♥t❡r❢❡r❡♥❝❡ ❝❤❛♥♥❡❧✿ ❙✉❜✲♦♣t✐♠❛❧✐t② ♦❢ t❤❡ ❍❛♥✕❑♦❜❛②❛s❤✐ r❡❣✐♦♥ • ❋❛♠✐❧② ♦❢ ♥♦♥✲❝♦♥✈❡① ♦♣t✐♠✐③❛t✐♦♥ ♣r♦❜❧❡♠s ⋆ ❘❡❧❛t✐♦♥ t♦ ♣r♦❜❧❡♠s ♦❢ ✐♥t❡r❡st ✐♥ ♦t❤❡r ✜❡❧❞s ⋆ ❯♥✐❢②✐♥❣ ♦❜s❡r✈❛t✐♦♥s ❛♥❞ s♦♠❡ ❝♦♥❥❡❝t✉r❡s ✶✻

  36. ▼■▼❖ ●❛✉ss✐❛♥ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧ ✿ ✇❤❡r❡ ❞❡♥♦t❡s t❤❡ ❛❞❞✐t✐✈❡ ♥♦✐s❡✳ ❱❡r② ✐♠♣♦rt❛♥t ❝❤❛♥♥❡❧ ❝❧❛ss ✐♥ ✇✐r❡❧❡ss ❝♦♠♠✉♥✐❝❛t✐♦♥ ▼♦❞❡❧s✿ ♠✉❧t✐✲❛♥t❡♥♥❛ tr❛♥s♠✐tt❡r✴r❡❝❡✐✈❡rs ✭❞♦✇♥❧✐♥❦✮ ▼■▼❖ ✭❱❡❝t♦r✮ ●❛✉ss✐❛♥ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧ Y n 1 M 0 , ˆ ˆ W a ( y 1 | x ) ❉❡❝♦❞❡r ✶ M 1 X n ( M 0 , M 1 , M 2 ) ❊♥❝♦❞❡r Y n 2 M 0 , ˜ ˜ W b ( y 2 | x ) ❉❡❝♦❞❡r ✷ M 2 ✶✼

  37. ▼■▼❖ ✭❱❡❝t♦r✮ ●❛✉ss✐❛♥ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧ Y n 1 M 0 , ˆ ˆ W a ( y 1 | x ) ❉❡❝♦❞❡r ✶ M 1 X n ( M 0 , M 1 , M 2 ) ❊♥❝♦❞❡r Y n 2 M 0 , ˜ ˜ W b ( y 2 | x ) ❉❡❝♦❞❡r ✷ M 2 ▼■▼❖ ●❛✉ss✐❛♥ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧ ✿ Y 1 = AX + Z Y 2 = BX + Z ✇❤❡r❡ Z ∼ N (0 , I ) ❞❡♥♦t❡s t❤❡ ❛❞❞✐t✐✈❡ ♥♦✐s❡✳ ❱❡r② ✐♠♣♦rt❛♥t ❝❤❛♥♥❡❧ ❝❧❛ss ✐♥ ✇✐r❡❧❡ss ❝♦♠♠✉♥✐❝❛t✐♦♥ ▼♦❞❡❧s✿ ♠✉❧t✐✲❛♥t❡♥♥❛ tr❛♥s♠✐tt❡r✴r❡❝❡✐✈❡rs ✭❞♦✇♥❧✐♥❦✮ ✶✼

  38. ●✐st ✿ ❙❤♦✇✐♥❣ ♦♣t✐♠❛❧✐t② ♦❢ ●❛✉ss✐❛♥ r❛♥❞♦♠ ✈❛r✐❛❜❧❡s ❢♦r ❛ ♥♦♥✲❝♦♥✈❡① ♦♣t✐♠✐③❛t✐♦♥ ♣r♦❜❧❡♠ ❘❡♠❛r❦ ✿ ■❞❡❛s ❞♦ ♥♦t ❡①t❡♥❞ t♦ s❤♦✇ ♦♣t✐♠❛❧✐t② ✇❤❡♥ t❤❡r❡ ✐s ❝♦♠♠♦♥ ♠❡ss❛❣❡✱ ✐✳❡✳ ❖♣t✐♠❛❧✐t② ✐♥ ❣❡♥❡r❛❧ ✭●❡♥❣ ❛♥❞ ◆❛✐r ✬✶✹✮ ●✐st ✿ ❉❡✈❡❧♦♣ ❛ t❡❝❤♥✐q✉❡ ❢♦r ♣r♦✈✐♥❣ ♦♣t✐♠❛❧✐t② ♦❢ ●❛✉ss✐❛♥ r❛♥❞♦♠ ✈❛r✐❛❜❧❡s ✭❢r♦♠ s✉❜✲❛❞❞✐t✐✈✐t②✮ ❊①♣❧❛✐♥ ♦✉r t❡❝❤♥✐q✉❡ ♦♥ ✭❢♦r s✐♠♣❧✐❝✐t②✮ ❍✐st♦r② ❖♣t✐♠❛❧✐t② ♦❢ ▼❛rt♦♥✬s ❜♦✉♥❞✱ R ( W a , W b ) ✱ ✇❛s ❡st❛❜❧✐s❤❡❞✿ • ❙❝❛❧❛r ❝❛s❡ ✭❇❡r❣♠❛♥s ✬✼✸✮ ✭❊♥tr♦♣② P♦✇❡r ■♥❡q✉❛❧✐t②✮ • ❘❡✈❡rs❡❧② ❞❡❣r❛❞❡❞ s❡tt✐♥❣ ✭P♦❧t②r❡✈ ✬✼✽✱ ❊❧ ●❛♠❛❧ ✬✽✶✮ • ❖♣t✐♠❛❧✐t② ♦♥ R 0 = 0 ✭❲❡✐♥❣❛rt❡♥ ❛♥❞ ❙t❡✐♥❜❡r❣ ❛♥❞ ❙❤❛♠❛✐ ✬✵✻✮ ⋆ ❇✉✐❧❞s ♦♥ ✐❞❡❛s ✐♥ P♦❧t②r❡✈ ⋆ ❚♦✉r ❞❡ ❢♦r❝❡ ✐♥ ♦♣t✐♠✐③❛t✐♦♥ ✶✽

  39. ❖♣t✐♠❛❧✐t② ✐♥ ❣❡♥❡r❛❧ ✭●❡♥❣ ❛♥❞ ◆❛✐r ✬✶✹✮ ●✐st ✿ ❉❡✈❡❧♦♣ ❛ t❡❝❤♥✐q✉❡ ❢♦r ♣r♦✈✐♥❣ ♦♣t✐♠❛❧✐t② ♦❢ ●❛✉ss✐❛♥ r❛♥❞♦♠ ✈❛r✐❛❜❧❡s ✭❢r♦♠ s✉❜✲❛❞❞✐t✐✈✐t②✮ ❊①♣❧❛✐♥ ♦✉r t❡❝❤♥✐q✉❡ ♦♥ ✭❢♦r s✐♠♣❧✐❝✐t②✮ ❍✐st♦r② ❖♣t✐♠❛❧✐t② ♦❢ ▼❛rt♦♥✬s ❜♦✉♥❞✱ R ( W a , W b ) ✱ ✇❛s ❡st❛❜❧✐s❤❡❞✿ • ❙❝❛❧❛r ❝❛s❡ ✭❇❡r❣♠❛♥s ✬✼✸✮ ✭❊♥tr♦♣② P♦✇❡r ■♥❡q✉❛❧✐t②✮ • ❘❡✈❡rs❡❧② ❞❡❣r❛❞❡❞ s❡tt✐♥❣ ✭P♦❧t②r❡✈ ✬✼✽✱ ❊❧ ●❛♠❛❧ ✬✽✶✮ • ❖♣t✐♠❛❧✐t② ♦♥ R 0 = 0 ✭❲❡✐♥❣❛rt❡♥ ❛♥❞ ❙t❡✐♥❜❡r❣ ❛♥❞ ❙❤❛♠❛✐ ✬✵✻✮ ⋆ ❇✉✐❧❞s ♦♥ ✐❞❡❛s ✐♥ P♦❧t②r❡✈ ⋆ ❚♦✉r ❞❡ ❢♦r❝❡ ✐♥ ♦♣t✐♠✐③❛t✐♦♥ ⋆ ●✐st ✿ ❙❤♦✇✐♥❣ ♦♣t✐♠❛❧✐t② ♦❢ ●❛✉ss✐❛♥ r❛♥❞♦♠ ✈❛r✐❛❜❧❡s ❢♦r ❛ ♥♦♥✲❝♦♥✈❡① ♦♣t✐♠✐③❛t✐♦♥ ♣r♦❜❧❡♠ ⋆ ❘❡♠❛r❦ ✿ ■❞❡❛s ❞♦ ♥♦t ❡①t❡♥❞ t♦ s❤♦✇ ♦♣t✐♠❛❧✐t② ✇❤❡♥ t❤❡r❡ ✐s ❝♦♠♠♦♥ ♠❡ss❛❣❡✱ ✐✳❡✳ R 0 � = 0 ✶✽

  40. ❊①♣❧❛✐♥ ♦✉r t❡❝❤♥✐q✉❡ ♦♥ ✭❢♦r s✐♠♣❧✐❝✐t②✮ ❍✐st♦r② ❖♣t✐♠❛❧✐t② ♦❢ ▼❛rt♦♥✬s ❜♦✉♥❞✱ R ( W a , W b ) ✱ ✇❛s ❡st❛❜❧✐s❤❡❞✿ • ❙❝❛❧❛r ❝❛s❡ ✭❇❡r❣♠❛♥s ✬✼✸✮ ✭❊♥tr♦♣② P♦✇❡r ■♥❡q✉❛❧✐t②✮ • ❘❡✈❡rs❡❧② ❞❡❣r❛❞❡❞ s❡tt✐♥❣ ✭P♦❧t②r❡✈ ✬✼✽✱ ❊❧ ●❛♠❛❧ ✬✽✶✮ • ❖♣t✐♠❛❧✐t② ♦♥ R 0 = 0 ✭❲❡✐♥❣❛rt❡♥ ❛♥❞ ❙t❡✐♥❜❡r❣ ❛♥❞ ❙❤❛♠❛✐ ✬✵✻✮ ⋆ ❇✉✐❧❞s ♦♥ ✐❞❡❛s ✐♥ P♦❧t②r❡✈ ⋆ ❚♦✉r ❞❡ ❢♦r❝❡ ✐♥ ♦♣t✐♠✐③❛t✐♦♥ ⋆ ●✐st ✿ ❙❤♦✇✐♥❣ ♦♣t✐♠❛❧✐t② ♦❢ ●❛✉ss✐❛♥ r❛♥❞♦♠ ✈❛r✐❛❜❧❡s ❢♦r ❛ ♥♦♥✲❝♦♥✈❡① ♦♣t✐♠✐③❛t✐♦♥ ♣r♦❜❧❡♠ ⋆ ❘❡♠❛r❦ ✿ ■❞❡❛s ❞♦ ♥♦t ❡①t❡♥❞ t♦ s❤♦✇ ♦♣t✐♠❛❧✐t② ✇❤❡♥ t❤❡r❡ ✐s ❝♦♠♠♦♥ ♠❡ss❛❣❡✱ ✐✳❡✳ R 0 � = 0 • ❖♣t✐♠❛❧✐t② ✐♥ ❣❡♥❡r❛❧ ✭●❡♥❣ ❛♥❞ ◆❛✐r ✬✶✹✮ ⋆ ●✐st ✿ ❉❡✈❡❧♦♣ ❛ t❡❝❤♥✐q✉❡ ❢♦r ♣r♦✈✐♥❣ ♦♣t✐♠❛❧✐t② ♦❢ ●❛✉ss✐❛♥ r❛♥❞♦♠ ✈❛r✐❛❜❧❡s ✭❢r♦♠ s✉❜✲❛❞❞✐t✐✈✐t②✮ ✶✽

  41. ❍✐st♦r② ❖♣t✐♠❛❧✐t② ♦❢ ▼❛rt♦♥✬s ❜♦✉♥❞✱ R ( W a , W b ) ✱ ✇❛s ❡st❛❜❧✐s❤❡❞✿ • ❙❝❛❧❛r ❝❛s❡ ✭❇❡r❣♠❛♥s ✬✼✸✮ ✭❊♥tr♦♣② P♦✇❡r ■♥❡q✉❛❧✐t②✮ • ❘❡✈❡rs❡❧② ❞❡❣r❛❞❡❞ s❡tt✐♥❣ ✭P♦❧t②r❡✈ ✬✼✽✱ ❊❧ ●❛♠❛❧ ✬✽✶✮ • ❖♣t✐♠❛❧✐t② ♦♥ R 0 = 0 ✭❲❡✐♥❣❛rt❡♥ ❛♥❞ ❙t❡✐♥❜❡r❣ ❛♥❞ ❙❤❛♠❛✐ ✬✵✻✮ ⋆ ❇✉✐❧❞s ♦♥ ✐❞❡❛s ✐♥ P♦❧t②r❡✈ ⋆ ❚♦✉r ❞❡ ❢♦r❝❡ ✐♥ ♦♣t✐♠✐③❛t✐♦♥ ⋆ ●✐st ✿ ❙❤♦✇✐♥❣ ♦♣t✐♠❛❧✐t② ♦❢ ●❛✉ss✐❛♥ r❛♥❞♦♠ ✈❛r✐❛❜❧❡s ❢♦r ❛ ♥♦♥✲❝♦♥✈❡① ♦♣t✐♠✐③❛t✐♦♥ ♣r♦❜❧❡♠ ⋆ ❘❡♠❛r❦ ✿ ■❞❡❛s ❞♦ ♥♦t ❡①t❡♥❞ t♦ s❤♦✇ ♦♣t✐♠❛❧✐t② ✇❤❡♥ t❤❡r❡ ✐s ❝♦♠♠♦♥ ♠❡ss❛❣❡✱ ✐✳❡✳ R 0 � = 0 • ❖♣t✐♠❛❧✐t② ✐♥ ❣❡♥❡r❛❧ ✭●❡♥❣ ❛♥❞ ◆❛✐r ✬✶✹✮ ⋆ ●✐st ✿ ❉❡✈❡❧♦♣ ❛ t❡❝❤♥✐q✉❡ ❢♦r ♣r♦✈✐♥❣ ♦♣t✐♠❛❧✐t② ♦❢ ●❛✉ss✐❛♥ r❛♥❞♦♠ ✈❛r✐❛❜❧❡s ✭❢r♦♠ s✉❜✲❛❞❞✐t✐✈✐t②✮ ❊①♣❧❛✐♥ ♦✉r t❡❝❤♥✐q✉❡ ♦♥ R 0 = 0 ✭❢♦r s✐♠♣❧✐❝✐t②✮ ✶✽

  42. ✭◆❛✐r ✬✶✸✮ ✉♣♣❡r ❝♦♥❝❛✈❡ ❡♥✈❡❧♦♣❡✿ ◆♦t ❡❛s② t♦ ❝♦♠♣✉t❡ ✭✐♥ ❣❡♥❡r❛❧✮ ❋♦r ✱ ❧❡t ❖✉t❡r ❜♦✉♥❞ ✭❑♦r♥❡r✲▼❛rt♦♥ ✬✼✾✮ ❚❤❡ s❡t ♦❢ r❛t❡ ♣❛✐rs ( R 1 , R 2 ) s❛t✐s❢②✐♥❣ R 2 ≤ I ( U ; Y 2 ) R 1 + R 2 ≤ I ( U ; Y 2 ) + I ( X ; Y 1 | U ) ❢♦r s♦♠❡ p ( u, x ) ✱ ✇❤❡r❡ E( � X � 2 ) ≤ P ❢♦r♠s ❛♥ ♦✉t❡r ❜♦✉♥❞ t♦ t❤❡ ❝❛♣❛❝✐t② r❡❣✐♦♥✳ ❉❡♥♦t❡ t❤✐s r❡❣✐♦♥ ❛s O ( W a , W b ) ✳ ✶✾

  43. ✭◆❛✐r ✬✶✸✮ ✉♣♣❡r ❝♦♥❝❛✈❡ ❡♥✈❡❧♦♣❡✿ ◆♦t ❡❛s② t♦ ❝♦♠♣✉t❡ ✭✐♥ ❣❡♥❡r❛❧✮ ❖✉t❡r ❜♦✉♥❞ ✭❑♦r♥❡r✲▼❛rt♦♥ ✬✼✾✮ ❚❤❡ s❡t ♦❢ r❛t❡ ♣❛✐rs ( R 1 , R 2 ) s❛t✐s❢②✐♥❣ R 2 ≤ I ( U ; Y 2 ) R 1 + R 2 ≤ I ( U ; Y 2 ) + I ( X ; Y 1 | U ) ❢♦r s♦♠❡ p ( u, x ) ✱ ✇❤❡r❡ E( � X � 2 ) ≤ P ❢♦r♠s ❛♥ ♦✉t❡r ❜♦✉♥❞ t♦ t❤❡ ❝❛♣❛❝✐t② r❡❣✐♦♥✳ ❉❡♥♦t❡ t❤✐s r❡❣✐♦♥ ❛s O ( W a , W b ) ✳ ❋♦r λ > 1 ✱ ❧❡t S λ ( W a , W b ) := ( R 1 ,R 2 ) ∈O R 1 + λR 2 max = max p ( u,x ) λI ( U ; Y 2 ) + I ( X ; Y 1 | U ) ✶✾

  44. ✉♣♣❡r ❝♦♥❝❛✈❡ ❡♥✈❡❧♦♣❡✿ ◆♦t ❡❛s② t♦ ❝♦♠♣✉t❡ ✭✐♥ ❣❡♥❡r❛❧✮ ❖✉t❡r ❜♦✉♥❞ ✭❑♦r♥❡r✲▼❛rt♦♥ ✬✼✾✮ ❚❤❡ s❡t ♦❢ r❛t❡ ♣❛✐rs ( R 1 , R 2 ) s❛t✐s❢②✐♥❣ R 2 ≤ I ( U ; Y 2 ) R 1 + R 2 ≤ I ( U ; Y 2 ) + I ( X ; Y 1 | U ) ❢♦r s♦♠❡ p ( u, x ) ✱ ✇❤❡r❡ E( � X � 2 ) ≤ P ❢♦r♠s ❛♥ ♦✉t❡r ❜♦✉♥❞ t♦ t❤❡ ❝❛♣❛❝✐t② r❡❣✐♦♥✳ ❉❡♥♦t❡ t❤✐s r❡❣✐♦♥ ❛s O ( W a , W b ) ✳ ❋♦r λ > 1 ✱ ❧❡t S λ ( W a , W b ) := ( R 1 ,R 2 ) ∈O R 1 + λR 2 max = max p ( u,x ) λI ( U ; Y 2 ) + I ( X ; Y 1 | U ) � � λI ( X ; Z ) + C µ X [ I ( X ; Y ) − λI ( X ; Z )] = max ✭◆❛✐r ✬✶✸✮ p ( x ) ✶✾

  45. ❖✉t❡r ❜♦✉♥❞ ✭❑♦r♥❡r✲▼❛rt♦♥ ✬✼✾✮ ❚❤❡ s❡t ♦❢ r❛t❡ ♣❛✐rs ( R 1 , R 2 ) s❛t✐s❢②✐♥❣ R 2 ≤ I ( U ; Y 2 ) R 1 + R 2 ≤ I ( U ; Y 2 ) + I ( X ; Y 1 | U ) ❢♦r s♦♠❡ p ( u, x ) ✱ ✇❤❡r❡ E( � X � 2 ) ≤ P ❢♦r♠s ❛♥ ♦✉t❡r ❜♦✉♥❞ t♦ t❤❡ ❝❛♣❛❝✐t② r❡❣✐♦♥✳ ❉❡♥♦t❡ t❤✐s r❡❣✐♦♥ ❛s O ( W a , W b ) ✳ ❋♦r λ > 1 ✱ ❧❡t S λ ( W a , W b ) := ( R 1 ,R 2 ) ∈O R 1 + λR 2 max = max p ( u,x ) λI ( U ; Y 2 ) + I ( X ; Y 1 | U ) � � λI ( X ; Z ) + C µ X [ I ( X ; Y ) − λI ( X ; Z )] = max ✭◆❛✐r ✬✶✸✮ p ( x ) x x f ( x ) ✉♣♣❡r ❝♦♥❝❛✈❡ ❡♥✈❡❧♦♣❡✿ C x [ f ] ✶✾ ◆♦t ❡❛s② t♦ ❝♦♠♣✉t❡ ✭✐♥ ❣❡♥❡r❛❧✮

  46. ❖♥❡ ❝❛♥ s❤♦✇ t❤❛t ✐❢ ●❛✉ss✐❛♥s ♠❛①✐♠✐③❡ C µ X [ h ( Y 1 ) − λh ( Y 2 )] t❤❡♥ ▼❛rt♦♥✬s ✐♥♥❡r ❜♦✉♥❞ ✐s ♦♣t✐♠❛❧ ✭♦♥ R 0 = 0 ✮ ❍❡r❡ h ( X ) ✐s t❤❡ ❞✐✛❡r❡♥t✐❛❧ ❡♥tr♦♣②✿ � − h ( X ) := f ( x ) log f ( x ) dx, ✇❤❡r❡ f ( x ) ✐s t❤❡ ❞❡♥s✐t② ❢✉♥❝t✐♦♥ ♦❢ X ✳ ❆ s✐♠✐❧❛r ✭♠♦r❡✲✐♥✈♦❧✈❡❞✮ ♣r♦❜❧❡♠ s❤♦✇s ✉♣ ✇❤❡♥ R 0 � = 0 ❆♥ ✐❞❡♥t✐❝❛❧ t❡❝❤♥✐q✉❡ ✭t♦ t❤❡ ♦♥❡ ■ ❛♠ ❣♦✐♥❣ t♦ ❞❡♠♦♥str❛t❡✮ ❡st❛❜❧✐s❤❡s t❤❛t ❛❧s♦ ✷✵

  47. ▲❡♠♠❛ ✿ ✐s s✉❜✲❛❞❞✐t✐✈❡✳ Pr♦♦❢ ✿ ❋♦r ❛♥② ●❛✉ss✐❛♥ ♦♣t✐♠❛❧✐t② ✈✐❛ s✉❜✲❛❞❞✐t✐✈✐t② ✭●❡♥❣✲◆❛✐r ✬✶✹✮ ▼❛①✐♠✐③❡✱ ❢♦r λ > 1 ✱ t❤❡ ✈❛❧✉❡ ♦❢ t❤❡ ❢✉♥❝t✐♦♥❛❧ C µ X [ h ( AX + Z ) − λh ( BX + Z )] ♦✈❡r X : ❊ ( XX T ) � K ✱ ✇❤❡r❡ A, B ❛r❡ ✐♥✈❡rt✐❜❧❡ ♠❛tr✐❝❡s ❛♥❞ Z ∼ N (0 , I ) ✳ ❲❡ ✇✐❧❧ s❡❡ t❤❛t t❤❡ ♠❛①✐♠✉♠ ✈❛❧✉❡ ✐s h ( AX ∗ + Z ) − λh ( BX ∗ + Z ) , ✇❤❡r❡ X ∗ ∼ N (0 , K ′ ) ❢♦r s♦♠❡ K ′ � K ✳ ✷✶

  48. ●❛✉ss✐❛♥ ♦♣t✐♠❛❧✐t② ✈✐❛ s✉❜✲❛❞❞✐t✐✈✐t② ✭●❡♥❣✲◆❛✐r ✬✶✹✮ ▼❛①✐♠✐③❡✱ ❢♦r λ > 1 ✱ t❤❡ ✈❛❧✉❡ ♦❢ t❤❡ ❢✉♥❝t✐♦♥❛❧ C µ X [ h ( AX + Z ) − λh ( BX + Z )] ♦✈❡r X : ❊ ( XX T ) � K ✱ ✇❤❡r❡ A, B ❛r❡ ✐♥✈❡rt✐❜❧❡ ♠❛tr✐❝❡s ❛♥❞ Z ∼ N (0 , I ) ✳ ❲❡ ✇✐❧❧ s❡❡ t❤❛t t❤❡ ♠❛①✐♠✉♠ ✈❛❧✉❡ ✐s h ( AX ∗ + Z ) − λh ( BX ∗ + Z ) , ✇❤❡r❡ X ∗ ∼ N (0 , K ′ ) ❢♦r s♦♠❡ K ′ � K ✳ ▲❡♠♠❛ ✿ C µ X [ h ( AX + Z ) − λh ( BX + Z )] ✐s s✉❜✲❛❞❞✐t✐✈❡✳ Pr♦♦❢ ✿ ❋♦r ❛♥② µ X 1 ,X 2 h ( AX 1 + Z 1 , AX 2 + Z 2 | U ) − λh ( BX 1 + Z 1 , BX 2 + Z 2 | U ) = h ( AX 1 + Z 1 | U, AX 2 + Z 2 ) − λh ( BX 1 + Z 1 | U, AX 2 + Z 2 ) + h ( AX 2 + Z 2 | U, BX 1 + Z 1 ) − λh ( BX 2 + Z 2 | U, BX 1 + Z 1 ) − ( λ − 1) I ( AX 2 + Z 2 ; BX 1 + Z 1 | U ) ✷✶

  49. ●❛✉ss✐❛♥ ♦♣t✐♠❛❧✐t② ✈✐❛ s✉❜✲❛❞❞✐t✐✈✐t② ✭●❡♥❣✲◆❛✐r ✬✶✹✮ ▼❛①✐♠✐③❡✱ ❢♦r λ > 1 ✱ t❤❡ ✈❛❧✉❡ ♦❢ t❤❡ ❢✉♥❝t✐♦♥❛❧ C µ X [ h ( AX + Z ) − λh ( BX + Z )] ♦✈❡r X : ❊ ( XX T ) � K ✱ ✇❤❡r❡ A, B ❛r❡ ✐♥✈❡rt✐❜❧❡ ♠❛tr✐❝❡s ❛♥❞ Z ∼ N (0 , I ) ✳ ❲❡ ✇✐❧❧ s❡❡ t❤❛t t❤❡ ♠❛①✐♠✉♠ ✈❛❧✉❡ ✐s h ( AX ∗ + Z ) − λh ( BX ∗ + Z ) , ✇❤❡r❡ X ∗ ∼ N (0 , K ′ ) ❢♦r s♦♠❡ K ′ � K ✳ ▲❡♠♠❛ ✿ C µ X [ h ( AX + Z ) − λh ( BX + Z )] ✐s s✉❜✲❛❞❞✐t✐✈❡✳ Pr♦♦❢ ✿ ❋♦r ❛♥② µ X 1 ,X 2 C µ X 1 ,X 2 [ h ( AX 1 + Z 1 , AX 2 + Z 2 ) − λh ( BX 1 + Z 1 , BX 2 + Z 2 )] ≤ C µ X 1 [ h ( AX 1 + Z 1 ) − λh ( BX 1 + Z 1 )] + C µ X 2 [ h ( AX 2 + Z 2 ) − λh ( BX 2 + Z 2 )] ✷✶

  50. ●❛✉ss✐❛♥ ♦♣t✐♠❛❧✐t② ✈✐❛ s✉❜✲❛❞❞✐t✐✈✐t② ✭●❡♥❣✲◆❛✐r ✬✶✹✮ ▼❛①✐♠✐③❡✱ ❢♦r λ > 1 ✱ t❤❡ ✈❛❧✉❡ ♦❢ t❤❡ ❢✉♥❝t✐♦♥❛❧ C µ X [ h ( AX + Z ) − λh ( BX + Z )] ♦✈❡r X : ❊ ( XX T ) � K ✱ ✇❤❡r❡ A, B ❛r❡ ✐♥✈❡rt✐❜❧❡ ♠❛tr✐❝❡s ❛♥❞ Z ∼ N (0 , I ) ✳ ❲❡ ✇✐❧❧ s❡❡ t❤❛t t❤❡ ♠❛①✐♠✉♠ ✈❛❧✉❡ ✐s h ( AX ∗ + Z ) − λh ( BX ∗ + Z ) , ✇❤❡r❡ X ∗ ∼ N (0 , K ′ ) ❢♦r s♦♠❡ K ′ � K ✳ ▲❡♠♠❛ ✿ C µ X [ h ( AX + Z ) − λh ( BX + Z )] ✐s s✉❜✲❛❞❞✐t✐✈❡✳ Pr♦♦❢ ✿ ❋♦r ❛♥② µ X 1 ,X 2 h ( AX 1 + Z 1 , AX 2 + Z 2 | U ) − λh ( BX 1 + Z 1 , BX 2 + Z 2 | U ) = h ( AX 1 + Z 1 | U, AX 2 + Z 2 ) − λh ( BX 1 + Z 1 | U, AX 2 + Z 2 ) + h ( AX 2 + Z 2 | U, BX 1 + Z 1 ) − λh ( BX 2 + Z 2 | U, BX 1 + Z 1 ) − ( λ − 1) I ( AX 2 + Z 2 ; BX 1 + Z 1 | U ) ✷✶

  51. ■♠♣❧✐❡s t❤❛t ❝♦♥❞✐t✐♦♥❡❞ ♦♥ ✿ ❛r❡ ●❛✉ss✐❛♥ ❈❤❛r❛❝t❡r✐③❛t✐♦♥ ♦❢ ●❛✉ss✐❛♥s ✭❇❡r♥st❡✐♥ ✬✹✵s✮ Pr♦♦❢✿ ❯s✐♥❣ ❝❤❛r❛❝t❡r✐st✐❝ ❢✉♥❝t✐♦♥s ✭❋♦✉r✐❡r tr❛♥s❢♦r♠s✮ ❚❤✐s t❡❝❤♥✐q✉❡ ❤❛s ❜❡❡♥ s✉❜s❡q✉❡♥t❧② ✉s❡❞ ❜② ♦t❤❡rs ✐♥ ✈❛r✐♦✉s ♦t❤❡r ✐♥st❛♥❝❡s✳ ◆♦t❡ ✿ ❚❤❡r❡ ❛r❡ s♦♠❡ s✐♠✐❧❛r✐t✐❡s ✇✐t❤ ✇♦r❦ ♦❢ ▲✐❡❜ ❛♥❞ ❇❛rt❤❡ ✭✾✵s✮ ❚❤❡② ❛❧s♦ ✉s❡ r♦t❛t✐♦♥s ✭❜✉t ♥♦t ✐♥❢♦r♠❛t✐♦♥ ♠❡❛s✉r❡s ❛♥❞ ✐ts ❛❧❣❡❜r❛✮ ◆♦t❡ ✿ ❚❤✉s✱ ❝♦♥❞✐t✐♦♥❡❞ ♦♥ ✿ ✭❢r♦♠ ❝♦♥str✉❝t✐♦♥✮ ✭❢r♦♠ ♣r♦♦❢ ♦❢ s✉❜✲❛❞❞✐t✐✈✐t②✮ ●❛✉ss✐❛♥ ♦♣t✐♠❛❧✐t②✿ ❝t❞✳✳ ▲❡t ( U † , X † ) ❜❡ ❛ ♠❛①✐♠✐③❡r✱ ✐✳❡✳ V = max µ X C µ X [ h ( AX + Z ) − λh ( BX + Z )] = h ( AX † + Z | U † ) − λh ( BX † + Z | U † ) . ▲❡t ( X a , U a ) ❛♥❞ ( X b , U b ) ❜❡ ✐✳✐✳❞✳ ❛❝❝♦r❞✐♥❣ t♦ ( U † , X † ) ✳ ✷✷

  52. ■♠♣❧✐❡s t❤❛t ❝♦♥❞✐t✐♦♥❡❞ ♦♥ ✿ ❛r❡ ●❛✉ss✐❛♥ ❈❤❛r❛❝t❡r✐③❛t✐♦♥ ♦❢ ●❛✉ss✐❛♥s ✭❇❡r♥st❡✐♥ ✬✹✵s✮ Pr♦♦❢✿ ❯s✐♥❣ ❝❤❛r❛❝t❡r✐st✐❝ ❢✉♥❝t✐♦♥s ✭❋♦✉r✐❡r tr❛♥s❢♦r♠s✮ ❚❤✐s t❡❝❤♥✐q✉❡ ❤❛s ❜❡❡♥ s✉❜s❡q✉❡♥t❧② ✉s❡❞ ❜② ♦t❤❡rs ✐♥ ✈❛r✐♦✉s ♦t❤❡r ✐♥st❛♥❝❡s✳ ◆♦t❡ ✿ ❚❤❡r❡ ❛r❡ s♦♠❡ s✐♠✐❧❛r✐t✐❡s ✇✐t❤ ✇♦r❦ ♦❢ ▲✐❡❜ ❛♥❞ ❇❛rt❤❡ ✭✾✵s✮ ❚❤❡② ❛❧s♦ ✉s❡ r♦t❛t✐♦♥s ✭❜✉t ♥♦t ✐♥❢♦r♠❛t✐♦♥ ♠❡❛s✉r❡s ❛♥❞ ✐ts ❛❧❣❡❜r❛✮ ❚❤❡r❡❢♦r❡ ✿ ✇❡ ❣❡t t❤❛t ❝♦♥❞✐t✐♦♥❡❞ ♦♥ ✿ ✳ ◆♦t❡ ✿ ❚❤✉s✱ ❝♦♥❞✐t✐♦♥❡❞ ♦♥ ✿ ✭❢r♦♠ ❝♦♥str✉❝t✐♦♥✮ ✭❢r♦♠ ♣r♦♦❢ ♦❢ s✉❜✲❛❞❞✐t✐✈✐t②✮ ●❛✉ss✐❛♥ ♦♣t✐♠❛❧✐t②✿ ❝t❞✳✳ ▲❡t ( U † , X † ) ❜❡ ❛ ♠❛①✐♠✐③❡r✱ ✐✳❡✳ V = max µ X C µ X [ h ( AX + Z ) − λh ( BX + Z )] = h ( AX † + Z | U † ) − λh ( BX † + Z | U † ) . ▲❡t ( X a , U a ) ❛♥❞ ( X b , U b ) ❜❡ ✐✳✐✳❞✳ ❛❝❝♦r❞✐♥❣ t♦ ( U † , X † ) ✳ ❙❡tt✐♥❣ U = ( U a , U b ) ✱ X + = X a + X b ❛♥❞ X − = X a − X b t❤❡ ♣r♦♦❢ ♦❢ s✉❜✲❛❞❞✐t✐✈✐t② √ √ 2 2 ②✐❡❧❞s � � 2 V = C µ X 1 ,X 2 [ h ( AX 1 + Z 1 , AX 2 + Z 2 ) − λh ( BX 1 + Z 1 , BX 2 + Z 2 )] � ( µ X + ,X − ) � � ≤ C µ X 1 [ h ( AX 1 + Z 1 ) − λh ( BX 1 + Z 1 )] � µ X + � � + C µ X 2 [ h ( AX 2 + Z 2 ) − λh ( BX 2 + Z 2 )] � µ X − − ( λ − 1) I ( AX − + Z 2 ; BX + + Z 1 | U a , U b ) ≤ V + V ✷✷

  53. ■♠♣❧✐❡s t❤❛t ❝♦♥❞✐t✐♦♥❡❞ ♦♥ ✿ ❛r❡ ●❛✉ss✐❛♥ ❈❤❛r❛❝t❡r✐③❛t✐♦♥ ♦❢ ●❛✉ss✐❛♥s ✭❇❡r♥st❡✐♥ ✬✹✵s✮ Pr♦♦❢✿ ❯s✐♥❣ ❝❤❛r❛❝t❡r✐st✐❝ ❢✉♥❝t✐♦♥s ✭❋♦✉r✐❡r tr❛♥s❢♦r♠s✮ ❚❤✐s t❡❝❤♥✐q✉❡ ❤❛s ❜❡❡♥ s✉❜s❡q✉❡♥t❧② ✉s❡❞ ❜② ♦t❤❡rs ✐♥ ✈❛r✐♦✉s ♦t❤❡r ✐♥st❛♥❝❡s✳ ◆♦t❡ ✿ ❚❤❡r❡ ❛r❡ s♦♠❡ s✐♠✐❧❛r✐t✐❡s ✇✐t❤ ✇♦r❦ ♦❢ ▲✐❡❜ ❛♥❞ ❇❛rt❤❡ ✭✾✵s✮ ❚❤❡② ❛❧s♦ ✉s❡ r♦t❛t✐♦♥s ✭❜✉t ♥♦t ✐♥❢♦r♠❛t✐♦♥ ♠❡❛s✉r❡s ❛♥❞ ✐ts ❛❧❣❡❜r❛✮ ◆♦t❡ ✿ ❚❤✉s✱ ❝♦♥❞✐t✐♦♥❡❞ ♦♥ ✿ ✭❢r♦♠ ❝♦♥str✉❝t✐♦♥✮ ✭❢r♦♠ ♣r♦♦❢ ♦❢ s✉❜✲❛❞❞✐t✐✈✐t②✮ ●❛✉ss✐❛♥ ♦♣t✐♠❛❧✐t②✿ ❝t❞✳✳ ▲❡t ( U † , X † ) ❜❡ ❛ ♠❛①✐♠✐③❡r✱ ✐✳❡✳ V = max µ X C µ X [ h ( AX + Z ) − λh ( BX + Z )] = h ( AX † + Z | U † ) − λh ( BX † + Z | U † ) . ▲❡t ( X a , U a ) ❛♥❞ ( X b , U b ) ❜❡ ✐✳✐✳❞✳ ❛❝❝♦r❞✐♥❣ t♦ ( U † , X † ) ✳ ❙❡tt✐♥❣ U = ( U a , U b ) ✱ X + = X a + X b ❛♥❞ X − = X a − X b t❤❡ ♣r♦♦❢ ♦❢ s✉❜✲❛❞❞✐t✐✈✐t② √ √ 2 2 ②✐❡❧❞s � � 2 V = C µ X 1 ,X 2 [ h ( AX 1 + Z 1 , AX 2 + Z 2 ) − λh ( BX 1 + Z 1 , BX 2 + Z 2 )] � ( µ X + ,X − ) � � ≤ C µ X 1 [ h ( AX 1 + Z 1 ) − λh ( BX 1 + Z 1 )] � µ X + � � + C µ X 2 [ h ( AX 2 + Z 2 ) − λh ( BX 2 + Z 2 )] � µ X − − ( λ − 1) I ( AX − + Z 2 ; BX + + Z 1 | U a , U b ) ≤ V + V ❚❤❡r❡❢♦r❡ ✿ ✇❡ ❣❡t t❤❛t ❝♦♥❞✐t✐♦♥❡❞ ♦♥ ( U a , U b ) ✿ X + ⊥ X − ✳ ✷✷

  54. ■♠♣❧✐❡s t❤❛t ❝♦♥❞✐t✐♦♥❡❞ ♦♥ ✿ ❛r❡ ●❛✉ss✐❛♥ ❈❤❛r❛❝t❡r✐③❛t✐♦♥ ♦❢ ●❛✉ss✐❛♥s ✭❇❡r♥st❡✐♥ ✬✹✵s✮ Pr♦♦❢✿ ❯s✐♥❣ ❝❤❛r❛❝t❡r✐st✐❝ ❢✉♥❝t✐♦♥s ✭❋♦✉r✐❡r tr❛♥s❢♦r♠s✮ ❚❤✐s t❡❝❤♥✐q✉❡ ❤❛s ❜❡❡♥ s✉❜s❡q✉❡♥t❧② ✉s❡❞ ❜② ♦t❤❡rs ✐♥ ✈❛r✐♦✉s ♦t❤❡r ✐♥st❛♥❝❡s✳ ◆♦t❡ ✿ ❚❤❡r❡ ❛r❡ s♦♠❡ s✐♠✐❧❛r✐t✐❡s ✇✐t❤ ✇♦r❦ ♦❢ ▲✐❡❜ ❛♥❞ ❇❛rt❤❡ ✭✾✵s✮ ❚❤❡② ❛❧s♦ ✉s❡ r♦t❛t✐♦♥s ✭❜✉t ♥♦t ✐♥❢♦r♠❛t✐♦♥ ♠❡❛s✉r❡s ❛♥❞ ✐ts ❛❧❣❡❜r❛✮ ●❛✉ss✐❛♥ ♦♣t✐♠❛❧✐t②✿ ❝t❞✳✳ ▲❡t ( U † , X † ) ❜❡ ❛ ♠❛①✐♠✐③❡r✱ ✐✳❡✳ V = max µ X C µ X [ h ( AX + Z ) − λh ( BX + Z )] = h ( AX † + Z | U † ) − λh ( BX † + Z | U † ) . ▲❡t ( X a , U a ) ❛♥❞ ( X b , U b ) ❜❡ ✐✳✐✳❞✳ ❛❝❝♦r❞✐♥❣ t♦ ( U † , X † ) ✳ ◆♦t❡ ✿ ❚❤✉s✱ ❝♦♥❞✐t✐♦♥❡❞ ♦♥ ( U a , U b ) ✿ • X a ⊥ X b ✭❢r♦♠ ❝♦♥str✉❝t✐♦♥✮ • ( X a + X b ) ⊥ ( X a − X b ) ✭❢r♦♠ ♣r♦♦❢ ♦❢ s✉❜✲❛❞❞✐t✐✈✐t②✮ ✷✷

  55. ❚❤✐s t❡❝❤♥✐q✉❡ ❤❛s ❜❡❡♥ s✉❜s❡q✉❡♥t❧② ✉s❡❞ ❜② ♦t❤❡rs ✐♥ ✈❛r✐♦✉s ♦t❤❡r ✐♥st❛♥❝❡s✳ ◆♦t❡ ✿ ❚❤❡r❡ ❛r❡ s♦♠❡ s✐♠✐❧❛r✐t✐❡s ✇✐t❤ ✇♦r❦ ♦❢ ▲✐❡❜ ❛♥❞ ❇❛rt❤❡ ✭✾✵s✮ ❚❤❡② ❛❧s♦ ✉s❡ r♦t❛t✐♦♥s ✭❜✉t ♥♦t ✐♥❢♦r♠❛t✐♦♥ ♠❡❛s✉r❡s ❛♥❞ ✐ts ❛❧❣❡❜r❛✮ ●❛✉ss✐❛♥ ♦♣t✐♠❛❧✐t②✿ ❝t❞✳✳ ▲❡t ( U † , X † ) ❜❡ ❛ ♠❛①✐♠✐③❡r✱ ✐✳❡✳ V = max µ X C µ X [ h ( AX + Z ) − λh ( BX + Z )] = h ( AX † + Z | U † ) − λh ( BX † + Z | U † ) . ▲❡t ( X a , U a ) ❛♥❞ ( X b , U b ) ❜❡ ✐✳✐✳❞✳ ❛❝❝♦r❞✐♥❣ t♦ ( U † , X † ) ✳ ◆♦t❡ ✿ ❚❤✉s✱ ❝♦♥❞✐t✐♦♥❡❞ ♦♥ ( U a , U b ) ✿ • X a ⊥ X b ✭❢r♦♠ ❝♦♥str✉❝t✐♦♥✮ • ( X a + X b ) ⊥ ( X a − X b ) ✭❢r♦♠ ♣r♦♦❢ ♦❢ s✉❜✲❛❞❞✐t✐✈✐t②✮ • ■♠♣❧✐❡s t❤❛t ❝♦♥❞✐t✐♦♥❡❞ ♦♥ ( U a , U b ) ✿ X a , X b ❛r❡ ●❛✉ss✐❛♥ ⋆ ❈❤❛r❛❝t❡r✐③❛t✐♦♥ ♦❢ ●❛✉ss✐❛♥s ✭❇❡r♥st❡✐♥ ✬✹✵s✮ ⋆ Pr♦♦❢✿ ❯s✐♥❣ ❝❤❛r❛❝t❡r✐st✐❝ ❢✉♥❝t✐♦♥s ✭❋♦✉r✐❡r tr❛♥s❢♦r♠s✮ ✷✷

  56. ●❛✉ss✐❛♥ ♦♣t✐♠❛❧✐t②✿ ❝t❞✳✳ ▲❡t ( U † , X † ) ❜❡ ❛ ♠❛①✐♠✐③❡r✱ ✐✳❡✳ V = max µ X C µ X [ h ( AX + Z ) − λh ( BX + Z )] = h ( AX † + Z | U † ) − λh ( BX † + Z | U † ) . ▲❡t ( X a , U a ) ❛♥❞ ( X b , U b ) ❜❡ ✐✳✐✳❞✳ ❛❝❝♦r❞✐♥❣ t♦ ( U † , X † ) ✳ ◆♦t❡ ✿ ❚❤✉s✱ ❝♦♥❞✐t✐♦♥❡❞ ♦♥ ( U a , U b ) ✿ • X a ⊥ X b ✭❢r♦♠ ❝♦♥str✉❝t✐♦♥✮ • ( X a + X b ) ⊥ ( X a − X b ) ✭❢r♦♠ ♣r♦♦❢ ♦❢ s✉❜✲❛❞❞✐t✐✈✐t②✮ • ■♠♣❧✐❡s t❤❛t ❝♦♥❞✐t✐♦♥❡❞ ♦♥ ( U a , U b ) ✿ X a , X b ❛r❡ ●❛✉ss✐❛♥ ⋆ ❈❤❛r❛❝t❡r✐③❛t✐♦♥ ♦❢ ●❛✉ss✐❛♥s ✭❇❡r♥st❡✐♥ ✬✹✵s✮ ⋆ Pr♦♦❢✿ ❯s✐♥❣ ❝❤❛r❛❝t❡r✐st✐❝ ❢✉♥❝t✐♦♥s ✭❋♦✉r✐❡r tr❛♥s❢♦r♠s✮ ❚❤✐s t❡❝❤♥✐q✉❡ ❤❛s ❜❡❡♥ s✉❜s❡q✉❡♥t❧② ✉s❡❞ ❜② ♦t❤❡rs ✐♥ ✈❛r✐♦✉s ♦t❤❡r ✐♥st❛♥❝❡s✳ ◆♦t❡ ✿ ❚❤❡r❡ ❛r❡ s♦♠❡ s✐♠✐❧❛r✐t✐❡s ✇✐t❤ ✇♦r❦ ♦❢ ▲✐❡❜ ❛♥❞ ❇❛rt❤❡ ✭✾✵s✮ ❚❤❡② ❛❧s♦ ✉s❡ r♦t❛t✐♦♥s ✭❜✉t ♥♦t ✐♥❢♦r♠❛t✐♦♥ ♠❡❛s✉r❡s ❛♥❞ ✐ts ❛❧❣❡❜r❛✮ ✷✷

  57. ❆✣r♠❛t✐✈❡ ✐❢ t❤❡ ❢♦❧❧♦✇✐♥❣ ❢✉♥❝t✐♦♥❛❧ s✉❜✲❛❞❞✐t✐✈❡❄ ❲❤② s❤♦✉❧❞ s♦♠❡♦♥❡ ❝❛r❡❄ ■❢ tr✉❡✱ s♦❧✈❡s t❤❡ ❝❛♣❛❝✐t② r❡❣✐♦♥ ❢♦r t❤❡ ●❛✉ss✐❛♥ ❩✲✐♥t❡r❢❡r❡♥❝❡ ❝❤❛♥♥❡❧ ❘❡❧❛t❡❞ t♦ r❡✈❡rs❡ ❊P■s✱ ❤②♣❡r♣❧❛♥❡ ❝♦♥❥❡❝t✉r❡✱ ❡t❝✳ ❆♥ ♦♣❡♥ q✉❡st✐♦♥ ❲❡ ❤❛✈❡ s❡❡♥ ✭②❡st❡r❞❛② ❛♥❞ t♦❞❛②✮ ❤♦✇ s✉❜✲❛❞❞✐t✐✈✐t② ✐♠♣❧✐❡s ●❛✉ss✐❛♥ ♦♣t✐♠❛❧✐t② ❖♣❡♥ q✉❡st✐♦♥ ❋♦r α, a ∈ (0 , 1) ✱ ❞♦ ●❛✉ss✐❛♥s ♠❛①✐♠✐③❡ t❤❡ ❢✉♥❝t✐♦♥❛❧ αh ( X 2 + aX 1 + Z ) + (1 − α ) h ( X 1 + Z ) − h ( aX 1 + Z ) ♦✈❡r X 1 ⊥ X 2 ✱ s✉❜❥❡❝t t♦ E( X 2 1 ) ≤ P 1 ✱ E( X 2 2 ) ≤ P 2 ✳ ❍❡r❡ Z ∼ N (0 , 1) ✐s ✐♥❞❡♣❡♥❞❡♥t ♦❢ X 1 , X 2 ✳ ✷✸

  58. ❲❤② s❤♦✉❧❞ s♦♠❡♦♥❡ ❝❛r❡❄ ■❢ tr✉❡✱ s♦❧✈❡s t❤❡ ❝❛♣❛❝✐t② r❡❣✐♦♥ ❢♦r t❤❡ ●❛✉ss✐❛♥ ❩✲✐♥t❡r❢❡r❡♥❝❡ ❝❤❛♥♥❡❧ ❘❡❧❛t❡❞ t♦ r❡✈❡rs❡ ❊P■s✱ ❤②♣❡r♣❧❛♥❡ ❝♦♥❥❡❝t✉r❡✱ ❡t❝✳ ❆♥ ♦♣❡♥ q✉❡st✐♦♥ ❲❡ ❤❛✈❡ s❡❡♥ ✭②❡st❡r❞❛② ❛♥❞ t♦❞❛②✮ ❤♦✇ s✉❜✲❛❞❞✐t✐✈✐t② ✐♠♣❧✐❡s ●❛✉ss✐❛♥ ♦♣t✐♠❛❧✐t② ❖♣❡♥ q✉❡st✐♦♥ ❋♦r α, a ∈ (0 , 1) ✱ ❞♦ ●❛✉ss✐❛♥s ♠❛①✐♠✐③❡ t❤❡ ❢✉♥❝t✐♦♥❛❧ αh ( X 2 + aX 1 + Z ) + (1 − α ) h ( X 1 + Z ) − h ( aX 1 + Z ) ♦✈❡r X 1 ⊥ X 2 ✱ s✉❜❥❡❝t t♦ E( X 2 1 ) ≤ P 1 ✱ E( X 2 2 ) ≤ P 2 ✳ ❍❡r❡ Z ∼ N (0 , 1) ✐s ✐♥❞❡♣❡♥❞❡♥t ♦❢ X 1 , X 2 ✳ ❆✣r♠❛t✐✈❡ ✐❢ t❤❡ ❢♦❧❧♦✇✐♥❣ ❢✉♥❝t✐♦♥❛❧ s✉❜✲❛❞❞✐t✐✈❡❄ � � C µ X 1 αh ( X 2 + aX 1 + Z ) + (1 − α ) h ( X 1 + Z ) − h ( aX 1 + Z ) ✷✸

  59. ❆♥ ♦♣❡♥ q✉❡st✐♦♥ ❲❡ ❤❛✈❡ s❡❡♥ ✭②❡st❡r❞❛② ❛♥❞ t♦❞❛②✮ ❤♦✇ s✉❜✲❛❞❞✐t✐✈✐t② ✐♠♣❧✐❡s ●❛✉ss✐❛♥ ♦♣t✐♠❛❧✐t② ❖♣❡♥ q✉❡st✐♦♥ ❋♦r α, a ∈ (0 , 1) ✱ ❞♦ ●❛✉ss✐❛♥s ♠❛①✐♠✐③❡ t❤❡ ❢✉♥❝t✐♦♥❛❧ αh ( X 2 + aX 1 + Z ) + (1 − α ) h ( X 1 + Z ) − h ( aX 1 + Z ) ♦✈❡r X 1 ⊥ X 2 ✱ s✉❜❥❡❝t t♦ E( X 2 1 ) ≤ P 1 ✱ E( X 2 2 ) ≤ P 2 ✳ ❍❡r❡ Z ∼ N (0 , 1) ✐s ✐♥❞❡♣❡♥❞❡♥t ♦❢ X 1 , X 2 ✳ ❆✣r♠❛t✐✈❡ ✐❢ t❤❡ ❢♦❧❧♦✇✐♥❣ ❢✉♥❝t✐♦♥❛❧ s✉❜✲❛❞❞✐t✐✈❡❄ � � C µ X 1 αh ( X 2 + aX 1 + Z ) + (1 − α ) h ( X 1 + Z ) − h ( aX 1 + Z ) ❲❤② s❤♦✉❧❞ s♦♠❡♦♥❡ ❝❛r❡❄ • ■❢ tr✉❡✱ s♦❧✈❡s t❤❡ ❝❛♣❛❝✐t② r❡❣✐♦♥ ❢♦r t❤❡ ●❛✉ss✐❛♥ ❩✲✐♥t❡r❢❡r❡♥❝❡ ❝❤❛♥♥❡❧ • ❘❡❧❛t❡❞ t♦ r❡✈❡rs❡ ❊P■s✱ ❤②♣❡r♣❧❛♥❡ ❝♦♥❥❡❝t✉r❡✱ ❡t❝✳ ✷✸

  60. ❖✉t❧✐♥❡ • ❇r♦❛❞❝❛st ❝❤❛♥♥❡❧✿ ❊st❛❜❧✐s❤✐♥❣ ♦♣t✐♠❛❧✐t② ♦❢ ▼❛rt♦♥✬s ❢♦r ▼■▼❖ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧ • ■♥t❡r❢❡r❡♥❝❡ ❝❤❛♥♥❡❧✿ ❙✉❜✲♦♣t✐♠❛❧✐t② ♦❢ t❤❡ ❍❛♥✕❑♦❜❛②❛s❤✐ r❡❣✐♦♥ • ❋❛♠✐❧② ♦❢ ♥♦♥✲❝♦♥✈❡① ♦♣t✐♠✐③❛t✐♦♥ ♣r♦❜❧❡♠s ⋆ ❘❡❧❛t✐♦♥ t♦ ♣r♦❜❧❡♠s ♦❢ ✐♥t❡r❡st ✐♥ ♦t❤❡r ✜❡❧❞s ⋆ ❯♥✐❢②✐♥❣ ♦❜s❡r✈❛t✐♦♥s ❛♥❞ s♦♠❡ ❝♦♥❥❡❝t✉r❡s ✷✹

  61. ■♥t❡r❢❡r❡♥❝❡ ❈❤❛♥♥❡❧ ✭❆❤❧s✇❡❞❡ ✬✼✹✮ X n Y n 1 1 W a ( y 1 | x 1 , x 2 ) ˆ M 1 ❊♥❝♦❞❡r ✶ ❉❡❝♦❞❡r ✶ M 1 Y n 2 ˆ W b ( y 2 | x 1 , x 2 ) M 2 ❊♥❝♦❞❡r ✷ ❉❡❝♦❞❡r ✷ M 2 X n 2 ✷✺

  62. ❋✐rst st❡♣✿ ❋✐♥❞ ❛ ❝❤❛♥♥❡❧ ❝❧❛ss ✇❤❡r❡ ❍❑ r❡❣✐♦♥ s✐♠♣❧✐✜❡s ❆◆❉ ②❡t ♥♦t t♦♦ tr✐✈✐❛❧ ❍❛♥✲❑♦❜❛②❛s❤✐ ❛❝❤✐❡✈❛❜❧❡ r❡❣✐♦♥ ✭✶✾✽✶✮ á ❧❛ ❈❤♦♥❣ ❡t✳ ❛❧✳ ❆ r❛t❡✲♣❛✐r ( R 1 , R 2 ) ✐s ❛❝❤✐❡✈❛❜❧❡ ❢♦r t❤❡ ✐♥t❡r❢❡r❡♥❝❡ ❝❤❛♥♥❡❧ ✐❢ R 1 < I ( X 1 ; Y 1 | U 2 , Q ) , R 2 < I ( X 2 ; Y 2 | U 1 , Q ) , R 1 + R 2 < I ( X 1 , U 2 ; Y 1 | Q ) + I ( X 2 ; Y 2 | U 1 , U 2 , Q ) , R 1 + R 2 < I ( X 2 , U 1 ; Y 2 | Q ) + I ( X 1 ; Y 1 | U 1 , U 2 , Q ) , R 1 + R 2 < I ( X 1 , U 2 ; Y 1 | U 1 , Q ) + I ( X 2 , U 1 ; Y 2 | U 2 , Q ) , 2 R 1 + R 2 < I ( X 1 , U 2 ; Y 1 | Q ) + I ( X 1 ; Y 1 | U 1 , U 2 , Q ) + I ( X 2 , U 1 ; Y 2 | U 2 , Q ) , R 1 + 2 R 2 < I ( X 2 , U 1 ; Y 2 | Q ) + I ( X 2 ; Y 2 | U 1 , U 2 , Q ) + I ( X 1 , U 2 ; Y 1 | U 1 , Q ) ❢♦r s♦♠❡ ♣♠❢ p ( q ) p ( u 1 , x 1 | q ) p ( u 2 , x 2 | q ) ✱ ✇❤❡r❡ | U 1 | ≤ | X 1 | + 4 ✱ | U 2 | ≤ | X 2 | + 4 ✱ ❛♥❞ | Q | ≤ 7 ✳ ❉❡♥♦t❡ t❤❡ ✭❝❧♦s✉r❡ ♦❢✮ r❡❣✐♦♥ ❛s R ( W a , W b ) ✳ ◆✉♠❡r✐❝❛❧❧② ✐♥❢❡❛s✐❜❧❡ t♦ ❝♦♠♣✉t❡ R ( W a W b ) ❡✈❡♥ ❢♦r ❣❡♥❡r✐❝ ❜✐♥❛r②✲✐♥♣✉t ❜✐♥❛r②✲♦✉t♣✉t ✐♥t❡r❢❡r❡♥❝❡ ❝❤❛♥♥❡❧s ✷✻

  63. ❍❛♥✲❑♦❜❛②❛s❤✐ ❛❝❤✐❡✈❛❜❧❡ r❡❣✐♦♥ ✭✶✾✽✶✮ á ❧❛ ❈❤♦♥❣ ❡t✳ ❛❧✳ ❆ r❛t❡✲♣❛✐r ( R 1 , R 2 ) ✐s ❛❝❤✐❡✈❛❜❧❡ ❢♦r t❤❡ ✐♥t❡r❢❡r❡♥❝❡ ❝❤❛♥♥❡❧ ✐❢ R 1 < I ( X 1 ; Y 1 | U 2 , Q ) , R 2 < I ( X 2 ; Y 2 | U 1 , Q ) , R 1 + R 2 < I ( X 1 , U 2 ; Y 1 | Q ) + I ( X 2 ; Y 2 | U 1 , U 2 , Q ) , R 1 + R 2 < I ( X 2 , U 1 ; Y 2 | Q ) + I ( X 1 ; Y 1 | U 1 , U 2 , Q ) , R 1 + R 2 < I ( X 1 , U 2 ; Y 1 | U 1 , Q ) + I ( X 2 , U 1 ; Y 2 | U 2 , Q ) , 2 R 1 + R 2 < I ( X 1 , U 2 ; Y 1 | Q ) + I ( X 1 ; Y 1 | U 1 , U 2 , Q ) + I ( X 2 , U 1 ; Y 2 | U 2 , Q ) , R 1 + 2 R 2 < I ( X 2 , U 1 ; Y 2 | Q ) + I ( X 2 ; Y 2 | U 1 , U 2 , Q ) + I ( X 1 , U 2 ; Y 1 | U 1 , Q ) ❢♦r s♦♠❡ ♣♠❢ p ( q ) p ( u 1 , x 1 | q ) p ( u 2 , x 2 | q ) ✱ ✇❤❡r❡ | U 1 | ≤ | X 1 | + 4 ✱ | U 2 | ≤ | X 2 | + 4 ✱ ❛♥❞ | Q | ≤ 7 ✳ ❉❡♥♦t❡ t❤❡ ✭❝❧♦s✉r❡ ♦❢✮ r❡❣✐♦♥ ❛s R ( W a , W b ) ✳ ◆✉♠❡r✐❝❛❧❧② ✐♥❢❡❛s✐❜❧❡ t♦ ❝♦♠♣✉t❡ R ( W a W b ) ❡✈❡♥ ❢♦r ❣❡♥❡r✐❝ ❜✐♥❛r②✲✐♥♣✉t ❜✐♥❛r②✲♦✉t♣✉t ✐♥t❡r❢❡r❡♥❝❡ ❝❤❛♥♥❡❧s ❋✐rst st❡♣✿ • ❋✐♥❞ ❛ ❝❤❛♥♥❡❧ ❝❧❛ss ✇❤❡r❡ ❍❑ r❡❣✐♦♥ s✐♠♣❧✐✜❡s ❆◆❉ ②❡t ♥♦t t♦♦ tr✐✈✐❛❧ ✷✻

  64. ❈❧❡❛♥ ❩ ■♥t❡r❢❡r❡♥❝❡ ❈❤❛♥♥❡❧ ✭❈❩■❈✮ ▼♦❞❡❧ X n Y n 1 1 ˆ M 1 W a ( y 1 | x 1 , x 2 ) ❊♥❝♦❞❡r ✶ ❉❡❝♦❞❡r ✶ M 1 Y n 2 = X n 2 ˆ M 2 ❊♥❝♦❞❡r ✷ ❉❡❝♦❞❡r ✷ M 2 X n 2 ❈❧❡❛♥ ❩✲✐♥t❡r❢❡r❡♥❝❡ ❝❤❛♥♥❡❧ ▲❡♠♠❛ ✿ ❆ r❛t❡✲♣❛✐r ( R 1 , R 2 ) ❜❡❧♦♥❣s t♦ ❍❛♥✲❑♦❜❛②❛s❤✐ r❡❣✐♦♥ ✐❢ ❛♥❞ ♦♥❧② ✐❢ R 1 < I ( X 1 ; Y 1 | U 2 , Q ) , R 2 < H ( X 2 | Q ) , R 1 + R 2 < I ( X 1 , U 2 ; Y 1 | Q ) + H ( X 2 | U 2 , Q ) , ❢♦r s♦♠❡ ♣♠❢ p ( q ) p ( x 1 | q ) p ( u 2 , x 2 | q ) ✱ ✇❤❡r❡ | U 2 | ≤ | X 2 | ❛♥❞ | Q | ≤ 2 ✳ ❉❡♥♦t❡ r❡❣✐♦♥✿ R ( W a ) ✷✼

  65. ▲❡♠♠❛ ✭s✉❜✲❛❞❞✐t✐✈✐t②✮ ✭◆❛✐r✲❳✐❛✲❨❛③❞❛♥♣❛♥❛❤ ✬✶✺✮✿ ■♠♣❧✐❡s ♦♣t✐♠❛❧✐t② ♦❢ ✱ ✳ ❋♦r ✱ ✐s ❣✐✈❡♥ ❜② ❚❡st✐♥❣ ♦♣t✐♠❛❧✐t② ❊q✉✐✈❛❧❡♥t t♦ t❡st ✐❢ S λ ( W a ⊗ W a ) = 2 S λ ( W a ) , ∀ W a , λ ≥ 0 , ✇❤❡r❡ S λ ( W a ) := ( R 1 ,R 2 ) ∈R ( W a ) λR 1 + R 2 . max ✷✽

  66. ▲❡♠♠❛ ✭s✉❜✲❛❞❞✐t✐✈✐t②✮ ✭◆❛✐r✲❳✐❛✲❨❛③❞❛♥♣❛♥❛❤ ✬✶✺✮✿ ■♠♣❧✐❡s ♦♣t✐♠❛❧✐t② ♦❢ ✱ ✳ ❚❡st✐♥❣ ♦♣t✐♠❛❧✐t② ❊q✉✐✈❛❧❡♥t t♦ t❡st ✐❢ S λ ( W a ⊗ W a ) = 2 S λ ( W a ) , ∀ W a , λ ≥ 0 , ✇❤❡r❡ S λ ( W a ) := ( R 1 ,R 2 ) ∈R ( W a ) λR 1 + R 2 . max ❋♦r λ ∈ [0 , 1] ✱ S λ ( W a ) ✐s ❣✐✈❡♥ ❜② � � (1 − λ ) H ( X 2 ) + λI ( X 1 , U 2 ; Y 1 ) + λH ( X 2 | U 2 ) max p 1 ( x 1 ) p 2 ( u 2 ,x 2 ) � � = max H ( X 2 ) + λI ( X 1 ; Y 1 ) p 1 ( x 1 ) p 2 ( x 2 ) ✷✽

  67. ❚❡st✐♥❣ ♦♣t✐♠❛❧✐t② ❊q✉✐✈❛❧❡♥t t♦ t❡st ✐❢ S λ ( W a ⊗ W a ) = 2 S λ ( W a ) , ∀ W a , λ ≥ 0 , ✇❤❡r❡ S λ ( W a ) := ( R 1 ,R 2 ) ∈R ( W a ) λR 1 + R 2 . max ❋♦r λ ∈ [0 , 1] ✱ S λ ( W a ) ✐s ❣✐✈❡♥ ❜② � � (1 − λ ) H ( X 2 ) + λI ( X 1 , U 2 ; Y 1 ) + λH ( X 2 | U 2 ) max p 1 ( x 1 ) p 2 ( u 2 ,x 2 ) � � = max H ( X 2 ) + λI ( X 1 ; Y 1 ) p 1 ( x 1 ) p 2 ( x 2 ) ▲❡♠♠❛ ✭s✉❜✲❛❞❞✐t✐✈✐t②✮ ✭◆❛✐r✲❳✐❛✲❨❛③❞❛♥♣❛♥❛❤ ✬✶✺✮✿ H ( X 21 , X 22 ) + λI ( X 11 , X 12 ; Y 11 , Y 12 ) � � � � ≤ H ( X 21 ) + λI ( X 11 ; Y 11 ) + H ( X 22 ) + λI ( X 12 ; Y 12 ) − (1 − λ ) I ( X 21 ; X 22 ) . ■♠♣❧✐❡s ♦♣t✐♠❛❧✐t② ♦❢ S λ ( W a ) ✱ λ ∈ [0 , 1] ✳ ✷✽

  68. ◗✉❡st✐♦♥ ✿ ❈❛♥ ✇❡ ♥✉♠❡r✐❝❛❧❧② t❡st ✐❢ ❄ ✐s ❛ ❜✐♥❛r② r❛♥❞♦♠ ✈❛r✐❛❜❧❡ ✭✐✳❡✳ ❝♦♥❝❛✈❡ ❡♥✈❡❧♦♣❡ ♦✈❡r s✐♥❣❧❡ ✈❛r✐❛❜❧❡✮ ✿ ❤❛s ❛t ♠♦st ✷ ✐♥✢❡①✐♦♥ ♣♦✐♥ts ❲❤❛t ❛❜♦✉t λ > 1 ❄ ❋♦r λ ≥ 1 ✱ S λ ( W a ) ✐s ❣✐✈❡♥ ❜② � � I ( X 1 , U 2 ; Y 1 ) + H ( X 2 | U 2 ) + ( λ − 1) I ( X 1 ; Y 1 | U 2 ) max p 1 ( x 1 ) p 2 ( u 2 ,x 2 ) � � I ( X 1 , X 2 ; Y 1 ) + C X 2 [( λ − 1) I ( X 1 ; Y 1 ) + H ( X 2 ) − I ( X 2 ; Y 1 | X 1 )] = max p 1 ( x 1 ) p 2 ( x 2 ) ✷✾

  69. ❲❤❛t ❛❜♦✉t λ > 1 ❄ ❋♦r λ ≥ 1 ✱ S λ ( W a ) ✐s ❣✐✈❡♥ ❜② � � I ( X 1 , U 2 ; Y 1 ) + H ( X 2 | U 2 ) + ( λ − 1) I ( X 1 ; Y 1 | U 2 ) max p 1 ( x 1 ) p 2 ( u 2 ,x 2 ) � � I ( X 1 , X 2 ; Y 1 ) + C X 2 [( λ − 1) I ( X 1 ; Y 1 ) + H ( X 2 ) − I ( X 2 ; Y 1 | X 1 )] = max p 1 ( x 1 ) p 2 ( x 2 ) ◗✉❡st✐♦♥ ✿ ❈❛♥ ✇❡ ♥✉♠❡r✐❝❛❧❧② t❡st ✐❢ S λ ( W a ⊗ W a ) = 2 S λ ( W a ) ❄ X 2 ✐s ❛ ❜✐♥❛r② r❛♥❞♦♠ ✈❛r✐❛❜❧❡ ✭✐✳❡✳ ❝♦♥❝❛✈❡ ❡♥✈❡❧♦♣❡ ♦✈❡r s✐♥❣❧❡ ✈❛r✐❛❜❧❡✮ ( λ − 1) I ( X 1 ; Y 1 ) + H ( X 2 ) − I ( X 2 ; Y 1 | X 1 ) ✿ ❤❛s ❛t ♠♦st ✷ ✐♥✢❡①✐♦♥ ♣♦✐♥ts ✷✾

  70. ❲❤❛t ❛❜♦✉t λ > 1 ❄ H ( X 2 ) + ( λ − 1) H ( Y 1 ) − λH ( Y 1 | X 1 ) P 2 ( X 2 ) [ H ( X 2 ) + ( λ − 1) H ( Y 1 ) − λH ( Y 1 | X 1 )] C P ( X 2 ) ❚❤❡ s❤❛♣❡ ♦❢ ❝♦♥❝❛✈❡ ❡♥✈❡❧♦♣❡ ❢♦r ❛ ❣❡♥❡r✐❝ ❜✐♥❛r② ❈❩■❈ ✷✾

  71. ❙✉❜✲♦♣t✐♠❛❧✐t② ♦❢ t❤❡ ❍❛♥✲❑♦❜❛②❛s❤✐ r❡❣✐♦♥ A ❍❑ 1 2 A ❚■◆ ( W ⊗ 2 ) W ( Y 1 = 0 | X 1 , X 2 ) λ ( W ) λ λ � 1 � 0 . 5 ✷ ✶✳✶✵✼✺✶✻ ✶✳✶✵✽✶✹✶ 1 0 � 0 . 12 � 0 . 89 ✾ ✶✳✵✼✹✹✽✹ ✶✳✵✼✺✺✹✹ 0 . 21 0 . 62 � 0 . 01 � 0 . 58 ✶✷ ✶✳✷✽✾✽✸✵ ✶✳✷✾✸✼✻✵ 0 . 20 0 . 74 � 0 . 78 � 0 . 07 ✶✹ ✶✳✹✷✻✺✷✻ ✶✳✹✸✷✹✶✾ 0 . 46 0 . 05 � 0 . 91 � 0 . 22 ✶✺ ✶✳✸✷✸✼✻✻ ✶✳✸✸✾✵✻✺ 0 . 66 0 . 15 � 0 . 91 � 0 . 13 ✶✻ ✶✳✺✶✺✹✷✶ ✶✳✺✸✹✼✷✹ 0 . 62 0 . 06 � 0 . 38 � 0 . 87 ✶✽ ✶✳✹✹✾✾✺✾ ✶✳✹✻✽✺✼✼ 0 . 12 0 . 79 ❈♦✉♥t❡r❡①❛♠♣❧❡s t♦ t❤❡ ♦♣t✐♠❛❧✐t② ♦❢ ❍❛♥✲❑♦❜❛②❛s❤✐ r❡❣✐♦♥✳ ◆♦t❡ ✿ ❋♦r t❤❡ ✜rst ❡①❛♠♣❧❡✱ ✇❡ ❝❛♥ ❝❛❧❝✉❧❛t❡ t❤❡ ❝♦♥❝❛✈❡ ❡♥✈❡❧♦♣❡ ❛♥❛❧②t✐❝❛❧❧②✳ ✸✵

  72. ▲❡t ❛♥❞ r❡s♣❡❝t✐✈❡❧② ❞❡♥♦t❡ ❛♥❞ ✇❤❡r❡ ✐s t❤❡ ❜✐♥❛r② ❡♥tr♦♣② ❢✉♥❝t✐♦♥ P❛rt✐❝✉❧❛r ❈❤❛♥♥❡❧ 1 2 0 0 0 0 X 1 Y 1 X 1 Y 1 1 2 1 1 1 1 X 2 = 0 X 2 = 1 • ❲❡ ❝♦♠♣✉t❡ max HK λR 1 + R 2 ❢♦r λ = 2 � �� � max H ( Y 1 ) + H ( X 2 ) + 2 H ( Y 1 ) − H ( Y 1 | X 1 ) C p 1 ( x 1 ) p 2 ( x 2 ) p 2 ( x 2 ) ✸✶

  73. P❛rt✐❝✉❧❛r ❈❤❛♥♥❡❧ 1 2 0 0 0 0 X 1 Y 1 X 1 Y 1 1 2 1 1 1 1 X 2 = 0 X 2 = 1 • ❲❡ ❝♦♠♣✉t❡ max HK λR 1 + R 2 ❢♦r λ = 2 f ( p, q ) � �� � max H ( Y 1 ) + H ( X 2 ) + 2 H ( Y 1 ) − H ( Y 1 | X 1 ) C p 1 ( x 1 ) p 2 ( x 2 ) p 2 ( x 2 ) • ▲❡t p ❛♥❞ q r❡s♣❡❝t✐✈❡❧② ❞❡♥♦t❡ Pr ( X 1 = 0) ❛♥❞ Pr ( X 2 = 0) q ) − 2 ph b ( q + 1 p ) h b ( q ) + h b ( q + p f ( p, q ) = (1 − 2¯ 2 ¯ ) 2 ✇❤❡r❡ h b ( . ) ✐s t❤❡ ❜✐♥❛r② ❡♥tr♦♣② ❢✉♥❝t✐♦♥ ✸✶

  74. P❛rt✐❝✉❧❛r ❝❤❛♥♥❡❧ ❝♦♥t✐♥✉❡❞ f ( p, q ) ✐s ❝♦♥❝❛✈❡ ✐♥ q ❢♦r p ≥ 1 2 ❛♥❞ ❢♦r 0 ≤ p < 1 2  f ( p, q ) q > 1 − 2 p  q [ f ( p, q )] = C f ( p, 1 − 2 p ) − f ( p, 0) q + f ( p, 0) q ∈ [0 , 1 − 2 p ]  1 − 2 p ✸✷

  75. P❛rt✐❝✉❧❛r ❝❤❛♥♥❡❧ ❝♦♥t✐♥✉❡❞ f ( p, q ) ✐s ❝♦♥❝❛✈❡ ✐♥ q ❢♦r p ≥ 1 2 ❛♥❞ ❢♦r 0 ≤ p < 1 2  f ( p, q ) q > 1 − 2 p  q [ f ( p, q )] = C f ( p, 1 − 2 p ) − f ( p, 0) q + f ( p, 0) q ∈ [0 , 1 − 2 p ]  1 − 2 p ✵✳✸ ✵✳✷✺ ✵✳✷ ✵✳✶✺ ✵✳✶ f (0 . 2 , q ) ✵✳✵✺ C q [ f (0 . 2 , q )] q ✵ 0 0 . 1 0 . 2 0 . 3 0 . 4 0 . 5 0 . 6 0 . 7 0 . 8 0 . 9 1 ✸✷

  76. P❛rt✐❝✉❧❛r ❝❤❛♥♥❡❧ ❝♦♥t✐♥✉❡❞ f ( p, q ) ✐s ❝♦♥❝❛✈❡ ✐♥ q ❢♦r p ≥ 1 2 ❛♥❞ ❢♦r 0 ≤ p < 1 2  f ( p, q ) q > 1 − 2 p  q [ f ( p, q )] = C f ( p, 1 − 2 p ) − f ( p, 0) q + f ( p, 0) q ∈ [0 , 1 − 2 p ]  1 − 2 p ❈♦r♦❧❧❛r② ▼❛①✐♠✉♠ ♦❢ 2 R 1 + R 2 ❢♦r t❤❡ ❍❛♥✕❑♦❜❛②❛s❤✐ r❡❣✐♦♥ ✐s ❡q✉❛❧ t♦ t❤❡ ♠❛①✐♠✉♠ ♦❢ T ( p, q ) ❢♦r ( p, q ) ∈ [0 , 1] × [0 , 1] ✱ ✇❤❡r❡  h b ( q + p 2 ¯ q ) + f ( p, q ) q ≥ min { 0 , 1 − 2 p }  T ( p, q ) = q ) + f ( p, 1 − 2 p ) − f ( p, 0) h b ( q + p 2 ¯ q + f ( p, 0) o.w.,  1 − 2 p q ) − 2 ph b ( q +1 p ) h b ( q ) + h b ( q + p ✇❤❡r❡ f ( p, q ) = (1 − 2¯ 2 ¯ 2 ) ✸✷

  77. P❧♦t ♦❢ T ( p, q ) ◆✉♠❡r✐❝❛❧ s❡❛r❝❤ ✐♥❞✐❝❛t❡s✿ max p,q T ( p, q ) = 1 . 107516 .. ❛t p = 0 . 5078 .. ❛♥❞ q = 0 . 4365 .. ✸✸

  78. P❛rt✐❝✉❧❛r ❝❤❛♥♥❡❧ ❝♦♥t✐♥✉❡❞ • ■♥t❡r✈❛❧ ❛r✐t❤♠❡t✐❝ ✐s ❛ ♠❡t❤♦❞ t♦ ♦❜t❛✐♥ ❢♦r♠❛❧ ❜♦✉♥❞s ❢♦r ❢✉♥❝t✐♦♥s ❝♦♥s✐st✐♥❣ ♦❢ ❜❛s✐❝ ❛r✐t❤♠❡t✐❝ ❢✉♥❝t✐♦♥s ❛♥❞ ❝♦♠♠♦♥❧② ✉s❡❞ ❢✉♥❝t✐♦♥s s✉❝❤ ❛s ❧♦❣❛r✐t❤♠s ❛♥❞ tr✐❣♦♥♦♠❡tr✐❝ ❢✉♥❝t✐♦♥s✳ • T ( p, q ) ♦♥❧② ✐♥❝❧✉❞❡s ❜❛s✐❝ ❛r✐t❤♠❡t✐❝ ❢✉♥❝t✐♦♥s ❛♥❞ ❧♦❣❛r✐t❤♠✳ • ❲❡ ✉s❡❞ ❏✉❧✐❛ ❜❛s❡❞ ✐♠♣❧❡♠❡♥t❛t✐♦♥ ♦❢ t❤✐s ❢♦r♠❛❧ ♠❡t❤♦❞ t♦ ♦❜t❛✐♥ max T ( p, q ) ∈ [1 . 10751 , 1 . 10769] • ❚❤❡ ✷✲❧❡tt❡r ❚■◆ ❛❝❤✐❡✈❡s 2 R 1 + R 2 = 1 . 108141 ❛t t❤❡ ❞✐str✐❜✉t✐♦♥ P (( X 11 , X 12 ) = (0 , 0)) = p P (( X 11 , X 12 ) = (1 , 1)) = 1 − p P (( X 21 , X 22 ) = (0 , 0)) = 0 . 36 q P (( X 21 , X 22 ) = (1 , 1)) = 1 − 1 . 64 q P (( X 21 , X 22 ) = (0 , 1)) = 0 . 64 q P (( X 21 , X 22 ) = (1 , 0)) = 0 . 64 q ✇❤❡r❡ p = 0 . 507829413 ✱ q = 0 . 436538150 • ❘❡♣❡t✐t✐♦♥ ❝♦❞✐♥❣ ❛❝r♦ss t✐♠❡ s❡❡♠s t♦ ♦✉t♣❡r❢♦r♠ ♠❡♠♦r②❧❡ss ❝♦❞✐♥❣ ✸✹

  79. ■❢ s✉❜✲❛❞❞✐t✐✈❡✱ t❤❡♥ ▼❛rt♦♥✬s r❡❣✐♦♥ ✐s ♦♣t✐♠❛❧ ❢♦r ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧ ■❢ ❡①❛♠♣❧❡ ✇❤❡r❡ ✐t ✐s s✉♣❡r✲❛❞❞✐t✐✈❡✱ t❤❡♥ ♦♥❡ s❤♦✉❧❞ ❜❡ ❛❜❧❡ t♦ ❞❡❞✉❝❡ ❛ ❝❤❛♥♥❡❧ ✇❤❡r❡ ▼❛rt♦♥✬s r❡❣✐♦♥ ✐s ♥♦t ♦♣t✐♠❛❧ ❘❡♠❛r❦s✿ ❈♦♥❥❡❝t✉r❡❞ t♦ ❜❡ s✉❜✲❛❞❞✐t✐✈❡ ✭❆♥❛♥t❤❛r❛♠✲●♦❤❛r✐✲◆❛✐r ✬✶✸✮ ❚♦ ❡✈❛❧✉❛t❡ t❤❡ ❝♦♥❝❛✈❡ ❡♥✈❡❧♦♣❡ ❙✉✣❝❡s t♦ ❝♦♥s✐❞❡r ✿ ✳ ❲❡ ❞✐❞ ♥♦t ❣❡t ❛♥② ❝♦♥tr❛❞✐❝t✐♦♥ t♦ s✉❜✲❛❞❞✐✈✐t② ❢♦r ❜✐♥❛r② ✐♥♣✉t ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧s ❈❛♥ ♣r♦✈❡ s✉❜✲❛❞❞✐t✐✈✐t② ✇❤❡♥ ♦r ✳ ❲❤❛t ❛❜♦✉t ▼❛rt♦♥✬s r❡❣✐♦♥ ❢♦r t❤❡ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧❄ ■s t❤❡ ❢♦❧❧♦✇✐♥❣ ❢✉♥❝t✐♦♥❛❧ s✉❜✲❛❞❞✐t✐✈❡ ♦r ✐s t❤❡r❡ ❛♥ ❡①❛♠♣❧❡ ✇❤❡r❡ ✐t ✐s s✉♣❡r✲❛❞❞✐t✐✈❡❄ ▲❡t W a ( y | x ) ❛♥❞ W b ( z | x ) ❜❡ ❣✐✈❡♥ ❝❤❛♥♥❡❧s✱ α ∈ [0 , 1] ✱ ❛♥❞ λ ≥ 1 ✳ � � C µ X ( λ − α ) H ( Y ) − αH ( Z ) + max p ( u,v | x ) { λI ( U ; Y ) + I ( V ; Z ) − I ( U ; V ) } ✸✺

  80. ❘❡♠❛r❦s✿ ❈♦♥❥❡❝t✉r❡❞ t♦ ❜❡ s✉❜✲❛❞❞✐t✐✈❡ ✭❆♥❛♥t❤❛r❛♠✲●♦❤❛r✐✲◆❛✐r ✬✶✸✮ ❚♦ ❡✈❛❧✉❛t❡ t❤❡ ❝♦♥❝❛✈❡ ❡♥✈❡❧♦♣❡ ❙✉✣❝❡s t♦ ❝♦♥s✐❞❡r ✿ ✳ ❲❡ ❞✐❞ ♥♦t ❣❡t ❛♥② ❝♦♥tr❛❞✐❝t✐♦♥ t♦ s✉❜✲❛❞❞✐✈✐t② ❢♦r ❜✐♥❛r② ✐♥♣✉t ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧s ❈❛♥ ♣r♦✈❡ s✉❜✲❛❞❞✐t✐✈✐t② ✇❤❡♥ ♦r ✳ ❲❤❛t ❛❜♦✉t ▼❛rt♦♥✬s r❡❣✐♦♥ ❢♦r t❤❡ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧❄ ■s t❤❡ ❢♦❧❧♦✇✐♥❣ ❢✉♥❝t✐♦♥❛❧ s✉❜✲❛❞❞✐t✐✈❡ ♦r ✐s t❤❡r❡ ❛♥ ❡①❛♠♣❧❡ ✇❤❡r❡ ✐t ✐s s✉♣❡r✲❛❞❞✐t✐✈❡❄ ▲❡t W a ( y | x ) ❛♥❞ W b ( z | x ) ❜❡ ❣✐✈❡♥ ❝❤❛♥♥❡❧s✱ α ∈ [0 , 1] ✱ ❛♥❞ λ ≥ 1 ✳ � � C µ X ( λ − α ) H ( Y ) − αH ( Z ) + max p ( u,v | x ) { λI ( U ; Y ) + I ( V ; Z ) − I ( U ; V ) } • ■❢ s✉❜✲❛❞❞✐t✐✈❡✱ t❤❡♥ ▼❛rt♦♥✬s r❡❣✐♦♥ ✐s ♦♣t✐♠❛❧ ❢♦r ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧ • ■❢ ∃ ❡①❛♠♣❧❡ ✇❤❡r❡ ✐t ✐s s✉♣❡r✲❛❞❞✐t✐✈❡✱ t❤❡♥ ♦♥❡ s❤♦✉❧❞ ❜❡ ❛❜❧❡ t♦ ❞❡❞✉❝❡ ❛ ❝❤❛♥♥❡❧ ✇❤❡r❡ ▼❛rt♦♥✬s r❡❣✐♦♥ ✐s ♥♦t ♦♣t✐♠❛❧ ✸✺

  81. ❲❤❛t ❛❜♦✉t ▼❛rt♦♥✬s r❡❣✐♦♥ ❢♦r t❤❡ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧❄ ■s t❤❡ ❢♦❧❧♦✇✐♥❣ ❢✉♥❝t✐♦♥❛❧ s✉❜✲❛❞❞✐t✐✈❡ ♦r ✐s t❤❡r❡ ❛♥ ❡①❛♠♣❧❡ ✇❤❡r❡ ✐t ✐s s✉♣❡r✲❛❞❞✐t✐✈❡❄ ▲❡t W a ( y | x ) ❛♥❞ W b ( z | x ) ❜❡ ❣✐✈❡♥ ❝❤❛♥♥❡❧s✱ α ∈ [0 , 1] ✱ ❛♥❞ λ ≥ 1 ✳ � � C µ X ( λ − α ) H ( Y ) − αH ( Z ) + max p ( u,v | x ) { λI ( U ; Y ) + I ( V ; Z ) − I ( U ; V ) } • ■❢ s✉❜✲❛❞❞✐t✐✈❡✱ t❤❡♥ ▼❛rt♦♥✬s r❡❣✐♦♥ ✐s ♦♣t✐♠❛❧ ❢♦r ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧ • ■❢ ∃ ❡①❛♠♣❧❡ ✇❤❡r❡ ✐t ✐s s✉♣❡r✲❛❞❞✐t✐✈❡✱ t❤❡♥ ♦♥❡ s❤♦✉❧❞ ❜❡ ❛❜❧❡ t♦ ❞❡❞✉❝❡ ❛ ❝❤❛♥♥❡❧ ✇❤❡r❡ ▼❛rt♦♥✬s r❡❣✐♦♥ ✐s ♥♦t ♦♣t✐♠❛❧ ❘❡♠❛r❦s✿ • ❈♦♥❥❡❝t✉r❡❞ t♦ ❜❡ s✉❜✲❛❞❞✐t✐✈❡ ✭❆♥❛♥t❤❛r❛♠✲●♦❤❛r✐✲◆❛✐r ✬✶✸✮ • ❚♦ ❡✈❛❧✉❛t❡ t❤❡ ❝♦♥❝❛✈❡ ❡♥✈❡❧♦♣❡ ⋆ ❙✉✣❝❡s t♦ ❝♦♥s✐❞❡r ( U, V ) ✿ | U | + | V | ≤ | X | + 1 ✳ ⋆ ❲❡ ❞✐❞ ♥♦t ❣❡t ❛♥② ❝♦♥tr❛❞✐❝t✐♦♥ t♦ s✉❜✲❛❞❞✐✈✐t② ❢♦r ❜✐♥❛r② ✐♥♣✉t ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧s • ❈❛♥ ♣r♦✈❡ s✉❜✲❛❞❞✐t✐✈✐t② ✇❤❡♥ α = 0 ♦r α = 1 ✳ ✸✺

  82. ❚❤✐s ✐❞❡❛ ✇❛s ❛❧s♦ ✉s❡❞ t♦ r❡s♦❧✈❡ ✽✳✷ ■s s✉♣❡r♣♦s✐t✐♦♥ ❝♦❞✐♥❣ ♦♣t✐♠❛❧ ❢♦r t❤❡ ❣❡♥❡r❛❧ ✸✲r❡❝❡✐✈❡r ❉▼✲❇❈ ✇✐t❤ ♦♥❡ ♠❡ss❛❣❡ t♦ ❛❧❧ t❤r❡❡ r❡❝❡✐✈❡rs ❛♥❞ ❛♥♦t❤❡r ♠❡ss❛❣❡ t♦ t✇♦ r❡❝❡✐✈❡rs❄ ◆❖ ✭◆❛✐r✱❨❛③❞❛♥♣❛♥❛❤ ✬✶✼✮ ❘❡♠❛r❦s • ■❞❡❛✿ ❚♦ ❞❡♠♦♥str❛t❡ s✉♣❡r✲❛❞❞✐t✐✈✐t② • ❉✐✣❝✉❧t② ✿ ■❞❡♥t✐❢② ❛ s✉✣❝✐❡♥t❧② s✐♠♣❧❡ ❝❧❛ss ✇❤❡r❡ ⋆ ❊✈❛❧✉❛t✐♦♥ ♦❢ t❤❡ r❡❣✐♦♥ ✐s ♣♦ss✐❜❧❡ ✿ ♥♦♥✲❝♦♥✈❡① ♦♣t✐♠✐③❛t✐♦♥ ⋆ ❙✉♣❡r✲❛❞❞✐t✐✈✐t② ❤♦❧❞s ✸✻

  83. ❘❡♠❛r❦s • ■❞❡❛✿ ❚♦ ❞❡♠♦♥str❛t❡ s✉♣❡r✲❛❞❞✐t✐✈✐t② • ❉✐✣❝✉❧t② ✿ ■❞❡♥t✐❢② ❛ s✉✣❝✐❡♥t❧② s✐♠♣❧❡ ❝❧❛ss ✇❤❡r❡ ⋆ ❊✈❛❧✉❛t✐♦♥ ♦❢ t❤❡ r❡❣✐♦♥ ✐s ♣♦ss✐❜❧❡ ✿ ♥♦♥✲❝♦♥✈❡① ♦♣t✐♠✐③❛t✐♦♥ ⋆ ❙✉♣❡r✲❛❞❞✐t✐✈✐t② ❤♦❧❞s ❚❤✐s ✐❞❡❛ ✇❛s ❛❧s♦ ✉s❡❞ t♦ r❡s♦❧✈❡ ✽✳✷ ■s s✉♣❡r♣♦s✐t✐♦♥ ❝♦❞✐♥❣ ♦♣t✐♠❛❧ ❢♦r t❤❡ ❣❡♥❡r❛❧ ✸✲r❡❝❡✐✈❡r ❉▼✲❇❈ ✇✐t❤ ♦♥❡ ♠❡ss❛❣❡ t♦ ❛❧❧ t❤r❡❡ r❡❝❡✐✈❡rs ❛♥❞ ❛♥♦t❤❡r ♠❡ss❛❣❡ t♦ t✇♦ r❡❝❡✐✈❡rs❄ ◆❖ ✭◆❛✐r✱❨❛③❞❛♥♣❛♥❛❤ ✬✶✼✮ ✸✻

  84. ❖✉t❧✐♥❡ • ❇r♦❛❞❝❛st ❝❤❛♥♥❡❧✿ ❊st❛❜❧✐s❤✐♥❣ ♦♣t✐♠❛❧✐t② ♦❢ ▼❛rt♦♥✬s ❢♦r ▼■▼❖ ❜r♦❛❞❝❛st ❝❤❛♥♥❡❧ • ■♥t❡r❢❡r❡♥❝❡ ❝❤❛♥♥❡❧✿ ❙✉❜✲♦♣t✐♠❛❧✐t② ♦❢ t❤❡ ❍❛♥✕❑♦❜❛②❛s❤✐ r❡❣✐♦♥ • ❋❛♠✐❧② ♦❢ ♥♦♥✲❝♦♥✈❡① ♦♣t✐♠✐③❛t✐♦♥ ♣r♦❜❧❡♠s ⋆ ❘❡❧❛t✐♦♥ t♦ ♣r♦❜❧❡♠s ♦❢ ✐♥t❡r❡st ✐♥ ♦t❤❡r ✜❡❧❞s ⋆ ❯♥✐❢②✐♥❣ ♦❜s❡r✈❛t✐♦♥s ❛♥❞ s♦♠❡ ❝♦♥❥❡❝t✉r❡s ✸✼

  85. ❯s✐♥❣ ❋❡♥❝❤❡❧ ❞✉❛❧✐t② t❤✐s ✐s s❛♠❡ ❛s ❆ s♣❡❝✐✜❝ ❢❛♠✐❧② ♦❢ ♥♦♥✲❝♦♥✈❡① ♦♣t✐♠✐③❛t✐♦♥ ♣r♦❜❧❡♠s ❙❤♦✇s ✉♣ ✿ ❚❡st✐♥❣ t❤❡ ♦♣t✐♠❛❧✐t② ♦❢ ❝♦❞✐♥❣ s❝❤❡♠❡s ❚❡st✐♥❣ ♦♣t✐♠❛❧✐t② ✭✉s✉❛❧❧②✮ r❡❞✉❝❡s t♦ t❡st✐♥❣ s✉❜✲❛❞❞✐t✐✈✐t② ♦❢ � � � C ν X , α S ∈ R . α S H ( X S ) S ⊆ [ n ] ✸✽

  86. ❆ s♣❡❝✐✜❝ ❢❛♠✐❧② ♦❢ ♥♦♥✲❝♦♥✈❡① ♦♣t✐♠✐③❛t✐♦♥ ♣r♦❜❧❡♠s ❙❤♦✇s ✉♣ ✿ ❚❡st✐♥❣ t❤❡ ♦♣t✐♠❛❧✐t② ♦❢ ❝♦❞✐♥❣ s❝❤❡♠❡s ❚❡st✐♥❣ ♦♣t✐♠❛❧✐t② ✭✉s✉❛❧❧②✮ r❡❞✉❝❡s t♦ t❡st✐♥❣ s✉❜✲❛❞❞✐t✐✈✐t② ♦❢ � � � C ν X , α S ∈ R . α S H ( X S ) S ⊆ [ n ] ❯s✐♥❣ ❋❡♥❝❤❡❧ ❞✉❛❧✐t② t❤✐s ✐s s❛♠❡ ❛s � G 1 ( γ 1 ) := max α S H ( X S ) − E( γ 1 ( X )) µ X S ⊆ [ n ] � α S H ( X S ) − E( γ 2 ( X )) G 2 ( γ 2 ) := max µ X S ⊆ [ n ] � G 12 ( γ 1 , γ 2 ) := max α S H ( X 1 S , X 2 S ) − E( γ 1 ( X 1 )) − E( γ 2 ( X 2 )) µ X 1 , X 2 S ⊆ [ n ] ■s G 12 ( γ 1 , γ 2 ) = G 1 ( γ 1 ) + G 2 ( γ 2 ) ∀ γ 1 , γ 2 ❄ ✐✳❡✳ ■s t❤❡ ♠❛①✐♠✐③❡r ♦❢ G 12 ❛ ♣r♦❞✉❝t ❞✐str✐❜✉t✐♦♥❄ ✸✽

  87. ❆ s♣❡❝✐✜❝ ❢❛♠✐❧② ♦❢ ♥♦♥✲❝♦♥✈❡① ♦♣t✐♠✐③❛t✐♦♥ ♣r♦❜❧❡♠s ❙❤♦✇s ✉♣ ✿ ❚❡st✐♥❣ t❤❡ ♦♣t✐♠❛❧✐t② ♦❢ ❝♦❞✐♥❣ s❝❤❡♠❡s ❚❡st✐♥❣ ♦♣t✐♠❛❧✐t② ✭✉s✉❛❧❧②✮ r❡❞✉❝❡s t♦ t❡st✐♥❣ s✉❜✲❛❞❞✐t✐✈✐t② ♦❢ � � � C ν X , α S ∈ R . α S H ( X S ) S ⊆ [ n ] ❯s✐♥❣ ❋❡♥❝❤❡❧ ❞✉❛❧✐t② t❤✐s ✐s s❛♠❡ ❛s � G 1 ( γ 1 ) := max α S H ( X S ) − E( γ 1 ( X )) µ X S ⊆ [ n ] � α S H ( X S ) − E( γ 2 ( X )) G 2 ( γ 2 ) := max µ X S ⊆ [ n ] � G 12 ( γ 1 , γ 2 ) := max α S H ( X 1 S , X 2 S ) − E( γ 1 ( X 1 )) − E( γ 2 ( X 2 )) µ X 1 , X 2 S ⊆ [ n ] ❆r❡ t❤❡r❡ ♦t❤❡r ✜❡❧❞s ✇❤❡r❡ t❤❡ s❛♠❡ ❢❛♠✐❧② s❤♦✇s ✉♣❄ ✸✽

  88. ❚❤✐s ✭s❡r❡♥❞✐♣✐t♦✉s✮ r❡❞✐s❝♦✈❡r② ♦❢ t❤❡ ❧✐♥❦ ❜❡t✇❡❡♥ ❤②♣❡r❝♦♥tr❛❝t✐✈✐t② ❛♥❞ ✐♥❢♦r♠❛t✐♦♥ ♠❡❛s✉r❡s ❛♥❞ t❤❡s❡ ❡q✉✐✈❛❧❡♥t ❝❤❛r❛❝t❡r✐③❛t✐♦♥s ✐s s♣✉rr✐♥❣ ❛ ❧♦t ♦❢ ✇♦r❦ ❍②♣❡r❝♦♥tr❛❝t✐✈✐t② ❙t✉❞✐❡❞ ✐♥ ❢✉♥❝t✐♦♥❛❧ ❛♥❛❧②s✐s✱ ❝s t❤❡♦r②✱ ❡t❝✳ ❉❡✜♥✐t✐♦♥ ( X, Y ) ∼ µ XY ✐s ( p, q ) ✲❤②♣❡r❝♦♥tr❛❝t✐✈❡ ❢♦r 1 ≤ q ≤ p ✐❢ � Tg � p ≤ � g � q ∀ g ( Y ) ✇❤❡r❡ T ✐s t❤❡ ▼❛r❦♦✈ ♦♣❡r❛t♦r ❝❤❛r❛❝t❡r✐③❡❞ ❜② µ Y | X 1 p ✳ ❍❡r❡ � Z � p = E ( | Z | p ) ✸✾

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

NEWSdm experiment Directional Dark Matter Search with Super-high resolution Nuclear Emulsion

NEWSdm experiment Directional Dark Matter Search with Super-high resolution Nuclear Emulsion

NEWSdm experiment Directional Dark Matter Search with Super-high resolution Nuclear Emulsion Tatsuhiro NAKA KMI, Nagoya University on behalf of NEWSdm collaboration 1 Direction sensitive dark matter detector with solid Direction sensitive

507 views • 24 slides
(z.c) ooF x Exo zt . Dai x (u,,unJ R= (t,,tr) Cfiz 4. Ve\\ 'f I er e., iAr.,r- a',ol

(z.c) ooF x Exo zt . Dai x (u,,unJ R= (t,,tr) Cfiz 4. Ve\\ 'f I er e., iAr.,r- a',ol

( 235 L' r&quot;th ODE Ercct f{cvr r e\-w,2les 1 fefcrrltal l A.i&quot; Rer&quot;/f. t j 6cie,-0. (, to- lioiJ l ?nctor *etloy'. rhf?trc6tl (z.c) ooF x Exo zt . Dai x (u,,unJ R= (t,,tr) Cfiz 4. Ve\\ 'f I er e., iAr.,r-

564 views • 19 slides
The fourth moment of Dirichlet L -functions along a coset and the Weyl bound Ian Petrow ETH Z

The fourth moment of Dirichlet L -functions along a coset and the Weyl bound Ian Petrow ETH Z

The fourth moment of Dirichlet L -functions along a coset and the Weyl bound Ian Petrow ETH Z urich Joint work with Matthew P. Young Ian Petrow (ETH Z urich) The 4th moment and the Weyl bound 1 / 16 The subconvexity problem Given an

897 views • 60 slides
Discovery Potential for Directional Dark Matter Detection with the NEWSdm Experiment Valerio

Discovery Potential for Directional Dark Matter Detection with the NEWSdm Experiment Valerio

Discovery Potential for Directional Dark Matter Detection with the NEWSdm Experiment Valerio Gentile 54 rd Rencontres de Moriond - Electroweak 2019 1 / 60 NEWSdm COLLABORATION 70 physicists ITAL Y 14 institutes INFN e Univ. Bari, INFN e

340 views • 17 slides
The Future of Solar Power Center on Global Energy Policy, Columbia University School of

The Future of Solar Power Center on Global Energy Policy, Columbia University School of

The Future of Solar Power Center on Global Energy Policy, Columbia University School of International and Public Affairs October 20, 2015 Varun Sivaram, Ph.D. Douglas Dillon Fellow Cleantech VC: The Wrong Model for Energy Innovation Council on

571 views • 22 slides
Third Quarter 2015 Earnings Conference Call and Webcast October 29, 2015 Forward Looking

Third Quarter 2015 Earnings Conference Call and Webcast October 29, 2015 Forward Looking

Third Quarter 2015 Earnings Conference Call and Webcast October 29, 2015 Forward Looking Statements This presentation contains contains forward- looking statements within the meaning of federal securities laws regarding MPLX LP (MPLX)

354 views • 14 slides
INDUSTRIAL MARKET REVIEW Chip Worley Bainbridge Patrick Mumey, SIOR Cavalier Worth Remick

INDUSTRIAL MARKET REVIEW Chip Worley Bainbridge Patrick Mumey, SIOR Cavalier Worth Remick

INDUSTRIAL MARKET REVIEW INDUSTRIAL MARKET REVIEW Industrial Reporters Ron Biesecker &amp; Christine Kaempfe Greenbrier INDUSTRIAL MARKET REVIEW Chip Worley Bainbridge Patrick Mumey, SIOR Cavalier Worth Remick Norfolk Commerce

187 views • 8 slides
Not everything that counts can be counted, and not everything that can be counted counts.

Not everything that counts can be counted, and not everything that can be counted counts.

Not everything that counts can be counted, and not everything that can be counted counts. Albert Einstein Tehnology Capit al and the US Current A ount Ellen R. McGrattan and Edward C. Prescott August 2008 www.minneapolisfed.org /

845 views • 83 slides
Housing and the Business Cycle Morris Davis and Jonathan Heathcote Winter 2009 Huw Lloyd-Ellis

Housing and the Business Cycle Morris Davis and Jonathan Heathcote Winter 2009 Huw Lloyd-Ellis

Housing and the Business Cycle Morris Davis and Jonathan Heathcote Winter 2009 Huw Lloyd-Ellis () ECON917 Winter 2009 1 / 21 Motivation Need to distinguish between housing and nonhousing investment , ! produced using dierent

575 views • 29 slides
Live-sculpting a Genode-based operating system Norman Feske &lt; norman.feske@genode-labs.com

Live-sculpting a Genode-based operating system Norman Feske &lt; norman.feske@genode-labs.com

Live-sculpting a Genode-based operating system Norman Feske &lt; norman.feske@genode-labs.com &gt; Outline 1. Base system 2. Storage, networking 3. Extending the system 4. Road Map for 2018 Live-sculpting a Genode-based operating system 2

534 views • 32 slides
Quantum Computng with VBA By Tams Bunth Sofware Engineer at Collabora Productiittz

Quantum Computng with VBA By Tams Bunth Sofware Engineer at Collabora Productiittz

Collabora Productivity Quantum Computng with VBA By Tams Bunth Sofware Engineer at Collabora Productiittz www.collaboraofci.co Collabora Productivity m VBA and ... Teaching people radical new watzs to model problems Great simulatin

555 views • 26 slides
Leonardo Nve Egea Leonardo Nve Egea lnve@s21sec.com 1. because Im sure that some people will

Leonardo Nve Egea Leonardo Nve Egea lnve@s21sec.com 1. because Im sure that some people will

Leonardo Nve Egea Leonardo Nve Egea lnve@s21sec.com 1. because Im sure that some people will publish more attacks. .2 because previously presentations about satellite. ll Warezzman (in 2004 at Undercon VIII first Spanish hacker CON )

1.19k views • 105 slides
Ray Tracing Assignment Goal is to reproduce the following So You Want to Write a Ray Tracer

Ray Tracing Assignment Goal is to reproduce the following So You Want to Write a Ray Tracer

Ray Tracing Assignment Goal is to reproduce the following So You Want to Write a Ray Tracer Checkpoint 7 Tone Reproduction Whitted, 1980 Ray Tracing Assignment Ray Tracing Assignment Seven checkpoints Seven checkpoints 1.

176 views • 5 slides
Transactional Privacy Unknotting the Privacy Tussle w. Economics

Transactional Privacy Unknotting the Privacy Tussle w. Economics

Transactional Privacy Unknotting the Privacy Tussle w. Economics Chris Riederer, Philippa Gill, Vijay Erramilli, Pablo Rodriguez, Balachander Krishnamurthy, Dina

539 views • 35 slides
Resurrecting the Role of the Product Market Wedge in Recessions Mark Bils, University of Rochester

Resurrecting the Role of the Product Market Wedge in Recessions Mark Bils, University of Rochester

Resurrecting the Role of the Product Market Wedge in Recessions Mark Bils, University of Rochester and NBER Pete Klenow, Stanford University and NBER Ben Malin, Federal Reserve Bank of Minneapolis 1 Federal Reserve Bank of San Francisco

830 views • 80 slides
FUJII, Shinobu National Institute of Technology (KOSEN), Oshima College Workshop on the

FUJII, Shinobu National Institute of Technology (KOSEN), Oshima College Workshop on the

Quartic CartanM unzner polynomials and Casimir operators FUJII, Shinobu National Institute of Technology (KOSEN), Oshima College Workshop on the isoparametric theory At Beijing Normal University June 6th., 2019 FUJII, Shinobu ( NIT

1.04k views • 66 slides
Discreteness and rationality of F -jumping numbers on rings with singularities Karl Schwede 1 ,

Discreteness and rationality of F -jumping numbers on rings with singularities Karl Schwede 1 ,

Motivation Discreteness and rationality on rings with singularities What about the non-(log)- Q -Gorenstein case? Discreteness and rationality of F -jumping numbers on rings with singularities Karl Schwede 1 , Wenliang Zhang 1 , Shunsuke Takagi 2

1.14k views • 62 slides
A glance at big-O notation. There are precise way of talking about the approximate properties of

A glance at big-O notation. There are precise way of talking about the approximate properties of

A glance at big-O notation. There are precise way of talking about the approximate properties of programs. We are going to use one, called big-O notation. If we write that the execution time of program P is ( ) we mean in the worst case, its

490 views • 27 slides
BACK GROUND L UT HE R COURT SOCI E T Y T he L uthe r Co urt So c ie ty (L CS) wa s e

BACK GROUND L UT HE R COURT SOCI E T Y T he L uthe r Co urt So c ie ty (L CS) wa s e

LUT HER C O URT SO C IET Y INT ER-G ENERAT IO NAL HO USING PRO JEC T SAANIC H, BC BACK GROUND L UT HE R COURT SOCI E T Y T he L uthe r Co urt So c ie ty (L CS) wa s e sta b lishe d in 1974, o ffspring o f the pa rish o f L

320 views • 17 slides
Trust-region interior-point method for large sparse l 1 optimization cek 1 L.Luk san, C.

Trust-region interior-point method for large sparse l 1 optimization cek 1 L.Luk san, C.

Trust-region interior-point method for large sparse l 1 optimization cek 1 L.Luk san, C. Matonoha, J. Vl Institute of Computer Science, Academy of Sciences of the Czech Republic, Pod Vod arenskou v e z 2, 182 07 Praha 8.

354 views • 16 slides
Carbon sanctuaries and development: preserving tropical forests in a rural vs diversified economy

Carbon sanctuaries and development: preserving tropical forests in a rural vs diversified economy

Analytical framework Preserving the forest Conclusion Carbon sanctuaries and development: preserving tropical forests in a rural vs diversified economy Jean-Marc Bourgeon 1 and H ene Ollivier 2 el` September 29th 2012 UNU-WIDER conference

540 views • 31 slides
Implementing a Basic Income Guarantee in Canada: Prospects and Problems Robin Boadway, Queens

Implementing a Basic Income Guarantee in Canada: Prospects and Problems Robin Boadway, Queens

Implementing a Basic Income Guarantee in Canada: Prospects and Problems Robin Boadway, Queens University Economics Seminar, Dalhousie University, Nov. 1, 2019 Based on a paper with Katherine Cuff and Kourtney Koebel written for CARE,

607 views • 36 slides
M o n t e C a r l o S i m u l a t i o n f o r N e x t G e n e r a

M o n t e C a r l o S i m u l a t i o n f o r N e x t G e n e r a

M o n t e C a r l o S i m u l a t i o n f o r N e x t G e n e r a t i o n S o u r c e a n d C h a n n e l C o d i n g o n O S G C o n n e c t A h m a d G o l m o h a

514 views • 27 slides
Testing Email Invitations in Testing Email Invitations in a Nonprobability Panel p y Elizabeth

Testing Email Invitations in Testing Email Invitations in a Nonprobability Panel p y Elizabeth

Testing Email Invitations in Testing Email Invitations in a Nonprobability Panel p y Elizabeth Nichols, Ryan King and Elizabeth Nichols, Ryan King and Jennifer Childs U S C U.S. Census Bureau B This presentation is released to inform

542 views • 27 slides

More recommend