[PPT] - t r PowerPoint Presentation, free download

SLIDE 1

❖♣❡♥✐♥❣ t❤❡ ❜❧❛❝❦ ❜♦① ♦❢ ❉❡❡♣ ◆❡✉r❛❧ ◆❡t✇♦r❦s ✈✐❛ ■♥❢♦r♠❛t✐♦♥

Òûùóê ✃èðèëë✱ ✸ êóðñ áàêàëàâðèàòà

Ñåìèíàð ➽Ñîâðåìåííûå ìåòîäû â Ò➮➾

✶✼ ôåâðàëÿ ✷✵✷✵ ã✳

SLIDE 2

❙✉♣❡r✈✐s❡❞ ❞❡❡♣ ❧❡❛r♥✐♥❣ ❛♥❞ ✐♥❢♦r♠❛t✐♦♥

Encoder Decoder

❲❡ ❛r❡ ✐♥t❡r❡st❡❞ ✐♥ ❣♦♦❞ r❡♣r❡s❡♥t❛t✐♦♥s✱ T(X)✱ ♦❢ t❤❡ ✐♥♣✉t ♣❛tt❡r♥s x ∈ X✱ t❤❛t ❡♥❛❜❧❡ ❣♦♦❞ ♣r❡❞✐❝t✐♦♥s ♦❢ t❤❡ ❧❛❜❡❧ y ∈ Y ✳ ▼♦r❡♦✈❡r✱ ✇❡ ✇❛♥t t♦ ❡✣❝✐❡♥t❧② ❧❡❛r♥ s✉❝❤ r❡♣r❡s❡♥t❛t✐♦♥s ❢r♦♠ ❛♥ ❡♠♣✐r✐❝❛❧ s❛♠♣❧❡ ♦❢ t❤❡ ✭✉♥❦♥♦✇♥✮ ❥♦✐♥t ❞✐str✐❜✉t✐♦♥ P(X, Y )✱ ✐♥ ❛ ✇❛② t❤❛t ♣r♦✈✐❞❡s ❣♦♦❞ ❣❡♥❡r❛❧✐③❛t✐♦♥✳

SLIDE 3

❙✉♣❡r✈✐s❡❞ ❞❡❡♣ ❧❡❛r♥✐♥❣ ❛♥❞ ✐♥❢♦r♠❛t✐♦♥

Encoder Decoder

❲❡ tr❡❛t t❤❡ ✇❤♦❧❡ ❧❛②❡r✱ T✱ ❛s ❛ s✐♥❣❧❡ r❛♥❞♦♠ ✈❛r✐❛❜❧❡✱ ❝❤❛r❛❝t❡r✐③❡❞ ❜② ✐ts ❡♥❝♦❞❡r✱ P(T|X)✱ ❛♥❞ ❞❡❝♦❞❡r✱ P(Y |T) ❞✐str✐❜✉t✐♦♥s✳ ❆s ✇❡ ❛r❡ ♦♥❧② ✐♥t❡r❡st❡❞ ✐♥ t❤❡ ✐♥❢♦r♠❛t✐♦♥ t❤❛t ✢♦✇s t❤r♦✉❣❤ t❤❡ ♥❡t✇♦r❦✱ ✇❡ q✉❛♥t✐❢② t❤❡ r❡♣r❡s❡♥t❛t✐♦♥s ❜② t✇♦ ♥✉♠❜❡rs✱ t❤❛t ❛r❡ ✐♥✈❛r✐❛♥t t♦ ❛♥② ✐♥✈❡rt✐❜❧❡ r❡✲♣❛r❛♠❡t❡r✐③❛t✐♦♥ ♦❢ T✱ t❤❡ ♠✉t✉❛❧ ✐♥❢♦r♠❛t✐♦♥ ♦❢ T ✇✐t❤ t❤❡ ✐♥♣✉t X ❛♥❞ t❤❡ ❞❡s✐r❡❞ ♦✉t♣✉t Y ✳

SLIDE 4

▼✉t✉❛❧ ✐♥❢♦r♠❛t✐♦♥

✐✈❡♥ ❛♥② t✇♦ r❛♥❞♦♠ ✈❛r✐❛❜❧❡s✱ X ❛♥❞ Y ✱ ✇✐t❤ ❛ ❥♦✐♥t ❞✐str✐❜✉t✐♦♥ p(x, y)✱ t❤❡✐r

▼✉t✉❛❧ ■♥❢♦r♠❛t✐♦♥ ✐s ❞❡✜♥❡❞ ❛s✿ I(X; Y ) = DKL[p(x, y)||p(x)p(y)] =

x∈X,y∈Y

p(x, y) log p (x, y) p (x) p (y)

✭✶✮

=

x∈X,y∈Y

p (x, y) log p (x|y) p (x)

= H(X) − H(X|Y ) ,

✭✷✮ ✇❤❡r❡ DKL[p||q] ✐s t❤❡ ❑✉❧❧❜❛❝❦✲▲✐❡❜❧❡r ❞✐✈❡r❣❡♥❝❡ ♦❢ t❤❡ ❞✐str✐❜✉t✐♦♥s p ❛♥❞ q✱ ❛♥❞ H(X) ❛♥❞ H(X|Y ) ❛r❡ t❤❡ ❡♥tr♦♣② ❛♥❞ ❝♦♥❞✐t✐♦♥❛❧ ❡♥tr♦♣② ♦❢ X ❛♥❞ Y ✱ r❡s♣❡❝t✐✈❡❧②✳ ❚❤❡ ♠✉t✉❛❧ ✐♥❢♦r♠❛t✐♦♥ q✉❛♥t✐✜❡s t❤❡ ♥✉♠❜❡r ♦❢ r❡❧❡✈❛♥t ❜✐ts t❤❛t t❤❡ ✐♥♣✉t ✈❛r✐❛❜❧❡ X ❝♦♥t❛✐♥s ❛❜♦✉t t❤❡ ❧❛❜❡❧ Y ✱ ♦♥ ❛✈❡r❛❣❡✳✳

SLIDE 5

❚❤❡ ■♥❢♦r♠❛t✐♦♥ P❧❛♥❡

❉❛t❛ Pr♦❝❡ss✐♥❣ ■♥❡q✉❛❧✐t② ✭❉P■✮✿ ❋♦r ❛♥② ✸ ✈❛r✐❛❜❧❡s t❤❛t ❢♦r♠ ❛ ▼❛r❦♦✈ ❝❤❛✐♥ X → Y → Z✱ I (X; Y ) ≥ I(X; Z) . ✭✸✮ ■♥❢♦r♠❛t✐♦♥ P❧❛♥❡

✐✈❡♥ P(X; Y )✱ T ✐s ✉♥✐q✉❡❧② ♠❛♣♣❡❞ t♦ ❛ ♣♦✐♥t ✐♥ t❤❡ ✐♥❢♦r♠❛t✐♦♥✲♣❧❛♥❡ ✇✐t❤

❝♦♦r❞✐♥❛t❡s (I(X; T), I(T; Y ))✳ ❲❤❡♥ ❛♣♣❧✐❡❞ t♦ t❤❡ ▼❛r❦♦✈ ❝❤❛✐♥ ♦❢ ❛ ❑✲❧❛②❡rs ❉◆◆✱ ✇✐t❤ Ti ❞❡♥♦t✐♥❣ t❤❡ ith ❤✐❞❞❡♥ ❧❛②❡r ❛s ❛ s✐♥❣❧❡ ♠✉❧t✐✈❛r✐❛t❡ ✈❛r✐❛❜❧❡✱ t❤❡ ❧❛②❡rs ❛r❡ ♠❛♣♣❡❞ t♦ K ♠♦♥♦t♦♥✐❝ ❝♦♥♥❡❝t❡❞ ♣♦✐♥ts ✐♥ t❤❡ ♣❧❛♥❡ ✲ ❤❡♥❝❡❢♦rt❤ ❛ ✉♥✐q✉❡ ✐♥❢♦r♠❛t✐♦♥ ♣❛t❤ ✲ ✇❤✐❝❤ s❛t✐s✜❡s t❤❡ ❢♦❧❧♦✇✐♥❣ ❉P■ ❝❤❛✐♥s✿ I(X; Y ) ≥ I(T1; Y ) ≥ I(T2; Y ) ≥ ... ≥ I(Tk; Y ) ≥ I( ˆ Y ; Y ) ✭✹✮ H(X) ≥ I(X; T1) ≥ I(X; T2) ≥ ... ≥ I(X; Tk) ≥ I(X; ˆ Y ). ✭✺✮

SLIDE 6

❚❤❡ ■♥❢♦r♠❛t✐♦♥ ❇♦tt❧❡♥❡❝❦ ♦♣t✐♠❛❧ r❡♣r❡s❡♥t❛t✐♦♥s

❲❤❛t ❝❤❛r❛❝t❡r✐③❡s t❤❡ ♦♣t✐♠❛❧ r❡♣r❡s❡♥t❛t✐♦♥s ♦❢ X ✇✳r✳t✳ Y ❄ ❙✉✣❝✐❡♥t st❛t✐st✐❝s✱ ✐♥ ♦✉r ❝♦♥t❡①t✱ ❛r❡ ♠❛♣s ♦r ♣❛rt✐t✐♦♥s ♦❢ X✱ S(X)✱ t❤❛t ❝❛♣t✉r❡ ❛❧❧ t❤❡ ✐♥❢♦r♠❛t✐♦♥ t❤❛t X ❤❛s ♦♥ Y ✳ ◆❛♠❡❧②✱ I(S(X); Y ) = I(X; Y ) ▼✐♥✐♠❛❧ s✉✣❝✐❡♥t st❛t✐st✐❝s✱ T(X)✱ ❛r❡ t❤❡ s✐♠♣❧❡st s✉✣❝✐❡♥t st❛t✐st✐❝s ❛♥❞ ✐♥❞✉❝❡ t❤❡ ❝♦❛rs❡st s✉✣❝✐❡♥t ♣❛rt✐t✐♦♥ ♦♥ X✳ T(X) = arg min

S(X):I(S(X);Y )=I(X;Y ) I(S(X); X).

✭✻✮ ❇✉t ✐t ❞♦❡s♥✬t ❛❧✇❛②s ❡①✐st ❛♥❞ ✐s ❤❛r❞ t♦ ✜♥❞ ❡①❛❝t❧②✱ s♦ ✇❡ r❡❧❛① t❤❡ ❝♦♥❞✐t✐♦♥✳

SLIDE 7

❚❤❡ ■♥❢♦r♠❛t✐♦♥ ❇♦tt❧❡♥❡❝❦ ♦♣t✐♠❛❧ r❡♣r❡s❡♥t❛t✐♦♥s

❚❤✐s ❧❡❛❞s t♦ t❤❡ ■♥❢♦r♠❛t✐♦♥ ❇♦tt❧❡♥❡❝❦ ✭■❇✮ tr❛❞❡♦✛✱ ✇❤✐❝❤ ♣r♦✈✐❞❡s ❛ ❝♦♠♣✉t❛t✐♦♥❛❧ ❢r❛♠❡✇♦r❦ ❢♦r ✜♥❞✐♥❣ ❛♣♣r♦①✐♠❛t❡ ♠✐♥✐♠❛❧ s✉✣❝✐❡♥t st❛t✐st✐❝s✱ ♦r t❤❡ ♦♣t✐♠❛❧ tr❛❞❡♦✛ ❜❡t✇❡❡♥ ❝♦♠♣r❡ss✐♦♥ ♦❢ X ❛♥❞ ♣r❡❞✐❝t✐♦♥ ♦❢ Y ✳ ❊✣❝✐❡♥t r❡♣r❡s❡♥t❛t✐♦♥s ❛r❡ ❛♣♣r♦①✐♠❛t❡ ♠✐♥✐♠❛❧ s✉✣❝✐❡♥t st❛t✐st✐❝s ✐♥ t❤❛t s❡♥s❡✳ ■❢ ✇❡ ❞❡✜♥❡ t ∈ T ❛s t❤❡ ❝♦♠♣r❡ss❡❞ r❡♣r❡s❡♥t❛t✐♦♥s ♦❢ x ∈ X✱ t❤❡ r❡♣r❡s❡♥t❛t✐♦♥ ♦❢ x ✐s ♥♦✇ ❞❡✜♥❡❞ ❜② t❤❡ ♠❛♣♣✐♥❣ p (t|x)✳ ❚❤✐s ■♥❢♦r♠❛t✐♦♥ ❇♦tt❧❡♥❡❝❦ tr❛❞❡♦✛ ✐s ❢♦r♠✉❧❛t❡❞ ❜② t❤❡ ❢♦❧❧♦✇✐♥❣ ♦♣t✐♠✐③❛t✐♦♥ ♣r♦❜❧❡♠✱ ❝❛rr✐❡❞ ✐♥❞❡♣❡♥❞❡♥t❧② ❢♦r t❤❡ ❞✐str✐❜✉t✐♦♥s✱ p(t|x), p(t), p(y|t)✱ ✇✐t❤ t❤❡ ▼❛r❦♦✈ ❝❤❛✐♥✿ Y → X → T✱ min

p(t|x),p(y|t),p(t) {I (X; T) − βI (T; Y )} .

✭✼✮ ❚❤❡ ▲❛❣r❛♥❣❡ ♠✉❧t✐♣❧✐❡r β ❞❡t❡r♠✐♥❡s t❤❡ ❧❡✈❡❧ ♦❢ r❡❧❡✈❛♥t ✐♥❢♦r♠❛t✐♦♥ ❝❛♣t✉r❡❞ ❜② t❤❡ r❡♣r❡s❡♥t❛t✐♦♥ T✱ I(T; Y )✱ ✇❤✐❝❤ ✐s ❞✐r❡❝t❧② r❡❧❛t❡❞ t♦ t❤❡ ❡rr♦r ✐♥ t❤❡ ❧❛❜❡❧ ♣r❡❞✐❝t✐♦♥ ❢r♦♠ t❤✐s r❡♣r❡s❡♥t❛t✐♦♥✳

SLIDE 8

■♥❢♦r♠❛t✐♦♥ ❝✉r✈❡

❚❤❡s❡ ❡q✉❛t✐♦♥s ❛r❡ s❛t✐s✜❡❞ ❛❧♦♥❣ t❤❡ ✐♥❢♦r♠❛t✐♦♥ ❝✉r✈❡✱ ✇❤✐❝❤ ✐s ❛ ♠♦♥♦t♦♥✐❝ ❝♦♥❝❛✈❡ ❧✐♥❡ ♦❢ ♦♣t✐♠❛❧ r❡♣r❡s❡♥t❛t✐♦♥s t❤❛t s❡♣❛r❛t❡s t❤❡ ❛❝❤✐❡✈❛❜❧❡ ❛♥❞ ✉♥❛❝❤✐❡✈❛❜❧❡ r❡❣✐♦♥s ✐♥ t❤❡ ✐♥❢♦r♠❛t✐♦♥✲♣❧❛♥❡✳ ❋♦r s♠♦♦t❤ P(X, Y ) ❞✐str✐❜✉t✐♦♥s❀ ✐✳❡✳✱ ✇❤❡♥ Y ✐s ♥♦t ❛ ❝♦♠♣❧❡t❡❧② ❞❡t❡r♠✐♥✐st✐❝ ❢✉♥❝t✐♦♥ ♦❢ X✱ t❤❡ ✐♥❢♦r♠❛t✐♦♥ ❝✉r✈❡ ✐s str✐❝t❧② ❝♦♥❝❛✈❡ ✇✐t❤ ❛ ✉♥✐q✉❡ s❧♦♣❡✱ β−1✱ ❛t ❡✈❡r② ♣♦✐♥t✱ ❛♥❞ ❛ ✜♥✐t❡ s❧♦♣❡ ❛t t❤❡ ♦r✐❣✐♥✳ ■♥ t❤❡s❡ ❝❛s❡s β ❞❡t❡r♠✐♥❡s ❛ s✐♥❣❧❡ ♣♦✐♥t✱ ♦♥ t❤❡ ✐♥❢♦r♠❛t✐♦♥ ❝✉r✈❡ ✇✐t❤ s♣❡❝✐✜❡❞ ❡♥❝♦❞❡r✱Pβ(T|X)✱ ❛♥❞ ❞❡❝♦❞❡r✱ Pβ(Y |T)

SLIDE 9