Strong Converse for Testing Against Independence over a Noisy - PowerPoint PPT Presentation

Strong Converse for Testing Against Independence over a Noisy Channel Sreejith Sreekumar and Deniz Gündüz Cornell University Imperial College London June 8, 2020 SS (Cornell) Strong Converse for TAI over Noisy Channel June 8, 2020 1 / 15

System Model ◮ Hypothesis test (HT): testing against independence (TAI). ◮ n i.i.d. samples U n and V n available at observer and detector, respectively. ◮ Observer transmits X n = f n ( U n ) to detector over a DMC P Y | X . ◮ Detector outputs decision ˆ H = g n ( V n , Y n ) . ◮ Motivation: Distributed statistical inference over noisy channels. SS (Cornell) Strong Converse for TAI over Noisy Channel June 8, 2020 2 / 15

Type I and Type II Error Probabilities Trade-off True hypothesis: H ∈ { 0 , 1 } . Type I and Type II Error Probabilities: α n ( f n , g n ) = P ( g n ( Y n , V n ) = 1 | H = 0 ) , β n ( f n , g n ) = P ( g n ( Y n , V n ) = 0 | H = 1 ) . Type II Error Exponent: κ ′ : ( κ ′ , ǫ ) ∈ R � � κ ( ǫ ) = sup , ( κ ′ , ǫ ) ∈ R ≥ 0 × [ 0 , 1 ] : ∃ { ( f n , g n ) } n ∈ N , α n ( f n , g n ) ≤ ǫ,     R =  . − 1 n log ( β n ( f n , g n )) ≥ κ ′ lim inf  n →∞ SS (Cornell) Strong Converse for TAI over Noisy Channel June 8, 2020 3 / 15

Existing Results: TAI over a rate-limited noiseless channel: ◮ Single-letter type II error exponent characterization [Ahlswede and Csiszár (1986)]: κ ( ǫ, R ) = ¯ ǫ → 0 ¯ lim θ ( P UV , R ) := max { I ( V ; W ) : I ( U ; W ) ≤ R , V − U − W } . κ ( ǫ, R ) = ¯ ◮ Strong converse holds: ¯ θ ( P UV , R ) , ∀ ǫ ∈ ( 0 , 1 ) . Type II error exponent independent of ǫ ∈ ( 0 , 1 ) . TAI over a noisy channel: ◮ Single-letter type II error exponent characterization [SS and Gündüz (2020)] ǫ → 0 κ ( ǫ ) = θ ( P UV , C ) := max { I ( V ; W ) : I ( U ; W ) ≤ C , V − U − W } . lim Does strong converse hold ? SS (Cornell) Strong Converse for TAI over Noisy Channel June 8, 2020 4 / 15

Proof of Strong Converse in Noiseless Channel Setting: Key tools Covering lemma: [Ahlswede and Csiszar (1986)] T n P : Type class (non-empty) of P on fi- nite alphabet X Non-empty subset B n ⊆ T n P SS (Cornell) Strong Converse for TAI over Noisy Channel June 8, 2020 5 / 15

Proof of Strong Converse in Noiseless Channel Setting: Key tools Covering lemma: [Ahlswede and Csiszar (1986)] T n P : Type class (non-empty) of P on fi- nite alphabet X Non-empty subset B n ⊆ T n P { π i , 1 ≤ i ≤ N } : Set of permutations SS (Cornell) Strong Converse for TAI over Noisy Channel June 8, 2020 5 / 15

Proof of Strong Converse in Noiseless Channel Setting: Key tools Covering lemma: [Ahlswede and Csiszar (1986)] T n P : Type class (non-empty) of P on fi- nite alphabet X Non-empty subset B n ⊆ T n P { π i , 1 ≤ i ≤ N } : Set of permutations SS (Cornell) Strong Converse for TAI over Noisy Channel June 8, 2020 5 / 15

Proof of Strong Converse in Noiseless Channel Setting: Key tools Covering lemma: [Ahlswede and Csiszar (1986)] T n P : Type class (non-empty) of P on fi- nite alphabet X Non-empty subset B n ⊆ T n P { π i , 1 ≤ i ≤ N } : Set of permutations SS (Cornell) Strong Converse for TAI over Noisy Channel June 8, 2020 5 / 15

Proof of Strong Converse in Noiseless Channel Setting: Key tools Covering lemma: [Ahlswede and Csiszar (1986)] T n P : Type class (non-empty) of P on fi- nite alphabet X Non-empty subset B n ⊆ T n P { π i , 1 ≤ i ≤ N } : Set of permutations N < |T n P | |B n | log |T n P | permutations exist that cover T n P . SS (Cornell) Strong Converse for TAI over Noisy Channel June 8, 2020 5 / 15

Proof of Strong Converse in Noiseless Channel Setting: Key tools Blowing-up lemma: [Ahlswede-Gács-Körner (1976), Marton (1996)] Let Z 1 , . . . , Z n be n independent r.v.’s taking values in a finite set Z . Then, for any set D ⊆ Z n with P Z n ( D ) > 0 and l n > Θ( √ n ) , P Z n (Γ l n ( D )) ≥ 1 − o ( 1 ) , Γ l ( D ) is Hamming l -neighborhood of D , i.e., n � � z n ∈ Z n : z i ) ≤ l for some z n ∈ D Γ l ( D ) := � ˜ ✶ ( z i � = ˜ . i = 1 SS (Cornell) Strong Converse for TAI over Noisy Channel June 8, 2020 6 / 15

Strong Converse Proof Outline (Noiseless channel) Given: Encoder f n and decision region A n ∈ M × V n such that α n ( f n , g n ) ≤ ǫ . Blowing-up lemma is used to show existence of message 1 m ∗ ∈ M , a set C n ⊆ f − 1 n ( m ∗ ) ⊆ U n and decision rule ¯ g n = 1 − ✶ ¯ A n such that given u n ∈ C n , ( n ) g n | U n = u n ) α n ( f n , ¯ − − → 0 , g n | U n = u n ) ≤ β n ( f n , g n ) e no ( 1 ) . β n ( f n , ¯ Covering lemma is used to guarantee existence of N ≈ e nR 2 permutations π i , i ∈ [ 1 : N ] , of C n that cover typical set T n ( P U , δ ) , δ > 0. M is set as index of permutation, and acceptance region as 3 n = � N i = 1 { i } × π i ( ¯ disjoint union of sets { i } × π i ( ¯ A ∗ � � A n ) A n ) . SS (Cornell) Strong Converse for TAI over Noisy Channel June 8, 2020 7 / 15

Strong Converse Proof Outline (Noiseless channel) Given: Encoder f n and decision region A n ∈ M × V n such that α n ( f n , g n ) ≤ ǫ . Blowing-up lemma is used to show existence of message 1 m ∗ ∈ M , a set C n ⊆ f − 1 n ( m ∗ ) ⊆ U n and decision rule ¯ g n = 1 − ✶ ¯ A n such that given u n ∈ C n , ( n ) g n | U n = u n ) α n ( f n , ¯ − − → 0 , g n | U n = u n ) ≤ β n ( f n , g n ) e no ( 1 ) . β n ( f n , ¯ Covering lemma is used to guarantee existence of N ≈ e nR 2 permutations π i , i ∈ [ 1 : N ] , of C n that cover typical set T n ( P U , δ ) , δ > 0. M is set as index of permutation, and acceptance region as 3 n = � N i = 1 { i } × π i ( ¯ disjoint union of sets { i } × π i ( ¯ A ∗ � � A n ) A n ) . SS (Cornell) Strong Converse for TAI over Noisy Channel June 8, 2020 7 / 15

Strong Converse Proof Outline (Noiseless channel) Given: Encoder f n and decision region A n ∈ M × V n such that α n ( f n , g n ) ≤ ǫ . Blowing-up lemma is used to show existence of message 1 m ∗ ∈ M , a set C n ⊆ f − 1 n ( m ∗ ) ⊆ U n and decision rule ¯ g n = 1 − ✶ ¯ A n such that given u n ∈ C n , ( n ) g n | U n = u n ) α n ( f n , ¯ − − → 0 , g n | U n = u n ) ≤ β n ( f n , g n ) e no ( 1 ) . β n ( f n , ¯ Covering lemma is used to guarantee existence of N ≈ e nR 2 permutations π i , i ∈ [ 1 : N ] , of C n that cover typical set T n ( P U , δ ) , δ > 0. M is set as index of permutation, and acceptance region as 3 n = � N i = 1 { i } × π i ( ¯ disjoint union of sets { i } × π i ( ¯ A ∗ � � A n ) A n ) . SS (Cornell) Strong Converse for TAI over Noisy Channel June 8, 2020 7 / 15

Strong Converse Proof Outline (Noiseless channel) Given: Encoder f n and decision region A n ∈ M × V n such that α n ( f n , g n ) ≤ ǫ . Blowing-up lemma is used to show existence of message 1 m ∗ ∈ M , a set C n ⊆ f − 1 n ( m ∗ ) ⊆ U n and decision rule ¯ g n = 1 − ✶ ¯ A n such that given u n ∈ C n , ( n ) g n | U n = u n ) α n ( f n , ¯ − − → 0 , g n | U n = u n ) ≤ β n ( f n , g n ) e no ( 1 ) . β n ( f n , ¯ Covering lemma is used to guarantee existence of N ≈ e nR 2 permutations π i , i ∈ [ 1 : N ] , of C n that cover typical set T n ( P U , δ ) , δ > 0. M is set as index of permutation, and acceptance region as 3 n = � N i = 1 { i } × π i ( ¯ disjoint union of sets { i } × π i ( ¯ A ∗ � � A n ) A n ) . SS (Cornell) Strong Converse for TAI over Noisy Channel June 8, 2020 7 / 15

Strong Converse: Noisy channel Noisy channel case: ◮ Mapping from encoder to channel output is stochastic ⇒ Message m ∗ as in the noisless channel case does not exist. ⇒ Acceptance region cannot be obtained as disjoint union of set permutations. Theorem (Strong Converse) κ ( ǫ ) = θ ( P UV , C ) := max { I ( V ; W ) : I ( U ; W ) ≤ C , V − U − W } , ∀ ǫ ∈ ( 0 , 1 ) . Proof uses a combination of Blowing-up lemma and change of measure technique [Tyagi and Watanabe (2020)]. SS (Cornell) Strong Converse for TAI over Noisy Channel June 8, 2020 8 / 15

Strong Converse: Noisy channel Noisy channel case: ◮ Mapping from encoder to channel output is stochastic ⇒ Message m ∗ as in the noisless channel case does not exist. ⇒ Acceptance region cannot be obtained as disjoint union of set permutations. Theorem (Strong Converse) κ ( ǫ ) = θ ( P UV , C ) := max { I ( V ; W ) : I ( U ; W ) ≤ C , V − U − W } , ∀ ǫ ∈ ( 0 , 1 ) . Proof uses a combination of Blowing-up lemma and change of measure technique [Tyagi and Watanabe (2020)]. SS (Cornell) Strong Converse for TAI over Noisy Channel June 8, 2020 8 / 15

Proof (Outline): Given: A sequence of ( f n , g n ) pairs specified by P X n | U n and A n such that α n ( f n , g n ) ≤ ǫ . Obtain reliable decoding set using Blowing-up lemma: 1 � A ( v n ) × { v n } , A n := v n ∈V n A ( v n ) := { y n ∈ Y n : ( y n , v n ) ∈ A n } . ( u n , v n , x n ) : P X n | U n ( x n | u n ) > 0 ,     Let B n :=  . P Y n | X n ( A ( v n ) | x n ) ≥ 1 − ǫ  2 ≥ 1 − ǫ � ( U n , V n , X n ) ∈ B n � � Then, P � H = 0 1 + ǫ . SS (Cornell) Strong Converse for TAI over Noisy Channel June 8, 2020 9 / 15