On Bounding the Union Probability Jun Yang 1 (Joint work with Fady - PowerPoint PPT Presentation

On Bounding the Union Probability Jun Yang 1 (Joint work with Fady Alajaji 2 and Glen Takahara 2 ) 1 Department of Statistical Sciences, University of Toronto, Canada 2 Department of Mathematics and Statistics, Queen’s University, Canada IEEE ISIT 2015, Hong Kong On Bounding P ( � N J. Yang, et al. i =1 A i ) ISIT 2015 1 / 29

Outline Problem Formulation 1 Recap: Bounds using { P ( A i ) } and { � j P ( A i ∩ A j ) } 2 New Bounds using { P ( A i ) } and { � j c j P ( A i ∩ A j ) } 3 New Bounds using { P ( A i ) } and { P ( A i ∩ A j ) } 4 Summary of Main Results 5 On Bounding P ( � N J. Yang, et al. i =1 A i ) ISIT 2015 2 / 29

Problem Formulation Outline Problem Formulation 1 Recap: Bounds using { P ( A i ) } and { � j P ( A i ∩ A j ) } 2 New Bounds using { P ( A i ) } and { � j c j P ( A i ∩ A j ) } 3 New Bounds using { P ( A i ) } and { P ( A i ∩ A j ) } 4 Summary of Main Results 5 On Bounding P ( � N J. Yang, et al. i =1 A i ) ISIT 2015 3 / 29

Problem Formulation Problem Formulation Bounds on the union probability are very useful in estimating the error probability in (coded or uncoded) communications systems. For example, � N � �� � � max P ( A i ) ≤ P ≤ min P ( A i ) , 1 . (1) A i i i =1 i � N � � � � P A i ≥ P ( A i ) − P ( A i ∩ A j ) . (2) i =1 i < j i Consider a finite family of events A 1 , . . . , A N in a finite discrete probability space (Ω , F , P ), where N is a fixed positive integer. �� N � We are interested in bounding P in terms of the individual i =1 A i event probabilities P ( A i )’s and the pairwise event probabilities P ( A i ∩ A j )’s. On Bounding P ( � N J. Yang, et al. i =1 A i ) ISIT 2015 4 / 29

Problem Formulation Dawson-Sankoff (DS) Bound, 1967 For each outcome x ∈ Ω, let the degree of x , denoted by deg( x ), be the number of A i ’s that contain x . Define a ( k ) := P ( { x ∈ � i A i , deg( x ) = k } ), then one can verify �� � N � = a ( k ) , P A i i k =1 N � � P ( A i ) = ka ( k ) , (3) k =1 i N � k � � � P ( A i ∩ A j ) = a ( k ) . 2 i < j k =2 On Bounding P ( � N J. Yang, et al. i =1 A i ) ISIT 2015 5 / 29

Problem Formulation Dawson-Sankoff (DS) Bound, 1967 �� � i P ( A i ) , � Using ( θ 1 , θ 2 ) := i < j P ( A i ∩ A j ) , the Dawson-Sankoff (DS) Bound: � N � κθ 2 (1 − κ ) θ 2 � 1 1 ≥ + , (4) P A i (2 − κ ) θ 1 + 2 θ 2 (1 − κ ) θ 1 + 2 θ 2 i =1 where κ = 2 θ 2 θ 1 − ⌊ 2 θ 2 θ 1 ⌋ and ⌊ x ⌋ denotes the largest integer less than or equal to x , is the solution of the linear programming (LP) problem: N N � � � min a ( k ) , ka ( k ) = P ( A i ) , s.t. { a ( k ) } k =1 k =1 i N (5) � k � � � a ( k ) = P ( A i ∩ A j ) , 2 k =2 i < j a ( k ) ≥ 0 , k = 1 , . . . , N . On Bounding P ( � N J. Yang, et al. i =1 A i ) ISIT 2015 6 / 29

Recap: Bounds using { P ( A i ) } and { � j P ( A i ∩ A j ) } Outline Problem Formulation 1 Recap: Bounds using { P ( A i ) } and { � j P ( A i ∩ A j ) } 2 New Bounds using { P ( A i ) } and { � j c j P ( A i ∩ A j ) } 3 New Bounds using { P ( A i ) } and { P ( A i ∩ A j ) } 4 Summary of Main Results 5 On Bounding P ( � N J. Yang, et al. i =1 A i ) ISIT 2015 7 / 29

Recap: Bounds using { P ( A i ) } and { � j P ( A i ∩ A j ) } Kuai-Alajaji-Takahara (KAT) Bound, 2000 Remind that a ( k ) := P ( { x ∈ � i A i , deg( x ) = k } ) Define a i ( k ) = P ( { x ∈ A i , deg( x ) = k ), one can verify �� � N a i ( k ) � � � � a i ( k ) = ka ( k ) , ⇒ P A i = a ( k ) = , k i =1 i k k i (6) N N � � � P ( A i ) = a i ( k ) , P ( A i ∩ A j ) = ( k − 1) a i ( k ) . k =1 j : j � = i k =2 We are able to use � � P ( A 1 ) , . . . , P ( A N ) , � j : j � =1 P ( A 1 ∩ A j ) , . . . , � j : j � = N P ( A N ∩ A j ) . On Bounding P ( � N J. Yang, et al. i =1 A i ) ISIT 2015 8 / 29

Recap: Bounds using { P ( A i ) } and { � j P ( A i ∩ A j ) } Kuai-Alajaji-Takahara (KAT) Bound, 2000 γ i := � j P ( A i ∩ A j ) = P ( A i ) + � Let α i := P ( A i ) , j : j � = i P ( A i ∩ A j ). The KAT bound is the solution of the following LP problem: N N N a i ( k ) � � � min , a i ( k ) = α i , i = 1 , . . . , N , s.t. k { a i ( k ) ≥ 0 } k =1 i =1 k =1 (7) N � ka i ( k ) = γ i , i = 1 , . . . , N . k =1 which is given by � N α i − ⌊ γ i γ i � N �� � � α i ⌋ 1 � � P A i ≥ α i ⌋ − α i , (8) ⌊ γ i (1 + ⌊ γ i α i ⌋ )( ⌊ γ i α i ⌋ ) i =1 i =1 where ⌊ x ⌋ is the largest positive integer less than or equal to x . On Bounding P ( � N J. Yang, et al. i =1 A i ) ISIT 2015 9 / 29

Recap: Bounds using { P ( A i ) } and { � j P ( A i ∩ A j ) } Lower Bounds which are sharper than KAT Bound Recall that a ( k ) := P ( { x ∈ � i A i , deg( x ) = k } ) and a i ( k ) = P ( { x ∈ A i , deg( x ) = k ), then we observe a ( k ) ≥ a i ( k ) for all � j a j ( k ) i and all k . Also, since a ( k ) = , one can write k � j a j ( k ) ≥ a i ( k ) k for all i and all k . As a special case for k = N , it reduces to a 1 ( N ) = a 2 ( N ) = · · · = a N ( N ) . On Bounding P ( � N J. Yang, et al. i =1 A i ) ISIT 2015 10 / 29

Recap: Bounds using { P ( A i ) } and { � j P ( A i ∩ A j ) } Optimal Lower Bound ℓ NEW-1 (in ISIT’14) The solution of the following LP problem: N N a i ( k ) � � min , k { a i ( k ) } k =1 i =1 N � s.t. a i ( k ) = P ( A i ) , i = 1 , . . . , N , k =1 (9) N � � ( k − 1) a i ( k ) = P ( A i ∩ A j ) , i = 1 , . . . , N , k =1 j : j � = i � j a j ( k ) ≥ a i ( k ) , i = 1 , . . . , N , k = 1 , . . . , N , k a i ( k ) ≥ 0 , k = 1 , . . . , N , i = 1 , . . . , N . is optimal in the class of lower bounds which are functions of � � P ( A 1 ) , . . . , P ( A N ) , � j : j � =1 P ( A 1 ∩ A j ) , . . . , � j : j � = N P ( A N ∩ A j ) . On Bounding P ( � N J. Yang, et al. i =1 A i ) ISIT 2015 11 / 29

Recap: Bounds using { P ( A i ) } and { � j P ( A i ∩ A j ) } Analytical Lower Bound ℓ NEW-2 (in ISIT’14) The new analytical lower bound is the solution of the LP problem: N N N a i ( k ) � � � min , a i ( k ) = P ( A i ) , i = 1 , . . . , N , s.t. k { a i ( k ) ≥ 0 } i =1 k =1 k =1 N (10) � � ( k − 1) a i ( k ) = P ( A i ∩ A j ) , i = 1 , . . . , N , k =1 j : j � = i a 1 ( N ) = a 2 ( N ) = · · · = a N ( N ) . The new analytical lower bound is given by � N   γ ′ i − χ ( γ ′   � N i ) i i 1  α ′ α ′  � �  α ′ ≥ δ + −  , (11) P A i  i χ ( γ ′ [1 + χ ( γ ′ i )][ χ ( γ ′ i i ) i i i )]  i =1 i =1 α ′ α ′ α ′ where δ := { max i [ γ i − ( N − 1) α i ] } + ≥ 0 , α ′ i := α i − δ, γ ′ i := γ i − N δ , and � n − 1 if x = n where n ≥ 2 is a integer χ ( x ) := ⌊ x ⌋ otherwise On Bounding P ( � N J. Yang, et al. i =1 A i ) ISIT 2015 12 / 29

New Bounds using { P ( A i ) } and { � j c j P ( A i ∩ A j ) } Outline Problem Formulation 1 Recap: Bounds using { P ( A i ) } and { � j P ( A i ∩ A j ) } 2 New Bounds using { P ( A i ) } and { � j c j P ( A i ∩ A j ) } 3 New Bounds using { P ( A i ) } and { P ( A i ∩ A j ) } 4 Summary of Main Results 5 On Bounding P ( � N J. Yang, et al. i =1 A i ) ISIT 2015 13 / 29

New Bounds using { P ( A i ) } and { � j c j P ( A i ∩ A j ) } Gallot-Kounias (GK) Bound, 1968 The GK bound is an analytical bound which fully uses { P ( A i ) } and { P ( A i ∩ A j ) } . Recently, it was re-visited by Feng-Li-Shen 1 that the GK bound can be obtained by � N � i c i P ( A i )] 2 [ � � ≥ ℓ GK = max k c k P ( A i ∩ A k ) . (12) P A i � � i c i c ∈ R N i − 1 Ignoring the maximization over c , the RHS is a lower bound using � i c i P ( A i ) and � k c k P ( A i ∩ A k ). 1 “Some inequalities in functional analysis, combinatorics, and probability theory”, The Electronic Journal of Combinatorics, 2010. On Bounding P ( � N J. Yang, et al. i =1 A i ) ISIT 2015 14 / 29

New Bounds using { P ( A i ) } and { � j c j P ( A i ∩ A j ) } �� N � New expressions for P i =1 A i Denote B as the collection of all non-empty subsets of { 1 , 2 , . . . , N } and let B ∈ B be a non-empty subset of { 1 , 2 , . . . , N } , p B := p ( { ω B , ω B ∈ A i for all i ∈ B , ω B / ∈ A i for all i / ∈ B } ) . (13) �� N � Then we have a new (novel) expression of P i =1 A i for any given c : � N � N � � c i p B � � � � = p B = . (14) P A i � k ∈ B c k i =1 B ∈ B i =1 B ∈ B : i ∈ B Furthermore, we have � P ( A i ) = p B , B ∈ B : i ∈ B (15) N �� � � � � � c k P ( A i ∩ A k ) = c k p B = c k p B . k =1 B : i ∈ B , k ∈ B B : i ∈ B k ∈ B k On Bounding P ( � N J. Yang, et al. i =1 A i ) ISIT 2015 15 / 29