Lower Bounds on the Probability of a Finite Union of Events Jun - PowerPoint PPT Presentation

Lower Bounds on the Probability of a Finite Union of Events Jun Yang (joint work with Fady Alajaji and Glen Takahara) Department of Mathematics and Statistics, Queen’s University, Kingston, Canada ISIT 2014 Lower Bounds on P ( � N Jun Yang, et al. (Queen’s University) i =1 A i ) ISIT 2014 1 / 19

Outline Problem Formulation 1 Existing Work 2 Dawson-Sankoff Bound Kuai-Alajaji-Takahara Bound New Lower Bounds 3 New Analytical Lower Bound New Optimal Lower Bound Numerical Examples 4 References 5 Lower Bounds on P ( � N Jun Yang, et al. (Queen’s University) i =1 A i ) ISIT 2014 2 / 19

Problem Formulation Problem Formulation Consider a finite family of events A 1 , . . . , A N in a finite probability space (Ω , F , P ), where N is a fixed positive integer. �� N � We are interested in lower bounds of P i =1 A i in terms of the individual event probabilities P ( A i )’s and the pairwise event probabilities P ( A i ∩ A j )’s. For example, � N � � ≥ max P ( A i ) . (1) P A i i i =1 � N � � � � ≥ P ( A i ) − P ( A i ∩ A j ) . (2) P A i i =1 i i < j Lower Bounds on P ( � N Jun Yang, et al. (Queen’s University) i =1 A i ) ISIT 2014 3 / 19

Problem Formulation Problem Formulation �� N � Assume a vector θ represents partial information of P i =1 A i . That is, each element of θ equals to a (linear) function of P ( A i )’s and P ( A i ∩ A j )’s. For example, θ = ( P ( A 1 ) , P ( A 2 ) , . . . , P ( A N )) . (3)   � �  . θ = P ( A i ) , P ( A i ∩ A j ) (4) i i < j �� N � Then a lower bound of P is a function of θ , ℓ ( θ ), such that i =1 A i � N � � P A i ≥ ℓ ( θ ) , (5) i =1 for any { A i } that satisfy the partial information represented by θ . Lower Bounds on P ( � N Jun Yang, et al. (Queen’s University) i =1 A i ) ISIT 2014 4 / 19

Problem Formulation Problem Formulation For a given definition of θ , for example, θ = ( P ( A 1 ) , · · · , P ( A N )), there are many lower bounds that are functions of only θ : � N � � P A i ≥ θ 1 = P ( A 1 ) , i =1 � N � � � i θ i i P ( A i ) � ≥ = (6) P A i , N N i =1 � N � � P A i ≥ max θ i = max P ( A i ) . i i i =1 What is the optimal lower bound in the class of lower bounds that are functions of θ ? Lower Bounds on P ( � N Jun Yang, et al. (Queen’s University) i =1 A i ) ISIT 2014 5 / 19

Problem Formulation Problem Formulation Let Θ denote the set of all possible values of θ (for a given definition �� N � of θ ) and L Θ the set of all lower bounds on P i =1 A i that are functions of only θ . Definition We say that a lower bound ℓ ⋆ ∈ L Θ is optimal in L Θ if ℓ ⋆ ( θ ) ≥ ℓ ( θ ) for all θ ∈ Θ and ℓ ∈ L Θ . Is ℓ ( θ ) = max i θ i = max i P ( A i ) optimal in the class of lower bounds that are functions of θ = ( P ( A 1 ) , . . . , P ( A N ))? How to prove a lower bound is optimal? Lower Bounds on P ( � N Jun Yang, et al. (Queen’s University) i =1 A i ) ISIT 2014 6 / 19

Problem Formulation Problem Formulation Definition We say that a lower bound ℓ ∈ L Θ is achievable if for every θ ∈ Θ, � N � � inf = ℓ ( θ ) , (7) P A i A 1 ,..., A N i =1 where the infimum ranges over all collections { A 1 , . . . , A N } , A i ∈ F , such that { A 1 , . . . , A N } is represented by θ . Lemma A lower bound ℓ ⋆ ∈ L Θ is optimal in L Θ if and only if it is achievable. Lower Bounds on P ( � N Jun Yang, et al. (Queen’s University) i =1 A i ) ISIT 2014 7 / 19

Problem Formulation Problem Formulation We can therefore prove optimality by proving achievability: Step 1: prove ℓ ( θ ) is a lower bound. Step 2: prove for any value of θ ∈ Θ, one can construct { A ∗ i } such i A ∗ that P ( � i ) = ℓ ( θ ). For example, �� N � ≥ max i P ( A i ) is the optimal lower bound in the class of P i =1 A i lower bounds that are functions of θ = ( P ( A 1 ) , . . . , P ( A N )). �� N � ≥ � i P ( A i ) − � P i =1 A i i < j P ( A i ∩ A j ) is not optimal lower bound in the class of lower bounds that are functions of �� � i P ( A i ) , � θ = i < j P ( A i ∩ A j ) . Lower Bounds on P ( � N Jun Yang, et al. (Queen’s University) i =1 A i ) ISIT 2014 8 / 19

Existing Work Dawson-Sankoff Bound Dawson-Sankoff (DS) Bound, 1967 For each outcome x ∈ F , let the degree of x , denoted by deg( x ), be the number of A i ’s that contain x . Define a k := P ( { x ∈ � i A i , deg( x ) = k } ), then one can verify �� � N � = P A i a k , i k =1 N � � P ( A i ) = ka k , (8) k =1 i N k ( k − 1) � � P ( A i ∩ A j ) = a k . 2 i < j k =2 Lower Bounds on P ( � N Jun Yang, et al. (Queen’s University) i =1 A i ) ISIT 2014 9 / 19

Existing Work Dawson-Sankoff Bound Dawson-Sankoff (DS) Bound, 1967 The Dawson-Sankoff (DS) bound is the solution of the following linear programming (LP) problem: N N � � � min ka k = P ( A i ) , a k , s.t. { a k ≥ 0 } k =1 k =1 i (9) N k ( k − 1) � � a k = P ( A i ∩ A j ) . 2 k =1 i < j The DS Bound is optimal in the class of lower bounds that are �� � i P ( A i ) , � functions of θ = i < j P ( A i ∩ A j ) =: ( θ 1 , θ 2 ), � N � κθ 2 (1 − κ ) θ 2 � 1 1 ≥ + , (10) P A i (2 − κ ) θ 1 + 2 θ 2 (1 − κ ) θ 1 + 2 θ 2 i =1 where κ = 2 θ 2 θ 1 − ⌊ 2 θ 2 θ 1 ⌋ and ⌊ x ⌋ denotes the largest integer less than or equal to x . Lower Bounds on P ( � N Jun Yang, et al. (Queen’s University) i =1 A i ) ISIT 2014 10 / 19

Existing Work Kuai-Alajaji-Takahara Bound Kuai-Alajaji-Takahara (KAT) Bound, 2000 Define a i ( k ) = P ( { x ∈ A i , deg( x ) = k ).Recall that a k := P ( { x ∈ � i A i , deg( x ) = k } ), one can verify N �� � a i ( k ) � � � � a i ( k ) = ka k , ⇒ P A i = a k = , k i =1 i k k i (11) N N � � � P ( A i ) = a i ( k ) , P ( A i ∩ A j ) = ( k − 1) a i ( k ) . k =1 j : j � = i k =2 The KAT bound is the solution of the following LP problem: N N N a i ( k ) � � � min a i ( k ) = P ( A i ) , i = 1 , . . . , N , , s.t. k { a i ( k ) ≥ 0 } k =1 i =1 k =1 N � � ( k − 1) a i ( k ) = P ( A i ∩ A j ) , i = 1 , . . . , N . k =1 j : j � = i (12) Lower Bounds on P ( � N Jun Yang, et al. (Queen’s University) i =1 A i ) ISIT 2014 11 / 19

Existing Work Kuai-Alajaji-Takahara Bound Kuai-Alajaji-Takahara (KAT) Bound, 2000 γ i := � j P ( A i ∩ A j ) = P ( A i ) + � Let α i := P ( A i ) , j : j � = i P ( A i ∩ A j ). The KAT bound, � N γ i α i − ⌊ γ i � N �� � � α i ⌋ 1 � � P A i ≥ α i ⌋ − α i , (13) ⌊ γ i (1 + ⌊ γ i α i ⌋ )( ⌊ γ i α i ⌋ ) i =1 i =1 where ⌊ x ⌋ is the largest positive integer less than or equal to x , is not optimal for θ = � � P ( A 1 ) , . . . , P ( A N ) , � j : j � =1 P ( A 1 ∩ A j ) , . . . , � j : j � = N P ( A N ∩ A j ) . Lower Bounds on P ( � N Jun Yang, et al. (Queen’s University) i =1 A i ) ISIT 2014 12 / 19

New Lower Bounds New Lower Bounds which are sharper than KAT Bound Recall that a i ( k ) = P ( { x ∈ A i , deg( x ) = k ), then we observe a i ( N ) = P ( { x ∈ A i , deg( x ) = N ) However, deg( x ) = N ⇔ x ∈ A i for all i , therefore a 1 ( N ) = a 2 ( N ) = · · · = a N ( N ) . Furthermore, by the definitions of a k := P ( { x ∈ � i A i , deg( x ) = k } ) and a i ( k ), we observe that a k ≥ a i ( k ) for all i and all k . Also, since � i a i ( k ) a k = , one can write k � i a i ( k ) ≥ a i ( k ) k for all i and all k . � i a i ( k ) Note that when k = N , ≥ a i ( k ) reduces to k a 1 ( N ) = a 2 ( N ) = · · · = a N ( N ). Lower Bounds on P ( � N Jun Yang, et al. (Queen’s University) i =1 A i ) ISIT 2014 13 / 19

New Lower Bounds New Analytical Lower Bound New analytical Lower Bound The new analytical lower bound is the solution of the LP problem: N N N a i ( k ) � � � min a i ( k ) = P ( A i ) , i = 1 , . . . , N , , s.t. k { a i ( k ) ≥ 0 } i =1 k =1 k =1 N (14) � � ( k − 1) a i ( k ) = P ( A i ∩ A j ) , i = 1 , . . . , N , k =1 j : j � = i a 1 ( N ) = a 2 ( N ) = · · · = a N ( N ) . The new analytical lower bound is given by � N   γ ′ i − χ ( γ ′   � N i ) i i 1  α ′ α ′  � �  α ′ ≥ δ + −  , (15) P A i  i χ ( γ ′ [1 + χ ( γ ′ i )][ χ ( γ ′ i i ) i i i )]  i =1 i =1 α ′ α ′ α ′ where δ := { max i [ γ i − ( N − 1) α i ] } + ≥ 0 , α ′ i := α i − δ, γ ′ i := γ i − N δ , and � n − 1 if x = n where n ≥ 2 is a integer χ ( x ) := ⌊ x ⌋ otherwise Lower Bounds on P ( � N Jun Yang, et al. (Queen’s University) i =1 A i ) ISIT 2014 14 / 19