Lower Bounds on the Probability of a Finite Union of Events Jun - - PowerPoint PPT Presentation

Lower Bounds on the Probability of a Finite Union of Events

Jun Yang (joint work with Fady Alajaji and Glen Takahara)

Department of Mathematics and Statistics, Queen’s University, Kingston, Canada

ISIT 2014

Jun Yang, et al. (Queen’s University) Lower Bounds on P(N

i=1 Ai )

ISIT 2014 1 / 19

Outline

1

Problem Formulation

2

Existing Work Dawson-Sankoff Bound Kuai-Alajaji-Takahara Bound

3

New Lower Bounds New Analytical Lower Bound New Optimal Lower Bound

4

Numerical Examples

5

References

Jun Yang, et al. (Queen’s University) Lower Bounds on P(N

i=1 Ai )

ISIT 2014 2 / 19

Problem Formulation

Problem Formulation

Consider a finite family of events A1, . . . , AN in a finite probability space (Ω, F, P), where N is a fixed positive integer. We are interested in lower bounds of P N

i=1 Ai

• in terms of the

individual event probabilities P(Ai)’s and the pairwise event probabilities P(Ai ∩ Aj)’s. For example, P N

• i=1

Ai

• ≥ max

i

P(Ai). (1) P N

• i=1

Ai

• ≥
• i

P(Ai) −

• i<j

P(Ai ∩ Aj). (2)

Jun Yang, et al. (Queen’s University) Lower Bounds on P(N

i=1 Ai )

ISIT 2014 3 / 19

Problem Formulation

Problem Formulation

Assume a vector θ represents partial information of P N

i=1 Ai

• .

That is, each element of θ equals to a (linear) function of P(Ai)’s and P(Ai ∩ Aj)’s. For example, θ = (P(A1), P(A2), . . . , P(AN)) . (3) θ =  

i

P(Ai),

• i<j

P(Ai ∩ Aj)   . (4) Then a lower bound of P N

i=1 Ai

• is a function of θ, ℓ(θ), such that

P N

• i=1

Ai

• ≥ ℓ(θ),

(5) for any {Ai} that satisfy the partial information represented by θ.

Jun Yang, et al. (Queen’s University) Lower Bounds on P(N

i=1 Ai )

ISIT 2014 4 / 19

Problem Formulation

Problem Formulation

For a given definition of θ, for example, θ = (P(A1), · · · , P(AN)), there are many lower bounds that are functions of only θ: P N

• i=1

Ai

• ≥ θ1 = P(A1),

P N

• i=1

Ai

• ≥
• i θi

N =

• i P(Ai)

N , P N

• i=1

Ai

• ≥ max

i

θi = max

i

P(Ai). (6) What is the optimal lower bound in the class of lower bounds that are functions of θ?

Jun Yang, et al. (Queen’s University) Lower Bounds on P(N

i=1 Ai )

ISIT 2014 5 / 19

Problem Formulation

Problem Formulation

Let Θ denote the set of all possible values of θ (for a given definition

• f θ) and LΘ the set of all lower bounds on P

N

i=1 Ai

• that are

functions of only θ. Definition We say that a lower bound ℓ⋆ ∈ LΘ is optimal in LΘ if ℓ⋆(θ) ≥ ℓ(θ) for all θ ∈ Θ and ℓ ∈ LΘ. Is ℓ(θ) = maxi θi = maxi P(Ai) optimal in the class of lower bounds that are functions of θ = (P(A1), . . . , P(AN))? How to prove a lower bound is optimal?

Jun Yang, et al. (Queen’s University) Lower Bounds on P(N

i=1 Ai )

ISIT 2014 6 / 19

Problem Formulation

Problem Formulation

Definition We say that a lower bound ℓ ∈ LΘ is achievable if for every θ ∈ Θ, inf

A1,...,AN

P N

• i=1

Ai

• = ℓ(θ),

(7) where the infimum ranges over all collections {A1, . . . , AN}, Ai ∈ F, such that {A1, . . . , AN} is represented by θ. Lemma A lower bound ℓ⋆ ∈ LΘ is optimal in LΘ if and only if it is achievable.

Jun Yang, et al. (Queen’s University) Lower Bounds on P(N

i=1 Ai )

ISIT 2014 7 / 19

Problem Formulation

Problem Formulation

We can therefore prove optimality by proving achievability: Step 1: prove ℓ(θ) is a lower bound. Step 2: prove for any value of θ ∈ Θ, one can construct {A∗

i } such

that P (

i A∗ i ) = ℓ(θ).

For example, P N

i=1 Ai

• ≥ maxi P(Ai) is the optimal lower bound in the class of

lower bounds that are functions of θ = (P(A1), . . . , P(AN)). P N

i=1 Ai

• ≥

i P(Ai) − i<j P(Ai ∩ Aj) is not optimal lower

bound in the class of lower bounds that are functions of θ =

• i P(Ai),

i<j P(Ai ∩ Aj)

• .

Jun Yang, et al. (Queen’s University) Lower Bounds on P(N

i=1 Ai )

ISIT 2014 8 / 19

Existing Work Dawson-Sankoff Bound

Dawson-Sankoff (DS) Bound, 1967

For each outcome x ∈ F, let the degree of x, denoted by deg(x), be the number of Ai’s that contain x. Define ak := P ({x ∈

i Ai, deg(x) = k}), then one can verify

P

• i

Ai

• =

N

• k=1

ak,

• i

P(Ai) =

N

• k=1

kak,

• i<j

P(Ai ∩ Aj) =

N

• k=2

k(k − 1) 2 ak. (8)

Jun Yang, et al. (Queen’s University) Lower Bounds on P(N

i=1 Ai )

ISIT 2014 9 / 19

Existing Work Dawson-Sankoff Bound

Dawson-Sankoff (DS) Bound, 1967

The Dawson-Sankoff (DS) bound is the solution of the following linear programming (LP) problem: min

{ak≥0} N

• k=1

ak, s.t.

N

• k=1

kak =

• i

P(Ai),

N

• k=1

k(k − 1) 2 ak =

• i<j

P(Ai ∩ Aj). (9) The DS Bound is optimal in the class of lower bounds that are functions of θ =

• i P(Ai),

i<j P(Ai ∩ Aj)

• =: (θ1, θ2),

P N

• i=1

Ai

• ≥

κθ2

1

(2 − κ)θ1 + 2θ2 + (1 − κ)θ2

1

(1 − κ)θ1 + 2θ2 , (10)

where κ = 2θ2

θ1 − ⌊ 2θ2 θ1 ⌋ and ⌊x⌋ denotes the largest integer less than

• r equal to x.

Jun Yang, et al. (Queen’s University) Lower Bounds on P(N

i=1 Ai )

ISIT 2014 10 / 19

Existing Work Kuai-Alajaji-Takahara Bound

Kuai-Alajaji-Takahara (KAT) Bound, 2000

Define ai(k) = P ({x ∈ Ai, deg(x) = k).Recall that ak := P ({x ∈

i Ai, deg(x) = k}), one can verify N

• i=1

ai(k) = kak, ⇒ P

• i

Ai

• =
• k

ak =

• k
• i

ai(k) k , P(Ai) =

N

• k=1

ai(k),

• j:j=i

P(Ai ∩ Aj) =

N

• k=2

(k − 1)ai(k). (11) The KAT bound is the solution of the following LP problem: min

{ai(k)≥0} N

• k=1

N

• i=1

ai(k) k , s.t.

N

• k=1

ai(k) = P(Ai), i = 1, . . . , N,

N

• k=1

(k − 1)ai(k) =

• j:j=i

P(Ai ∩ Aj), i = 1, . . . , N. (12)

Jun Yang, et al. (Queen’s University) Lower Bounds on P(N

i=1 Ai )

ISIT 2014 11 / 19

Existing Work Kuai-Alajaji-Takahara Bound

Kuai-Alajaji-Takahara (KAT) Bound, 2000

Let αi := P(Ai), γi :=

j P(Ai ∩ Aj) = P(Ai) + j:j=i P(Ai ∩ Aj).

The KAT bound, P N

• i=1

Ai

• ≥

N

• i=1
• 1

⌊ γi

αi ⌋ − γi αi − ⌊ γi αi ⌋

(1 + ⌊ γi

αi ⌋)(⌊ γi αi ⌋)

• αi
• ,

(13) where ⌊x⌋ is the largest positive integer less than or equal to x, is not optimal for θ =

• P(A1), . . . , P(AN),

j:j=1 P(A1 ∩ Aj), . . . , j:j=N P(AN ∩ Aj)

• .

Jun Yang, et al. (Queen’s University) Lower Bounds on P(N

i=1 Ai )

ISIT 2014 12 / 19

New Lower Bounds

New Lower Bounds which are sharper than KAT Bound

Recall that ai(k) = P ({x ∈ Ai, deg(x) = k), then we observe ai(N) = P ({x ∈ Ai, deg(x) = N) However, deg(x) = N ⇔ x ∈ Ai for all i, therefore a1(N) = a2(N) = · · · = aN(N). Furthermore, by the definitions of ak := P ({x ∈

i Ai, deg(x) = k})

and ai(k), we observe that ak ≥ ai(k) for all i and all k. Also, since ak =

• i ai(k)

k

, one can write

• i ai(k)

k ≥ ai(k) for all i and all k. Note that when k = N,

• i ai(k)

k

≥ ai(k) reduces to a1(N) = a2(N) = · · · = aN(N).

Jun Yang, et al. (Queen’s University) Lower Bounds on P(N

i=1 Ai )

ISIT 2014 13 / 19

New Lower Bounds New Analytical Lower Bound

New analytical Lower Bound

The new analytical lower bound is the solution of the LP problem:

min

{ai(k)≥0} N

• k=1

N

• i=1

ai(k) k , s.t.

N

• k=1

ai(k) = P(Ai), i = 1, . . . , N,

N

• k=1

(k − 1)ai(k) =

• j:j=i

P(Ai ∩ Aj), i = 1, . . . , N, a1(N) = a2(N) = · · · = aN(N). (14)

The new analytical lower bound is given by

P N

• i=1

Ai

• ≥ δ +

N

• i=1

     1 χ( γ′

i

α′

i )

−

γ′

i

α′

i − χ( γ′ i

α′

i )

[1 + χ( γ′

i

α′

i )][χ( γ′ i

α′

i )]

  α′

i

   , (15)

where δ := {maxi [γi − (N − 1)αi]}+ ≥ 0, α′

i := αi − δ, γ′ i := γi − Nδ, and

χ(x) :=

• n − 1

if x = n where n ≥ 2 is a integer ⌊x⌋

• therwise

Jun Yang, et al. (Queen’s University) Lower Bounds on P(N

i=1 Ai )

ISIT 2014 14 / 19

New Lower Bounds New Analytical Lower Bound

New Analytical Lower Bound

Let ℓNEW denote the new analytical bound and ℓKAT denote the KAT lower bound. Then the improvement of the new analytical bound over the existing KAT bound, i.e., ℓNEW − ℓKAT, satisfies the following inequality ℓNEW − ℓKAT ≥   

N

• i=1
• N − χ( γi

αi )

N − χ( γi

αi ) − 1

• χ( γi

αi )

• χ( γi

αi ) + 1

• 

  δ N ≥ 0. (16) The new analytical lower bound is still not optimal for θ =

• P(A1), . . . , P(AN),

j:j=1 P(A1 ∩ Aj), . . . , j:j=N P(AN ∩ Aj)

• .

Jun Yang, et al. (Queen’s University) Lower Bounds on P(N

i=1 Ai )

ISIT 2014 15 / 19

New Lower Bounds New Optimal Lower Bound

New Optimal Lower Bound

The solution of the following LP problem: min

{ai(k)} N

• k=1

N

• i=1

ai(k) k , s.t.

N

• k=1

ai(k) = P(Ai), i = 1, . . . , N,

N

• k=1

(k − 1)ai(k) =

• j:j=i

P(Ai ∩ Aj), i = 1, . . . , N,

• i ai(k)

k ≥ ai(k), i = 1, . . . , N, k = 1, . . . , N, ai(k) ≥ 0, k = 1, . . . , N, i = 1, . . . , N. (17) is optimal in the class of lower bounds which are functions of θ =

• P(A1), . . . , P(AN),

j:j=1 P(A1 ∩ Aj), . . . , j:j=N P(AN ∩ Aj)

• .

Jun Yang, et al. (Queen’s University) Lower Bounds on P(N

i=1 Ai )

ISIT 2014 16 / 19

New Lower Bounds New Optimal Lower Bound

New Optimal Lower Bound

We gave a construction proof for the achievability in the paper. We could also obtain the optimal upper bound for θ =

• P(A1), . . . , P(AN),

j:j=1 P(A1 ∩ Aj), . . . , j:j=N P(AN ∩ Aj)

• .

All bounds can be applied to any general probability of error estimation problem, including channel coding.

Jun Yang, et al. (Queen’s University) Lower Bounds on P(N

i=1 Ai )

ISIT 2014 17 / 19

Numerical Examples

Numerical Examples

Table: Comparison of Lower Bounds.

System P N

i=1 Ai

• DS

KAT New Bound 1 New Bound 2 I 0.7890 0.7007 0.7247 0.7247 0.7487 II 0.6740 0.6150 0.6227 0.6227 0.6398 III 0.7890 0.6933 0.7222 0.7222 0.7427 IV 0.9687 0.8879 0.8909 0.8909 0.9044 V 0.3900 0.3800 0.3833 0.3900 0.3900 VI 0.3252 0.2706 0.2769 0.3205 0.3252 VII 0.5346 0.3989 0.4434 0.4562 0.5090 VIII 0.5854 0.5395 0.5412 0.5464 0.5513

Jun Yang, et al. (Queen’s University) Lower Bounds on P(N

i=1 Ai )

ISIT 2014 18 / 19

References

References

Dawson-Sankoff (DS) Bound, 1967

• D. Dawson, D. Sankoff, An inequality for probabilities, Proceedings of

the American Mathematical Society 18 (3) (1967) 504-507. Kuai-Alajaji-Takahara (KAT) Bound, 2000

• H. Kuai, F. Alajaji, G. Takahara, A lower bound on the probability of

a finite union of events, Discrete Mathematics 215 (1-3) (2000) 147-158. The New Bounds

• J. Yang, F. Alajaji, G. Takahara, Lower Bounds on the Probability of

a Finite Union of Events, submitted. Online: http://arxiv.org/abs/1401.5543

Jun Yang, et al. (Queen’s University) Lower Bounds on P(N

i=1 Ai )

ISIT 2014 19 / 19