[PPT] - On Bounding the Union Probability Jun Yang 1 (Joint work with Fady PowerPoint Presentation

SLIDE 1

On Bounding the Union Probability

Jun Yang 1 (Joint work with Fady Alajaji2 and Glen Takahara2)

1Department of Statistical Sciences, University of Toronto, Canada 2Department of Mathematics and Statistics, Queen’s University, Canada

IEEE ISIT 2015, Hong Kong

J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 1 / 29

SLIDE 2

Outline

1

Problem Formulation

2

Recap: Bounds using {P(Ai)} and {

j P(Ai ∩ Aj)} 3

New Bounds using {P(Ai)} and {

j cjP(Ai ∩ Aj)} 4

New Bounds using {P(Ai)} and {P(Ai ∩ Aj)}

5

Summary of Main Results

J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 2 / 29

SLIDE 3

Problem Formulation

Outline

1

Problem Formulation

2

Recap: Bounds using {P(Ai)} and {

j P(Ai ∩ Aj)} 3

New Bounds using {P(Ai)} and {

j cjP(Ai ∩ Aj)} 4

New Bounds using {P(Ai)} and {P(Ai ∩ Aj)}

5

Summary of Main Results

J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 3 / 29

SLIDE 4

Problem Formulation

Bounds on the union probability are very useful in estimating the error probability in (coded or uncoded) communications systems. For example, max

i

P(Ai) ≤ P N

i=1

Ai

≤ min
i

P(Ai), 1

.

(1) P N

i=1

Ai

≥
i

P(Ai) −

i<j

P(Ai ∩ Aj). (2) Consider a finite family of events A1, . . . , AN in a finite discrete probability space (Ω, F, P), where N is a fixed positive integer. We are interested in bounding P N

i=1 Ai

in terms of the individual

event probabilities P(Ai)’s and the pairwise event probabilities P(Ai ∩ Aj)’s.

J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 4 / 29

SLIDE 5

Problem Formulation

Dawson-Sankoff (DS) Bound, 1967

For each outcome x ∈ Ω, let the degree of x, denoted by deg(x), be the number of Ai’s that contain x. Define a(k) := P ({x ∈

i Ai, deg(x) = k}), then one can verify

P

i

Ai

=

N

k=1

a(k),

i

P(Ai) =

N

k=1

ka(k),

i<j

P(Ai ∩ Aj) =

N

k=2

k 2

a(k).

(3)

J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 5 / 29

SLIDE 6

Problem Formulation

Dawson-Sankoff (DS) Bound, 1967

Using (θ1, θ2) :=

i P(Ai),

i<j P(Ai ∩ Aj)

, the Dawson-Sankoff

(DS) Bound: P N

i=1

Ai

≥

κθ2

1

(2 − κ)θ1 + 2θ2 + (1 − κ)θ2

1

(1 − κ)θ1 + 2θ2 , (4) where κ = 2θ2

θ1 − ⌊ 2θ2 θ1 ⌋ and ⌊x⌋ denotes the largest integer less than

r equal to x, is the solution of the linear programming (LP) problem:

min

{a(k)} N

k=1

a(k), s.t.

N

k=1

ka(k) =

i

P(Ai),

N

k=2

k 2

a(k) =
i<j

P(Ai ∩ Aj), a(k) ≥ 0, k = 1, . . . , N. (5)

J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 6 / 29

SLIDE 7

Recap: Bounds using {P(Ai )} and {

j P(Ai ∩ Aj )}

Outline

1

Problem Formulation

2

Recap: Bounds using {P(Ai)} and {

j P(Ai ∩ Aj)} 3

New Bounds using {P(Ai)} and {

j cjP(Ai ∩ Aj)} 4

New Bounds using {P(Ai)} and {P(Ai ∩ Aj)}

5

Summary of Main Results

J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 7 / 29

SLIDE 8

Recap: Bounds using {P(Ai )} and {

j P(Ai ∩ Aj )}

Kuai-Alajaji-Takahara (KAT) Bound, 2000

Remind that a(k) := P ({x ∈

i Ai, deg(x) = k})

Define ai(k) = P ({x ∈ Ai, deg(x) = k), one can verify

N

i=1

ai(k) = ka(k), ⇒ P

i

Ai

=
k

a(k) =

k
i

ai(k) k , P(Ai) =

N

k=1

ai(k),

j:j=i

P(Ai ∩ Aj) =

N

k=2

(k − 1)ai(k). (6)

We are able to use

P(A1), . . . , P(AN),

j:j=1 P(A1 ∩ Aj), . . . , j:j=N P(AN ∩ Aj)

.
J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 8 / 29

SLIDE 9

Recap: Bounds using {P(Ai )} and {

j P(Ai ∩ Aj )}

Kuai-Alajaji-Takahara (KAT) Bound, 2000

Let αi := P(Ai), γi :=

j P(Ai ∩ Aj) = P(Ai) + j:j=i P(Ai ∩ Aj).

The KAT bound is the solution of the following LP problem:

min

{ai(k)≥0} N

k=1

N

i=1

ai(k) k , s.t.

N

k=1

ai(k) = αi, i = 1, . . . , N,

N

k=1

kai(k) = γi, i = 1, . . . , N. (7)

which is given by P N

i=1

Ai

≥

N

i=1
1

⌊ γi

αi ⌋ − γi αi − ⌊ γi αi ⌋

(1 + ⌊ γi

αi ⌋)(⌊ γi αi ⌋)

αi
,

(8) where ⌊x⌋ is the largest positive integer less than or equal to x.

J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 9 / 29

SLIDE 10

Recap: Bounds using {P(Ai )} and {

j P(Ai ∩ Aj )}

Lower Bounds which are sharper than KAT Bound

Recall that a(k) := P ({x ∈

i Ai, deg(x) = k}) and

ai(k) = P ({x ∈ Ai, deg(x) = k), then we observe a(k) ≥ ai(k) for all i and all k. Also, since a(k) =

j aj(k)

k

, one can write

j aj(k)

k ≥ ai(k) for all i and all k. As a special case for k = N, it reduces to a1(N) = a2(N) = · · · = aN(N).

J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 10 / 29

SLIDE 11

Recap: Bounds using {P(Ai )} and {

j P(Ai ∩ Aj )}

Optimal Lower Bound ℓNEW-1 (in ISIT’14)

The solution of the following LP problem: min

{ai(k)} N

k=1

N

i=1

ai(k) k , s.t.

N

k=1

ai(k) = P(Ai), i = 1, . . . , N,

N

k=1

(k − 1)ai(k) =

j:j=i

P(Ai ∩ Aj), i = 1, . . . , N,

j aj(k)

k ≥ ai(k), i = 1, . . . , N, k = 1, . . . , N, ai(k) ≥ 0, k = 1, . . . , N, i = 1, . . . , N. (9) is optimal in the class of lower bounds which are functions of

P(A1), . . . , P(AN),

j:j=1 P(A1 ∩ Aj), . . . , j:j=N P(AN ∩ Aj)

.
J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 11 / 29

SLIDE 12

Recap: Bounds using {P(Ai )} and {

j P(Ai ∩ Aj )}

Analytical Lower Bound ℓNEW-2 (in ISIT’14)

The new analytical lower bound is the solution of the LP problem:

min

{ai(k)≥0} N

k=1

N

i=1

ai(k) k , s.t.

N

k=1

ai(k) = P(Ai), i = 1, . . . , N,

N

k=1

(k − 1)ai(k) =

j:j=i

P(Ai ∩ Aj), i = 1, . . . , N, a1(N) = a2(N) = · · · = aN(N). (10)

The new analytical lower bound is given by

P N

i=1

Ai

≥ δ +

N

i=1

     1 χ( γ′

i

α′

i )

−

γ′

i

α′

i − χ( γ′ i

α′

i )

[1 + χ( γ′

i

α′

i )][χ( γ′ i

α′

i )]

  α′

i

   , (11)

where δ := {maxi [γi − (N − 1)αi]}+ ≥ 0, α′

i := αi − δ, γ′ i := γi − Nδ, and

χ(x) :=

n − 1

if x = n where n ≥ 2 is a integer ⌊x⌋

therwise
J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 12 / 29

SLIDE 13

New Bounds using {P(Ai )} and {

j cj P(Ai ∩ Aj )}

Outline

1

Problem Formulation

2

Recap: Bounds using {P(Ai)} and {

j P(Ai ∩ Aj)} 3

New Bounds using {P(Ai)} and {

j cjP(Ai ∩ Aj)} 4

New Bounds using {P(Ai)} and {P(Ai ∩ Aj)}

5

Summary of Main Results

J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 13 / 29

SLIDE 14

New Bounds using {P(Ai )} and {

j cj P(Ai ∩ Aj )}

Gallot-Kounias (GK) Bound, 1968

The GK bound is an analytical bound which fully uses {P(Ai)} and {P(Ai ∩ Aj)}. Recently, it was re-visited by Feng-Li-Shen1 that the GK bound can be obtained by P N

i−1

Ai

≥ ℓGK = max

c∈RN

[

i ciP(Ai)]2

i ci
k ckP(Ai ∩ Ak).

(12) Ignoring the maximization over c, the RHS is a lower bound using

i ciP(Ai) and

k ckP(Ai ∩ Ak).

1“Some inequalities in functional analysis, combinatorics, and probability theory”,

The Electronic Journal of Combinatorics, 2010.

J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 14 / 29

SLIDE 15

New Bounds using {P(Ai )} and {

j cj P(Ai ∩ Aj )}

New expressions for P N

i=1 Ai

Denote B as the collection of all non-empty subsets of {1, 2, . . . , N} and

let B ∈ B be a non-empty subset of {1, 2, . . . , N}, pB := p ({ωB, ωB ∈ Ai for all i ∈ B, ωB / ∈ Ai for all i / ∈ B}) . (13) Then we have a new (novel) expression of P N

i=1 Ai

for any given c:

P N

i=1

Ai

=
B∈B

pB =

N

i=1
B∈B:i∈B

cipB

k∈B ck
.

(14) Furthermore, we have P(Ai) =

B∈B:i∈B

pB,

k

ckP(Ai ∩ Ak) =

N

k=1
B:i∈B,k∈B

ckpB =

B:i∈B
k∈B

ck

pB.

(15)

J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 15 / 29

SLIDE 16

New Bounds using {P(Ai )} and {

j cj P(Ai ∩ Aj )}

New expressions for P N

i=1 Ai

If c ∈ RN

+, by Cauchy-Schwarz inequality

P

i

Ai

=

N

i=1
B∈B:i∈B

cipB

k∈B ck
≥

N

i=1

c2

i P(Ai)2

ci

k ckP(Ai ∩ Ak). (16)

Note that we can use Cauchy-Schwarz Inequality again to get P

i

Ai

≥

N

i=1

c2

i P(Ai)2

ci

k ckP(Ai ∩ Ak) ≥

[

i ciP(Ai)]2

i
k cickP(Ai ∩ Ak).

(17)

J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 16 / 29

SLIDE 17

New Bounds using {P(Ai )} and {

j cj P(Ai ∩ Aj )}

New Class of Lower Bounds ℓNEW-3(c)

The new class of lower bounds ℓNEW-3(c) is given by P N

i=1

Ai

≥

N

i=1

ℓi(c) =: ℓNEW-3(c), (18) where ℓi(c) := min

{pB:i∈B}

B:i∈B

cipB

k∈B ck

s.t.

B:i∈B

pB = P(Ai),

B:i∈B
k∈B ck

ci

pB = 1

ci

k

ckP(Ai ∩ Ak), pB ≥ 0, for all B ∈ B such that i ∈ B. (19)

J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 17 / 29

SLIDE 18

New Bounds using {P(Ai )} and {

j cj P(Ai ∩ Aj )}

New Class of Lower Bounds ℓNEW-3(c)

The solution of ℓi(c) is given by

ℓi(c) = P(Ai)   ci

k∈B(i)

1 ck

+ ci

k∈B(i)

2 ck

− ci

k ckP(Ai ∩ Ak)

P(Ai)

k∈B(i)

1 ck

k∈B(i)

2 ck



 (20)

where B(i)

1

and B(i)

2

are subsets of {1, . . . , N}. For c ∈ RN

+,

B(i)

1

= arg max

{B:i∈B}

k∈B ck

ci s.t.

k∈B ck

ci ≤

k ckP(Ai ∩ Ak)

ciP(Ai) , B(i)

2

= arg min

{B:i∈B}

k∈B ck

ci s.t.

k∈B ck

ci ≥

k ckP(Ai ∩ Ak)

ciP(Ai) , (21)

which are 0/1 Knapsack Problems (Pseudo-Polynomial and Polynomial-time approx). ℓNEW-3(κ1) = ℓKAT for any κ = 0.

J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 18 / 29

SLIDE 19

New Bounds using {P(Ai )} and {

j cj P(Ai ∩ Aj )}

Another Class of Lower Bounds ℓNEW-4(c) for c ∈ RN

+

Defining B− = B \ {1, . . . , N}, ˜ γi :=

k ckP(Ai ∩ Ak), ˜

αi := P(Ai) and

˜ δ := ˜ γi − (

k ck − mink ck) ˜

αi mink ck + , (22)

another class is given by ℓNEW-4(c) := ˜ δ + N

i=1 ℓ′ i(c, ˜

δ), where ℓ′

i(c, x) = [P(Ai) − x]

·   ci

k∈B(i)

1 ck

+ ci

k∈B(i)

2 ck

− ci

k ck [P(Ai ∩ Ak) − x]

[P(Ai) − x]

k∈B(i)

1 ck

k∈B(i)

2 ck



 . (23) ℓNEW-4(κ1) = ℓNEW-2 for any κ > 0; ℓNEW-4(c) ≥ ℓNEW-3(c) if c ∈ RN

+; ℓNEW-4(˜

c) ≥ ℓNEW-3(˜ c) ≥ ℓGK if ˜ c ∈ RN

+;

Optimal c in either class is still open.

J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 19 / 29

SLIDE 20

New Bounds using {P(Ai )} and {P(Ai ∩ Aj )}

Outline

1

Problem Formulation

2

Recap: Bounds using {P(Ai)} and {

j P(Ai ∩ Aj)} 3

New Bounds using {P(Ai)} and {

j cjP(Ai ∩ Aj)} 4

New Bounds using {P(Ai)} and {P(Ai ∩ Aj)}

5

Summary of Main Results

J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 20 / 29

SLIDE 21

New Bounds using {P(Ai )} and {P(Ai ∩ Aj )}

Optimal Bounds with Exponential Complexity

The following (exhaustive) LP problem with 2N − 1 number of variables gives the optimal lower/upper bound established using {P(Ai)} and {P(Ai ∩ Aj)}: min

{pB,B∈B} /

max

{pB,B∈B}

B∈B

pB, s.t.

i,j∈B,B∈B

pB = P(Ai ∩ Aj), i, j ∈ {1, . . . , N}, pB ≥ 0, B ∈ B. (24) The optimality of (24) can be easily proved by showing its achievability: for each pB, construct an outcome ωB such that p(ωB) = pB and let ωB ∈ Ai, ∀i ∈ B. However, the computational complexity of the optimal lower bound (24) is exponential.

J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 21 / 29

SLIDE 22

New Bounds using {P(Ai )} and {P(Ai ∩ Aj )}

A Relaxed Problem

Consider the following relaxed problem:

min

{pB,B∈B} /

max

{pB,B∈B}

B∈B

pB, s.t.

i,j∈B,B∈B

pB = P(Ai ∩ Aj), i, j ∈ {1, . . . , N},

B:i,j,l∈B,|B|=k

pB ≥ 0,

B:i,j∈B,l /

∈B,|B|=k

pB ≥ 0,

B:i∈B,j,l /

∈B,|B|=k

pB ≥ 0,

B:i,j,l /

∈B,|B|=k

pB ≥ 0, ∀i, j, l, k ∈ {1, . . . , N}. (25)

The solution of problem (25) coincides with the optimal lower/upper bound by (24) when N ≤ 7.

J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 22 / 29

SLIDE 23

New Bounds using {P(Ai )} and {P(Ai ∩ Aj )}

The optimal feasible point of (25) is also optimal in

min

{pB,B∈B} /

max

{pB,B∈B}

B∈B

pB, s.t.

i,j∈B,B∈B

pB = P(Ai ∩ Aj), i, j ∈ {1, . . . , N},

B:i,j,l∈B,|B|=k

pB +

B:i,j∈B,l /

∈B,|B|=k

pB ≥ 0,

B:l∈B,i,j /

∈B,|B|=k

pB +

B:i,j,l /

∈B,|B|=k

pB ≥ 0,

B:i,j,l∈B,|B|=k

pB +

B:i,j,l /

∈B,|B|=k

pB ≥ 0,

B:i,j∈B,l /

∈B,|B|=k

pB +

B:l∈B,i,j /

∈B,|B|=k

pB ≥ 0,

B:i,j∈B,|B|=k

pB +

B:i∈B,j,l /

∈B,|B|=k

pB ≥ 0, ∀i, j, l, k ∈ {1, . . . , N}. (26)

J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 23 / 29

SLIDE 24

New Bounds using {P(Ai )} and {P(Ai ∩ Aj )}

New Numerical Bounds

Define aij(k) := P ({x ∈ Ai ∩ Aj, deg(x) = k}) , i, j, k ∈ {1, · · · , N}. (27) Consider aij(k) as (N−1)3+N+3

2

variables. Then a(k) and ai(k) are linear functions of aij(k):

N

j=1

aij(k) k = P ({x ∈ Ai, deg(x) = k}) = ai(k). (28)

J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 24 / 29

SLIDE 25

New Bounds using {P(Ai )} and {P(Ai ∩ Aj )}

min

{pB ,B∈B} /

max

{pB ,B∈B}

B∈B

pB =

k
i
j

aij(k) k2 , s.t.

k

aij(k) =

i,j∈B,B∈B

pB = P(Ai ∩ Aj), i, j ∈ {1, . . . , N}, aij(k) =

B:i,j,l∈B,|B|=k

pB +

B:i,j∈B,l /

∈B,|B|=k

pB ≥ 0, a(k) − ai(k) − aj(k) + aij(k) =

B:l∈B,i,j /

∈B,|B|=k

pB +

B:i,j,l /

∈B,|B|=k

pB ≥ 0, a(k) − al(k) − ai(k) − aj(k) +aij(k) + ail(k) + ajl(k) =

B:i,j,l∈B,|B|=k

pB +

B:i,j,l /

∈B,|B|=k

pB ≥ 0, al(k) + aij(k) − ail(k) − ajl(k) =

B:i,j∈B,l /

∈B,|B|=k

pB +

B:l∈B,i,j /

∈B,|B|=k

pB ≥ 0, ai(k) − aij(k) =

B:i,l∈B,j /

∈B,|B|=k

pB +

B:i∈B,j,l /

∈B,|B|=k

pB ≥ 0, ∀i, j, l, k ∈ {1, . . . , N}. (29)

J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 25 / 29

SLIDE 26

Summary of Main Results

Outline

1

Problem Formulation

2

Recap: Bounds using {P(Ai)} and {

j P(Ai ∩ Aj)} 3

New Bounds using {P(Ai)} and {

j cjP(Ai ∩ Aj)} 4

New Bounds using {P(Ai)} and {P(Ai ∩ Aj)}

5

Summary of Main Results

J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 26 / 29

SLIDE 27

Summary of Main Results

1 In ISIT’14: Bounds using {P(Ai)} and {

j P(Ai ∩ Aj)}

Optimal Numerical Bound ℓNEW-1 (LP with N2 − N + 1 variables); Analytical Lower Bound ℓNEW-2; ℓNEW-1 ≥ ℓNEW-2 ≥ ℓKAT.

2 New Bounds using {P(Ai)} and {

j cjP(Ai ∩ Aj)}

New Class of Lower Bounds ℓNEW-3(c) (Pseudo-polynomial if c ∈ RN

+);

New Class of Lower Bounds ℓNEW-4(c) for c ∈ RN

+

(Pseudo-polynomial); ℓNEW-3(κ1) = ℓKAT, ℓNEW-4(κ1) = ℓNEW-2; ℓNEW-4(c) ≥ ℓNEW-3(c) if c ∈ RN

+;

ℓNEW-4(˜ c) ≥ ℓNEW-3(˜ c) ≥ ℓGK if ˜ c ∈ RN

+

3 New Bounds using {P(Ai)} and {P(Ai ∩ Aj)}

New Numerical Bound ℓNEW-5 (LP with (N−1)3+N+3

2

variables); ℓNEW-5 = ℓOPT when N ≤ 7; ℓNEW-5 ≥ ℓNEW-1.

J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 27 / 29

SLIDE 28

Summary of Main Results

Comparisons of lower bounds 2

System I II* III* IV V VI VII VIII* N 6 6 6 7 3 4 4 4 P N

i=1 Ai

0.7890

0.6740 0.7890 0.9687 0.3900 0.3252 0.5346 0.5854 KAT 0.7247 0.6227 0.7222 0.8909 0.3833 0.2769 0.4434 0.5412 GK 0.7601 0.6510 0.7508 0.9231 0.3813 0.2972 0.4750 0.5390 ℓNEW-2 0.7247 0.6227 0.7222 0.8909 0.3900 0.3205 0.4562 0.5464 ℓNEW-1 0.7487 0.6398 0.7427 0.9044 0.3900 0.3252 0.5090 0.5531 ℓNEW-4 (˜ c+) 0.7638 0.6517 0.7512 0.9231 0.3900 0.2951 0.4905 0.5412 ℓNEW-4 (rd) 0.7783 0.6633 0.7810 0.9501 0.3900 0.3203 0.4992 0.5666 ℓNEW-5 0.7890 0.6740 0.7890 0.9687 0.3900 0.3252 0.5090 0.5673

2In the table, * indicates ˜

c ∈ RN

+ and a bold number indicates coincidence with the

ptimal bound (24).
J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 28 / 29

SLIDE 29

Summary of Main Results

References

ℓNEW-1 and ℓNEW-2: [1] J. Yang, F. Alajaji, and G. Takahara, Lower bounds on the probability of a finite union of events, http://arxiv.org/abs/1401.5543 [2] —, New bounds on the probability of a finite union of events, ISIT’14. ℓNEW-4 and ℓNEW-5: [3] —, On Bounding the Union Probability, ISIT’15.

Thank you!

J. Yang, et al.

On Bounding P(N

i=1 Ai )

ISIT 2015 29 / 29