[PPT] - On Bounding the Union Probability Jun Yang (joint work with Fady PowerPoint Presentation

SLIDE 1

On Bounding the Union Probability

Jun Yang (joint work with Fady Alajaji and Glen Takahara)

Department of Statistical Sciences, University of Toronto

May 23, 2015

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 1 / 30

SLIDE 2

Outline

1

Problem Formulation

2

New Bounds using {P(Ai)} and {

j P(Ai ∩ Aj)} 3

New Bounds using {P(Ai)} and {

j cjP(Ai ∩ Aj)} 4

New Bounds using {P(Ai)} and {P(Ai ∩ Aj)}

5

Summary of Main Results

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 2 / 30

SLIDE 3

Problem Formulation

Outline

1

Problem Formulation

2

New Bounds using {P(Ai)} and {

j P(Ai ∩ Aj)} 3

New Bounds using {P(Ai)} and {

j cjP(Ai ∩ Aj)} 4

New Bounds using {P(Ai)} and {P(Ai ∩ Aj)}

5

Summary of Main Results

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 3 / 30

SLIDE 4

Problem Formulation

Consider a finite family of events A1, . . . , AN in a finite probability space (Ω, F, P), where N is a fixed positive integer. We are interested in bounding P N

i=1 Ai

in terms of the individual

event probabilities P(Ai)’s and the pairwise event probabilities P(Ai ∩ Aj)’s. For example, max

i

P(Ai) ≤ P N

i=1

Ai

≤ min
i

P(Ai), 1

.

(1) P N

i=1

Ai

≥
i

P(Ai) −

i<j

P(Ai ∩ Aj). (2)

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 4 / 30

SLIDE 5

Problem Formulation

Dawson-Sankoff (DS) Bound, 1967

For each outcome x ∈ F, let the degree of x, denoted by deg(x), be the number of Ai’s that contain x. Define a(k) := P ({x ∈

i Ai, deg(x) = k}), then one can verify

P

i

Ai

=

N

k=1

a(k),

i

P(Ai) =

N

k=1

ka(k),

i<j

P(Ai ∩ Aj) =

N

k=2

k 2

a(k).

(3)

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 5 / 30

SLIDE 6

Problem Formulation

Dawson-Sankoff (DS) Bound, 1967

Using (θ1, θ2) :=

i P(Ai),

i<j P(Ai ∩ Aj)

, the Dawson-Sankoff

(DS) Bound: P N

i=1

Ai

≥

κθ2

1

(2 − κ)θ1 + 2θ2 + (1 − κ)θ2

1

(1 − κ)θ1 + 2θ2 , (4) where κ = 2θ2

θ1 − ⌊ 2θ2 θ1 ⌋ and ⌊x⌋ denotes the largest integer less than

r equal to x, is the solution of the linear programming (LP) problem:

min

{a(k)} N

k=1

a(k), s.t.

N

k=1

ka(k) = θ1,

N

k=2

k 2

a(k) = θ2,

a(k) ≥ 0, k = 1, . . . , N. (5)

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 6 / 30

SLIDE 7

New Bounds using {P(Ai )} and {

j P(Ai ∩ Aj )}

Outline

1

Problem Formulation

2

New Bounds using {P(Ai)} and {

j P(Ai ∩ Aj)} 3

New Bounds using {P(Ai)} and {

j cjP(Ai ∩ Aj)} 4

New Bounds using {P(Ai)} and {P(Ai ∩ Aj)}

5

Summary of Main Results

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 7 / 30

SLIDE 8

New Bounds using {P(Ai )} and {

j P(Ai ∩ Aj )}

Kuai-Alajaji-Takahara (KAT) Bound, 2000

Remind that a(k) := P ({x ∈

i Ai, deg(x) = k})

Define ai(k) = P ({x ∈ Ai, deg(x) = k), one can verify

N

i=1

ai(k) = ka(k), ⇒ P

i

Ai

=
k

a(k) =

k
i

ai(k) k , P(Ai) =

N

k=1

ai(k),

j:j=i

P(Ai ∩ Aj) =

N

k=2

(k − 1)ai(k). (6)

We are able to use

P(A1), . . . , P(AN),

j:j=1 P(A1 ∩ Aj), . . . , j:j=N P(AN ∩ Aj)

.

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 8 / 30

SLIDE 9

New Bounds using {P(Ai )} and {

j P(Ai ∩ Aj )}

Kuai-Alajaji-Takahara (KAT) Bound, 2000

Let αi := P(Ai), γi :=

j P(Ai ∩ Aj) = P(Ai) + j:j=i P(Ai ∩ Aj).

The KAT bound is the solution of the following LP problem:

min

{ai(k)≥0} N

k=1

N

i=1

ai(k) k , s.t.

N

k=1

ai(k) = αi, i = 1, . . . , N,

N

k=1

kai(k) = γi, i = 1, . . . , N. (7)

which is given by P N

i=1

Ai

≥

N

i=1
1

⌊ γi

αi ⌋ − γi αi − ⌊ γi αi ⌋

(1 + ⌊ γi

αi ⌋)(⌊ γi αi ⌋)

αi
,

(8) where ⌊x⌋ is the largest positive integer less than or equal to x.

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 9 / 30

SLIDE 10

New Bounds using {P(Ai )} and {

j P(Ai ∩ Aj )}

New Lower Bounds which are sharper than KAT Bound

Recall that a(k) := P ({x ∈

i Ai, deg(x) = k}) and

ai(k) = P ({x ∈ Ai, deg(x) = k), then we observe a(k) ≥ ai(k) for all i and all k. Also, since a(k) =

i ai(k)

k

, one can write

i ai(k)

k ≥ ai(k) for all i and all k. As a special case for k = N, it reduces to a1(N) = a2(N) = · · · = aN(N).

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 10 / 30

SLIDE 11

New Bounds using {P(Ai )} and {

j P(Ai ∩ Aj )}

New Optimal Lower Bound ℓNEW-1

The solution of the following LP problem: min

{ai(k)} N

k=1

N

i=1

ai(k) k , s.t.

N

k=1

ai(k) = P(Ai), i = 1, . . . , N,

N

k=1

(k − 1)ai(k) =

j:j=i

P(Ai ∩ Aj), i = 1, . . . , N,

i ai(k)

k ≥ ai(k), i = 1, . . . , N, k = 1, . . . , N, ai(k) ≥ 0, k = 1, . . . , N, i = 1, . . . , N. (9) is optimal in the class of lower bounds which are functions of

P(A1), . . . , P(AN),

j:j=1 P(A1 ∩ Aj), . . . , j:j=N P(AN ∩ Aj)

.

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 11 / 30

SLIDE 12

New Bounds using {P(Ai )} and {

j P(Ai ∩ Aj )}

New Analytical Lower Bound ℓNEW-2

The new analytical lower bound is the solution of the LP problem:

min

{ai(k)≥0} N

k=1

N

i=1

ai(k) k , s.t.

N

k=1

ai(k) = P(Ai), i = 1, . . . , N,

N

k=1

(k − 1)ai(k) =

j:j=i

P(Ai ∩ Aj), i = 1, . . . , N, a1(N) = a2(N) = · · · = aN(N). (10)

The new analytical lower bound is given by

P N

i=1

Ai

≥ δ +

N

i=1

     1 χ( γ′

i

α′

i )

−

γ′

i

α′

i − χ( γ′ i

α′

i )

[1 + χ( γ′

i

α′

i )][χ( γ′ i

α′

i )]

  α′

i

   , (11)

where δ := {maxi [γi − (N − 1)αi]}+ ≥ 0, α′

i := αi − δ, γ′ i := γi − Nδ, and

χ(x) :=

n − 1

if x = n where n ≥ 2 is a integer ⌊x⌋

therwise

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 12 / 30

SLIDE 13

New Bounds using {P(Ai )} and {

j cj P(Ai ∩ Aj )}

Outline

1

Problem Formulation

2

New Bounds using {P(Ai)} and {

j P(Ai ∩ Aj)} 3

New Bounds using {P(Ai)} and {

j cjP(Ai ∩ Aj)} 4

New Bounds using {P(Ai)} and {P(Ai ∩ Aj)}

5

Summary of Main Results

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 13 / 30

SLIDE 14

New Bounds using {P(Ai )} and {

j cj P(Ai ∩ Aj )}

Gallot-Kounias (GK) Bound, 1968

The GK bound is an analytical bound which fully uses {P(Ai)} and {P(Ai ∩ Aj)}. Recently, it was re-visited by Feng-Li-Shen1 that the GK bound can be obtained by P N

i−1

Ai

≥ ℓGK = max

c∈RN

[

i ciP(Ai)]2

i ci
k ckP(Ai ∩ Ak).

(12) If ignoring the maximization over c, the RHS is a lower bound using

i ciP(Ai) and

k ckP(Ai ∩ Ak).

1“Some inequalities in functional analysis, combinatorics, and probability theory”,

The Electronic Journal of Combinatorics, 2010.

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 14 / 30

SLIDE 15

New Bounds using {P(Ai )} and {

j cj P(Ai ∩ Aj )}

New expressions for P N

i=1 Ai

Denote B as the collection of all non-empty subsets of {1, 2, . . . , N} and

let B ∈ B be a non-empty subset of {1, 2, . . . , N}, pB := p ({ωB, ωB ∈ Ai for all i ∈ B, ωB / ∈ Ai for all i / ∈ B}) . (13) Then we have a new (novel) expression of P N

i=1 Ai

for any given c:

P N

i=1

Ai

=
B∈B

pB =

N

i=1
B∈B:i∈B

cipB

k∈B ck
.

(14) Furthermore, we have P(Ai) =

B∈B:i∈B

pB,

k

ckP(Ai ∩ Ak) =

N

k=1
B:i∈B,k∈B

ckpB =

B:i∈B
k∈B

ck

pB.

(15)

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 15 / 30

SLIDE 16

New Bounds using {P(Ai )} and {

j cj P(Ai ∩ Aj )}

New expressions for P N

i=1 Ai

If c ∈ RN

+, by Cauchy-Schwarz inequality

P

i

Ai

=

N

i=1
B∈B:i∈B

cipB

k∈B ck
≥

N

i=1

c2

i P(Ai)2

ci

k ckP(Ai ∩ Ak). (16)

Note that we can use Cauchy-Schwarz Inequality again to get P

i

Ai

≥ max

c∈RN

+

N

i=1

c2

i P(Ai)2

ci

k ckP(Ai ∩ Ak) ≥ max

c∈RN

+

[

i ciP(Ai)]2

i
k cickP(Ai ∩ Ak).

(17)

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 16 / 30

SLIDE 17

New Bounds using {P(Ai )} and {

j cj P(Ai ∩ Aj )}

New Class of Lower Bounds ℓNEW-3(c)

The new class of lower bounds ℓNEW-3(c) is given by P N

i=1

Ai

≥

N

i=1

ℓi(c) =: ℓNEW-3(c), (18) where ℓi(c) := min

{pB:i∈B}

B:i∈B

cipB

k∈B ck

s.t.

B:i∈B

pB = P(Ai),

B:i∈B
k∈B ck

ci

pB = 1

ci

k

ckP(Ai ∩ Ak), pB ≥ 0, for all B ∈ B such that i ∈ B. (19)

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 17 / 30

SLIDE 18

New Bounds using {P(Ai )} and {

j cj P(Ai ∩ Aj )}

New Class of Lower Bounds ℓNEW-3(c)

The solution of ℓi(c) is given by

ℓi(c) = P(Ai)   ci

k∈B(i)

1 ck

+ ci

k∈B(i)

2 ck

− ci

k ckP(Ai ∩ Ak)

P(Ai)

k∈B(i)

1 ck

k∈B(i)

2 ck



 (20)

where B(i)

1

and B(i)

2

are subsets of {1, . . . , N}. For c ∈ RN

+,

B(i)

1

= arg max

{B:i∈B}

k∈B ck

ci s.t.

k∈B ck

ci ≤

k ckP(Ai ∩ Ak)

ciP(Ai) , B(i)

2

= arg min

{B:i∈B}

k∈B ck

ci s.t.

k∈B ck

ci ≥

k ckP(Ai ∩ Ak)

ciP(Ai) , (21)

which are 0/1 Knapsack Problems (Pseudo-Polynomial and Polynomial-time approx). ℓNEW-3(κ1) = ℓKAT for any κ = 0.

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 18 / 30

SLIDE 19

New Bounds using {P(Ai )} and {

j cj P(Ai ∩ Aj )}

Another Class of Lower Bounds ℓNEW-4(c) for c ∈ RN

+

Defining B− = B \ {1, . . . , N}, ˜ γi :=

k ckP(Ai ∩ Ak), ˜

αi := P(Ai) and

˜ δ := ˜ γi − (

k ck − mink ck) ˜

αi mink ck + , (22)

another class is given by ℓNEW-4(c) := ˜ δ + N

i=1 ℓ′ i(c, ˜

δ), where ℓ′

i(c, x) = [P(Ai) − x]

·   ci

k∈B(i)

1 ck

+ ci

k∈B(i)

2 ck

− ci

k ck [P(Ai ∩ Ak) − x]

[P(Ai) − x]

k∈B(i)

1 ck

k∈B(i)

2 ck



 . (23) ℓNEW-4(κ1) = ℓNEW-2 for any κ > 0; ℓNEW-4(c) ≥ ℓNEW-3(c) if c ∈ RN

+; ℓNEW-4(˜

c) ≥ ℓNEW-3(˜ c) ≥ ℓGK if ˜ c ∈ RN

+;

Optimal c in either class is still open.

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 19 / 30

SLIDE 20

New Bounds using {P(Ai )} and {P(Ai ∩ Aj )}

Outline

1

Problem Formulation

2

New Bounds using {P(Ai)} and {

j P(Ai ∩ Aj)} 3

New Bounds using {P(Ai)} and {

j cjP(Ai ∩ Aj)} 4

New Bounds using {P(Ai)} and {P(Ai ∩ Aj)}

5

Summary of Main Results

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 20 / 30

SLIDE 21

New Bounds using {P(Ai )} and {P(Ai ∩ Aj )}

Optimal Bounds with Exponential Complexity

The following (exhaustive) LP problem with 2N number of variables gives the optimal lower/upper bound established using {P(Ai)} and {P(Ai ∩ Aj)}: min

{pB,B∈B} /

max

{pB,B∈B}

B∈B

pB, s.t.

i,j∈B,B∈B

pB = P(Ai ∩ Aj), i, j ∈ {1, . . . , N}, pB ≥ 0, B ∈ B. (24) The optimality of (24) can be easily proved by showing its achievability: for each pB, construct an outcome ωB such that p(ωB) = pB and let ωB ∈ Ai, ∀i ∈ B. However, the computational complexity of the optimal lower bound (24) is exponential.

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 21 / 30

SLIDE 22

New Bounds using {P(Ai )} and {P(Ai ∩ Aj )}

A Relaxed Problem

Consider the following relaxed problem:

min

{pB,B∈B} /

max

{pB,B∈B}

B∈B

pB, s.t.

i,j∈B,B∈B

pB = P(Ai ∩ Aj), i, j ∈ {1, . . . , N},

B:i,j,l∈B,|B|=k

pB ≥ 0,

B:i,j∈B,l /

∈B,|B|=k

pB ≥ 0,

B:i∈B,j,l /

∈B,|B|=k

pB ≥ 0,

B:i,j,l /

∈B,|B|=k

pB ≥ 0, ∀i, j, l, k ∈ {1, . . . , N}. (25)

The solution of problem (25) coincides with the optimal lower/upper bound by (24) when N ≤ 7.

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 22 / 30

SLIDE 23

New Bounds using {P(Ai )} and {P(Ai ∩ Aj )}

The optimal feasible point of (25) is also optimal in

min

{pB,B∈B} /

max

{pB,B∈B}

B∈B

pB, s.t.

i,j∈B,B∈B

pB = P(Ai ∩ Aj), i, j ∈ {1, . . . , N},

B:i,j,l∈B,|B|=k

pB +

B:i,j∈B,l /

∈B,|B|=k

pB ≥ 0,

B:l∈B,i,j /

∈B,|B|=k

pB +

B:i,j,l /

∈B,|B|=k

pB ≥ 0,

B:i,j,l∈B,|B|=k

pB +

B:i,j,l /

∈B,|B|=k

pB ≥ 0,

B:i,j∈B,l /

∈B,|B|=k

pB +

B:l∈B,i,j /

∈B,|B|=k

pB ≥ 0,

B:i,j∈B,|B|=k

pB +

B:i∈B,j,l /

∈B,|B|=k

pB ≥ 0, ∀i, j, l, k ∈ {1, . . . , N}. (26)

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 23 / 30

SLIDE 24

New Bounds using {P(Ai )} and {P(Ai ∩ Aj )}

New Numerical Bounds

Define aij(k) := P ({x ∈ Ai ∩ Aj, deg(x) = k}) , i, j, k ∈ {1, · · · , N}. (27) Consider aij(k) as (N−1)3+N+3

2

variables. Then a(k) and ai(k) are linear functions of aij(k):

N

j=1

aij(k) k = P ({x ∈ Ai, deg(x) = k}) = ai(k). (28)

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 24 / 30

SLIDE 25

New Bounds using {P(Ai )} and {P(Ai ∩ Aj )}

min

{pB,B∈B} /

max

{pB,B∈B}

B∈B

pB =

k
i
j

aij(k) k2 , s.t.

k

aij(k) =

i,j∈B,B∈B

pB = P(Ai ∩ Aj), i, j ∈ {1, . . . , N}, aij(k) =

B:i,j,l∈B,|B|=k

pB +

B:i,j∈B,l /

∈B,|B|=k

pB ≥ 0, a(k) − ai(k) − aj(k) + aij(k) =

B:l∈B,i,j /

∈B,|B|=k

pB +

B:i,j,l /

∈B,|B|=k

pB ≥ 0, a(k) − al(k) − ai(k) − aj(k) +aij(k) + ail(k) + ajl(k) =

B:i,j,l∈B,|B|=k

pB +

B:i,j,l /

∈B,|B|=k

pB ≥ 0, al(k) + aij(k) − ail(k) − ajl(k) =

B:i,j∈B,l /

∈B,|B|=k

pB +

B:l∈B,i,j /

∈B,|B|=k

pB ≥ 0, ai(k) − aij(k) =

B:i,l∈B,j /

∈B,|B|=k

pB +

B:i∈B,j,l /

∈B,|B|=k

pB ≥ 0, ∀i, j, l, k ∈ {1, . . . , N}. (29)

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 25 / 30

SLIDE 26

Summary of Main Results

Outline

1

Problem Formulation

2

New Bounds using {P(Ai)} and {

j P(Ai ∩ Aj)} 3

New Bounds using {P(Ai)} and {

j cjP(Ai ∩ Aj)} 4

New Bounds using {P(Ai)} and {P(Ai ∩ Aj)}

5

Summary of Main Results

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 26 / 30

SLIDE 27

Summary of Main Results

1 New Bounds using {P(Ai)} and {

j P(Ai ∩ Aj)}

Optimal Numerical Bound ℓNEW-1 (LP with N2 − N + 1 variables); Analytical Lower Bound ℓNEW-2; ℓNEW-1 ≥ ℓNEW-2 ≥ ℓKAT.

2 New Bounds using {P(Ai)} and {

j cjP(Ai ∩ Aj)}

New Class of Lower Bounds ℓNEW-3(c) (Pseudo-polynomial if c ∈ RN

+);

New Class of Lower Bounds ℓNEW-4(c) for c ∈ RN

+

(Pseudo-polynomial); ℓNEW-3(κ1) = ℓKAT, ℓNEW-4(κ1) = ℓNEW-2; ℓNEW-4(c) ≥ ℓNEW-3(c) if c ∈ RN

+;

ℓNEW-4(˜ c) ≥ ℓNEW-3(˜ c) ≥ ℓGK if ˜ c ∈ RN

+

3 New Bounds using {P(Ai)} and {P(Ai ∩ Aj)}

New Numerical Bound ℓNEW-5 (LP with (N−1)3+N+3

2

variables); ℓNEW-5 = ℓOPT when N ≤ 7; ℓNEW-5 ≥ ℓNEW-1.

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 27 / 30

SLIDE 28

Summary of Main Results

Comparisons of lower bounds

System I II* III* IV V VI VII VIII* N 6 6 6 7 3 4 4 4 P N

i=1 Ai

0.7890

0.6740 0.7890 0.9687 0.3900 0.3252 0.5346 0.5854 KAT 0.7247 0.6227 0.7222 0.8909 0.3833 0.2769 0.4434 0.5412 GK 0.7601 0.6510 0.7508 0.9231 0.3813 0.2972 0.4750 0.5390 ℓNEW-2 0.7247 0.6227 0.7222 0.8909 0.3900 0.3205 0.4562 0.5464 ℓNEW-1 0.7487 0.6398 0.7427 0.9044 0.3900 0.3252 0.5090 0.5531 ℓNEW-4 (˜ c+) 0.7638 0.6517 0.7512 0.9231 0.3900 0.2951 0.4905 0.5412 ℓNEW-4 (rd) 0.7783 0.6633 0.7810 0.9501 0.3900 0.3203 0.4992 0.5666 ℓNEW-5 0.7890 0.6740 0.7890 0.9687 0.3900 0.3252 0.5090 0.5673

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 28 / 30

SLIDE 29

Summary of Main Results

References

ℓNEW-1 and ℓNEW-2: [1] J. Yang, F. Alajaji, and G. Takahara, Lower bounds on the probability of a finite union of events, http://arxiv.org/abs/1401.5543 [2] —, New bounds on the probability of a finite union of events, ISIT’14. ℓNEW-4 and ℓNEW-5: [3] —, On Bounding the Union Probability, ISIT’15.

Thank you!

Jun Yang (University of Toronto) On Bounding P(N

i=1 Ai )

May 23, 2015 29 / 30