Counting Iftach Haitner Tel Aviv University. December 2, 2014 - - PowerPoint PPT Presentation

▶
counting
SMART_READER_LITE
LIVE PREVIEW

Counting Iftach Haitner Tel Aviv University. December 2, 2014 - - PowerPoint PPT Presentation

Apr 01, 2023 249 likes •2.14k views

Application of Information Theory, Lecture 6 Counting Iftach Haitner Tel Aviv University. December 2, 2014 Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 1 / 25 Section 1 Graph Homomorphisms Iftach

slide-1
SLIDE 1

Application of Information Theory, Lecture 6

Counting

Iftach Haitner

Tel Aviv University.

December 2, 2014

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 1 / 25

slide-2
SLIDE 2

Section 1 Graph Homomorphisms

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 2 / 25

slide-3
SLIDE 3

Counting # of graph homomorphisms

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 3 / 25

slide-4
SLIDE 4

Counting # of graph homomorphisms

◮ T = (VT, ET) — directed graph (no self loops)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 3 / 25

slide-5
SLIDE 5

Counting # of graph homomorphisms

◮ T = (VT, ET) — directed graph (no self loops) ◮ G = (VG, EG) 1 2 3

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 3 / 25

slide-6
SLIDE 6

Counting # of graph homomorphisms

◮ T = (VT, ET) — directed graph (no self loops) ◮ G = (VG, EG) 1 2 3 ◮ H = (VH, EH) 1 2 3

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 3 / 25

slide-7
SLIDE 7

Counting # of graph homomorphisms

◮ T = (VT, ET) — directed graph (no self loops) ◮ G = (VG, EG) 1 2 3 ◮ H = (VH, EH) 1 2 3 ◮ (x1, x2, x3) is an homomorphism of G in T, if x1, x2, x3 ∈ VT and

(i, j) ∈ EG = ⇒ (xi, xj) ∈ ET (might be x1 = x2)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 3 / 25

slide-8
SLIDE 8

Counting # of graph homomorphisms

◮ T = (VT, ET) — directed graph (no self loops) ◮ G = (VG, EG) 1 2 3 ◮ H = (VH, EH) 1 2 3 ◮ (x1, x2, x3) is an homomorphism of G in T, if x1, x2, x3 ∈ VT and

(i, j) ∈ EG = ⇒ (xi, xj) ∈ ET (might be x1 = x2)

◮ Example: see board

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 3 / 25

slide-9
SLIDE 9

Counting # of graph homomorphisms

◮ T = (VT, ET) — directed graph (no self loops) ◮ G = (VG, EG) 1 2 3 ◮ H = (VH, EH) 1 2 3 ◮ (x1, x2, x3) is an homomorphism of G in T, if x1, x2, x3 ∈ VT and

(i, j) ∈ EG = ⇒ (xi, xj) ∈ ET (might be x1 = x2)

◮ Example: see board ◮ Hom(X, T): all homomorphisms of X in T

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 3 / 25

slide-10
SLIDE 10

Counting # of graph homomorphisms

◮ T = (VT, ET) — directed graph (no self loops) ◮ G = (VG, EG) 1 2 3 ◮ H = (VH, EH) 1 2 3 ◮ (x1, x2, x3) is an homomorphism of G in T, if x1, x2, x3 ∈ VT and

(i, j) ∈ EG = ⇒ (xi, xj) ∈ ET (might be x1 = x2)

◮ Example: see board ◮ Hom(X, T): all homomorphisms of X in T ◮ Claim |Hom(H, T)| ≤ |Hom(G, T)|

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 3 / 25

slide-11
SLIDE 11

Counting # of graph homomorphisms

◮ T = (VT, ET) — directed graph (no self loops) ◮ G = (VG, EG) 1 2 3 ◮ H = (VH, EH) 1 2 3 ◮ (x1, x2, x3) is an homomorphism of G in T, if x1, x2, x3 ∈ VT and

(i, j) ∈ EG = ⇒ (xi, xj) ∈ ET (might be x1 = x2)

◮ Example: see board ◮ Hom(X, T): all homomorphisms of X in T ◮ Claim |Hom(H, T)| ≤ |Hom(G, T)| ◮ Trivial if G would be a subgraph of G

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 3 / 25

slide-12
SLIDE 12

Counting # of graph homomorphisms

◮ T = (VT, ET) — directed graph (no self loops) ◮ G = (VG, EG) 1 2 3 ◮ H = (VH, EH) 1 2 3 ◮ (x1, x2, x3) is an homomorphism of G in T, if x1, x2, x3 ∈ VT and

(i, j) ∈ EG = ⇒ (xi, xj) ∈ ET (might be x1 = x2)

◮ Example: see board ◮ Hom(X, T): all homomorphisms of X in T ◮ Claim |Hom(H, T)| ≤ |Hom(G, T)| ◮ Trivial if G would be a subgraph of G ◮ Special case of a more general theorem

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 3 / 25

slide-13
SLIDE 13

Proving the claim

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 4 / 25

slide-14
SLIDE 14

Proving the claim

◮ (X1, X2, X3) ← Hom(H, T)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 4 / 25

slide-15
SLIDE 15

Proving the claim

◮ (X1, X2, X3) ← Hom(H, T) ◮ log |Hom(H, T)| = H(X1, X2, X3)

= H(X1) + H(X2|X1) + H(X3|X1, X2) ≤ H(X1) + H(X2|X1) + H(X3|X2)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 4 / 25

slide-16
SLIDE 16

Proving the claim

◮ (X1, X2, X3) ← Hom(H, T) ◮ log |Hom(H, T)| = H(X1, X2, X3)

= H(X1) + H(X2|X1) + H(X3|X1, X2) ≤ H(X1) + H(X2|X1) + H(X3|X2)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 4 / 25

slide-17
SLIDE 17

Proving the claim

◮ (X1, X2, X3) ← Hom(H, T) ◮ log |Hom(H, T)| = H(X1, X2, X3)

= H(X1) + H(X2|X1) + H(X3|X1, X2) ≤ H(X1) + H(X2|X1) + H(X3|X2) = H(X1) + 2 · H(X2|X1) (by symmetry of H)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 4 / 25

slide-18
SLIDE 18

Proving the claim

◮ (X1, X2, X3) ← Hom(H, T) ◮ log |Hom(H, T)| = H(X1, X2, X3)

= H(X1) + H(X2|X1) + H(X3|X1, X2) ≤ H(X1) + H(X2|X1) + H(X3|X2) = H(X1) + 2 · H(X2|X1) (by symmetry of H)

◮ Let D2(x) be the distribution of X2|X1 = x, and let X ′

2 ∼ D2(X1)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 4 / 25

slide-19
SLIDE 19

Proving the claim

◮ (X1, X2, X3) ← Hom(H, T) ◮ log |Hom(H, T)| = H(X1, X2, X3)

= H(X1) + H(X2|X1) + H(X3|X1, X2) ≤ H(X1) + H(X2|X1) + H(X3|X2) = H(X1) + 2 · H(X2|X1) (by symmetry of H)

◮ Let D2(x) be the distribution of X2|X1 = x, and let X ′

2 ∼ D2(X1)

◮

H(X1, X2, X ′

2) = H(X1) + H(X2|X1) + H(X ′ 2|X1, X2)

= H(X1) + H(X2|X1) + H(X ′

2|X1)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 4 / 25

slide-20
SLIDE 20

Proving the claim

◮ (X1, X2, X3) ← Hom(H, T) ◮ log |Hom(H, T)| = H(X1, X2, X3)

= H(X1) + H(X2|X1) + H(X3|X1, X2) ≤ H(X1) + H(X2|X1) + H(X3|X2) = H(X1) + 2 · H(X2|X1) (by symmetry of H)

◮ Let D2(x) be the distribution of X2|X1 = x, and let X ′

2 ∼ D2(X1)

◮

H(X1, X2, X ′

2) = H(X1) + H(X2|X1) + H(X ′ 2|X1, X2)

= H(X1) + H(X2|X1) + H(X ′

2|X1)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 4 / 25

slide-21
SLIDE 21

Proving the claim

◮ (X1, X2, X3) ← Hom(H, T) ◮ log |Hom(H, T)| = H(X1, X2, X3)

= H(X1) + H(X2|X1) + H(X3|X1, X2) ≤ H(X1) + H(X2|X1) + H(X3|X2) = H(X1) + 2 · H(X2|X1) (by symmetry of H)

◮ Let D2(x) be the distribution of X2|X1 = x, and let X ′

2 ∼ D2(X1)

◮

H(X1, X2, X ′

2) = H(X1) + H(X2|X1) + H(X ′ 2|X1, X2)

= H(X1) + H(X2|X1) + H(X ′

2|X1)

= H(X1) + 2 · H(X2|X1)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 4 / 25

slide-22
SLIDE 22

Proving the claim

◮ (X1, X2, X3) ← Hom(H, T) ◮ log |Hom(H, T)| = H(X1, X2, X3)

= H(X1) + H(X2|X1) + H(X3|X1, X2) ≤ H(X1) + H(X2|X1) + H(X3|X2) = H(X1) + 2 · H(X2|X1) (by symmetry of H)

◮ Let D2(x) be the distribution of X2|X1 = x, and let X ′

2 ∼ D2(X1)

◮

H(X1, X2, X ′

2) = H(X1) + H(X2|X1) + H(X ′ 2|X1, X2)

= H(X1) + H(X2|X1) + H(X ′

2|X1)

= H(X1) + 2 · H(X2|X1)

◮ (X1, X2) ∈ ET and (X1, X ′

2) ∈ ET

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 4 / 25

slide-23
SLIDE 23

Proving the claim

◮ (X1, X2, X3) ← Hom(H, T) ◮ log |Hom(H, T)| = H(X1, X2, X3)

= H(X1) + H(X2|X1) + H(X3|X1, X2) ≤ H(X1) + H(X2|X1) + H(X3|X2) = H(X1) + 2 · H(X2|X1) (by symmetry of H)

◮ Let D2(x) be the distribution of X2|X1 = x, and let X ′

2 ∼ D2(X1)

◮

H(X1, X2, X ′

2) = H(X1) + H(X2|X1) + H(X ′ 2|X1, X2)

= H(X1) + H(X2|X1) + H(X ′

2|X1)

= H(X1) + 2 · H(X2|X1)

◮ (X1, X2) ∈ ET and (X1, X ′

2) ∈ ET

= ⇒ (X1, X2, X ′

2) ∈ Hom(G, T)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 4 / 25

slide-24
SLIDE 24

Proving the claim

◮ (X1, X2, X3) ← Hom(H, T) ◮ log |Hom(H, T)| = H(X1, X2, X3)

= H(X1) + H(X2|X1) + H(X3|X1, X2) ≤ H(X1) + H(X2|X1) + H(X3|X2) = H(X1) + 2 · H(X2|X1) (by symmetry of H)

◮ Let D2(x) be the distribution of X2|X1 = x, and let X ′

2 ∼ D2(X1)

◮

H(X1, X2, X ′

2) = H(X1) + H(X2|X1) + H(X ′ 2|X1, X2)

= H(X1) + H(X2|X1) + H(X ′

2|X1)

= H(X1) + 2 · H(X2|X1)

◮ (X1, X2) ∈ ET and (X1, X ′

2) ∈ ET

= ⇒ (X1, X2, X ′

2) ∈ Hom(G, T)

= ⇒ H(X1, X2, X ′

2) ≤ log |Hom(G, T)|

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 4 / 25

slide-25
SLIDE 25

Proving the claim

◮ (X1, X2, X3) ← Hom(H, T) ◮ log |Hom(H, T)| = H(X1, X2, X3)

= H(X1) + H(X2|X1) + H(X3|X1, X2) ≤ H(X1) + H(X2|X1) + H(X3|X2) = H(X1) + 2 · H(X2|X1) (by symmetry of H)

◮ Let D2(x) be the distribution of X2|X1 = x, and let X ′

2 ∼ D2(X1)

◮

H(X1, X2, X ′

2) = H(X1) + H(X2|X1) + H(X ′ 2|X1, X2)

= H(X1) + H(X2|X1) + H(X ′

2|X1)

= H(X1) + 2 · H(X2|X1)

◮ (X1, X2) ∈ ET and (X1, X ′

2) ∈ ET

= ⇒ (X1, X2, X ′

2) ∈ Hom(G, T)

= ⇒ H(X1, X2, X ′

2) ≤ log |Hom(G, T)|

= ⇒ log |Hom(H, T)| ≤ log |Hom(G, T)|.

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 4 / 25

slide-26
SLIDE 26

Section 2 Perfect Matchings

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 5 / 25

slide-27
SLIDE 27

Bregman’s theorem

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 6 / 25

slide-28
SLIDE 28

Bregman’s theorem

For bi-partite graph G = (A, B, E), let d(v) = |N(v) = {u ∈ B : (v, u) ∈ E}|

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 6 / 25

slide-29
SLIDE 29

Bregman’s theorem

For bi-partite graph G = (A, B, E), let d(v) = |N(v) = {u ∈ B : (v, u) ∈ E}| Theorem 1 Let G = (A, B, E) be bi-partite graph with |A| = |B|. Then P(G) — the number

  • f perfect matching in G — is at most

v∈A(d(v)!)1/d(v).

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 6 / 25

slide-30
SLIDE 30

Bregman’s theorem

For bi-partite graph G = (A, B, E), let d(v) = |N(v) = {u ∈ B : (v, u) ∈ E}| Theorem 1 Let G = (A, B, E) be bi-partite graph with |A| = |B|. Then P(G) — the number

  • f perfect matching in G — is at most

v∈A(d(v)!)1/d(v).

◮ Let A = B = [n] = {1, . . . , n}

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 6 / 25

slide-31
SLIDE 31

Bregman’s theorem

For bi-partite graph G = (A, B, E), let d(v) = |N(v) = {u ∈ B : (v, u) ∈ E}| Theorem 1 Let G = (A, B, E) be bi-partite graph with |A| = |B|. Then P(G) — the number

  • f perfect matching in G — is at most

v∈A(d(v)!)1/d(v).

◮ Let A = B = [n] = {1, . . . , n} ◮ It is clear that P(G) ≤

i∈[n] d(i):

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 6 / 25

slide-32
SLIDE 32

Bregman’s theorem

For bi-partite graph G = (A, B, E), let d(v) = |N(v) = {u ∈ B : (v, u) ∈ E}| Theorem 1 Let G = (A, B, E) be bi-partite graph with |A| = |B|. Then P(G) — the number

  • f perfect matching in G — is at most

v∈A(d(v)!)1/d(v).

◮ Let A = B = [n] = {1, . . . , n} ◮ It is clear that P(G) ≤

i∈[n] d(i):

◮ Let M be the perfect matchings in G.

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 6 / 25

slide-33
SLIDE 33

Bregman’s theorem

For bi-partite graph G = (A, B, E), let d(v) = |N(v) = {u ∈ B : (v, u) ∈ E}| Theorem 1 Let G = (A, B, E) be bi-partite graph with |A| = |B|. Then P(G) — the number

  • f perfect matching in G — is at most

v∈A(d(v)!)1/d(v).

◮ Let A = B = [n] = {1, . . . , n} ◮ It is clear that P(G) ≤

i∈[n] d(i):

◮ Let M be the perfect matchings in G. ◮ For m ∈ M let m(i) be the node in B matched with i by m.

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 6 / 25

slide-34
SLIDE 34

Bregman’s theorem

For bi-partite graph G = (A, B, E), let d(v) = |N(v) = {u ∈ B : (v, u) ∈ E}| Theorem 1 Let G = (A, B, E) be bi-partite graph with |A| = |B|. Then P(G) — the number

  • f perfect matching in G — is at most

v∈A(d(v)!)1/d(v).

◮ Let A = B = [n] = {1, . . . , n} ◮ It is clear that P(G) ≤

i∈[n] d(i):

◮ Let M be the perfect matchings in G. ◮ For m ∈ M let m(i) be the node in B matched with i by m. ◮ Let M ← M.

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 6 / 25

slide-35
SLIDE 35

Bregman’s theorem

For bi-partite graph G = (A, B, E), let d(v) = |N(v) = {u ∈ B : (v, u) ∈ E}| Theorem 1 Let G = (A, B, E) be bi-partite graph with |A| = |B|. Then P(G) — the number

  • f perfect matching in G — is at most

v∈A(d(v)!)1/d(v).

◮ Let A = B = [n] = {1, . . . , n} ◮ It is clear that P(G) ≤

i∈[n] d(i):

◮ Let M be the perfect matchings in G. ◮ For m ∈ M let m(i) be the node in B matched with i by m. ◮ Let M ← M.

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 6 / 25

slide-36
SLIDE 36

Bregman’s theorem

For bi-partite graph G = (A, B, E), let d(v) = |N(v) = {u ∈ B : (v, u) ∈ E}| Theorem 1 Let G = (A, B, E) be bi-partite graph with |A| = |B|. Then P(G) — the number

  • f perfect matching in G — is at most

v∈A(d(v)!)1/d(v).

◮ Let A = B = [n] = {1, . . . , n} ◮ It is clear that P(G) ≤

i∈[n] d(i):

◮ Let M be the perfect matchings in G. ◮ For m ∈ M let m(i) be the node in B matched with i by m. ◮ Let M ← M. Hence,

log |M|=H(M)= H(M(1)) + H(M(2)|M(1)) + . . . + H(M(n)|M(1), . . . , M(n − 1))

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 6 / 25

slide-37
SLIDE 37

Bregman’s theorem

For bi-partite graph G = (A, B, E), let d(v) = |N(v) = {u ∈ B : (v, u) ∈ E}| Theorem 1 Let G = (A, B, E) be bi-partite graph with |A| = |B|. Then P(G) — the number

  • f perfect matching in G — is at most

v∈A(d(v)!)1/d(v).

◮ Let A = B = [n] = {1, . . . , n} ◮ It is clear that P(G) ≤

i∈[n] d(i):

◮ Let M be the perfect matchings in G. ◮ For m ∈ M let m(i) be the node in B matched with i by m. ◮ Let M ← M. Hence,

log |M|=H(M)= H(M(1)) + H(M(2)|M(1)) + . . . + H(M(n)|M(1), . . . , M(n − 1)) ≤ H(M(1)) + H(M(2)) + . . . + H(M(n))

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 6 / 25

slide-38
SLIDE 38

Bregman’s theorem

For bi-partite graph G = (A, B, E), let d(v) = |N(v) = {u ∈ B : (v, u) ∈ E}| Theorem 1 Let G = (A, B, E) be bi-partite graph with |A| = |B|. Then P(G) — the number

  • f perfect matching in G — is at most

v∈A(d(v)!)1/d(v).

◮ Let A = B = [n] = {1, . . . , n} ◮ It is clear that P(G) ≤

i∈[n] d(i):

◮ Let M be the perfect matchings in G. ◮ For m ∈ M let m(i) be the node in B matched with i by m. ◮ Let M ← M. Hence,

log |M|=H(M)= H(M(1)) + H(M(2)|M(1)) + . . . + H(M(n)|M(1), . . . , M(n − 1)) ≤ H(M(1)) + H(M(2)) + . . . + H(M(n)) ≤ log d(1) + log d(2) + . . . + log d(n)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 6 / 25

slide-39
SLIDE 39

Bregman’s theorem

For bi-partite graph G = (A, B, E), let d(v) = |N(v) = {u ∈ B : (v, u) ∈ E}| Theorem 1 Let G = (A, B, E) be bi-partite graph with |A| = |B|. Then P(G) — the number

  • f perfect matching in G — is at most

v∈A(d(v)!)1/d(v).

◮ Let A = B = [n] = {1, . . . , n} ◮ It is clear that P(G) ≤

i∈[n] d(i):

◮ Let M be the perfect matchings in G. ◮ For m ∈ M let m(i) be the node in B matched with i by m. ◮ Let M ← M. Hence,

log |M|=H(M)= H(M(1)) + H(M(2)|M(1)) + . . . + H(M(n)|M(1), . . . , M(n − 1)) ≤ H(M(1)) + H(M(2)) + . . . + H(M(n)) ≤ log d(1) + log d(2) + . . . + log d(n) =

  • i∈[n]

log d(i)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 6 / 25

slide-40
SLIDE 40

Proving Bregman’s theorem

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 7 / 25

slide-41
SLIDE 41

Proving Bregman’s theorem

◮ Key observations:

H(M(i|M(1), . . . , M(i − 1)) ≤ log |N(i) \ {M(1), . . . , M(i − 1)}|

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 7 / 25

slide-42
SLIDE 42

Proving Bregman’s theorem

◮ Key observations:

H(M(i|M(1), . . . , M(i − 1)) ≤ log |N(i) \ {M(1), . . . , M(i − 1)}|

◮ Let P be the set of all permutation over [n].

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 7 / 25

slide-43
SLIDE 43

Proving Bregman’s theorem

◮ Key observations:

H(M(i|M(1), . . . , M(i − 1)) ≤ log |N(i) \ {M(1), . . . , M(i − 1)}|

◮ Let P be the set of all permutation over [n].

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 7 / 25

slide-44
SLIDE 44

Proving Bregman’s theorem

◮ Key observations:

H(M(i|M(1), . . . , M(i − 1)) ≤ log |N(i) \ {M(1), . . . , M(i − 1)}|

◮ Let P be the set of all permutation over [n]. For p ∈ P:

H(M) = H(M(p(1))) + . . . + H(M(p(n))|M(p(1)), . . . , M(p(n − 1)))

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 7 / 25

slide-45
SLIDE 45

Proving Bregman’s theorem

◮ Key observations:

H(M(i|M(1), . . . , M(i − 1)) ≤ log |N(i) \ {M(1), . . . , M(i − 1)}|

◮ Let P be the set of all permutation over [n]. For p ∈ P:

H(M) = H(M(p(1))) + . . . + H(M(p(n))|M(p(1)), . . . , M(p(n − 1)))

◮ Sp(i) = {1, . . . , p−1(i) − 1} — matchings appear above before i

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 7 / 25

slide-46
SLIDE 46

Proving Bregman’s theorem

◮ Key observations:

H(M(i|M(1), . . . , M(i − 1)) ≤ log |N(i) \ {M(1), . . . , M(i − 1)}|

◮ Let P be the set of all permutation over [n]. For p ∈ P:

H(M) = H(M(p(1))) + . . . + H(M(p(n))|M(p(1)), . . . , M(p(n − 1)))

◮ Sp(i) = {1, . . . , p−1(i) − 1} — matchings appear above before i ◮ H(M) = n

i=1 H(M(i)|M(Sp(i)))

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 7 / 25

slide-47
SLIDE 47

Proving Bregman’s theorem

◮ Key observations:

H(M(i|M(1), . . . , M(i − 1)) ≤ log |N(i) \ {M(1), . . . , M(i − 1)}|

◮ Let P be the set of all permutation over [n]. For p ∈ P:

H(M) = H(M(p(1))) + . . . + H(M(p(n))|M(p(1)), . . . , M(p(n − 1)))

◮ Sp(i) = {1, . . . , p−1(i) − 1} — matchings appear above before i ◮ H(M) = n

i=1 H(M(i)|M(Sp(i)))

◮ For m ∈ M and P ← P:

|N(i) \ m(SP(i))| is uniform over {1, . . . , d(i)}

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 7 / 25

slide-48
SLIDE 48

Proving Bregman’s theorem

◮ Key observations:

H(M(i|M(1), . . . , M(i − 1)) ≤ log |N(i) \ {M(1), . . . , M(i − 1)}|

◮ Let P be the set of all permutation over [n]. For p ∈ P:

H(M) = H(M(p(1))) + . . . + H(M(p(n))|M(p(1)), . . . , M(p(n − 1)))

◮ Sp(i) = {1, . . . , p−1(i) − 1} — matchings appear above before i ◮ H(M) = n

i=1 H(M(i)|M(Sp(i)))

◮ For m ∈ M and P ← P:

|N(i) \ m(SP(i))| is uniform over {1, . . . , d(i)} = ⇒ EP [H(M(i) | M(SP(i)))] ≤

1 d(i)

d(i)

k=1 log k

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 7 / 25

slide-49
SLIDE 49

Proving Bregman’s theorem

◮ Key observations:

H(M(i|M(1), . . . , M(i − 1)) ≤ log |N(i) \ {M(1), . . . , M(i − 1)}|

◮ Let P be the set of all permutation over [n]. For p ∈ P:

H(M) = H(M(p(1))) + . . . + H(M(p(n))|M(p(1)), . . . , M(p(n − 1)))

◮ Sp(i) = {1, . . . , p−1(i) − 1} — matchings appear above before i ◮ H(M) = n

i=1 H(M(i)|M(Sp(i)))

◮ For m ∈ M and P ← P:

|N(i) \ m(SP(i))| is uniform over {1, . . . , d(i)} = ⇒ EP [H(M(i) | M(SP(i)))] ≤

1 d(i)

d(i)

k=1 log k

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 7 / 25

slide-50
SLIDE 50

Proving Bregman’s theorem

◮ Key observations:

H(M(i|M(1), . . . , M(i − 1)) ≤ log |N(i) \ {M(1), . . . , M(i − 1)}|

◮ Let P be the set of all permutation over [n]. For p ∈ P:

H(M) = H(M(p(1))) + . . . + H(M(p(n))|M(p(1)), . . . , M(p(n − 1)))

◮ Sp(i) = {1, . . . , p−1(i) − 1} — matchings appear above before i ◮ H(M) = n

i=1 H(M(i)|M(Sp(i)))

◮ For m ∈ M and P ← P:

|N(i) \ m(SP(i))| is uniform over {1, . . . , d(i)} = ⇒ EP [H(M(i) | M(SP(i)))] ≤

1 d(i)

d(i)

k=1 log k = log

  • (d(i)!)1/d(i)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 7 / 25

slide-51
SLIDE 51

Proving Bregman’s theorem

◮ Key observations:

H(M(i|M(1), . . . , M(i − 1)) ≤ log |N(i) \ {M(1), . . . , M(i − 1)}|

◮ Let P be the set of all permutation over [n]. For p ∈ P:

H(M) = H(M(p(1))) + . . . + H(M(p(n))|M(p(1)), . . . , M(p(n − 1)))

◮ Sp(i) = {1, . . . , p−1(i) − 1} — matchings appear above before i ◮ H(M) = n

i=1 H(M(i)|M(Sp(i)))

◮ For m ∈ M and P ← P:

|N(i) \ m(SP(i))| is uniform over {1, . . . , d(i)} = ⇒ EP [H(M(i) | M(SP(i)))] ≤

1 d(i)

d(i)

k=1 log k = log

  • (d(i)!)1/d(i)

= ⇒ H(M) = E

P

n

  • i=1

H(M(i)|M(SP(i)))

  • Iftach Haitner (TAU)

Application of Information Theory, Lecture 6 December 2, 2014 7 / 25

slide-52
SLIDE 52

Proving Bregman’s theorem

◮ Key observations:

H(M(i|M(1), . . . , M(i − 1)) ≤ log |N(i) \ {M(1), . . . , M(i − 1)}|

◮ Let P be the set of all permutation over [n]. For p ∈ P:

H(M) = H(M(p(1))) + . . . + H(M(p(n))|M(p(1)), . . . , M(p(n − 1)))

◮ Sp(i) = {1, . . . , p−1(i) − 1} — matchings appear above before i ◮ H(M) = n

i=1 H(M(i)|M(Sp(i)))

◮ For m ∈ M and P ← P:

|N(i) \ m(SP(i))| is uniform over {1, . . . , d(i)} = ⇒ EP [H(M(i) | M(SP(i)))] ≤

1 d(i)

d(i)

k=1 log k = log

  • (d(i)!)1/d(i)

= ⇒ H(M) = E

P

n

  • i=1

H(M(i)|M(SP(i)))

  • Iftach Haitner (TAU)

Application of Information Theory, Lecture 6 December 2, 2014 7 / 25

slide-53
SLIDE 53

Proving Bregman’s theorem

◮ Key observations:

H(M(i|M(1), . . . , M(i − 1)) ≤ log |N(i) \ {M(1), . . . , M(i − 1)}|

◮ Let P be the set of all permutation over [n]. For p ∈ P:

H(M) = H(M(p(1))) + . . . + H(M(p(n))|M(p(1)), . . . , M(p(n − 1)))

◮ Sp(i) = {1, . . . , p−1(i) − 1} — matchings appear above before i ◮ H(M) = n

i=1 H(M(i)|M(Sp(i)))

◮ For m ∈ M and P ← P:

|N(i) \ m(SP(i))| is uniform over {1, . . . , d(i)} = ⇒ EP [H(M(i) | M(SP(i)))] ≤

1 d(i)

d(i)

k=1 log k = log

  • (d(i)!)1/d(i)

= ⇒ H(M) = E

P

n

  • i=1

H(M(i)|M(SP(i)))

  • =

n

  • i=1

E

P [H(M(i)|JP(i))]

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 7 / 25

slide-54
SLIDE 54

Proving Bregman’s theorem

◮ Key observations:

H(M(i|M(1), . . . , M(i − 1)) ≤ log |N(i) \ {M(1), . . . , M(i − 1)}|

◮ Let P be the set of all permutation over [n]. For p ∈ P:

H(M) = H(M(p(1))) + . . . + H(M(p(n))|M(p(1)), . . . , M(p(n − 1)))

◮ Sp(i) = {1, . . . , p−1(i) − 1} — matchings appear above before i ◮ H(M) = n

i=1 H(M(i)|M(Sp(i)))

◮ For m ∈ M and P ← P:

|N(i) \ m(SP(i))| is uniform over {1, . . . , d(i)} = ⇒ EP [H(M(i) | M(SP(i)))] ≤

1 d(i)

d(i)

k=1 log k = log

  • (d(i)!)1/d(i)

= ⇒ H(M) = E

P

n

  • i=1

H(M(i)|M(SP(i)))

  • =

n

  • i=1

E

P [H(M(i)|JP(i))]

≤

  • i∈[n]

log

  • (d(i)!)1/d(i)

.

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 7 / 25

slide-55
SLIDE 55

Section 3 Shearer’s Lemma

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 8 / 25

slide-56
SLIDE 56

H(X1, X2, X3) Vs. H(X1, X2) + H(X2, X3) + H(X3, X1)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 9 / 25

slide-57
SLIDE 57

H(X1, X2, X3) Vs. H(X1, X2) + H(X2, X3) + H(X3, X1)

◮ How does H(X1, X2, X3) compares to H(X1, X2) + H(X2, X3) + H(X3, X1)?

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 9 / 25

slide-58
SLIDE 58

H(X1, X2, X3) Vs. H(X1, X2) + H(X2, X3) + H(X3, X1)

◮ How does H(X1, X2, X3) compares to H(X1, X2) + H(X2, X3) + H(X3, X1)? ◮ If X1, X2, X3 are independence, then

H(X1, X2, X3) = 1

2 (H(X1, X2) + H(X2, X3) + H(X3, X1))

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 9 / 25

slide-59
SLIDE 59

H(X1, X2, X3) Vs. H(X1, X2) + H(X2, X3) + H(X3, X1)

◮ How does H(X1, X2, X3) compares to H(X1, X2) + H(X2, X3) + H(X3, X1)? ◮ If X1, X2, X3 are independence, then

H(X1, X2, X3) = 1

2 (H(X1, X2) + H(X2, X3) + H(X3, X1))

◮ In general: H(X1, X2, X3) ≤ 1

2 (H(X1, X2) + H(X2, X3) + H(X3, X1))

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 9 / 25

slide-60
SLIDE 60

H(X1, X2, X3) Vs. H(X1, X2) + H(X2, X3) + H(X3, X1)

◮ How does H(X1, X2, X3) compares to H(X1, X2) + H(X2, X3) + H(X3, X1)? ◮ If X1, X2, X3 are independence, then

H(X1, X2, X3) = 1

2 (H(X1, X2) + H(X2, X3) + H(X3, X1))

◮ In general: H(X1, X2, X3) ≤ 1

2 (H(X1, X2) + H(X2, X3) + H(X3, X1))

◮ A tighter bounds than H(X1) + H(X2) + H(X3)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 9 / 25

slide-61
SLIDE 61

H(X1, X2, X3) Vs. H(X1, X2) + H(X2, X3) + H(X3, X1)

◮ How does H(X1, X2, X3) compares to H(X1, X2) + H(X2, X3) + H(X3, X1)? ◮ If X1, X2, X3 are independence, then

H(X1, X2, X3) = 1

2 (H(X1, X2) + H(X2, X3) + H(X3, X1))

◮ In general: H(X1, X2, X3) ≤ 1

2 (H(X1, X2) + H(X2, X3) + H(X3, X1))

◮ A tighter bounds than H(X1) + H(X2) + H(X3) ◮ Proof:

2H(X1, X2, X3) = 2H(X1) +2H(X2|X1) +2H(X3|X1, X2)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 9 / 25

slide-62
SLIDE 62

H(X1, X2, X3) Vs. H(X1, X2) + H(X2, X3) + H(X3, X1)

◮ How does H(X1, X2, X3) compares to H(X1, X2) + H(X2, X3) + H(X3, X1)? ◮ If X1, X2, X3 are independence, then

H(X1, X2, X3) = 1

2 (H(X1, X2) + H(X2, X3) + H(X3, X1))

◮ In general: H(X1, X2, X3) ≤ 1

2 (H(X1, X2) + H(X2, X3) + H(X3, X1))

◮ A tighter bounds than H(X1) + H(X2) + H(X3) ◮ Proof:

2H(X1, X2, X3) = 2H(X1) +2H(X2|X1) +2H(X3|X1, X2)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 9 / 25

slide-63
SLIDE 63

H(X1, X2, X3) Vs. H(X1, X2) + H(X2, X3) + H(X3, X1)

◮ How does H(X1, X2, X3) compares to H(X1, X2) + H(X2, X3) + H(X3, X1)? ◮ If X1, X2, X3 are independence, then

H(X1, X2, X3) = 1

2 (H(X1, X2) + H(X2, X3) + H(X3, X1))

◮ In general: H(X1, X2, X3) ≤ 1

2 (H(X1, X2) + H(X2, X3) + H(X3, X1))

◮ A tighter bounds than H(X1) + H(X2) + H(X3) ◮ Proof:

2H(X1, X2, X3) = 2H(X1) +2H(X2|X1) +2H(X3|X1, X2) H(X2, X3) = H(X1) +H(X2|X1)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 9 / 25

slide-64
SLIDE 64

H(X1, X2, X3) Vs. H(X1, X2) + H(X2, X3) + H(X3, X1)

◮ How does H(X1, X2, X3) compares to H(X1, X2) + H(X2, X3) + H(X3, X1)? ◮ If X1, X2, X3 are independence, then

H(X1, X2, X3) = 1

2 (H(X1, X2) + H(X2, X3) + H(X3, X1))

◮ In general: H(X1, X2, X3) ≤ 1

2 (H(X1, X2) + H(X2, X3) + H(X3, X1))

◮ A tighter bounds than H(X1) + H(X2) + H(X3) ◮ Proof:

2H(X1, X2, X3) = 2H(X1) +2H(X2|X1) +2H(X3|X1, X2) H(X2, X3) = H(X1) +H(X2|X1) H(X2, X3) = +H(X2) +H(X3|X2)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 9 / 25

slide-65
SLIDE 65

H(X1, X2, X3) Vs. H(X1, X2) + H(X2, X3) + H(X3, X1)

◮ How does H(X1, X2, X3) compares to H(X1, X2) + H(X2, X3) + H(X3, X1)? ◮ If X1, X2, X3 are independence, then

H(X1, X2, X3) = 1

2 (H(X1, X2) + H(X2, X3) + H(X3, X1))

◮ In general: H(X1, X2, X3) ≤ 1

2 (H(X1, X2) + H(X2, X3) + H(X3, X1))

◮ A tighter bounds than H(X1) + H(X2) + H(X3) ◮ Proof:

2H(X1, X2, X3) = 2H(X1) +2H(X2|X1) +2H(X3|X1, X2) H(X2, X3) = H(X1) +H(X2|X1) H(X2, X3) = +H(X2) +H(X3|X2) H(X1, X3) = H(X1) +H(X3|X1)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 9 / 25

slide-66
SLIDE 66

H(X1, X2, X3) Vs. H(X1, X2) + H(X2, X3) + H(X3, X1)

◮ How does H(X1, X2, X3) compares to H(X1, X2) + H(X2, X3) + H(X3, X1)? ◮ If X1, X2, X3 are independence, then

H(X1, X2, X3) = 1

2 (H(X1, X2) + H(X2, X3) + H(X3, X1))

◮ In general: H(X1, X2, X3) ≤ 1

2 (H(X1, X2) + H(X2, X3) + H(X3, X1))

◮ A tighter bounds than H(X1) + H(X2) + H(X3) ◮ Proof:

2H(X1, X2, X3) = 2H(X1) +2H(X2|X1) +2H(X3|X1, X2) H(X2, X3) = H(X1) +H(X2|X1) H(X2, X3) = +H(X2) +H(X3|X2) H(X1, X3) = H(X1) +H(X3|X1)

◮ but

H(X2|X1) ≤ H(X2) H(X3|X1, X2) ≤ H(X3|X1) H(X3|X1, X2) ≤ H(X3|X2)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 9 / 25

slide-67
SLIDE 67

Shearer’s lemma

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 10 / 25

slide-68
SLIDE 68

Shearer’s lemma

◮ Let X = (X1, . . . , Xn)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 10 / 25

slide-69
SLIDE 69

Shearer’s lemma

◮ Let X = (X1, . . . , Xn) ◮ For S = {i1, . . . , ik} ⊆ [n], let XS = (Xi1, . . . , Xik )

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 10 / 25

slide-70
SLIDE 70

Shearer’s lemma

◮ Let X = (X1, . . . , Xn) ◮ For S = {i1, . . . , ik} ⊆ [n], let XS = (Xi1, . . . , Xik ) ◮ Example: X1,3 = (X1, X3)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 10 / 25

slide-71
SLIDE 71

Shearer’s lemma

◮ Let X = (X1, . . . , Xn) ◮ For S = {i1, . . . , ik} ⊆ [n], let XS = (Xi1, . . . , Xik ) ◮ Example: X1,3 = (X1, X3)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 10 / 25

slide-72
SLIDE 72

Shearer’s lemma

◮ Let X = (X1, . . . , Xn) ◮ For S = {i1, . . . , ik} ⊆ [n], let XS = (Xi1, . . . , Xik ) ◮ Example: X1,3 = (X1, X3)

Lemma 2 (Shearer’s lemma) Let X = (X1, . . . , Xn) be a rv and let F be a family of subset of [n] s.t. each i ∈ [n] appears in at least m subset of F. Then H(X) ≤ 1

m

  • F∈F H(XF).

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 10 / 25

slide-73
SLIDE 73

Shearer’s lemma

◮ Let X = (X1, . . . , Xn) ◮ For S = {i1, . . . , ik} ⊆ [n], let XS = (Xi1, . . . , Xik ) ◮ Example: X1,3 = (X1, X3)

Lemma 2 (Shearer’s lemma) Let X = (X1, . . . , Xn) be a rv and let F be a family of subset of [n] s.t. each i ∈ [n] appears in at least m subset of F. Then H(X) ≤ 1

m

  • F∈F H(XF).

Proof:

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 10 / 25

slide-74
SLIDE 74

Shearer’s lemma

◮ Let X = (X1, . . . , Xn) ◮ For S = {i1, . . . , ik} ⊆ [n], let XS = (Xi1, . . . , Xik ) ◮ Example: X1,3 = (X1, X3)

Lemma 2 (Shearer’s lemma) Let X = (X1, . . . , Xn) be a rv and let F be a family of subset of [n] s.t. each i ∈ [n] appears in at least m subset of F. Then H(X) ≤ 1

m

  • F∈F H(XF).

Proof:

◮ H(X) = n

i=1 H(Xi|{Xℓ : ℓ < i})

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 10 / 25

slide-75
SLIDE 75

Shearer’s lemma

◮ Let X = (X1, . . . , Xn) ◮ For S = {i1, . . . , ik} ⊆ [n], let XS = (Xi1, . . . , Xik ) ◮ Example: X1,3 = (X1, X3)

Lemma 2 (Shearer’s lemma) Let X = (X1, . . . , Xn) be a rv and let F be a family of subset of [n] s.t. each i ∈ [n] appears in at least m subset of F. Then H(X) ≤ 1

m

  • F∈F H(XF).

Proof:

◮ H(X) = n

i=1 H(Xi|{Xℓ : ℓ < i})

◮ H(XF) =

i∈F H(Xi|{Xℓ : ℓ < i ∧ ℓ ∈ F})

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 10 / 25

slide-76
SLIDE 76

Shearer’s lemma

◮ Let X = (X1, . . . , Xn) ◮ For S = {i1, . . . , ik} ⊆ [n], let XS = (Xi1, . . . , Xik ) ◮ Example: X1,3 = (X1, X3)

Lemma 2 (Shearer’s lemma) Let X = (X1, . . . , Xn) be a rv and let F be a family of subset of [n] s.t. each i ∈ [n] appears in at least m subset of F. Then H(X) ≤ 1

m

  • F∈F H(XF).

Proof:

◮ H(X) = n

i=1 H(Xi|{Xℓ : ℓ < i})

◮ H(XF) =

i∈F H(Xi|{Xℓ : ℓ < i ∧ ℓ ∈ F})

◮ Hence,

  • F∈F

H(XF) ≥

n

  • i=1

m

  • j=1

H(Xi|{Xℓ : ℓ < i ∧ ℓ ∈ Fi,m})

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 10 / 25

slide-77
SLIDE 77

Shearer’s lemma

◮ Let X = (X1, . . . , Xn) ◮ For S = {i1, . . . , ik} ⊆ [n], let XS = (Xi1, . . . , Xik ) ◮ Example: X1,3 = (X1, X3)

Lemma 2 (Shearer’s lemma) Let X = (X1, . . . , Xn) be a rv and let F be a family of subset of [n] s.t. each i ∈ [n] appears in at least m subset of F. Then H(X) ≤ 1

m

  • F∈F H(XF).

Proof:

◮ H(X) = n

i=1 H(Xi|{Xℓ : ℓ < i})

◮ H(XF) =

i∈F H(Xi|{Xℓ : ℓ < i ∧ ℓ ∈ F})

◮ Hence,

  • F∈F

H(XF) ≥

n

  • i=1

m

  • j=1

H(Xi|{Xℓ : ℓ < i ∧ ℓ ∈ Fi,m})

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 10 / 25

slide-78
SLIDE 78

Shearer’s lemma

◮ Let X = (X1, . . . , Xn) ◮ For S = {i1, . . . , ik} ⊆ [n], let XS = (Xi1, . . . , Xik ) ◮ Example: X1,3 = (X1, X3)

Lemma 2 (Shearer’s lemma) Let X = (X1, . . . , Xn) be a rv and let F be a family of subset of [n] s.t. each i ∈ [n] appears in at least m subset of F. Then H(X) ≤ 1

m

  • F∈F H(XF).

Proof:

◮ H(X) = n

i=1 H(Xi|{Xℓ : ℓ < i})

◮ H(XF) =

i∈F H(Xi|{Xℓ : ℓ < i ∧ ℓ ∈ F})

◮ Hence,

  • F∈F

H(XF) ≥

n

  • i=1

m

  • j=1

H(Xi|{Xℓ : ℓ < i ∧ ℓ ∈ Fi,m}) ≥ m ·

n

  • i=1

H(Xi|{Xℓ : ℓ < i})

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 10 / 25

slide-79
SLIDE 79

Shearer’s lemma

◮ Let X = (X1, . . . , Xn) ◮ For S = {i1, . . . , ik} ⊆ [n], let XS = (Xi1, . . . , Xik ) ◮ Example: X1,3 = (X1, X3)

Lemma 2 (Shearer’s lemma) Let X = (X1, . . . , Xn) be a rv and let F be a family of subset of [n] s.t. each i ∈ [n] appears in at least m subset of F. Then H(X) ≤ 1

m

  • F∈F H(XF).

Proof:

◮ H(X) = n

i=1 H(Xi|{Xℓ : ℓ < i})

◮ H(XF) =

i∈F H(Xi|{Xℓ : ℓ < i ∧ ℓ ∈ F})

◮ Hence,

  • F∈F

H(XF) ≥

n

  • i=1

m

  • j=1

H(Xi|{Xℓ : ℓ < i ∧ ℓ ∈ Fi,m}) ≥ m ·

n

  • i=1

H(Xi|{Xℓ : ℓ < i}) = m · H(X)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 10 / 25

slide-80
SLIDE 80

Corollary

Corollary 3 Let F = {F ⊆ [n]: |F| = k}. Then H(X) ≤ n

k · 1

(

n k) ·

F∈F H(XF) = n k · EF←F [H(XF)].

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 11 / 25

slide-81
SLIDE 81

Corollary

Corollary 3 Let F = {F ⊆ [n]: |F| = k}. Then H(X) ≤ n

k · 1

(

n k) ·

F∈F H(XF) = n k · EF←F [H(XF)].

Proof: k

n ·

n

k

  • is the # of times i appears in F.

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 11 / 25

slide-82
SLIDE 82

Corollary

Corollary 3 Let F = {F ⊆ [n]: |F| = k}. Then H(X) ≤ n

k · 1

(

n k) ·

F∈F H(XF) = n k · EF←F [H(XF)].

Proof: k

n ·

n

k

  • is the # of times i appears in F.

Implications:

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 11 / 25

slide-83
SLIDE 83

Corollary

Corollary 3 Let F = {F ⊆ [n]: |F| = k}. Then H(X) ≤ n

k · 1

(

n k) ·

F∈F H(XF) = n k · EF←F [H(XF)].

Proof: k

n ·

n

k

  • is the # of times i appears in F.

Implications:

◮ Let Q ⊆ {0, 1}n and X = (X1, . . . , Xn) ← Q

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 11 / 25

slide-84
SLIDE 84

Corollary

Corollary 3 Let F = {F ⊆ [n]: |F| = k}. Then H(X) ≤ n

k · 1

(

n k) ·

F∈F H(XF) = n k · EF←F [H(XF)].

Proof: k

n ·

n

k

  • is the # of times i appears in F.

Implications:

◮ Let Q ⊆ {0, 1}n and X = (X1, . . . , Xn) ← Q ◮ |Q| ≤ 2

n k ·EF←F[H(XF )]

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 11 / 25

slide-85
SLIDE 85

Corollary

Corollary 3 Let F = {F ⊆ [n]: |F| = k}. Then H(X) ≤ n

k · 1

(

n k) ·

F∈F H(XF) = n k · EF←F [H(XF)].

Proof: k

n ·

n

k

  • is the # of times i appears in F.

Implications:

◮ Let Q ⊆ {0, 1}n and X = (X1, . . . , Xn) ← Q ◮ |Q| ≤ 2

n k ·EF←F[H(XF )]

◮ EF [H(XF)] is small =

⇒ Q is small

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 11 / 25

slide-86
SLIDE 86

Corollary

Corollary 3 Let F = {F ⊆ [n]: |F| = k}. Then H(X) ≤ n

k · 1

(

n k) ·

F∈F H(XF) = n k · EF←F [H(XF)].

Proof: k

n ·

n

k

  • is the # of times i appears in F.

Implications:

◮ Let Q ⊆ {0, 1}n and X = (X1, . . . , Xn) ← Q ◮ |Q| ≤ 2

n k ·EF←F[H(XF )]

◮ EF [H(XF)] is small =

⇒ Q is small

◮ Q is large =

⇒ EF [H(XF)] is large

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 11 / 25

slide-87
SLIDE 87

Example

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 12 / 25

slide-88
SLIDE 88

Example

◮ Q ⊆ {0, 1}n with |Q| = 2n/2 = 2n−1; X ← Q.

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 12 / 25

slide-89
SLIDE 89

Example

◮ Q ⊆ {0, 1}n with |Q| = 2n/2 = 2n−1; X ← Q. ◮ F = {F ⊆ [n]: |F| = k}

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 12 / 25

slide-90
SLIDE 90

Example

◮ Q ⊆ {0, 1}n with |Q| = 2n/2 = 2n−1; X ← Q. ◮ F = {F ⊆ [n]: |F| = k} ◮ By Corollary 3, log |Q| = n − 1 ≤ n

k · EF←F [H(XF)]

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 12 / 25

slide-91
SLIDE 91

Example

◮ Q ⊆ {0, 1}n with |Q| = 2n/2 = 2n−1; X ← Q. ◮ F = {F ⊆ [n]: |F| = k} ◮ By Corollary 3, log |Q| = n − 1 ≤ n

k · EF←F [H(XF)]

= ⇒ EF [H(XF)] ≥ k(1 − 1

n) = n − k n

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 12 / 25

slide-92
SLIDE 92

Example

◮ Q ⊆ {0, 1}n with |Q| = 2n/2 = 2n−1; X ← Q. ◮ F = {F ⊆ [n]: |F| = k} ◮ By Corollary 3, log |Q| = n − 1 ≤ n

k · EF←F [H(XF)]

= ⇒ EF [H(XF)] ≥ k(1 − 1

n) = n − k n

= ⇒ ∃F ∈ F s.t. H(XF) ≥ n − k

n

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 12 / 25

slide-93
SLIDE 93

Example

◮ Q ⊆ {0, 1}n with |Q| = 2n/2 = 2n−1; X ← Q. ◮ F = {F ⊆ [n]: |F| = k} ◮ By Corollary 3, log |Q| = n − 1 ≤ n

k · EF←F [H(XF)]

= ⇒ EF [H(XF)] ≥ k(1 − 1

n) = n − k n

= ⇒ ∃F ∈ F s.t. H(XF) ≥ n − k

n

◮ Assume n = 1000 and k = 5, hence H(XF) ≥ 5 −

1 200

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 12 / 25

slide-94
SLIDE 94

Example

◮ Q ⊆ {0, 1}n with |Q| = 2n/2 = 2n−1; X ← Q. ◮ F = {F ⊆ [n]: |F| = k} ◮ By Corollary 3, log |Q| = n − 1 ≤ n

k · EF←F [H(XF)]

= ⇒ EF [H(XF)] ≥ k(1 − 1

n) = n − k n

= ⇒ ∃F ∈ F s.t. H(XF) ≥ n − k

n

◮ Assume n = 1000 and k = 5, hence H(XF) ≥ 5 −

1 200

◮ XF can take at least 25− 1

200 = 2− 1 200 · 25 > 31 (and hence 32) values

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 12 / 25

slide-95
SLIDE 95

Example

◮ Q ⊆ {0, 1}n with |Q| = 2n/2 = 2n−1; X ← Q. ◮ F = {F ⊆ [n]: |F| = k} ◮ By Corollary 3, log |Q| = n − 1 ≤ n

k · EF←F [H(XF)]

= ⇒ EF [H(XF)] ≥ k(1 − 1

n) = n − k n

= ⇒ ∃F ∈ F s.t. H(XF) ≥ n − k

n

◮ Assume n = 1000 and k = 5, hence H(XF) ≥ 5 −

1 200

◮ XF can take at least 25− 1

200 = 2− 1 200 · 25 > 31 (and hence 32) values

◮ Stronger conclusion: XF is close to the uniform distribution.

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 12 / 25

slide-96
SLIDE 96

More generally

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 13 / 25

slide-97
SLIDE 97

More generally

◮ |Q| ≥

1 2d · 2n; X ← Q

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 13 / 25

slide-98
SLIDE 98

More generally

◮ |Q| ≥

1 2d · 2n; X ← Q

◮ F = {F ⊆ [n]: |F| = k}

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 13 / 25

slide-99
SLIDE 99

More generally

◮ |Q| ≥

1 2d · 2n; X ← Q

◮ F = {F ⊆ [n]: |F| = k} ◮ n − d ≤ H(X) ≤ n

k · 1 |F| · F∈F H(XF)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 13 / 25

slide-100
SLIDE 100

More generally

◮ |Q| ≥

1 2d · 2n; X ← Q

◮ F = {F ⊆ [n]: |F| = k} ◮ n − d ≤ H(X) ≤ n

k · 1 |F| · F∈F H(XF)

= ⇒

1 |F| · F∈F H(XF) ≥ k − dk n

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 13 / 25

slide-101
SLIDE 101

More generally

◮ |Q| ≥

1 2d · 2n; X ← Q

◮ F = {F ⊆ [n]: |F| = k} ◮ n − d ≤ H(X) ≤ n

k · 1 |F| · F∈F H(XF)

= ⇒

1 |F| · F∈F H(XF) ≥ k − dk n

= ⇒ EF←F [H(XF)] ≥ k − dk

n

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 13 / 25

slide-102
SLIDE 102

More generally

◮ |Q| ≥

1 2d · 2n; X ← Q

◮ F = {F ⊆ [n]: |F| = k} ◮ n − d ≤ H(X) ≤ n

k · 1 |F| · F∈F H(XF)

= ⇒

1 |F| · F∈F H(XF) ≥ k − dk n

= ⇒ EF←F [H(XF)] ≥ k − dk

n

◮ If dk << n, then a typical XF is close to the uniform distribution

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 13 / 25

slide-103
SLIDE 103

Section 4 Statistical Distance

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 14 / 25

slide-104
SLIDE 104

Statistical distance

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 15 / 25

slide-105
SLIDE 105

Statistical distance

◮ Let p = (p1, . . . , pm) and q = (q1, . . . , qm) be distributions over [m]

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 15 / 25

slide-106
SLIDE 106

Statistical distance

◮ Let p = (p1, . . . , pm) and q = (q1, . . . , qm) be distributions over [m] ◮ Their statistical distance (also known as, variation distance) is defined by

SD(p, q) := 1 2

  • i∈[m]

|pi − qi|

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 15 / 25

slide-107
SLIDE 107

Statistical distance

◮ Let p = (p1, . . . , pm) and q = (q1, . . . , qm) be distributions over [m] ◮ Their statistical distance (also known as, variation distance) is defined by

SD(p, q) := 1 2

  • i∈[m]

|pi − qi|

◮ This is simply the L1 norm between the distribution vectors

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 15 / 25

slide-108
SLIDE 108

Statistical distance

◮ Let p = (p1, . . . , pm) and q = (q1, . . . , qm) be distributions over [m] ◮ Their statistical distance (also known as, variation distance) is defined by

SD(p, q) := 1 2

  • i∈[m]

|pi − qi|

◮ This is simply the L1 norm between the distribution vectors ◮ We will see other “distance" measures for distributions next lecture

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 15 / 25

slide-109
SLIDE 109

Statistical distance

◮ Let p = (p1, . . . , pm) and q = (q1, . . . , qm) be distributions over [m] ◮ Their statistical distance (also known as, variation distance) is defined by

SD(p, q) := 1 2

  • i∈[m]

|pi − qi|

◮ This is simply the L1 norm between the distribution vectors ◮ We will see other “distance" measures for distributions next lecture ◮ For Z ∼ p and Y ∼ q, let SD(X, Y) = SD(p, q)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 15 / 25

slide-110
SLIDE 110

Statistical distance

◮ Let p = (p1, . . . , pm) and q = (q1, . . . , qm) be distributions over [m] ◮ Their statistical distance (also known as, variation distance) is defined by

SD(p, q) := 1 2

  • i∈[m]

|pi − qi|

◮ This is simply the L1 norm between the distribution vectors ◮ We will see other “distance" measures for distributions next lecture ◮ For Z ∼ p and Y ∼ q, let SD(X, Y) = SD(p, q) ◮ Claim (HW): SD(p, q) = maxS⊆[m]

  • i∈S pi −

i∈S qi

  • Iftach Haitner (TAU)

Application of Information Theory, Lecture 6 December 2, 2014 15 / 25

slide-111
SLIDE 111

Statistical distance

◮ Let p = (p1, . . . , pm) and q = (q1, . . . , qm) be distributions over [m] ◮ Their statistical distance (also known as, variation distance) is defined by

SD(p, q) := 1 2

  • i∈[m]

|pi − qi|

◮ This is simply the L1 norm between the distribution vectors ◮ We will see other “distance" measures for distributions next lecture ◮ For Z ∼ p and Y ∼ q, let SD(X, Y) = SD(p, q) ◮ Claim (HW): SD(p, q) = maxS⊆[m]

  • i∈S pi −

i∈S qi

  • ◮ Hence, SD(p, q) = maxD (PrX∼p [D(X) = 1] − PrX∼q [D(X) = 1])

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 15 / 25

slide-112
SLIDE 112

Statistical distance

◮ Let p = (p1, . . . , pm) and q = (q1, . . . , qm) be distributions over [m] ◮ Their statistical distance (also known as, variation distance) is defined by

SD(p, q) := 1 2

  • i∈[m]

|pi − qi|

◮ This is simply the L1 norm between the distribution vectors ◮ We will see other “distance" measures for distributions next lecture ◮ For Z ∼ p and Y ∼ q, let SD(X, Y) = SD(p, q) ◮ Claim (HW): SD(p, q) = maxS⊆[m]

  • i∈S pi −

i∈S qi

  • ◮ Hence, SD(p, q) = maxD (PrX∼p [D(X) = 1] − PrX∼q [D(X) = 1])

◮ Interpretation

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 15 / 25

slide-113
SLIDE 113

Distance from the uniform distribution

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 16 / 25

slide-114
SLIDE 114

Distance from the uniform distribution

◮ Let X be rv over [m]

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 16 / 25

slide-115
SLIDE 115

Distance from the uniform distribution

◮ Let X be rv over [m] ◮ H(X) ≤ log m

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 16 / 25

slide-116
SLIDE 116

Distance from the uniform distribution

◮ Let X be rv over [m] ◮ H(X) ≤ log m ◮ H(X) = m ←

→ X is uniform over [m]

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 16 / 25

slide-117
SLIDE 117

Distance from the uniform distribution

◮ Let X be rv over [m] ◮ H(X) ≤ log m ◮ H(X) = m ←

→ X is uniform over [m] Theorem 4 (Next lecture) Let X rv over [m]. Assume H(X) ≥ log m − ε, then SD(X, ∼ [m]) ≤ √ ε · 2 · ln 2 = O(ε)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 16 / 25

slide-118
SLIDE 118

Section 5 Gold Coins

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 17 / 25

slide-119
SLIDE 119

# of gold coins in a cube

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 18 / 25

slide-120
SLIDE 120

# of gold coins in a cube

◮ Q — (finite) set of points in R3

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 18 / 25

slide-121
SLIDE 121

# of gold coins in a cube

◮ Q — (finite) set of points in R3

◮ Projection of Q on xy — 6 Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 18 / 25

slide-122
SLIDE 122

# of gold coins in a cube

◮ Q — (finite) set of points in R3

◮ Projection of Q on xy — 6 ◮ Projection of Q on xz — 8 Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 18 / 25

slide-123
SLIDE 123

# of gold coins in a cube

◮ Q — (finite) set of points in R3

◮ Projection of Q on xy — 6 ◮ Projection of Q on xz — 8 ◮ Projection of Q on yz — 12 Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 18 / 25

slide-124
SLIDE 124

# of gold coins in a cube

◮ Q — (finite) set of points in R3

◮ Projection of Q on xy — 6 ◮ Projection of Q on xz — 8 ◮ Projection of Q on yz — 12

◮ Can we bound |Q|?

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 18 / 25

slide-125
SLIDE 125

# of gold coins in a cube

◮ Q — (finite) set of points in R3

◮ Projection of Q on xy — 6 ◮ Projection of Q on xz — 8 ◮ Projection of Q on yz — 12

◮ Can we bound |Q|? ◮ The real story

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 18 / 25

slide-126
SLIDE 126

# of gold coins in a cube

◮ Q — (finite) set of points in R3

◮ Projection of Q on xy — 6 ◮ Projection of Q on xz — 8 ◮ Projection of Q on yz — 12

◮ Can we bound |Q|? ◮ The real story ◮ X = (X1, X2, X3) ← Q

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 18 / 25

slide-127
SLIDE 127

# of gold coins in a cube

◮ Q — (finite) set of points in R3

◮ Projection of Q on xy — 6 ◮ Projection of Q on xz — 8 ◮ Projection of Q on yz — 12

◮ Can we bound |Q|? ◮ The real story ◮ X = (X1, X2, X3) ← Q ◮

log |Q| = H(X) ≤ 1 2(H(X1, X2) + H(X1, X3) + H(X2, X3))

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 18 / 25

slide-128
SLIDE 128

# of gold coins in a cube

◮ Q — (finite) set of points in R3

◮ Projection of Q on xy — 6 ◮ Projection of Q on xz — 8 ◮ Projection of Q on yz — 12

◮ Can we bound |Q|? ◮ The real story ◮ X = (X1, X2, X3) ← Q ◮

log |Q| = H(X) ≤ 1 2(H(X1, X2) + H(X1, X3) + H(X2, X3))

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 18 / 25

slide-129
SLIDE 129

# of gold coins in a cube

◮ Q — (finite) set of points in R3

◮ Projection of Q on xy — 6 ◮ Projection of Q on xz — 8 ◮ Projection of Q on yz — 12

◮ Can we bound |Q|? ◮ The real story ◮ X = (X1, X2, X3) ← Q ◮

log |Q| = H(X) ≤ 1 2(H(X1, X2) + H(X1, X3) + H(X2, X3)) ≤ 1 2(log 6 + log 8 + log 12)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 18 / 25

slide-130
SLIDE 130

# of gold coins in a cube

◮ Q — (finite) set of points in R3

◮ Projection of Q on xy — 6 ◮ Projection of Q on xz — 8 ◮ Projection of Q on yz — 12

◮ Can we bound |Q|? ◮ The real story ◮ X = (X1, X2, X3) ← Q ◮

log |Q| = H(X) ≤ 1 2(H(X1, X2) + H(X1, X3) + H(X2, X3)) ≤ 1 2(log 6 + log 8 + log 12) ≤ 1 2(log 6 · 8 · 12)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 18 / 25

slide-131
SLIDE 131

# of gold coins in a cube

◮ Q — (finite) set of points in R3

◮ Projection of Q on xy — 6 ◮ Projection of Q on xz — 8 ◮ Projection of Q on yz — 12

◮ Can we bound |Q|? ◮ The real story ◮ X = (X1, X2, X3) ← Q ◮

log |Q| = H(X) ≤ 1 2(H(X1, X2) + H(X1, X3) + H(X2, X3)) ≤ 1 2(log 6 + log 8 + log 12) ≤ 1 2(log 6 · 8 · 12)

◮ Hence, |Q| ≤

√ 6 · 8 · 12 = 24

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 18 / 25

slide-132
SLIDE 132

# of gold coins in a cube

◮ Q — (finite) set of points in R3

◮ Projection of Q on xy — 6 ◮ Projection of Q on xz — 8 ◮ Projection of Q on yz — 12

◮ Can we bound |Q|? ◮ The real story ◮ X = (X1, X2, X3) ← Q ◮

log |Q| = H(X) ≤ 1 2(H(X1, X2) + H(X1, X3) + H(X2, X3)) ≤ 1 2(log 6 + log 8 + log 12) ≤ 1 2(log 6 · 8 · 12)

◮ Hence, |Q| ≤

√ 6 · 8 · 12 = 24

◮ Tight!

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 18 / 25

slide-133
SLIDE 133

# of gold coins in a cube

◮ Q — (finite) set of points in R3

◮ Projection of Q on xy — 6 ◮ Projection of Q on xz — 8 ◮ Projection of Q on yz — 12

◮ Can we bound |Q|? ◮ The real story ◮ X = (X1, X2, X3) ← Q ◮

log |Q| = H(X) ≤ 1 2(H(X1, X2) + H(X1, X3) + H(X2, X3)) ≤ 1 2(log 6 + log 8 + log 12) ≤ 1 2(log 6 · 8 · 12)

◮ Hence, |Q| ≤

√ 6 · 8 · 12 = 24

◮ Tight!

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 18 / 25

slide-134
SLIDE 134

# of gold coins in a cube

◮ Q — (finite) set of points in R3

◮ Projection of Q on xy — 6 ◮ Projection of Q on xz — 8 ◮ Projection of Q on yz — 12

◮ Can we bound |Q|? ◮ The real story ◮ X = (X1, X2, X3) ← Q ◮

log |Q| = H(X) ≤ 1 2(H(X1, X2) + H(X1, X3) + H(X2, X3)) ≤ 1 2(log 6 + log 8 + log 12) ≤ 1 2(log 6 · 8 · 12)

◮ Hence, |Q| ≤

√ 6 · 8 · 12 = 24

◮ Tight! 2 × 3 × 4

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 18 / 25

slide-135
SLIDE 135

# of gold coins, the hyperspace case

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 19 / 25

slide-136
SLIDE 136

# of gold coins, the hyperspace case

◮ Q — (finite) set of points in Rn

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 19 / 25

slide-137
SLIDE 137

# of gold coins, the hyperspace case

◮ Q — (finite) set of points in Rn ◮ mi— # of coins in projection on (1, . . . , i − 1, i + 1, . . . , n)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 19 / 25

slide-138
SLIDE 138

# of gold coins, the hyperspace case

◮ Q — (finite) set of points in Rn ◮ mi— # of coins in projection on (1, . . . , i − 1, i + 1, . . . , n) ◮ Claim: |Q| ≤ (

i∈[n] mi)1/(n−1)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 19 / 25

slide-139
SLIDE 139

# of gold coins, the hyperspace case

◮ Q — (finite) set of points in Rn ◮ mi— # of coins in projection on (1, . . . , i − 1, i + 1, . . . , n) ◮ Claim: |Q| ≤ (

i∈[n] mi)1/(n−1)

◮ Proof: X = (X1, . . . , Xn) ← Q, X−i = (X1, . . . , Xi−1, Xi+1, . . . , Xn)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 19 / 25

slide-140
SLIDE 140

# of gold coins, the hyperspace case

◮ Q — (finite) set of points in Rn ◮ mi— # of coins in projection on (1, . . . , i − 1, i + 1, . . . , n) ◮ Claim: |Q| ≤ (

i∈[n] mi)1/(n−1)

◮ Proof: X = (X1, . . . , Xn) ← Q, X−i = (X1, . . . , Xi−1, Xi+1, . . . , Xn) ◮ log |Q| = H(X) ≤

1 n−1

  • i H(X−i)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 19 / 25

slide-141
SLIDE 141

# of gold coins, the hyperspace case

◮ Q — (finite) set of points in Rn ◮ mi— # of coins in projection on (1, . . . , i − 1, i + 1, . . . , n) ◮ Claim: |Q| ≤ (

i∈[n] mi)1/(n−1)

◮ Proof: X = (X1, . . . , Xn) ← Q, X−i = (X1, . . . , Xi−1, Xi+1, . . . , Xn) ◮ log |Q| = H(X) ≤

1 n−1

  • i H(X−i)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 19 / 25

slide-142
SLIDE 142

# of gold coins, the hyperspace case

◮ Q — (finite) set of points in Rn ◮ mi— # of coins in projection on (1, . . . , i − 1, i + 1, . . . , n) ◮ Claim: |Q| ≤ (

i∈[n] mi)1/(n−1)

◮ Proof: X = (X1, . . . , Xn) ← Q, X−i = (X1, . . . , Xi−1, Xi+1, . . . , Xn) ◮ log |Q| = H(X) ≤

1 n−1

  • i H(X−i) ≤

1 n−1

  • i log mi

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 19 / 25

slide-143
SLIDE 143

Section 6 Intersecting Graphs

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 20 / 25

slide-144
SLIDE 144

Another corollary of Shearer’s lemma

Corollary 5 Let A and F be collections of subsets of [n], and for F ∈ F let AF be the collection {A ∩ F : A ∈ A}. Assume that each element of [n] appears in at least m subsets of F, then |A|m ≤

F∈F |AF|.

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 21 / 25

slide-145
SLIDE 145

Another corollary of Shearer’s lemma

Corollary 5 Let A and F be collections of subsets of [n], and for F ∈ F let AF be the collection {A ∩ F : A ∈ A}. Assume that each element of [n] appears in at least m subsets of F, then |A|m ≤

F∈F |AF|.

Proof:

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 21 / 25

slide-146
SLIDE 146

Another corollary of Shearer’s lemma

Corollary 5 Let A and F be collections of subsets of [n], and for F ∈ F let AF be the collection {A ∩ F : A ∈ A}. Assume that each element of [n] appears in at least m subsets of F, then |A|m ≤

F∈F |AF|.

Proof:

◮ Let Y ← A, let Xi = 1 iff i ∈ Y, and X = (X1, . . . , Xn).

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 21 / 25

slide-147
SLIDE 147

Another corollary of Shearer’s lemma

Corollary 5 Let A and F be collections of subsets of [n], and for F ∈ F let AF be the collection {A ∩ F : A ∈ A}. Assume that each element of [n] appears in at least m subsets of F, then |A|m ≤

F∈F |AF|.

Proof:

◮ Let Y ← A, let Xi = 1 iff i ∈ Y, and X = (X1, . . . , Xn). ◮ log |A| = H(Y) = H(X)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 21 / 25

slide-148
SLIDE 148

Another corollary of Shearer’s lemma

Corollary 5 Let A and F be collections of subsets of [n], and for F ∈ F let AF be the collection {A ∩ F : A ∈ A}. Assume that each element of [n] appears in at least m subsets of F, then |A|m ≤

F∈F |AF|.

Proof:

◮ Let Y ← A, let Xi = 1 iff i ∈ Y, and X = (X1, . . . , Xn). ◮ log |A| = H(Y) = H(X)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 21 / 25

slide-149
SLIDE 149

Another corollary of Shearer’s lemma

Corollary 5 Let A and F be collections of subsets of [n], and for F ∈ F let AF be the collection {A ∩ F : A ∈ A}. Assume that each element of [n] appears in at least m subsets of F, then |A|m ≤

F∈F |AF|.

Proof:

◮ Let Y ← A, let Xi = 1 iff i ∈ Y, and X = (X1, . . . , Xn). ◮ log |A| = H(Y) = H(X)

( Y ← → X)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 21 / 25

slide-150
SLIDE 150

Another corollary of Shearer’s lemma

Corollary 5 Let A and F be collections of subsets of [n], and for F ∈ F let AF be the collection {A ∩ F : A ∈ A}. Assume that each element of [n] appears in at least m subsets of F, then |A|m ≤

F∈F |AF|.

Proof:

◮ Let Y ← A, let Xi = 1 iff i ∈ Y, and X = (X1, . . . , Xn). ◮ log |A| = H(Y) = H(X)

( Y ← → X)

◮ log |AF| ≥ H(XF)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 21 / 25

slide-151
SLIDE 151

Another corollary of Shearer’s lemma

Corollary 5 Let A and F be collections of subsets of [n], and for F ∈ F let AF be the collection {A ∩ F : A ∈ A}. Assume that each element of [n] appears in at least m subsets of F, then |A|m ≤

F∈F |AF|.

Proof:

◮ Let Y ← A, let Xi = 1 iff i ∈ Y, and X = (X1, . . . , Xn). ◮ log |A| = H(Y) = H(X)

( Y ← → X)

◮ log |AF| ≥ H(XF)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 21 / 25

slide-152
SLIDE 152

Another corollary of Shearer’s lemma

Corollary 5 Let A and F be collections of subsets of [n], and for F ∈ F let AF be the collection {A ∩ F : A ∈ A}. Assume that each element of [n] appears in at least m subsets of F, then |A|m ≤

F∈F |AF|.

Proof:

◮ Let Y ← A, let Xi = 1 iff i ∈ Y, and X = (X1, . . . , Xn). ◮ log |A| = H(Y) = H(X)

( Y ← → X)

◮ log |AF| ≥ H(XF)

( Supp(XF) ⊆ AF)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 21 / 25

slide-153
SLIDE 153

Another corollary of Shearer’s lemma

Corollary 5 Let A and F be collections of subsets of [n], and for F ∈ F let AF be the collection {A ∩ F : A ∈ A}. Assume that each element of [n] appears in at least m subsets of F, then |A|m ≤

F∈F |AF|.

Proof:

◮ Let Y ← A, let Xi = 1 iff i ∈ Y, and X = (X1, . . . , Xn). ◮ log |A| = H(Y) = H(X)

( Y ← → X)

◮ log |AF| ≥ H(XF)

( Supp(XF) ⊆ AF)

◮ By Shearer’s lemma, H(X) ≤ 1

m

  • F∈F H(XF).

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 21 / 25

slide-154
SLIDE 154

# of intersecting graphs

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 22 / 25

slide-155
SLIDE 155

# of intersecting graphs

Theorem 6 Let G be a family of graphs over [n], s.t. G ∩ G′ contains a triangle for each G, G′ ∈ G. Then |G| ≤ 2(

n 2)−2.

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 22 / 25

slide-156
SLIDE 156

# of intersecting graphs

Theorem 6 Let G be a family of graphs over [n], s.t. G ∩ G′ contains a triangle for each G, G′ ∈ G. Then |G| ≤ 2(

n 2)−2.

This improves over |G| ≤ 2(

n 2)−1, which follows from G ∩ G′ = ∅.

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 22 / 25

slide-157
SLIDE 157

# of intersecting graphs

Theorem 6 Let G be a family of graphs over [n], s.t. G ∩ G′ contains a triangle for each G, G′ ∈ G. Then |G| ≤ 2(

n 2)−2.

This improves over |G| ≤ 2(

n 2)−1, which follows from G ∩ G′ = ∅.

(wlg. all graph shares the same edge)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 22 / 25

slide-158
SLIDE 158

# of intersecting graphs

Theorem 6 Let G be a family of graphs over [n], s.t. G ∩ G′ contains a triangle for each G, G′ ∈ G. Then |G| ≤ 2(

n 2)−2.

This improves over |G| ≤ 2(

n 2)−1, which follows from G ∩ G′ = ∅.

(wlg. all graph shares the same edge) Proof:

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 22 / 25

slide-159
SLIDE 159

# of intersecting graphs

Theorem 6 Let G be a family of graphs over [n], s.t. G ∩ G′ contains a triangle for each G, G′ ∈ G. Then |G| ≤ 2(

n 2)−2.

This improves over |G| ≤ 2(

n 2)−1, which follows from G ∩ G′ = ∅.

(wlg. all graph shares the same edge) Proof:

◮ For ⌊n/2⌋-size set S ⊂ [n], let F = F(S) be the union of the cliques S

and [n] \ S

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 22 / 25

slide-160
SLIDE 160

# of intersecting graphs

Theorem 6 Let G be a family of graphs over [n], s.t. G ∩ G′ contains a triangle for each G, G′ ∈ G. Then |G| ≤ 2(

n 2)−2.

This improves over |G| ≤ 2(

n 2)−1, which follows from G ∩ G′ = ∅.

(wlg. all graph shares the same edge) Proof:

◮ For ⌊n/2⌋-size set S ⊂ [n], let F = F(S) be the union of the cliques S

and [n] \ S

◮ F ∩ G ∩ G′ = ∅ for any G, G′ ∈ G and S as above

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 22 / 25

slide-161
SLIDE 161

# of intersecting graphs

Theorem 6 Let G be a family of graphs over [n], s.t. G ∩ G′ contains a triangle for each G, G′ ∈ G. Then |G| ≤ 2(

n 2)−2.

This improves over |G| ≤ 2(

n 2)−1, which follows from G ∩ G′ = ∅.

(wlg. all graph shares the same edge) Proof:

◮ For ⌊n/2⌋-size set S ⊂ [n], let F = F(S) be the union of the cliques S

and [n] \ S

◮ F ∩ G ∩ G′ = ∅ for any G, G′ ∈ G and S as above ◮ Hence |GF := G ∩ F : G ∈ G| ≤ 2|F|−1

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 22 / 25

slide-162
SLIDE 162

# of intersecting graphs

Theorem 6 Let G be a family of graphs over [n], s.t. G ∩ G′ contains a triangle for each G, G′ ∈ G. Then |G| ≤ 2(

n 2)−2.

This improves over |G| ≤ 2(

n 2)−1, which follows from G ∩ G′ = ∅.

(wlg. all graph shares the same edge) Proof:

◮ For ⌊n/2⌋-size set S ⊂ [n], let F = F(S) be the union of the cliques S

and [n] \ S

◮ F ∩ G ∩ G′ = ∅ for any G, G′ ∈ G and S as above ◮ Hence |GF := G ∩ F : G ∈ G| ≤ 2|F|−1 ◮ Let m =

n

2

  • and m′ = |F|

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 22 / 25

slide-163
SLIDE 163

# of intersecting graphs

Theorem 6 Let G be a family of graphs over [n], s.t. G ∩ G′ contains a triangle for each G, G′ ∈ G. Then |G| ≤ 2(

n 2)−2.

This improves over |G| ≤ 2(

n 2)−1, which follows from G ∩ G′ = ∅.

(wlg. all graph shares the same edge) Proof:

◮ For ⌊n/2⌋-size set S ⊂ [n], let F = F(S) be the union of the cliques S

and [n] \ S

◮ F ∩ G ∩ G′ = ∅ for any G, G′ ∈ G and S as above ◮ Hence |GF := G ∩ F : G ∈ G| ≤ 2|F|−1 ◮ Let m =

n

2

  • and m′ = |F|

◮ Each edge over [n] × [n], appears in m′

m of graphs {F(S)}S⊂[n]: |S|=⌊n/2⌋.

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 22 / 25

slide-164
SLIDE 164

# of intersecting graphs

Theorem 6 Let G be a family of graphs over [n], s.t. G ∩ G′ contains a triangle for each G, G′ ∈ G. Then |G| ≤ 2(

n 2)−2.

This improves over |G| ≤ 2(

n 2)−1, which follows from G ∩ G′ = ∅.

(wlg. all graph shares the same edge) Proof:

◮ For ⌊n/2⌋-size set S ⊂ [n], let F = F(S) be the union of the cliques S

and [n] \ S

◮ F ∩ G ∩ G′ = ∅ for any G, G′ ∈ G and S as above ◮ Hence |GF := G ∩ F : G ∈ G| ≤ 2|F|−1 ◮ Let m =

n

2

  • and m′ = |F|

◮ Each edge over [n] × [n], appears in m′

m of graphs {F(S)}S⊂[n]: |S|=⌊n/2⌋.

◮ By Corollary 5, |G|

m′ m ·( n ⌊n/2⌋) ≤ (2m′−1)( n ⌊n/2⌋)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 22 / 25

slide-165
SLIDE 165

# of intersecting graphs

Theorem 6 Let G be a family of graphs over [n], s.t. G ∩ G′ contains a triangle for each G, G′ ∈ G. Then |G| ≤ 2(

n 2)−2.

This improves over |G| ≤ 2(

n 2)−1, which follows from G ∩ G′ = ∅.

(wlg. all graph shares the same edge) Proof:

◮ For ⌊n/2⌋-size set S ⊂ [n], let F = F(S) be the union of the cliques S

and [n] \ S

◮ F ∩ G ∩ G′ = ∅ for any G, G′ ∈ G and S as above ◮ Hence |GF := G ∩ F : G ∈ G| ≤ 2|F|−1 ◮ Let m =

n

2

  • and m′ = |F|

◮ Each edge over [n] × [n], appears in m′

m of graphs {F(S)}S⊂[n]: |S|=⌊n/2⌋.

◮ By Corollary 5, |G|

m′ m ·( n ⌊n/2⌋) ≤ (2m′−1)( n ⌊n/2⌋)

◮ Hence, |G| ≤ 2m− m

m′ ≤ 2( n 2)−2

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 22 / 25

slide-166
SLIDE 166

Section 7 Independent Sets

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 23 / 25

slide-167
SLIDE 167

# of independent sets in bi-partite graphs

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 24 / 25

slide-168
SLIDE 168

# of independent sets in bi-partite graphs

Theorem 7 Let G = (A, B, E) be an n-regular graph with |A| = |B| = m. Then the number

  • f independent sets in G is at most (2n+1 − 1)m.

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 24 / 25

slide-169
SLIDE 169

# of independent sets in bi-partite graphs

Theorem 7 Let G = (A, B, E) be an n-regular graph with |A| = |B| = m. Then the number

  • f independent sets in G is at most (2n+1 − 1)m.

Proof: I — set of independent sets in G.

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 24 / 25

slide-170
SLIDE 170

# of independent sets in bi-partite graphs

Theorem 7 Let G = (A, B, E) be an n-regular graph with |A| = |B| = m. Then the number

  • f independent sets in G is at most (2n+1 − 1)m.

Proof: I — set of independent sets in G.

◮ Let I ← I, let Xv = 1 iff v ∈ I, and XS = {Xv : v ∈ S}.

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 24 / 25

slide-171
SLIDE 171

# of independent sets in bi-partite graphs

Theorem 7 Let G = (A, B, E) be an n-regular graph with |A| = |B| = m. Then the number

  • f independent sets in G is at most (2n+1 − 1)m.

Proof: I — set of independent sets in G.

◮ Let I ← I, let Xv = 1 iff v ∈ I, and XS = {Xv : v ∈ S}. ◮ H(I) = H(XA|XB) + H(XB)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 24 / 25

slide-172
SLIDE 172

# of independent sets in bi-partite graphs

Theorem 7 Let G = (A, B, E) be an n-regular graph with |A| = |B| = m. Then the number

  • f independent sets in G is at most (2n+1 − 1)m.

Proof: I — set of independent sets in G.

◮ Let I ← I, let Xv = 1 iff v ∈ I, and XS = {Xv : v ∈ S}. ◮ H(I) = H(XA|XB) + H(XB)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 24 / 25

slide-173
SLIDE 173

# of independent sets in bi-partite graphs

Theorem 7 Let G = (A, B, E) be an n-regular graph with |A| = |B| = m. Then the number

  • f independent sets in G is at most (2n+1 − 1)m.

Proof: I — set of independent sets in G.

◮ Let I ← I, let Xv = 1 iff v ∈ I, and XS = {Xv : v ∈ S}. ◮ H(I) = H(XA|XB) + H(XB)

≤

  • v∈A

H(Xv|XB) + 1 n

  • v∈A

H(XN(v)) (rhs by Sherer’s Lemma)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 24 / 25

slide-174
SLIDE 174

# of independent sets in bi-partite graphs

Theorem 7 Let G = (A, B, E) be an n-regular graph with |A| = |B| = m. Then the number

  • f independent sets in G is at most (2n+1 − 1)m.

Proof: I — set of independent sets in G.

◮ Let I ← I, let Xv = 1 iff v ∈ I, and XS = {Xv : v ∈ S}. ◮ H(I) = H(XA|XB) + H(XB)

≤

  • v∈A

H(Xv|XB) + 1 n

  • v∈A

H(XN(v)) (rhs by Sherer’s Lemma) ≤

  • v∈A
  • H(Xv|N(v)) + 1

nH(XN(v))

  • Iftach Haitner (TAU)

Application of Information Theory, Lecture 6 December 2, 2014 24 / 25

slide-175
SLIDE 175

# of independent sets in bi-partite graphs

Theorem 7 Let G = (A, B, E) be an n-regular graph with |A| = |B| = m. Then the number

  • f independent sets in G is at most (2n+1 − 1)m.

Proof: I — set of independent sets in G.

◮ Let I ← I, let Xv = 1 iff v ∈ I, and XS = {Xv : v ∈ S}. ◮ H(I) = H(XA|XB) + H(XB)

≤

  • v∈A

H(Xv|XB) + 1 n

  • v∈A

H(XN(v)) (rhs by Sherer’s Lemma) ≤

  • v∈A
  • H(Xv|N(v)) + 1

nH(XN(v))

  • ◮ Fix v ∈ A. Let χv =
  • 0,

XN(v) = 0|N(v)| 1,

  • therwise.

, and p = p(v) = Pr [χv = 0]

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 24 / 25

slide-176
SLIDE 176

# of independent sets in bi-partite graphs

Theorem 7 Let G = (A, B, E) be an n-regular graph with |A| = |B| = m. Then the number

  • f independent sets in G is at most (2n+1 − 1)m.

Proof: I — set of independent sets in G.

◮ Let I ← I, let Xv = 1 iff v ∈ I, and XS = {Xv : v ∈ S}. ◮ H(I) = H(XA|XB) + H(XB)

≤

  • v∈A

H(Xv|XB) + 1 n

  • v∈A

H(XN(v)) (rhs by Sherer’s Lemma) ≤

  • v∈A
  • H(Xv|N(v)) + 1

nH(XN(v))

  • ◮ Fix v ∈ A. Let χv =
  • 0,

XN(v) = 0|N(v)| 1,

  • therwise.

, and p = p(v) = Pr [χv = 0]

◮ H(Xv|XN(v)) ≤ H(Xv|χv)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 24 / 25

slide-177
SLIDE 177

# of independent sets in bi-partite graphs

Theorem 7 Let G = (A, B, E) be an n-regular graph with |A| = |B| = m. Then the number

  • f independent sets in G is at most (2n+1 − 1)m.

Proof: I — set of independent sets in G.

◮ Let I ← I, let Xv = 1 iff v ∈ I, and XS = {Xv : v ∈ S}. ◮ H(I) = H(XA|XB) + H(XB)

≤

  • v∈A

H(Xv|XB) + 1 n

  • v∈A

H(XN(v)) (rhs by Sherer’s Lemma) ≤

  • v∈A
  • H(Xv|N(v)) + 1

nH(XN(v))

  • ◮ Fix v ∈ A. Let χv =
  • 0,

XN(v) = 0|N(v)| 1,

  • therwise.

, and p = p(v) = Pr [χv = 0]

◮ H(Xv|XN(v)) ≤ H(Xv|χv)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 24 / 25

slide-178
SLIDE 178

# of independent sets in bi-partite graphs

Theorem 7 Let G = (A, B, E) be an n-regular graph with |A| = |B| = m. Then the number

  • f independent sets in G is at most (2n+1 − 1)m.

Proof: I — set of independent sets in G.

◮ Let I ← I, let Xv = 1 iff v ∈ I, and XS = {Xv : v ∈ S}. ◮ H(I) = H(XA|XB) + H(XB)

≤

  • v∈A

H(Xv|XB) + 1 n

  • v∈A

H(XN(v)) (rhs by Sherer’s Lemma) ≤

  • v∈A
  • H(Xv|N(v)) + 1

nH(XN(v))

  • ◮ Fix v ∈ A. Let χv =
  • 0,

XN(v) = 0|N(v)| 1,

  • therwise.

, and p = p(v) = Pr [χv = 0]

◮ H(Xv|XN(v)) ≤ H(Xv|χv) ≤ p

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 24 / 25

slide-179
SLIDE 179

# of independent sets in bi-partite graphs

Theorem 7 Let G = (A, B, E) be an n-regular graph with |A| = |B| = m. Then the number

  • f independent sets in G is at most (2n+1 − 1)m.

Proof: I — set of independent sets in G.

◮ Let I ← I, let Xv = 1 iff v ∈ I, and XS = {Xv : v ∈ S}. ◮ H(I) = H(XA|XB) + H(XB)

≤

  • v∈A

H(Xv|XB) + 1 n

  • v∈A

H(XN(v)) (rhs by Sherer’s Lemma) ≤

  • v∈A
  • H(Xv|N(v)) + 1

nH(XN(v))

  • ◮ Fix v ∈ A. Let χv =
  • 0,

XN(v) = 0|N(v)| 1,

  • therwise.

, and p = p(v) = Pr [χv = 0]

◮ H(Xv|XN(v)) ≤ H(Xv|χv) ≤ p ◮ H(XN(v)) = H(XN(v)χv)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 24 / 25

slide-180
SLIDE 180

# of independent sets in bi-partite graphs

Theorem 7 Let G = (A, B, E) be an n-regular graph with |A| = |B| = m. Then the number

  • f independent sets in G is at most (2n+1 − 1)m.

Proof: I — set of independent sets in G.

◮ Let I ← I, let Xv = 1 iff v ∈ I, and XS = {Xv : v ∈ S}. ◮ H(I) = H(XA|XB) + H(XB)

≤

  • v∈A

H(Xv|XB) + 1 n

  • v∈A

H(XN(v)) (rhs by Sherer’s Lemma) ≤

  • v∈A
  • H(Xv|N(v)) + 1

nH(XN(v))

  • ◮ Fix v ∈ A. Let χv =
  • 0,

XN(v) = 0|N(v)| 1,

  • therwise.

, and p = p(v) = Pr [χv = 0]

◮ H(Xv|XN(v)) ≤ H(Xv|χv) ≤ p ◮ H(XN(v)) = H(XN(v)χv)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 24 / 25

slide-181
SLIDE 181

# of independent sets in bi-partite graphs

Theorem 7 Let G = (A, B, E) be an n-regular graph with |A| = |B| = m. Then the number

  • f independent sets in G is at most (2n+1 − 1)m.

Proof: I — set of independent sets in G.

◮ Let I ← I, let Xv = 1 iff v ∈ I, and XS = {Xv : v ∈ S}. ◮ H(I) = H(XA|XB) + H(XB)

≤

  • v∈A

H(Xv|XB) + 1 n

  • v∈A

H(XN(v)) (rhs by Sherer’s Lemma) ≤

  • v∈A
  • H(Xv|N(v)) + 1

nH(XN(v))

  • ◮ Fix v ∈ A. Let χv =
  • 0,

XN(v) = 0|N(v)| 1,

  • therwise.

, and p = p(v) = Pr [χv = 0]

◮ H(Xv|XN(v)) ≤ H(Xv|χv) ≤ p ◮ H(XN(v)) = H(XN(v)χv) = H(χv)+H(XN(v)|χv)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 24 / 25

slide-182
SLIDE 182

# of independent sets in bi-partite graphs

Theorem 7 Let G = (A, B, E) be an n-regular graph with |A| = |B| = m. Then the number

  • f independent sets in G is at most (2n+1 − 1)m.

Proof: I — set of independent sets in G.

◮ Let I ← I, let Xv = 1 iff v ∈ I, and XS = {Xv : v ∈ S}. ◮ H(I) = H(XA|XB) + H(XB)

≤

  • v∈A

H(Xv|XB) + 1 n

  • v∈A

H(XN(v)) (rhs by Sherer’s Lemma) ≤

  • v∈A
  • H(Xv|N(v)) + 1

nH(XN(v))

  • ◮ Fix v ∈ A. Let χv =
  • 0,

XN(v) = 0|N(v)| 1,

  • therwise.

, and p = p(v) = Pr [χv = 0]

◮ H(Xv|XN(v)) ≤ H(Xv|χv) ≤ p ◮ H(XN(v)) = H(XN(v)χv) = H(χv)+H(XN(v)|χv) ≤ h(p)+(1−p) log(2n−1)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 24 / 25

slide-183
SLIDE 183

# of independent sets in bi-partite graphs

Theorem 7 Let G = (A, B, E) be an n-regular graph with |A| = |B| = m. Then the number

  • f independent sets in G is at most (2n+1 − 1)m.

Proof: I — set of independent sets in G.

◮ Let I ← I, let Xv = 1 iff v ∈ I, and XS = {Xv : v ∈ S}. ◮ H(I) = H(XA|XB) + H(XB)

≤

  • v∈A

H(Xv|XB) + 1 n

  • v∈A

H(XN(v)) (rhs by Sherer’s Lemma) ≤

  • v∈A
  • H(Xv|N(v)) + 1

nH(XN(v))

  • ◮ Fix v ∈ A. Let χv =
  • 0,

XN(v) = 0|N(v)| 1,

  • therwise.

, and p = p(v) = Pr [χv = 0]

◮ H(Xv|XN(v)) ≤ H(Xv|χv) ≤ p ◮ H(XN(v)) = H(XN(v)χv) = H(χv)+H(XN(v)|χv) ≤ h(p)+(1−p) log(2n−1) ◮ Hence H(I) ≤

v∈A p(v) + 1 n (h(p(v)) + (1 − p(v)) log(2n − 1))

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 24 / 25

slide-184
SLIDE 184

# of independent sets in bi-partite graphs, cont.

◮ log |I| = H(I) ≤

v∈A p(v) + 1 n (h(p(v)) + (1 − p(v)) log(2n − 1))

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 25 / 25

slide-185
SLIDE 185

# of independent sets in bi-partite graphs, cont.

◮ log |I| = H(I) ≤

v∈A p(v) + 1 n (h(p(v)) + (1 − p(v)) log(2n − 1))

◮ Let f(t) := t + 1

n (h(t) + (1 − p(t)) log(2n − 1))

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 25 / 25

slide-186
SLIDE 186

# of independent sets in bi-partite graphs, cont.

◮ log |I| = H(I) ≤

v∈A p(v) + 1 n (h(p(v)) + (1 − p(v)) log(2n − 1))

◮ Let f(t) := t + 1

n (h(t) + (1 − p(t)) log(2n − 1))

◮ By calculus, maxt∈[0,1] f(t) = 1

n log(2n+1 − 1)

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 25 / 25

slide-187
SLIDE 187

# of independent sets in bi-partite graphs, cont.

◮ log |I| = H(I) ≤

v∈A p(v) + 1 n (h(p(v)) + (1 − p(v)) log(2n − 1))

◮ Let f(t) := t + 1

n (h(t) + (1 − p(t)) log(2n − 1))

◮ By calculus, maxt∈[0,1] f(t) = 1

n log(2n+1 − 1)

◮ Hence, log |I| ≤ m

n log(2n+1 − 1).

Iftach Haitner (TAU) Application of Information Theory, Lecture 6 December 2, 2014 25 / 25