[PPT] - Graph ensemble design for channel coding A. Montanari 1 A. Amraoui 2 PowerPoint Presentation

SLIDE 1

Graph ensemble design for channel coding

A. Montanari1
A. Amraoui2
T. Richardson3
R. Urbanke2
A. Dembo4

1ENS, France → Stanford, USA 2EPFL, Switzerland, 3Flarion, USA 4Stanford, USA

October 18, 2006

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 2

Outline

1

The optimization problem

2

A probabilistic strategy

2

The approximate formula

3

Future directions

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 3

The optimization problem

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 4

The object to be optimized: A code, i.e. a graph

1 2 · · · n 1 2 · · · m

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 5

The objective function

Let S ⊆ [n] be random with density ǫ ∈ [0, 1]... PB(G) = Pǫ {S contains a ‘stopping set’}

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 6

The objective function

up-degree ≥ 2

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 7

The objective function

up-degree ≥ 2

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 8

The objective function

up-degree ≥ 2

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 9

If you did not get it

Bipartite graph G ↔ hypergraph H PB = Pǫ{a random subgraph of H contains a 2-core}

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 10

A probabilistic strategy

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 11

General strategy

1 Define a graph ensemble with parameters n, m,

λ = (λ1, λ2, . . . , λk), ρ = (ρ1, ρ2, . . . , ρk).

2 Prove an pproximate formula Eλ,ρPB ≃ Q(λ, ρ). [*] 3 Find (λ∗, ρ∗) = arg min Q(λ, ρ). 4 Sample G from the (λ∗, ρ∗)-ensemble and check it.

[Concentration]

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 12

General strategy

1 Define a graph ensemble with parameters n, m,

λ = (λ1, λ2, . . . , λk), ρ = (ρ1, ρ2, . . . , ρk).

2 Prove an pproximate formula Eλ,ρPB ≃ Q(λ, ρ). [*] 3 Find (λ∗, ρ∗) = arg min Q(λ, ρ). 4 Sample G from the (λ∗, ρ∗)-ensemble and check it.

[Concentration]

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 13

General strategy

1 Define a graph ensemble with parameters n, m,

λ = (λ1, λ2, . . . , λk), ρ = (ρ1, ρ2, . . . , ρk).

2 Prove an pproximate formula Eλ,ρPB ≃ Q(λ, ρ). [*] 3 Find (λ∗, ρ∗) = arg min Q(λ, ρ). 4 Sample G from the (λ∗, ρ∗)-ensemble and check it.

[Concentration]

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 14

General strategy

1 Define a graph ensemble with parameters n, m,

λ = (λ1, λ2, . . . , λk), ρ = (ρ1, ρ2, . . . , ρk).

2 Prove an pproximate formula Eλ,ρPB ≃ Q(λ, ρ). [*] 3 Find (λ∗, ρ∗) = arg min Q(λ, ρ). 4 Sample G from the (λ∗, ρ∗)-ensemble and check it.

[Concentration]

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 15

General strategy

1 Define a graph ensemble with parameters n, m,

λ = (λ1, λ2, . . . , λk), ρ = (ρ1, ρ2, . . . , ρk).

2 Prove an pproximate formula Eλ,ρPB ≃ Q(λ, ρ). [*] 3 Find (λ∗, ρ∗) = arg min Q(λ, ρ). 4 Sample G from the (λ∗, ρ∗)-ensemble and check it.

[Concentration]

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 16

A ‘standard’ ensemble

G random (configuration model) with →Up-degree distribution: ρ = (ρ1, . . . , ρk) →Down-degree distribution: λ = (λ1, . . . , λk) Good for n = ∞! [Luby et al.]

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 17

Approximate formula for Eλ,ρPB

LATER!!!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 18

Implementation [Amraoui/Montanari/Urbanke]

Sample Run: Minimize m/n, given PB = 10−4 ǫ = 0.5 n = 5000 Largest degree 13 Expurgation 6 [I did not explain this]

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 19

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 40.58 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=0

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 20

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 40.97 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=5

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 21

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 41.34 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=10

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 22

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 41.68 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=15

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 23

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 42.01 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=20

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 24

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 42.33 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=25

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 25

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 43.63 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=30

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 26

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 46.01 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=35

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 27

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 49.59 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=40

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 28

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 53.04 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=45

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 29

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 55.55 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=50

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 30

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 57.90 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=55

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 31

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 60.43 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=60

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 32

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 63.49 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=65

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 33

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 65.22 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=70

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 34

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 66.88 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=75

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 35

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 67.98 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=80

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 36

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 70.08 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=85

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 37

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 72.22 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=115

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 38

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 76.77 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=255

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 39

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 78.68 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=355

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 40

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 79.15 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=390

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 41

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 79.72 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=445

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 42

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 80.65 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=500

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 43

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 81.43 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=555

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 44

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 81.66 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=580

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 45

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 81.84 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=595

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 46

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 82.01 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=610

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 47

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 82.13 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=625

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 48

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 82.03 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=660

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 49

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 82.09 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=700

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 50

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 82.13 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=713

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 51

Example Run

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 10-6 10-5 10-4 10-3 10-2 10-1 82.13 % 0.0 1.0 rate/capacity 2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10 11 12 13 contribution to error floor 6 8 10 12 14 16 18 20 22 24 26

0.0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

0.1
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01

counter:=713

λ = 0.0739196x + 0.65789x2 + 0.2681x12, ρ = 0.390753x4 + 0.361589x5 + 0.247658x9.

Play it Again Sam!

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 52

The approximate formula

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 53

For the sake of simplicity

I’ll just consider the interesting part of the formula. Up-degree Poisson, down regular Want to approximate the probability that a random l-hypergraph contains a 2-core

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 54

For the sake of simplicity

I’ll just consider the interesting part of the formula. Up-degree Poisson, down regular Want to approximate the probability that a random l-hypergraph contains a 2-core

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 55

Theorem (Luby et al., 1997, Darling/Norris, 2003) Consider a random l-hypergraph over n edges and m = n/γ vertices. There exists γd > 0, such that PB(γ, n) → 0 if γ < γd and P(γ, n) → 1 if γ > γd. Furthermore γd = sup{γ | x > 1 − e−lγxl−1 ∀x ∈ (0, 1]} .

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 56

Theorem (Luby et al., 1997, Darling/Norris, 2003) Consider a random l-hypergraph over n edges and m = n/γ vertices. There exists γd > 0, such that PB(γ, n) → 0 if γ < γd and P(γ, n) → 1 if γ > γd. Furthermore γd = sup{γ | x > 1 − e−lγxl−1 ∀x ∈ (0, 1]} .

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 57

Theorem (Luby et al., 1997, Darling/Norris, 2003) Consider a random l-hypergraph over n edges and m = n/γ vertices. There exists γd > 0, such that PB(γ, n) → 0 if γ < γd and P(γ, n) → 1 if γ > γd. Furthermore γd = sup{γ | x > 1 − e−lγxl−1 ∀x ∈ (0, 1]} .

0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 x

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 58

Theorem (Luby et al., 1997, Darling/Norris, 2003) Consider a random l-hypergraph over n edges and m = n/γ vertices. There exists γd > 0, such that PB(γ, n) → 0 if γ < γd and P(γ, n) → 1 if γ > γd. Furthermore γd = sup{γ | x > 1 − e−lγxl−1 ∀x ∈ (0, 1]} .

0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 x

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 59

Theorem (Luby et al., 1997, Darling/Norris, 2003) Consider a random l-hypergraph over n edges and m = n/γ vertices. There exists γd > 0, such that PB(γ, n) → 0 if γ < γd and P(γ, n) → 1 if γ > γd. Furthermore γd = sup{γ | x > 1 − e−lγxl−1 ∀x ∈ (0, 1]} .

0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 x

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 60

Theorem (Luby et al., 1997, Darling/Norris, 2003) Consider a random l-hypergraph over n edges and m = n/γ vertices. There exists γd > 0, such that PB(γ, n) → 0 if γ < γd and P(γ, n) → 1 if γ > γd. Furthermore γd = sup{γ | x > 1 − e−lγxl−1 ∀x ∈ (0, 1]} .

0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 x

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 61

Theorem (Luby et al., 1997, Darling/Norris, 2003) Consider a random l-hypergraph over n edges and m = n/γ vertices. There exists γd > 0, such that PB(γ, n) → 0 if γ < γd and P(γ, n) → 1 if γ > γd. Furthermore γd = sup{γ | x > 1 − e−lγxl−1 ∀x ∈ (0, 1]} .

0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 x

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 62

Pittel, Spencer, Wormald, 1996: Analogous [classic!] theorem for k-cores of random graphs.

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 63

Finite-size scaling

Theorem (Dembo/Montanari 2006) Let γn = γd + r n−1/2. Then P(γn, n) = Φ(r/α) + βΩ Φ′(r/α) n−1/6 + o(n−1/6) , where Φ(x) is the error function, Ω ≡ ∞

0 [1 − K(z)2] dz, and

K(z) ≡ 1 2 Ai(iy)Bi(21/3z + iy) − Ai(21/3z + iy)Bi(iy) Ai(iy) dy . Finally α and β. . . [solution ODE’s]

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 64

Data

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.6 0.7 0.8 0.9 1 P γ 100 200 300 400 600

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 65

Scaling: Without correction

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

3 -2.5 -2 -1.5 -1 -0.5 0

0.5 1 1.5 2 2.5 3 P N1/2(γ-γd)/α 100 200 300 400 600 Q(-x)

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 66

Scaling: With correction

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

3 -2.5 -2 -1.5 -1 -0.5 0

0.5 1 1.5 2 2.5 3 P N1/2(γ-γd+βN-2/3)/α 100 200 300 400 600 Q(-x)

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 67

Analytical (!) expression for the scaling parameters

α = ρ(¯ x)2 − ρ(¯ x2) + ρ′(¯ x)(1 − 2xρ(¯ x)) − ¯ x2ρ′(¯ x2) L′(1)λ(y)2ρ′(¯ x)2 + ǫ2λ(y)2 − ǫ2λ(y2) − y2ǫ2λ′(y2) L′(1)λ(y)2 1/2 , β = . . .

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 68

For those who have read Wormald et al.

Basic ideas in the proof Graph-pruning procedure as a Markov process in R2k. Prove a local CLT for the state. Approximate by parabola + Brownian near critical point (KMT).

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 69

0.05 0.1 0.15 0.2 n-t 0.2 0.4 0.6 0.8 z2 0.15 0.1 z1 v

(z1(t), z2(t)) discrete time Markov process PB = Pgauss

min

t

z1(t) ≤ 0

.
A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 70

Correction term

z1 t t∗ tmin O(n2/3) O(n1/3) Model: z1(t) = t2/2n + BM(t) ⇒ O(n−1/2) · O(n1/3) = O(n−1/6) corrections.

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 71

Future directions

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding

SLIDE 72

Future directions

Other ensembles (Turbo, RA, multi-edge, etc.) Code dependency whithin the ensemble. General memoryless channels.

A. Montanari A. Amraoui, T. Richardson, R. Urbanke A. Dembo

Graph ensemble design for channel coding