Computable Lower Bounds for Capacities of Input-Driven Finite-State - - PowerPoint PPT Presentation

▶
computable lower bounds for capacities of input driven
SMART_READER_LITE
LIVE PREVIEW

Computable Lower Bounds for Capacities of Input-Driven Finite-State - - PowerPoint PPT Presentation

Dec 09, 2022 48 likes •772 views

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Computable Lower Bounds for Capacities of Input-Driven Finite-State Channels V. Arvind Rameshwar Navin Kashyap Department of Electrical Communication

slide-1
SLIDE 1

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Computable Lower Bounds for Capacities of Input-Driven Finite-State Channels

  • V. Arvind Rameshwar

Navin Kashyap

Department of Electrical Communication Engineering Indian Institute of Science, Bengaluru

ISIT 2020

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-2
SLIDE 2

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Outline

1

Background Motivation System Model

2

Ideas and Directions

3

Single-Letter Lower Bound

4

Applications Input-constrained BEC(ǫ) Input-constrained BSC(p)

5

Future Work

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-3
SLIDE 3

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Motivation System Model

FSCs and Capacity

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-4
SLIDE 4

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Motivation System Model

FSCs and Capacity

Discrete Finite-State Channels (FSCs) are used to model:

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-5
SLIDE 5

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Motivation System Model

FSCs and Capacity

Discrete Finite-State Channels (FSCs) are used to model:

Inter-Symbol Interference in Magnetic and Optical Recording [e.g., Immink, Siegel, Wolf, ’98] Inter-Cell Interference in NAND Flash Memories [e.g., Li, Kavˇ ci´ c, Han, ’16] Fading in Mobile Radio Channels [e.g., Semmar, Lecours, Chouinard, Ahern, ’91]

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-6
SLIDE 6

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Motivation System Model

FSCs and Capacity

Discrete Finite-State Channels (FSCs) are used to model:

Inter-Symbol Interference in Magnetic and Optical Recording [e.g., Immink, Siegel, Wolf, ’98] Inter-Cell Interference in NAND Flash Memories [e.g., Li, Kavˇ ci´ c, Han, ’16] Fading in Mobile Radio Channels [e.g., Semmar, Lecours, Chouinard, Ahern, ’91]

Computing Capacities of FSCs

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-7
SLIDE 7

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Motivation System Model

FSCs and Capacity

Discrete Finite-State Channels (FSCs) are used to model:

Inter-Symbol Interference in Magnetic and Optical Recording [e.g., Immink, Siegel, Wolf, ’98] Inter-Cell Interference in NAND Flash Memories [e.g., Li, Kavˇ ci´ c, Han, ’16] Fading in Mobile Radio Channels [e.g., Semmar, Lecours, Chouinard, Ahern, ’91]

Computing Capacities of FSCs → Capacity-achieving coding schemes

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-8
SLIDE 8

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Motivation System Model

FSCs and Capacity

Discrete Finite-State Channels (FSCs) are used to model:

Inter-Symbol Interference in Magnetic and Optical Recording [e.g., Immink, Siegel, Wolf, ’98] Inter-Cell Interference in NAND Flash Memories [e.g., Li, Kavˇ ci´ c, Han, ’16] Fading in Mobile Radio Channels [e.g., Semmar, Lecours, Chouinard, Ahern, ’91]

Computing Capacities of FSCs → Capacity-achieving coding schemes For even simple input-constrained DMCs, Computing Capacity ≡ Computing H(Y)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-9
SLIDE 9

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Motivation System Model

FSCs and Capacity

Discrete Finite-State Channels (FSCs) are used to model:

Inter-Symbol Interference in Magnetic and Optical Recording [e.g., Immink, Siegel, Wolf, ’98] Inter-Cell Interference in NAND Flash Memories [e.g., Li, Kavˇ ci´ c, Han, ’16] Fading in Mobile Radio Channels [e.g., Semmar, Lecours, Chouinard, Ahern, ’91]

Computing Capacities of FSCs → Capacity-achieving coding schemes For even simple input-constrained DMCs, Computing Capacity ≡ Computing H(Y) Hard!

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-10
SLIDE 10

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Motivation System Model

FSCs and Capacity

Discrete Finite-State Channels (FSCs) are used to model:

Inter-Symbol Interference in Magnetic and Optical Recording [e.g., Immink, Siegel, Wolf, ’98] Inter-Cell Interference in NAND Flash Memories [e.g., Li, Kavˇ ci´ c, Han, ’16] Fading in Mobile Radio Channels [e.g., Semmar, Lecours, Chouinard, Ahern, ’91]

Computing Capacities of FSCs → Capacity-achieving coding schemes For even simple input-constrained DMCs, Computing Capacity ≡ Computing H(Y) Hard! Goal: Get good bounds on capacity without feedback

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-11
SLIDE 11

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Motivation System Model

The Setup

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-12
SLIDE 12

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Motivation System Model

The Setup

DMC: P(yn|xn) =

n

  • i=1

P(yi|xi)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-13
SLIDE 13

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Motivation System Model

The Setup

Generic FSC: P(yn, sn|xn, s0) =

n

  • i=1

P(yi, si|si−1, xi)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-14
SLIDE 14

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Motivation System Model

The Setup

Generic FSC: P(yn, sn|xn, s0) =

n

  • i=1

P(yi, si|xi, si−1) Input-Driven FSC: P(yn, sn|xn, s0) =

n

  • i=1

P(yi|xi, si−1)✶{si = f (si−1, xi)}

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-15
SLIDE 15

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Motivation System Model

Examples

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-16
SLIDE 16

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Motivation System Model

Examples

Input-Constrained DMCs:

(d, ∞)-RLL input constraint:

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-17
SLIDE 17

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Motivation System Model

Examples

Input-Constrained DMCs:

(d, ∞)-RLL input constraint: (d, k)-RLL input constraint:

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-18
SLIDE 18

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Motivation System Model

Examples

Input-Constrained DMCs:

(d, ∞)-RLL input constraint: (d, k)-RLL input constraint:

Other Channels:

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-19
SLIDE 19

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Motivation System Model

Examples

Input-Constrained DMCs:

(d, ∞)-RLL input constraint: (d, k)-RLL input constraint:

Other Channels:

Flash-Memory Channel (101 → 111 w.p. ǫ): si = (xi, xi−1)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-20
SLIDE 20

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Motivation System Model

Examples

Input-Constrained DMCs:

(d, ∞)-RLL input constraint: (d, k)-RLL input constraint:

Other Channels:

Flash-Memory Channel (101 → 111 w.p. ǫ): si = (xi, xi−1) ISI: yi =

m

  • k=0

hkxi−k + zi si = (xi−m, . . . , xi−1)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-21
SLIDE 21

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Motivation System Model

Examples

Input-Constrained DMCs:

(d, ∞)-RLL input constraint: (d, k)-RLL input constraint:

Other Channels:

Flash-Memory Channel (101 → 111 w.p. ǫ): si = (xi, xi−1) ISI: yi =

m

  • k=0

hkxi−k + zi si = (xi−m, . . . , xi−1)

Assumption: s0 known to encoder and decoder

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-22
SLIDE 22

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Motivation System Model

Summary of Results

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-23
SLIDE 23

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Motivation System Model

Summary of Results

C = lim

N→∞

max

Q(xN|s0)

1 N I(X N; Y N|s0)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-24
SLIDE 24

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Motivation System Model

Summary of Results

C = lim

N→∞

max

Q(xN|s0)

1 N I(X N; Y N|s0)

Upper Bounds FB Capacity [Sabag et al., ’16, ’18]:

1

(1, ∞)-RLL input-constrained BEC

2

(1, ∞)-RLL input-constrained BIBO Dual-Capacity [Thangaraj, ’17] Dual-Capacity + DP [Huleihel et al., ’19]

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-25
SLIDE 25

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Motivation System Model

Summary of Results

C = lim

N→∞

max

Q(xN|s0)

1 N I(X N; Y N|s0)

Upper Bounds FB Capacity [Sabag et al., ’16, ’18]:

1

(1, ∞)-RLL input-constrained BEC

2

(1, ∞)-RLL input-constrained BIBO Dual-Capacity [Thangaraj, ’17] Dual-Capacity + DP [Huleihel et al., ’19] Lower Bounds Simulation-Based:

1

M-C [Arnold et al., ’06]

2

GBA [Vontobel et al., ’08]

3

  • Stoch. Approx. [Han, ’15]

Analytical:

1

Asymptotics of BSC, BEC [Han and Marcus, ’09] [Li and Han, ’18]

2

Markov inputs for BSC [Zehavi and Wolf, ’88]

3

General input-driven (this paper)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-26
SLIDE 26

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Key Ideas

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-27
SLIDE 27

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Key Ideas

Recall: ⋆ Reverse Directed Information: I(Y N → X N|s0) =

N

  • t=1

I(Y t; Xt|X t−1, s0) ⋆ (Delayed) Forward Directed Information: I(X N−1 → Y N|s0) =

N

  • t=1

I(X t−1; Yt|Y t−1, s0)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-28
SLIDE 28

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Ideas (Contd. . .)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-29
SLIDE 29

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Ideas (Contd. . .)

Theorem (Conservation Law (Massey, ‘05)) I(X N; Y N|s0) = I(Y N → X N|s0) + I(X N−1 → Y N|s0) ≥ I(Y N → X N|s0)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-30
SLIDE 30

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Ideas (Contd. . .)

Theorem (Conservation Law (Massey, ‘05)) I(X N; Y N|s0) = I(Y N → X N|s0) + I(X N−1 → Y N|s0) ≥ I(Y N → X N|s0) Can we massage this LB into a computable expression?

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-31
SLIDE 31

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Ideas (Contd. . .)

Theorem (Conservation Law (Massey, ‘05)) I(X N; Y N|s0) = I(Y N → X N|s0) + I(X N−1 → Y N|s0) ≥ I(Y N → X N|s0) Can we massage this LB into a computable expression? Yes!

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-32
SLIDE 32

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Steps Towards Single-Letterization

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-33
SLIDE 33

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Steps Towards Single-Letterization

Step 1:

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-34
SLIDE 34

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Steps Towards Single-Letterization

Step 1: I(X N; Y N|s0) ≥ I(Y N → X N|s0)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-35
SLIDE 35

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Steps Towards Single-Letterization

Step 1: I(X N; Y N|s0) ≥ I(Y N → X N|s0) =

N

  • t=1

I(Xt; Y t|X t−1, s0)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-36
SLIDE 36

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Steps Towards Single-Letterization

Step 1: I(X N; Y N|s0) ≥ I(Y N → X N|s0) ≥

N

  • t=1

I(Xt; Yt|X t−1, s0)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-37
SLIDE 37

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Steps Towards Single-Letterization

Step 1: I(X N; Y N|s0) ≥ I(Y N → X N|s0) ≥

N

  • t=1

I(Xt; Yt|X t−1, s0) Step 2:

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-38
SLIDE 38

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Steps Towards Single-Letterization

Step 1: I(X N; Y N|s0) ≥ I(Y N → X N|s0) ≥

N

  • t=1

I(Xt; Yt|X t−1, s0) Step 2: C = lim

N→∞

max

Q(xN|s0)

1 N I(X N; Y N|s0)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-39
SLIDE 39

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Steps Towards Single-Letterization

Step 1: I(X N; Y N|s0) ≥ I(Y N → X N|s0) ≥

N

  • t=1

I(Xt; Yt|X t−1, s0) Step 2: C = lim

N→∞

max

{Q(xt|xt−1,s0)}N

t=1

1 N I(X N; Y N|s0)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-40
SLIDE 40

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Steps Towards Single-Letterization

Step 1: I(X N; Y N|s0) ≥ I(Y N → X N|s0) ≥

N

  • t=1

I(Xt; Yt|X t−1, s0) Step 2: C = lim

N→∞

max

{Q(xt|xt−1,s0)}N

t=1

1 N I(X N; Y N|s0) ≥ lim

N→∞

max

{Q(xt|xt−1,s0)}N

t=1

1 N

N

  • t=1

I(Xt; Yt|X t−1, s0)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-41
SLIDE 41

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Steps Towards Single-Letterization (Contd. . .)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-42
SLIDE 42

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Steps Towards Single-Letterization (Contd. . .)

Step 3:

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-43
SLIDE 43

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Steps Towards Single-Letterization (Contd. . .)

Step 3: C ≥ lim

N→∞

max

{Q(xt|xt−1,s0)}N

t=1

1 N

N

  • t=1

I(Xt; Yt|X t−1, s0)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-44
SLIDE 44

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Steps Towards Single-Letterization (Contd. . .)

Step 3: C ≥ lim

N→∞

max

{Q(xt|xt−1,s0)}N

t=1

1 N

N

  • t=1

I(Xt; Yt|X t−1, s0) = sup

{Q(xt|xt−1,s0)}N

t=1

lim inf

N→∞

1 N

N

  • t=1

I(Xt; Yt|X t−1, s0) [Permuter et al., ‘08]

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-45
SLIDE 45

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Steps Towards Single-Letterization (Contd. . .)

Step 3: C ≥ lim

N→∞

max

{Q(xt|xt−1,s0)}N

t=1

1 N

N

  • t=1

I(Xt; Yt|X t−1, s0) = sup

{Q(xt|xt−1,s0)}N

t=1

lim inf

N→∞

1 N

N

  • t=1

I(Xt; Yt|X t−1, s0) [Permuter et al., ‘08] ≥ sup

{Q(xt|st−1)}N

t=1

lim inf

N→∞

1 N

N

  • t=1

I(Xt; Yt|X t−1, s0)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-46
SLIDE 46

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Steps Towards Single-Letterization (Contd. . .)

Step 3: C ≥ lim

N→∞

max

{Q(xt|xt−1,s0)}N

t=1

1 N

N

  • t=1

I(Xt; Yt|X t−1, s0) = sup

{Q(xt|xt−1,s0)}N

t=1

lim inf

N→∞

1 N

N

  • t=1

I(Xt; Yt|X t−1, s0) [Permuter et al., ‘08] ≥ sup

{Q(xt|st−1)}N

t=1

lim inf

N→∞

1 N

N

  • t=1

I(Xt; Yt|X t−1, s0) = sup

{Q(xt|st−1)}N

t=1

lim inf

N→∞

1 N

N

  • t=1

I(Xt; Yt|St−1)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-47
SLIDE 47

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Steps Towards Single-Letterization (Contd. . .)

Step 3: C ≥ lim

N→∞

max

{Q(xt|xt−1,s0)}N

t=1

1 N

N

  • t=1

I(Xt; Yt|X t−1, s0) = sup

{Q(xt|xt−1,s0)}N

t=1

lim inf

N→∞

1 N

N

  • t=1

I(Xt; Yt|X t−1, s0) [Permuter et al., ‘08] ≥ sup

{Q(xt|st−1)}N

t=1

lim inf

N→∞

1 N

N

  • t=1

I(Xt; Yt|X t−1, s0) = sup

{Q(xt|st−1)}N

t=1

lim inf

N→∞

1 N

N

  • t=1

I(Xt; Yt|St−1) (Due to nature of channel and i/p distributions optimized over)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-48
SLIDE 48

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Steps Towards Single-Letterization (Contd. . .)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-49
SLIDE 49

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Steps Towards Single-Letterization (Contd. . .)

Notation: ⋆ P {Q(x|s)} : M.C. on S induced by Q is aperiodic and has a unique stationary distribution.

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-50
SLIDE 50

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Steps Towards Single-Letterization (Contd. . .)

Notation: ⋆ P {Q(x|s)} : M.C. on S induced by Q is aperiodic and has a unique stationary distribution. Step 4:

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-51
SLIDE 51

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Steps Towards Single-Letterization (Contd. . .)

Notation: ⋆ P {Q(x|s)} : M.C. on S induced by Q is aperiodic and has a unique stationary distribution. Step 4: C ≥ sup

{Q(xt|st−1)}N

t=1

lim inf

N→∞

1 N

N

  • t=1

I(Xt; Yt|St−1)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-52
SLIDE 52

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Steps Towards Single-Letterization (Contd. . .)

Notation: ⋆ P {Q(x|s)} : M.C. on S induced by Q is aperiodic and has a unique stationary distribution. Step 4: C ≥ sup

{Q(xt|st−1)}N

t=1

lim inf

N→∞

1 N

N

  • t=1

I(Xt; Yt|St−1) ≥ sup

{Q(x|s)∈P}

I(X; Y |S)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-53
SLIDE 53

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Remarks

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-54
SLIDE 54

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Remarks

Derived lower bound can be achieved directly by random coding, where the inputs are generated according to Q∗(xi|si−1) (where Q∗ ∈ P maximizes I(X; Y |S)), and the decoder performs ML decoding.

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-55
SLIDE 55

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Remarks

Derived lower bound can be achieved directly by random coding, where the inputs are generated according to Q∗(xi|si−1) (where Q∗ ∈ P maximizes I(X; Y |S)), and the decoder performs ML decoding. For the BEC(ǫ) with an input constraint presented by an irreducible deterministic graph on S, I(X; Y |S) = H(X|S) − H(X|Y , S) = H(X|S) − ǫH(X|Y =?, S) = H(X|S)(1 − ǫ). Hence, sup{Q(x|s)∈P} I(X; Y |S) = C0(1 − ǫ), where C0 is the noiseless capacity of the input-constraint.

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-56
SLIDE 56

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Input-constrained BEC(ǫ) Input-constrained BSC(p)

Application: (d, ∞)-RLL Input-Constrained BEC(ǫ)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-57
SLIDE 57

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Input-constrained BEC(ǫ) Input-constrained BSC(p)

Application: (d, ∞)-RLL Input-Constrained BEC(ǫ)

Theorem The capacity of the (d, ∞)-RLL input-constrained binary erasure channel with erasure probability ǫ satisfies C BEC(ǫ)

d,∞

≥ Cd,∞ · (1 − ǫ), where Cd,∞ = max

a∈[0,1] hb(a) ad+1 is the (noiseless) capacity of the

(d, ∞)-RLL constraint.

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-58
SLIDE 58

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Input-constrained BEC(ǫ) Input-constrained BSC(p)

Application: (d, k)-RLL Input-Constrained BEC(ǫ)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-59
SLIDE 59

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Input-constrained BEC(ǫ) Input-constrained BSC(p)

Application: (d, k)-RLL Input-Constrained BEC(ǫ)

Theorem The capacity of the (d, k)-RLL input-constrained (k < ∞) binary erasure channel with erasure probability ǫ satisfies C BEC(ǫ)

d,k

≥ Cd,k · (1 − ǫ), where Cd,k = max

ad,...,ak−1

k−1

  • i=d

hb(ai)

i−1

  • j=d

(1−aj) d+1+

k−1

  • i=d

i

  • j=d

(1−aj)

is the (noiseless) capacity

  • f the (d, k)-RLL constraint.
  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-60
SLIDE 60

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Input-constrained BEC(ǫ) Input-constrained BSC(p)

Plots: (1, ∞)-RLL Input-Constrained BEC(ǫ)

0.2 0.4 0.6 0.8 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 Our Lower Bound Feedback Capacity UB Thangaraj's UB

Figure: Comparison of our lower bound with the feedback capacity and dual-capacity upper bounds.

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-61
SLIDE 61

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Input-constrained BEC(ǫ) Input-constrained BSC(p)

Application: (d, k)-RLL Input-Constrained BSC(p)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-62
SLIDE 62

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Input-constrained BEC(ǫ) Input-constrained BSC(p)

Application: (d, k)-RLL Input-Constrained BSC(p)

Theorem The capacity of the (d, k)-RLL input-constrained (k < ∞) binary symmetric channel with cross-over probability p satisfies C BSC(p)

d,k

≥ max

ad,...,ak−1∈[0,1] k−1

  • i=d

hb(aip + ¯ ai ¯ p) − hb(p) i−1

  • j=d

(1 − aj) d + 1 +

k−1

  • i=d

i

  • j=d

(1 − aj) , where ¯ α := 1 − α.

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-63
SLIDE 63

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Input-constrained BEC(ǫ) Input-constrained BSC(p)

Plots: (0, k)-RLL Input-Constrained BSC(p)

Figure: Our lower bounds for the (0, 1), (0, 2) and (0, 3)-RLL input-constrained BSC(p).

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-64
SLIDE 64

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Input-constrained BEC(ǫ) Input-constrained BSC(p)

Application: (d, ∞)-RLL Input-Constrained BSC(p)

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-65
SLIDE 65

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Input-constrained BEC(ǫ) Input-constrained BSC(p)

Application: (d, ∞)-RLL Input-Constrained BSC(p)

Theorem The capacity of the (d, ∞)-RLL input-constrained binary symmetric channel with cross-over probability p obeys C BSC(p)

d,∞

≥ max

a∈[0,1]

hb(ap + ¯ a¯ p) − hb(p) ad + 1 , where ¯ α := 1 − α.

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-66
SLIDE 66

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Input-constrained BEC(ǫ) Input-constrained BSC(p)

Plots: (d, ∞)-RLL Input-Constrained BSC(p)

Figure: Our lower bounds for the (1, ∞), (2, ∞), (3, ∞)-RLL input-constrained BSC(p).

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-67
SLIDE 67

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work Input-constrained BEC(ǫ) Input-constrained BSC(p)

Plots: (1, ∞)-RLL Input-Constrained BSC(p)

Figure: Comparison of our lower bound with the feedback capacity and dual-capacity upper bounds.

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-68
SLIDE 68

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Future Work

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-69
SLIDE 69

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Future Work

Deriving efficient coding schemes that meet the lower bounds.

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-70
SLIDE 70

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Future Work

Deriving efficient coding schemes that meet the lower bounds. Improving our lower bound by estimating delayed forward directed information at the input distribution that achieves

  • ur lower bound.
  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-71
SLIDE 71

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Future Work

Deriving efficient coding schemes that meet the lower bounds. Improving our lower bound by estimating delayed forward directed information at the input distribution that achieves

  • ur lower bound.

Extending our lower bound to 2D input-constrained DMCs.

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs

slide-72
SLIDE 72

Background Ideas and Directions Single-Letter Lower Bound Applications Future Work

Questions?

  • V. Arvind Rameshwar, Navin Kashyap

Computable Lower Bounds for Capacities of Input-Driven FSCs