An Explicit Formula for the Zero-Error Feedback Capacity of a Class - - PowerPoint PPT Presentation

An Explicit Formula for the Zero-Error Feedback Capacity of a Class of Finite-State Additive Noise Channels

Amir Saberi *, Farhad Farokhi and Girish Nair

EEE Department, The University of Melbourne Email: a.saberi @ student.unimelb.edu.au ISIT 2020

June 5, 2020

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 1 / 17

Discrete finite-state additive noise channel

Xi ⊕ Zi Yi = Xi ⊕ Zi

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 2 / 17

Discrete finite-state additive noise channel

Xi ⊕ Zi Yi = Xi ⊕ Zi Zi ⊥ X1:i ⊕ is mod q addition Xi, Yi, Zi ∈ X := {0, 1, . . . , q − 1}

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 2 / 17

Discrete finite-state additive noise channel

Xi ⊕ Zi Yi = Xi ⊕ Zi Aljajy’95: C =Cf = log q − H(Z) H(Z) : The noise process entropy rate

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 2 / 17

Outline

1

Channel model and definitions

2

Results on zero-error capacities

3

Examples

4

Sketch of the proofs

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 3 / 17

Finite-state machine

A directed graph G = (S, E) with vertex set S = {0, 1, . . . , |S| − 1} denoting states edge set E ⊆ S × S denoting possible transitions between two states.

Si = 0 Si = 1

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 4 / 17

For a finite-state machine

Topological entropy

h := lim

N→∞

1 N log

• set of all state seq. of length N
• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 5 / 17

For a finite-state machine

Topological entropy

h := lim

N→∞

1 N log

• set of all state seq. of length N
• h = log λP , (λP : Perron value)
• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 5 / 17

For a finite-state machine

Topological entropy

h := lim

N→∞

1 N log

• set of all state seq. of length N
• h = log λP , (λP : Perron value)

Si = 0 Si = 1

⇒ A =

• 1

1 1

• ⇒

λP = max |λ(A)| = 1 + √ 5 2

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 5 / 17

Finite-state additive noise channel

Yi = Xi ⊕ Zi, i ∈ N, Xi, Yi, Zi ∈ X := {0, 1, . . . , q − 1}, ⊕ is mod q addition; correlated additive noise Zi is governed by a finite-state machine; each outgoing edge from a state si corresponds to different values zi

• f the noise.

An example channel at which no consecutive errors are allowed:

Si = 0 Si = 1

Zi = 0 Zi = 1 Zi = 0

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 6 / 17

Encoder Channel Decoder M Xi Yi ˆ M

Zero-error capacity

C0 := sup

n∈N, F∈F

log |F| n F ⊆ X n is the set of all n−block codes yielding zero decoding errors; for any channel noise sequence and channel initial state; no state information is available at the encoder and decoder.

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 7 / 17

Encoder Channel Decoder Delay M Xi Yi ˆ M

Zero-error feedback capacity

C0f := sup

n∈N, F1:n∈FM

log |M| n The zero-error capacity of the channel having a feedback from the output. xi(m) = fm,i(y1:i−1), i = 1, . . . , n, m ∈ M F1:n = {fm,1:n|m ∈ M}, where fm,i is the encoding function.

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 8 / 17

For a finite-state additive noise channel

Theorem

The zero-error feedback capacity is either zero or C0f = log q − h(Z) Moreover, C0 ≥ log q − 2h(Z)

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 9 / 17

For a finite-state additive noise channel

Theorem

The zero-error feedback capacity is either zero or C0f = log q − h(Z) Moreover, C0 ≥ log q − 2h(Z)

Theorem

C0 = 0 ⇐ ⇒ C0f = 0.

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 9 / 17

Examples

Noise finite-state machine Capacity (q = 2) Capacity (q > 2)

Si = 0 Si = 1

1 C0 = C0f = 0 C0f = log q

φ,

C0 ≥ log q

φ2

Si = 0 Si = 1 Si = 2

1 C0f = 0.594, C0 ≥ 0.189 C0f = log

q 1.325,

C0 ≥ log

q 1.755

φ := 1+

√ 5 2

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 10 / 17

Examples

1 2 3 4 1 2 3 4

S = 0

1 2 3 4 1 2 3 4

S = 1 1 − p p 1

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 11 / 17

Examples

1 2 3 4 1 2 3 4

S = 0

1 2 3 4 1 2 3 4

S = 1 1 − p p 1

¯ Si = 0 ¯ Si = 1

Zi = 0 Zi = 1 Zi = 0

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 11 / 17

Examples

1 2 3 4 1 2 3 4

S = 0

1 2 3 4 1 2 3 4

S = 1 1 − p p 1

¯ Si = 0 ¯ Si = 1

Zi = 0 Zi = 1 Zi = 0 log 5 φ2 ≤ C0 ≤ C0f = log 5 φ

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 11 / 17

Sketch of the proof

|Y (k)| =

• {y1:k|M = m}
• =
• set of all state seq. length k
• ∝ λk

P

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 12 / 17

Sketch of the proof

|Y (k)| =

• {y1:k|M = m}
• =
• set of all state seq. length k
• ∝ λk

P

Converse: # of distinguisable messages ≤ qk/λk

P

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 12 / 17

Sketch of the proof

|Y (k)| =

• {y1:k|M = m}
• =
• set of all state seq. length k
• ∝ λk

P

Achievability: Parity check symbols: which output (from {1, . . . , |Y (k)|} ) was received.

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 12 / 17

Sketch of the proof

|Y (k)| =

• {y1:k|M = m}
• =
• set of all state seq. length k
• ∝ λk

P

Achievability: Parity check symbols: which output (from {1, . . . , |Y (k)|} ) was received. C0f − δ = log |Y (k)| n − k ≈ kh n − k

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 12 / 17

Sketch of the proof

C0f ≥ k n log q C0f − δ C0f − δ + h log q

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 13 / 17

Sketch of the proof

C0f ≥ k n log q C0f − δ C0f − δ + h log q ⇒ C0f ≥ log q − log λ − δ

• 1 −

1 C0f

• log q −

constant

n

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 13 / 17

Theorem

C0 = C0f = 0 ⇐ ⇒ ∀ d1:n ∈ X n, n ∈ N, ∃ a walk on the coupled graph with the label sequence d1:n.

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 14 / 17

Theorem

C0 = C0f = 0 ⇐ ⇒ ∀ d1:n ∈ X n, n ∈ N, ∃ a walk on the coupled graph with the label sequence d1:n.

Coupled graph

Gc = G × G (tensor product);

◮ Vertex set: V = S × S; ◮ Vertices (i, j) (k, m) iff edges from i k and j m;

Each edge label is Eik ⊖ Ejm.

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 14 / 17

Theorem

C0 = C0f = 0 ⇐ ⇒ ∀ d1:n ∈ X n, n ∈ N, ∃ a walk on the coupled graph with the label sequence d1:n.

Coupled graph

Gc = G × G (tensor product);

◮ Vertex set: V = S × S; ◮ Vertices (i, j) (k, m) iff edges from i k and j m;

Each edge label is Eik ⊖ Ejm.

Si = 0 Si = 1

1

(0, 0) (1, 1) (1, 0) (0, 1)

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 14 / 17

Theorem

C0 = C0f = 0 ⇐ ⇒ ∀ d1:n ∈ X n, n ∈ N, ∃ a walk on the coupled graph with the label sequence d1:n. y1:n = fm,1:n(z1:n−1) ⊕ z1:n 1 1 1 1y′

1:n = fm′,1:n(z′ 1:n−1) ⊕ z′ 1:n (0, 0) (1, 1) (1, 0) (0, 1)

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 15 / 17

Theorem

C0 = C0f = 0 ⇐ ⇒ ∀ d1:n ∈ X n, n ∈ N, ∃ a walk on the coupled graph with the label sequence d1:n. y1:n = fm,1:n(z1:n−1) ⊕ z1:n 1 1 1 1y′

1:n = fm′,1:n(z′ 1:n−1) ⊕ z′ 1:n (0, 0) (1, 1) (1, 0) (0, 1)

d1:n := fm′,1:n(z′

1:n−1) ⊖ fm,1:n(z1:n−1)

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 15 / 17

Theorem

C0 = C0f = 0 ⇐ ⇒ ∀ d1:n ∈ X n, n ∈ N, ∃ a walk on the coupled graph with the label sequence d1:n. y1:n = fm,1:n(z1:n−1) ⊕ z1:n 1 1 1 1y′

1:n = fm′,1:n(z′ 1:n−1) ⊕ z′ 1:n (0, 0) (1, 1) (1, 0) (0, 1)

d1:n := fm′,1:n(z′

1:n−1) ⊖ fm,1:n(z1:n−1)

. . . . . . . . . . . .d1:n = z1:n ⊖ z′

1:n ⇐

⇒ y1:n = y′

1:n

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 15 / 17

Summary

For a class of additive noise channels and based on topological properties

A formula for computing C0f; A lower bound on C0; Conditions on C0 = C0f = 0; Revealing a close connection between the topological entropy of the underlying noise process and the zero-error communication.

Future work

Extending the results to a more general class of channels; Finitely checkable topological condition on C0 = C0f = 0. E: a.saberi @ student.unimelb.edu.au

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 16 / 17

See you next year in Melbourne!

• A. Saberi (U of Melbourne)

C0f = 1 − h(Z) June 5, 2020 17 / 17