Zero-Error Coding with a Generator Set of Variable-Length Words - PowerPoint PPT Presentation

Zero-Error Coding with a Generator Set of Variable-Length Words Nicolas Charpenay, Maël le Treust 2020 IEEE International Symposium on Information Theory Nicolas Charpenay, Maël le Treust Zero-Error Coding with a Generator Set of Variable-Length Words 1 / 25

Introduction Zero-error information theory (channel coding) ⬩ Transmission over a discrete memoryless channel ⬩ Correct decoding of each codeword with probability 1 ▶ Zero error capacity : Maximal asymptotic feasible rate with these constraints ? State of the art ⬩ Shannon (1956) defined zero-error capacity and built the first tools ⬩ Lovasz (1979) built the θ function (bound on zero-error capacity) ⬩ Answers for specific cases (channels with 4 or less inputs, bipartite channel graphs, etc...) ▶ For an arbitrary DMC, determining the zero-error capacity is a wide open problem Nicolas Charpenay, Maël le Treust Zero-Error Coding with a Generator Set of Variable-Length Words 2 / 25

Zero-error information theory Definition [channel graph] Given a discrete memoryless channel with transition probabilities W = ( p Y y ( x )) x ∈ X , y ∈ Y , we define the channel graph : G W ≐ ( V ( G W ) , E ( G W )) V ( G W ) = X with and xx ′ ∈ E ( G W ) iff ∃ y ∈ Y , W x , y , W x ′ , y > 0 (1) ▶ Examples : If ǫ ∈ ( 0 , 1 ) then 1 − ǫ 2 2 2 A A ǫ ǫ / 2 both have the ǫ 1 and 1 B 1 channel graph 1 − ǫ 1 − ǫ ǫ / 2 0 B 0 C 0 X Y X Y Nicolas Charpenay, Maël le Treust Zero-Error Coding with a Generator Set of Variable-Length Words 3 / 25

Zero-error information theory Definition [independent set] Consider a graph G = ( V , E ) : ⬩ An independent set S of G is a subset of V verifying : ∀ s , s ′ ∈ Q , ss ′ ∉ E . ⬩ The independence number : α ( G ) ≐ S independent set of G # S max (2) ▶ Interpretation : α ( G W ) is the maximum zero-error rate for one use of the channel W . Nicolas Charpenay, Maël le Treust Zero-Error Coding with a Generator Set of Variable-Length Words 4 / 25

Zero-error information theory Definition [strong product ⊠ ] Let G = ( V , E ) and G ′ = ( V ′ , E ′ ) , we define : G ⊠ G ′ ≐ ( V ( G ⊠ G ′ ) , E ( G ⊠ G ′ )) V ( G ⊠ G ′ ) = V × V ′ where v ′ 1 v ′ 2 ∈ E ′ ( v 1 , v ′ 1 )( v 2 , v ′ 2 ) ∈ E ( G ⊠ G ′ ) iff ( v 1 v 2 ∈ E ) AND ( ) and or v ′ 1 = v ′ or v 1 = v 2 2 (3) 2 , then G ⊠ G ′ is the king’s ▶ Example : Take G = G ′ = 0 1 graph : 0,2 1,2 2,2 ( Here it gives the channel graph for 2 channel uses. ) 0,1 1,1 2,1 0,0 1,0 2,0 Nicolas Charpenay, Maël le Treust Zero-Error Coding with a Generator Set of Variable-Length Words 5 / 25

Zero-error information theory ▶ Interpretation : G ⊠ T ����������������������������������������������������������������������������������������� = G W ⊠ ... ⊠ G W is the channel graph for T channel uses . W T times Thus the following are equivalent : ⬩ xx ′ ∈ E ( G ⊠ T W ) ⬩ ( x t ) t ≤ T and ( x ′ t ) t ≤ T lead to the same output sequence with positive probability ⬩ For all t ≤ T , there exists y t such that W x t , y t > 0 and W x ′ t , y t > 0 Nicolas Charpenay, Maël le Treust Zero-Error Coding with a Generator Set of Variable-Length Words 6 / 25

Zero-error capacity Definition (cf. Shannon [4]) Given a channel W we define its zero-error capacity : C 0 ( W ) = lim T log ( α ( G ⊠ T W )) T log ( α ( G ⊠ T W )) 1 1 [ Fekete ] sup (4) = T → ∞ T ∈ N Fekete’s Lemma ▶ For all subadditive sequence u (i.e. : u n + m ≤ u n + u m for all m , n ), u n u n lim n exists and is equal to inf n (it can be −∞ ). n → ∞ n ▶ Remark : Note that log ( α ( G ⊠ T W )) is superadditive. Idea : independent sets generate a product independent set in the product graph. ▶ Interpretation : C 0 ( W ) is the asymptotic maximum zero-error rate : limit of the rates of ( S t ) t ∈ N , where S t is a max independent set of G ⊠ T for all t . W Nicolas Charpenay, Maël le Treust Zero-Error Coding with a Generator Set of Variable-Length Words 7 / 25

Our main example ▶ Generated by a 5-symbol noisy typewriter union a 1-symbol perfect channel. 5 5 2 3 4 4 has the channel graph C 5 ⊞ 1 : 0 1 3 3 4 5 √ 2 2 ▶ Known zero-error capacity : log ( 5 + 1 ) 1 1 √ ( consequence of Shannon’s theorem for ⊞ 5 ) and Lovasz’ theorem : C 0 ( C 5 ) = log 0 0 Y X Nicolas Charpenay, Maël le Treust Zero-Error Coding with a Generator Set of Variable-Length Words 8 / 25

Codes with variable length Set of words and slicing For a given finite set S , we define the set of words over S : with S 0 = { ǫ } ( ǫ denotes the empty word ) S ∗ ≐ ⋃ S L (5) L ∈ N For w ∈ C , ∣ w ∣ is the length of w . Given S ′ ⊆ S ∗ , let : [ L ] ≐ { w ∈ S ′ ∣ ∣ w ∣ = L } S ′ (6) Nicolas Charpenay, Maël le Treust Zero-Error Coding with a Generator Set of Variable-Length Words 9 / 25

Example of words concatenation Take C = { 0 } ∪ { 11 , 23 , 35 , 42 , 54 } ( ⊆ � 0 , 5 � ∗ ) : Possible words (concatenation) : ⋮ ⋮ ⋮ # C ∗ [ 3 ] = 11 540 420 350 230 110 054 042 035 023 011 000 # C ∗ [ 2 ] = 6 54 42 35 23 11 00 # C ∗ [ 1 ] = 1 0 # C ∗ [ 0 ] = 1 ǫ Nicolas Charpenay, Maël le Treust Zero-Error Coding with a Generator Set of Variable-Length Words 10 / 25

First contribution Nicolas Charpenay, Maël le Treust Zero-Error Coding with a Generator Set of Variable-Length Words 11 / 25

Codes with variable length Definition [variable length code] ⬩ A variable length code for a channel ( W x , y ) x ∈ X , y ∈ Y is a finite subset C of X ∗ (called generator set ). ⬩ C is zero-error over W if it verifies : ∀ c , c ′ ∈ C such that c ≠ c ′ and ⊠ ∣ c ∣ ∣ c ∣ ≤ ∣ c ′ ∣ , cc ′ ∶ ∣ c ∣ ∉ E ( G ) with auto-adjacency convention. W Definition [variable length code rate] Let C be a zero-error variable length code over W , we define its rate : r ( C ) ≐ lim 1 L log # C ∗ [ L ] (7) L → ∞ Then we define the average number of distinguishable inputs : √ ν ( C ) ≐ lim [ L ] = 2 r ( C ) # C ∗ L (8) L → ∞ Nicolas Charpenay, Maël le Treust Zero-Error Coding with a Generator Set of Variable-Length Words 12 / 25

Codes with variable length Proposition C 0 ( W ) = sup C r ( C ) ▶ Interpretation : Considering variable-length coding is considering families of fixed-length codes → same supremum rate. Nicolas Charpenay, Maël le Treust Zero-Error Coding with a Generator Set of Variable-Length Words 13 / 25

Codes with variable length Theorem [rate computation] Consider a zero-error code with : ⬩ Generator set C nonempty ⬩ Max (resp. min) reached length l (resp. l ) ⬩ r ( C ) = log ν ( C ) > 0 its zero-error rate Then ν ( C ) is the unique positive solution of X l = ∑ l i = l # C [ i ] X l − i , where C [ i ] = { c ∈ C ∣ ∣ c ∣ = i } . ▶ Remark : An equivalent formulation is the linear recursive sequence for # C ∗ [ i ] : l # C ∗ [ L ] = ∑ # C [ l ] # C ∗ [ L − l ] (9) l = l The rate is the maximum eigenvalue of the transition matrix. Nicolas Charpenay, Maël le Treust Zero-Error Coding with a Generator Set of Variable-Length Words 14 / 25

Codes with variable length ▶ Examples : Consider the channel C 5 ⊞ 1 , and two possible zero-error variable-length codes C ≐ { 0 , 11 , 23 , 35 , 42 , 54 } , C ′ ≐ { 11 , 23 , 35 , 42 , 54 , 001 , 003 } . Here are the rates of some generated fixed-length codes : [ 3 ] = 11; rate = log ( 11 )/ 3 ≃ 1 . 153 # C ∗ L = 3 ∶ [ 4 ] = 41; rate = log ( 41 )/ 4 ≃ 1 . 339 # C ∗ L = 4 ∶ [ 5 ] = 96; rate = log ( 96 )/ 5 ≃ 1 . 317 1 # C ∗ L = 5 ∶ 2 [ 3 ] = 2; rate = log ( 2 )/ 3 ≃ 0 . 333 0 5 # C ′∗ L = 3 ∶ [ 4 ] = 25; rate = log ( 25 )/ 4 ≃ 1 . 161 3 # C ′∗ L = 4 ∶ [ 5 ] = 20; rate = log ( 20 )/ 5 ≃ 1 . 864 4 # C ′∗ L = 5 ∶ C = { 0 , 11 , 23 , 35 , 42 , 54 } C ′ = { 11 , 23 , 35 , 42 , 54 , 001 , 003 } Generator sets : X 2 = X + 5 X 3 = 5 X + 2 √ Polynomial equation : √ ν ( C ) = 1 + ν ( C ′ ) = 1 + 21 √ Value of ν : 2 √ log ( 1 + ) ≃ 1 . 481 log ( 1 + 2 ) ≃ 1 . 272 2 21 √ Asymptotic rate : These rates are inferior to the known capacity log ( 1 + 2 5 ) ≃ 1 . 694 Nicolas Charpenay, Maël le Treust Zero-Error Coding with a Generator Set of Variable-Length Words 15 / 25

Codes with variable length Nicolas Charpenay, Maël le Treust Zero-Error Coding with a Generator Set of Variable-Length Words 16 / 25

Codes with variable length ▶ f 1 , f 2 , f 3 obtained by studying secondary eigenvalues. Nicolas Charpenay, Maël le Treust Zero-Error Coding with a Generator Set of Variable-Length Words 17 / 25

Second contribution Nicolas Charpenay, Maël le Treust Zero-Error Coding with a Generator Set of Variable-Length Words 18 / 25