What are the best codes? def A q ( n , d ) = the largest # of vectors of length n over an alphabet with q letters so that any two of them are distance d apart def V q ( n , d ) = volume of the Hamming sphere of radius d in the space of n -tuples over an alphabet with q letters
What are the best codes? def A q ( n , d ) = the largest # of vectors of length n over an alphabet with q letters so that any two of them are distance d apart def V q ( n , d ) = volume of the Hamming sphere of radius d in the space of n -tuples over an alphabet with q letters Theorem (Gilbert-Varshamov bound) q n q n A q ( n , d ) � = V q ( n , d − 1 ) � n � d − 1 ∑ ( q − 1 ) i i i = 0 E.N. Gilbert , A comparison of signaling alphabets, Bell Systems Technical Journal , October 1952.
Proof of the GV bound Greedy construction algorithm. Take an arbitrary vector from the space, adjoin it to the code being constructed, and remove from the space the Hamming sphere or radius d − 1 around it. Repeat. d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 · · · · · · d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 q n q n q n d 1 d 1 d 1 If after M steps there is nothing left, then the spheres of radius d − 1 about the M codewords cover the space, so M V q ( n , d − 1 ) � q n .
Proof of the GV bound Greedy construction algorithm. Take an arbitrary vector from the space, adjoin it to the code being constructed, and remove from the space the Hamming sphere or radius d − 1 around it. Repeat. d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 · · · · · · d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 d 1 q n q n q n d 1 d 1 d 1 If after M steps there is nothing left, then the spheres of radius d − 1 about the M codewords cover the space, so M V q ( n , d − 1 ) � q n . Open problem: Can we do better asymptotically?
Asymptotic improvements of the GV bound R 1 0.9 Tsfasman−Vladuts−Zink Improving on the Gilbert-Var- 0.8 shamov bound asymptotically 0.7 is a notoriously difficult task! Xing / Elkies 0.6 0.5 0.4 0.3 Gilbert−Varshamov 0.2 0.1 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 δ M.A. Tsfasman, S.G. Vlˇ adu¸ t, and T. Zink , Modular curves, Shimura curves, and Goppa codes better than the Varshamov-Gilbert bound, Math. Nachrichten , 104 , (1982), 13–28. C. Xing , Nonlinear codes from algebraic curves improving the Tsfasman-Vlˇ adu¸ t-Zink bound, IEEE Transactions on Information Theory , 49 , (2003), 1653–1657. N. Elkies , Still better codes from modular curves, preprint arXiv: math.NT/0308046 , 2003.
Asymptotic improvements of the GV bound R 1 0.9 Tsfasman−Vladuts−Zink Improving on the Gilbert-Var- 0.8 shamov bound asymptotically 0.7 is a notoriously difficult task! Xing / Elkies 0.6 0.5 0.4 For q � 45 , no asymptotic 0.3 Gilbert−Varshamov improvements of the GV 0.2 bound are yet known... 0.1 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 δ M.A. Tsfasman, S.G. Vlˇ adu¸ t, and T. Zink , Modular curves, Shimura curves, and Goppa codes better than the Varshamov-Gilbert bound, Math. Nachrichten , 104 , (1982), 13–28. C. Xing , Nonlinear codes from algebraic curves improving the Tsfasman-Vlˇ adu¸ t-Zink bound, IEEE Transactions on Information Theory , 49 , (2003), 1653–1657. N. Elkies , Still better codes from modular curves, preprint arXiv: math.NT/0308046 , 2003.
The GV bound for binary codes For binary codes, the Gilbert-Varshamov bound takes the simple form: 2 n A 2 ( n , d ) � f GV ( n , d ) def = V 2 ( n , d − 1 ) Since log 2 V 2 ( n , d ) � H ( d / n ) , this implies that for all n and all d � n / 2, there exist binary codes of rate R 2 ( n , d ) � 1 − H ( d / n ) . A well-known conjecture The Gilbert-Varshamov bound is asympto- tically exact for binary codes. V.D. Goppa , Bounds for codes, Doklady Academii Nauk , 1993, reports that he has a “wonderful proof” of this conjecture.
The GV bound for binary codes For binary codes, the Gilbert-Varshamov bound takes the simple form: 2 n A 2 ( n , d ) � f GV ( n , d ) def = V 2 ( n , d − 1 ) Since log 2 V 2 ( n , d ) � H ( d / n ) , this implies that for all n and all d � n / 2, there exist binary codes of rate R 2 ( n , d ) � 1 − H ( d / n ) . A well-known conjecture The Gilbert-Varshamov bound is asympto- tically exact for binary codes. V.D. Goppa , Bounds for codes, Doklady Academii Nauk , 1993, reports that he has a “wonderful proof” of this conjecture. Open problem: prove or disprove this conjecture!
Recent improvement of GV bound Theorem (Asymptotic improvement of Gilbert-Varshamov bound) Given positive integers n and d , with d � n , let e ( n , d ) denote the fol- lowing quantity: min { w , i } � n �� w �� n − w � � n � d d d 1 − 1 ∑ ∑ ∑ ∑ e ( n , d ) def = i − j 6 w j 6 w w = 1 w = 1 i = 1 j = ⌈ w + i − d ⌉ 2 Then: � 2 n · log 2 V ( n , d − 1 ) − log 2 e ( n , d − 1 ) A 2 ( n , d ) � V ( n , d − 1 ) 10 � �� � � �� � f GV ( n , d ) improvement over the GV bound by a factor linear in n T. Jiang and A. Vardy , Asymptotic improvement of the Gilbert-Varshamov bound on the size of binary codes, IEEE Trans. Inform. Theory , July 2004.
Proof of the new bound Definition (Gilbert graph) The Gilbert graph G G is defined as follows: V ( G G ) = all binary vectors of length n , and { x , y } ∈ E ( G G ) ⇐ ⇒ d ( x , y ) � d − 1
Proof of the new bound Definition (Gilbert graph) The Gilbert graph G G is defined as follows: V ( G G ) = all binary vectors of length n , and { x , y } ∈ E ( G G ) ⇐ ⇒ d ( x , y ) � d − 1 Then A 2 ( n , d ) is simply the independence number α ( G G ) of this graph.
Proof of the new bound Definition (Gilbert graph) The Gilbert graph G G is defined as follows: V ( G G ) = all binary vectors of length n , and { x , y } ∈ E ( G G ) ⇐ ⇒ d ( x , y ) � d − 1 Then A 2 ( n , d ) is simply the independence number α ( G G ) of this graph. Theorem (Generalization of Ajtai, Komlós, and Szemerédi bound) For any ∆ -regular graph with with at most T triangles, we have � � � � α ( G ) � | V ( G ) | T / log 2 ∆ − 1 2 log 2 10 ∆ | V ( G ) |
Proof of the new bound Definition (Gilbert graph) The Gilbert graph G G is defined as follows: V ( G G ) = all binary vectors of length n , and { x , y } ∈ E ( G G ) ⇐ ⇒ d ( x , y ) � d − 1 Then A 2 ( n , d ) is simply the independence number α ( G G ) of this graph. Theorem (Generalization of Ajtai, Komlós, and Szemerédi bound) For any ∆ -regular graph with with at most T triangles, we have � � � � α ( G ) � | V ( G ) | T / log 2 ∆ − 1 2 log 2 10 ∆ | V ( G ) | It remains to count the number of triangles in the Gilbert graph G G . This number is precisely the e ( n , d ) in the previous theorem.
New bound versus old conjecture How does this relate to the famous conjecture that the Gilbert- Varshamov bound is asymptotically exact for binary codes? Conjecture A. The Gilbert-Varshamov bound on the size of binary codes, namely A 2 ( n , d ) , is asymptotically exact. That is A 2 ( n , d ) f GV ( n , d ) = const lim n → ∞ Our result implies that this is certainly false . The limit does not exist. In fact, we prove that: log 2 A 2 ( n , d ) � log 2 f GV ( n , d ) + log n + const Conjecture B. The Gilbert-Varshamov bound on the rate of binary codes R 2 ( n , d ) = log 2 A 2 ( n , d ) / n is asymptotically exact. This could still be true!
New bound versus old conjecture How does this relate to the famous conjecture that the Gilbert- Varshamov bound is asymptotically exact for binary codes? Conjecture A. The Gilbert-Varshamov bound on the size of binary codes, namely A 2 ( n , d ) , is asymptotically exact. That is A 2 ( n , d ) f GV ( n , d ) = const lim n → ∞ Our result implies that this is certainly false . The limit does not exist. In fact, we prove that: log 2 A 2 ( n , d ) � log 2 f GV ( n , d ) + log n + const Conjecture B. The Gilbert-Varshamov bound on the rate of binary codes R 2 ( n , d ) = log 2 A 2 ( n , d ) / n is asymptotically exact. This could still be true!
New bound versus old conjecture How does this relate to the famous conjecture that the Gilbert- Varshamov bound is asymptotically exact for binary codes? Conjecture A. The Gilbert-Varshamov bound on the size of binary codes, namely A 2 ( n , d ) , is asymptotically exact. That is A 2 ( n , d ) f GV ( n , d ) = const lim n → ∞ Our result implies that this is certainly false . The limit does not exist. In fact, we prove that: log 2 A 2 ( n , d ) � log 2 f GV ( n , d ) + log n + const Conjecture B. The Gilbert-Varshamov bound on the rate of binary codes R 2 ( n , d ) = log 2 A 2 ( n , d ) / n is asymptotically exact. This could still be true!
Hamming bound and perfect codes 2 n 2 n � A 2 ( n , 2 e + 1 ) � V 2 ( n , 2 e ) V 2 ( n , e ) � �� � � �� � Hamming GV bound bound Definition (Perfect codes) Codes that attain the Hamming bound with equality are called perfect. What perfect binary codes are out there? Trivial codes: the whole space, any single codeword, the ( n , 1, n ) repetition code for all odd n Nontrivial codes: the ( n , n − m , 3 ) Hamming codes for n = 2 m − 1 and nonlinear perfect codes with the same parameters, the unique ( 23, 12, 7 ) binary Golay code
Hamming bound and perfect codes Definition (Perfect codes) Codes that attain the Hamming bound with equality are called perfect. What perfect binary codes are out there? Trivial codes: the whole space, any single codeword, the ( n , 1, n ) repetition code for all odd n Nontrivial codes: the ( n , n − m , 3 ) Hamming codes for n = 2 m − 1 and nonlinear perfect codes with the same parameters, the unique ( 23, 12, 7 ) binary Golay code Theorem (Complete characterization of perfect binary codes) There are no more perfect binary codes! — Van Lint, Tietäväinen, Zinoviev, and others, 1974
Perfect codes in other metric spaces? Instead of the Hamming space F n 2 consider the Johnson space J ( n , w ) of all binary vectors of length n and constant weight w . The sphere-pack- ing bound in this space is given by: � n � | J ( n , w ) | w A 2 ( n , 4 e + 2, w ) � = � w �� n − w � e ∑ V 2 ( n , e , w ) i i i = 0 Definition (Perfect codes in the Johnson scheme) Codes in J ( n , w ) that attain this bound with equality are called perfect. What perfect codes are out there? Trivial codes: the whole space, any single codeword, any pair of disjoint codewords for n = 2 w , with w odd. Nontrivial codes: Are there any?
The Delsarte Conjecture After having recalled that there are “very few” perfect codes in the Hamming schemes, one must say that there is not a single one known in the Johnson schemes. It is temp- ting to risk the conjecture that such codes do not exist. Philippe Delsarte , An algebraic approach to association schemes and coding theory, Philips Journal Research , October 1973. Open problem: prove or disprove this conjecture!
The Delsarte Conjecture After having recalled that there are “very few” perfect codes in the Hamming schemes, one must say that there is not a single one known in the Johnson schemes. It is temp- ting to risk the conjecture that such codes do not exist. Philippe Delsarte , An algebraic approach to association schemes and coding theory, Philips Journal Research , October 1973. Open problem: prove or disprove this conjecture! Biggs (1973), Bannai (1977), Hammond (1982) Roos (1983), Martin (1992), Etzion (1996) Ahlswede, Aydinian, and Khachatrian (2001) Etzion (2001), Etzion and Schwartz (2004) Shimabukuro (2005)
Recent result on the Delsarte conjecture Theorem If there exists a prime p such that e ≡ − 1 ( mod p 2 ) , for example if e = 3, 7, 8, 11, 15, 17, 19 . . . , then there can be only finitely many non- trivial e -perfect codes in J ( n , w ) . In particular, there are no nontrivial 3 -perfect, 7 -perfect, or 8 -perfect codes in J ( n , w ) for all n and w . T. Etzion and M. Schwartz , Perfect constant-weight codes, IEEE Trans. Information Theory , September 2004. Proof. Follows by showing that if there is an e -perfect code in J ( n , w ) , then the polynomial: � e − i � X + 1 � w − i �� n − w + i − X − 1 � e ∑ ∑ P n , w ( X ) def � − 1 � i = i + j i j i = 0 j = 0 must have integer zeros ϕ in the range e < ϕ < w . Etzion and Schwartz conjecture that P n , w ( X ) does not have any integer zeros if e > 2.
Singleton bound and MDS codes List of all the codewords · · · } d − 1 A q ( n , d ) � q n − d + 1 � n − d + 1 all these columns are distinct Definition (MDS codes) Codes that attain the Singleton bound with equality are called max- imum distance separable or simply MDS.
Singleton bound and MDS codes List of all the codewords · · · } d − 1 A q ( n , d ) � q n − d + 1 � n − d + 1 all these columns are distinct Definition (MDS codes) Codes that attain the Singleton bound with equality are called max- imum distance separable or simply MDS. What kind of MDS codes are out there? A cyclic code of prime length p over GF ( q ) is MDS for almost all q A random code over GF ( q ) is MDS with probability → 1 as q → ∞ Reed-Solomon codes and generalized Reed-Solomon codes
Some properties of MDS codes MDS codes have many beautiful and useful properties: Any k positions form an inform- ation set (linearly independent) Any d positions support a code- word of minimum weight The weight distribution of MDS codes is completely determined Trellis structure of MDS codes is also completely determined But what about their length? The MDS conjecture If C is an ( n , k , d ) MDS code with 1 < k < n − 1, then n � q + 1 with two exceptions: the ( q + 2, 3, q ) code and its dual, if q = 2 m .
Some properties of MDS codes MDS codes have many beautiful and useful properties: Any k positions form an inform- ation set (linearly independent) Any d positions support a code- word of minimum weight The weight distribution of MDS codes is completely determined Trellis structure of MDS codes is also completely determined But what about their length? The MDS conjecture If C is an ( n , k , d ) MDS code with 1 < k < n − 1, then n � q + 1 with two exceptions: the ( q + 2, 3, q ) code and its dual, if q = 2 m .
Some properties of MDS codes MDS codes have many beautiful and useful properties: Any k positions form an inform- ation set (linearly independent) Any d positions support a code- word of minimum weight The weight distribution of MDS codes is completely determined Trellis structure of MDS codes is also completely determined But what about their length? The MDS conjecture If C is an ( n , k , d ) MDS code with 1 < k < n − 1, then n � q + 1 with two exceptions: the ( q + 2, 3, q ) code and its dual, if q = 2 m .
The MDS Conjecture The MDS conjecture If C is an ( n , k , d ) MDS code with 1 < k < n − 1, then n � q + 1 with two exceptions: the ( q + 2, 3, q ) code and its dual, if q = 2 m . Equivalent conjecture: linear algebra Let V be a vector space over F q and let S be a set of vectors of V such that any k of them form a basis for V . Then |S| � q + 1. Equivalent conjecture: matrix theory Let M be a k × m matrix over F q such that every square subma- trix of M is nonsingular. Then m + k � q + 1. Equivalent conjecture: orthogonal arrays Let OA be an orthogonal array with q levels, strength k , and in-
The MDS Conjecture The MDS conjecture If C is an ( n , k , d ) MDS code with 1 < k < n − 1, then n � q + 1 with two exceptions: the ( q + 2, 3, q ) code and its dual, if q = 2 m . Equivalent conjecture: linear algebra Let V be a vector space over F q and let S be a set of vectors of V such that any k of them form a basis for V . Then |S| � q + 1. Equivalent conjecture: matrix theory Let M be a k × m matrix over F q such that every square subma- trix of M is nonsingular. Then m + k � q + 1. Equivalent conjecture: orthogonal arrays Let OA be an orthogonal array with q levels, strength k , and in-
The MDS Conjecture The MDS conjecture If C is an ( n , k , d ) MDS code with 1 < k < n − 1, then n � q + 1 with two exceptions: the ( q + 2, 3, q ) code and its dual, if q = 2 m . Equivalent conjecture: linear algebra Let V be a vector space over F q and let S be a set of vectors of V such that any k of them form a basis for V . Then |S| � q + 1. Equivalent conjecture: matrix theory Let M be a k × m matrix over F q such that every square subma- trix of M is nonsingular. Then m + k � q + 1. Equivalent conjecture: orthogonal arrays Let OA be an orthogonal array with q levels, strength k , and in-
The MDS Conjecture The MDS conjecture If C is an ( n , k , d ) MDS code with 1 < k < n − 1, then n � q + 1 with two exceptions: the ( q + 2, 3, q ) code and its dual, if q = 2 m . Equivalent conjecture: matrix theory Let M be a k × m matrix over F q such that every square subma- trix of M is nonsingular. Then m + k � q + 1. Equivalent conjecture: orthogonal arrays Let OA be an orthogonal array with q levels, strength k , and in- dex one. Then the number of constraints in OA is at most q + 1. Equivalent conjecture: projective geometry Let A be an arc in the projective geometry PG ( k − 1, q ) . Then the
The MDS Conjecture The MDS conjecture If C is an ( n , k , d ) MDS code with 1 < k < n − 1, then n � q + 1 with two exceptions: the ( q + 2, 3, q ) code and its dual, if q = 2 m . Equivalent conjecture: orthogonal arrays Let OA be an orthogonal array with q levels, strength k , and in- dex one. Then the number of constraints in OA is at most q + 1. Equivalent conjecture: projective geometry Let A be an arc in the projective geometry PG ( k − 1, q ) . Then the number of points of A is at most q + 1.
The MDS Conjecture The MDS conjecture If C is an ( n , k , d ) MDS code with 1 < k < n − 1, then n � q + 1 with two exceptions: the ( q + 2, 3, q ) code and its dual, if q = 2 m . Equivalent conjecture: projective geometry Let A be an arc in the projective geometry PG ( k − 1, q ) . Then the number of points of A is at most q + 1. All this is known to be true if q � 27, if k � 5, if √ q > 4 k − 9 and q is odd, and in certain other cases. Segre, Singleton, Casse, Hirchfeld, Roth, many others, 1955 —
The MDS Conjecture The MDS conjecture If C is an ( n , k , d ) MDS code with 1 < k < n − 1, then n � q + 1 with two exceptions: the ( q + 2, 3, q ) code and its dual, if q = 2 m . Equivalent conjecture: projective geometry Let A be an arc in the projective geometry PG ( k − 1, q ) . Then the number of points of A is at most q + 1. All this is known to be true if q � 27, if k � 5, if √ q > 4 k − 9 and q is odd, and in certain other cases. Segre, Singleton, Casse, Hirchfeld, Roth, many others, 1955 — Open problem: prove or disprove any of these conjectures!
Information Theory and Applications (ITA) University of California San Diego
Information Theory and Applications (ITA) Center at UCSD will offer prizes for the solution of impor- tant problems in information theory: Information Theory and Applications (ITA) University of California San Diego
Information Theory and Applications (ITA) Center at UCSD will offer prizes for the solution of impor- tant problems in information theory: The asymptotic GV conjecture $1,000 The Delsarte conjecture $1,000 The MDS conjecture $1,000 details soon at http://ita.ucsd.edu Information Theory and Applications (ITA) University of California San Diego
Information Theory and Applications (ITA) Center at UCSD will offer prizes for the solution of impor- tant problems in information theory: The asymptotic GV conjecture $1,000 The Delsarte conjecture $1,000 The MDS conjecture $1,000 details soon at http://ita.ucsd.edu Do these problems have potential applications in practice? Information Theory and Applications (ITA) University of California San Diego
Information Theory and Applications (ITA) Center at UCSD will offer prizes for the solution of impor- tant problems in information theory: The asymptotic GV conjecture $1,000 The Delsarte conjecture $1,000 The MDS conjecture $1,000 details soon at http://ita.ucsd.edu Do these problems have potential applications in practice? There is much pleasure to be gained from useless knowledge! — Bertrand Russel, 1912 Information Theory and Applications (ITA) University of California San Diego
Recent Advances in Algebraic List-Decoding of Reed-Solomon Codes
The best algebraic codes Millions of error-correcting codes are decoded every minute , with efficient algorithms implemented in custom VLSI circuits. At least 75% of these VLSI circuits decode Reed-Solomon codes. I.S. Reed and G. Solomon , Polynomial codes over certain finite fields, Journal Society Indust. Appl. Math. 8 , pp. 300-304, June 1960.
Construction of Reed-Solomon codes We describe the code via its encoder mapping E : F k q �→ F n q . Fix in- tegers k � n � q and n distinct elements x 1 , x 2 , . . . x n ∈ F q . Then u 0 , u 1 , . . . , u k − 1 k information symbols ⇓ f u ( X ) = u 0 + u 1 X + · · · + u k − 1 X k − 1 ⇓ c 1 = f u ( x 1 ) , c 2 = f u ( x 2 ) , · · · , c n = f u ( x n ) ⇓ ( c 1 , c 2 , . . . , c n ) n codeword symbols Thus Reed-Solomon codes are linear. They have rate R = k / n and distance d = n − k + 1, which is the best possible (MDS).
Algebraic decoding of Reed-Solomon codes Every codeword of a Reed-Solomon code C q ( n , k ) consists of some n values of a polynomial f ( X ) of degree < k . This polynomial can be uniquely recovered by interpolation from any k of its values. Thus a Reed-Solomon code C q ( n , k ) can correct up to n − k erasures or, equivalently, up to ( n − k ) / 2 = ( d − 1 ) / 2 errors . The Berlekamp-Massey algorithm is a very efficient way of doing this. It has applications outside of coding theory as well.
Algebraic decoding of Reed-Solomon codes Every codeword of a Reed-Solomon code C q ( n , k ) consists of some n values of a polynomial f ( X ) of degree < k . This polynomial can be uniquely recovered by interpolation from any k of its values. Thus a Reed-Solomon code C q ( n , k ) can correct up to n − k erasures or, equivalently, up to ( n − k ) / 2 = ( d − 1 ) / 2 errors . The Berlekamp-Massey algorithm is a very efficient way of doing this. It has applications outside of coding theory as well.
Algebraic decoding of Reed-Solomon codes Every codeword of a Reed-Solomon code C q ( n , k ) consists of some n values of a polynomial f ( X ) of degree < k . This polynomial can be uniquely recovered by interpolation from any k of its values. Thus a Reed-Solomon code C q ( n , k ) can correct up to n − k erasures or, equivalently, up to ( n − k ) / 2 = ( d − 1 ) / 2 errors . b b b b b b n n−1 4 3 2 1 Error−locator polynomial The Berlekamp-Massey algorithm is a very efficient way of doing this. It has applications outside of coding theory as well.
Algebraic decoding of Reed-Solomon codes Every codeword of a Reed-Solomon code C q ( n , k ) consists of some n values of a polynomial f ( X ) of degree < k . This polynomial can be uniquely recovered by interpolation from any k of its values. Thus a Reed-Solomon code C q ( n , k ) can correct up to n − k erasures or, equivalently, up to ( n − k ) / 2 = ( d − 1 ) / 2 errors . b b b b b b n n−1 4 3 2 1 Error−locator polynomial The Berlekamp-Massey algorithm is a very efficient way of doing this. It has applications outside of coding theory as well. Clearly, this is the best possible.
Algebraic decoding of Reed-Solomon codes Every codeword of a Reed-Solomon code C q ( n , k ) consists of some n values of a polynomial f ( X ) of degree < k . This polynomial can be uniquely recovered by interpolation from any k of its values. Thus a Reed-Solomon code C q ( n , k ) can correct up to n − k erasures or, equivalently, up to ( n − k ) / 2 = ( d − 1 ) / 2 errors . b b b b b b n n−1 4 3 2 1 Error−locator polynomial The Berlekamp-Massey algorithm is a very efficient way of doing this. It has applications outside of coding theory as well. Clearly, this is the best possible. Or is it?
Correcting more errors than thought possible The 2002 Nevanlinna Prize went to M. Sudan with the citation “ ...in the theory of error-correcting codes, Sudan showed that certain coding methods could correct many more errors than was previously thought possible .” 1 0.9 Fraction of errors corrected Sudan 0.8 0.7 Guruswami−Sudan 0.6 0.5 0.4 0.3 0.2 Berlekamp−Massey 0.1 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Rate M. Sudan , Decoding of Reed-Solomon codes beyond the error correction bound, Journal of Complexity , 1997. V. Guruswami and M. Sudan , Improved decoding of Reed-Solomon and algebraic-geometric codes, IEEE Trans. Information Theory , 1999.
How does it work: the principle Every codeword of the Reed-Solomon code C q ( n , k ) corresponds to a polynomial. The unknown trans- mitted codeword can be represented by the algebraic curve Y − f ( X ) of degree at most k − 1. Bézout’s Theorem Two algebraic curves of degrees d and δ intersect in δ d points, and cannot meet in more than δ d points unless the equations defining them have a common factor. E. Bézout , Théorie générale des équations algébriques , Paris, 1779.
How does it work: the principle Every codeword of the Reed-Solomon code C q ( n , k ) corresponds to a polynomial. The unknown trans- mitted codeword can be represented by the algebraic curve Y − f ( X ) of degree at most k − 1. Bézout’s Theorem Two algebraic curves of degrees d and δ intersect in δ d points, and cannot meet in more than δ d points unless the equations defining them have a common factor. E. Bézout , Théorie générale des équations algébriques , Paris, 1779. Application of Bézout’s Theorem for decoding If we could construct Q ( X , Y ) ∈ F q [ X , Y ] which defines a curve of de- gree δ that intersects Y − f ( X ) in more than ( k − 1 ) δ points (including points at ∞ ), then Y − f ( X ) can be recovered as a factor of Q ( X , Y ) !
A couple of quotations The real mathematics of the real mathematicians, the mathematics of Fermat and Euler and Gauss and Abel and Riemann and Bézout, is almost wholly useless. — G.H. Hardy , A Mathematician’s Apology , 1941 This is right where I started being so favorably im- pressed: the KV algorithm is fully 2 dB better than what I’m using, and the advantage holds up over a wide range of SNRs and error rates. The use of your Reed-Solomon decoder in this program has been a spectacular success. Many dozens (perhaps hundreds?) of Earth-Moon-Earth contacts are be- ing made with it every day now, all over the world. — Joseph H. Taylor , Nobel Laureate, 2004
Reed-Solomon decoding: toy example Suppose k = 2, so that the Reed-Solomon codewords are lines f ( X ) = aX + b . Given 14 interpolation points, we want to com- pute all lines passing through at least 5 of these points.
Reed-Solomon decoding: toy example Suppose k = 2, so that the Reed-Solomon codewords are lines f ( X ) = aX + b . Given 14 interpolation points, we want to com- pute all lines passing through at least 5 of these points. Compute a polynomial Q ( X , Y ) of degree < 5 such that Q ( α i , β i ) = 0 for all the 14 points: Q ( X , Y ) = Y 4 − X 4 − Y 2 + X 2
Reed-Solomon decoding: toy example Suppose k = 2, so that the Reed-Solomon codewords are lines f ( X ) = aX + b . Given 14 interpolation points, we want to com- pute all lines passing through at least 5 of these points. Compute a polynomial Q ( X , Y ) of degree < 5 such that Q ( α i , β i ) = 0 for all the 14 points: Q ( X , Y ) = Y 4 − X 4 − Y 2 + X 2 Let’s plot all the zeros of Q ( X , Y ) . All the relevant lines now emerge!
Reed-Solomon decoding: toy example Suppose k = 2, so that the Reed-Solomon codewords are lines f ( X ) = aX + b . Given 14 interpolation points, we want to com- pute all lines passing through at least 5 of these points. Compute a polynomial Q ( X , Y ) of degree < 5 such that Q ( α i , β i ) = 0 for all the 14 points: Q ( X , Y ) = Y 4 − X 4 − Y 2 + X 2 Let’s plot all the zeros of Q ( X , Y ) . All the relevant lines now emerge! Formally, Q ( X , Y ) factors as: ( Y + X )( Y − X )( X 2 + Y 2 − 1 ) Bézout’s Theorem says it must be so, since deg Q × deg f = 4 is strictly less than the number of intersection points, which is 5.
Key decoding problems Channel output —— ↓ Multiplicity assignment Assign interpolation weights M = [ m i , j ] ⇓ Polynomial interpolation Interpolate through M = [ m i , j ] to compute Q ( X , Y ) ⇓ Partial factorization � � |Q ( X , Y ) Given Q ( X , Y ) , find factors Y − f ( X ) ↓
Key decoding problems Channel output —— ↓ Multiplicity assignment Determines deco- der performance Assign interpolation weights M = [ m i , j ] ⇓ Polynomial interpolation Interpolate through M = [ m i , j ] to compute Q ( X , Y ) Determines deco- ⇓ der complexity Partial factorization � � |Q ( X , Y ) Given Q ( X , Y ) , find factors Y − f ( X ) ↓ Decoder output = F ( multiplicity assignment )
Key decoding problems Channel output —— ↓ Multiplicity assignment Determines deco- der performance Assign interpolation weights M = [ m i , j ] ⇓ Polynomial interpolation Interpolate through M = [ m i , j ] to compute Q ( X , Y ) Determines deco- ⇓ der complexity Partial factorization � � |Q ( X , Y ) Given Q ( X , Y ) , find factors Y − f ( X ) ↓ Decoder output = F ( multiplicity assignment )
Algebraic soft-decision decoding 0 10 A soft-decision decoder Hard−decision makes use of probabilis- −1 10 tic information available at the output of almost −2 10 every channel. Frame Error Rate Soft−decision −3 10 −4 10 Berlekamp−Welch Guruswami−Sudan (m = ∞ ) 1.5 dB GMD −5 10 Soft−decoding (L ≤ 32) Soft−decoding (L = ∞ ) −6 10 15 15.5 16 16.5 17 17.5 18 SNR [dB] R. Koetter and A. Vardy , Algebraic soft-decision decoding of Reed-Solomon codes, IEEE Transactions on Information Theory , 49 , November 2003.
Algebraic soft-decision decoding 0 10 A soft-decision decoder Hard−decision makes use of probabilis- −1 10 tic information available at the output of almost −2 10 every channel. Frame Error Rate Soft−decision −3 10 To decode, we’ll have to −4 10 convert channel probab- Berlekamp−Welch Guruswami−Sudan (m = ∞ ) 1.5 dB GMD ilities into algebraic in- −5 10 Soft−decoding (L ≤ 32) Soft−decoding (L = ∞ ) terpolation conditions. −6 10 15 15.5 16 16.5 17 17.5 18 SNR [dB] R. Koetter and A. Vardy , Algebraic soft-decision decoding of Reed-Solomon codes, IEEE Transactions on Information Theory , 49 , November 2003.
Proportional multiplicity assignment The # of intersections between Y − f ( X ) and Q ( X , Y ) is a random vari- able S M whose distribution depends on the channel observations and the multiplicity assignment. Which assignment maximizes its mean?
Proportional multiplicity assignment The # of intersections between Y − f ( X ) and Q ( X , Y ) is a random vari- able S M whose distribution depends on the channel observations and the multiplicity assignment. Which assignment maximizes its mean? Channel reliabilities Interpolation multiplicities p 1,1 · · · p 1, n m 1,1 · · · m 1, n . . ⇒ . . ... ... . . . . R = M = = ⌊ λ R⌋ . . . . p q ,1 · · · p q , n m q ,1 · · · m q , n m i , j = ⌊ λ p i , j ⌋ with
Proportional multiplicity assignment The # of intersections between Y − f ( X ) and Q ( X , Y ) is a random vari- able S M whose distribution depends on the channel observations and the multiplicity assignment. Which assignment maximizes its mean? Channel reliabilities Interpolation multiplicities p 1,1 · · · p 1, n m 1,1 · · · m 1, n . . ⇒ . . ... ... . . . . R = M = = ⌊ λ R⌋ . . . . p q ,1 · · · p q , n m q ,1 · · · m q , n m i , j = ⌊ λ p i , j ⌋ with Proportional Probability density multiplicity assignment Theorem ∆(Μ) The probability of decoding failure can be expressed as Pr {S M � ∆ ( M ) } . The pro- portional multiplicity assignment maxim- izes the mean of S M for a given ∆ ( M ) . Score random variable S M
Algebraic soft-decision decoder in real life 0 10 Berlekamp−Welch Sudan−Guruswami −1 Koetter−Vardy 10 Gaussian −2 10 Codeword Error Rate −3 10 Codeword error rate −4 10 −5 10 Gaussian Approximation −6 10 −7 10 Proportional Assignment −8 10 Guruswami−Sudan −9 10 −10 10 5.2 5.4 5.6 5.8 6 6.2 6.4 6.6 6.8 7 7.2 SNR[dB] SNR [dB] Performance of the ( 468, 420, 49 ) Reed-Solomon code on a BPSK modulated AWGN channel
Algebraic soft-decision decoder in real life ( ) ( ) 0 10 ( ) b 0 0 X q 0 Berlekamp−Welch s , 0 ( ) 0 q Sudan−Guruswami s , 1 ( ) 0 −1 q ( ) ( ) Koetter−Vardy 10 s , r b r 0 X Gaussian ⊗ −2 ⊗ 10 HME ( ) ⊕ a , b Codeword Error Rate d D 0 −3 ( ) 10 ~ ⊗ ~ [ ] c a , b , s , 0 , y ~ x s − a x + D MACE 0 Codeword error rate 1 −4 10 ( ) ( ) b 1 X ( ) 1 ( ) ( ) 0 q 1 ( ) ( ) q q 1 s 0 , b r 1 X s 1 , s , r −5 + ( ) ⊗ α ⋅ 10 2 ⊗ Gaussian Approximation 0 ⊕ ( ) −6 a , b 10 + ( ) d α ⋅ D 1 D 2 ⊗ [ ] − 1 b −7 10 ~ ( ) y ~ c a , b , s , 1 , y t Proportional Assignment MACE 1 −8 10 X D [ ] t − b Guruswami−Sudan −9 10 [ ] r − b ( ) ( ) b 0 −10 X 10 0 r 5.2 5.4 5.6 5.8 6 6.2 6.4 6.6 6.8 7 7.2 ( ) + ( ) SNR[dB] SNR [dB] α ⋅ q 0 r ( ) ( ) s , ( ) ( ) q 1 r q , r 2 b r 0 X X s , s r D ⊗ Performance of the ( 468, 420, 49 ) ( ) ~ ⊗ c a , b , s , r , y ( ) ⊕ d a , b D Reed-Solomon code on a BPSK r ⊗ modulated AWGN channel MACE r The VLSI architecture is designed for data throughput of over 3.0 Gbps, at hardware cost of 3-4 times that of a conventional Berlekamp-Massey decoder. Work in progress: VHDL description and ASIC design. J. Ma, A. Vardy and Z. Wang , Efficient fast interpolation architecture for soft- decision decoding of RS codes, IEEE Symp. Circuits and Systems , May 2006.
Beyond the Guruswami-Sudan radius? √ Since 1999, the Guruswami-Sudan decoding radius τ GS = 1 − R was the best known. As of a few months ago, we can do much better! Key idea: multivariate interpolation decoding interpolation ✇ ✇ decoding in three or more dimensions univariate interpolation Key idea: new family of Reed-Solomon-like codes Given information symbols u 0 , u 1 , . . . , u k − 1 , form the correspon- 1 ding polynomial f ( X ) = u 0 + u 1 X + · · · + u k − 1 X k − 1 . � a mod e ( X ) , where e ( X ) is a fixed irre- � Compute g ( X ) : = f ( X ) 2 ducible polynomial of degree k and a is an integer parameter. Transmit the evaluation of f ( X ) + α g ( X ) , where { 1, α } is a basis 3 � � for F n f ( x 1 ) + α g ( x 1 ) , . . . , f ( x n ) + α g ( x n ) q 2 over F q , namely .
Beyond the Guruswami-Sudan radius? √ Since 1999, the Guruswami-Sudan decoding radius τ GS = 1 − R was the best known. As of a few months ago, we can do much better! Key idea: multivariate interpolation decoding interpolation ✇ ✇ decoding in three or more dimensions univariate interpolation Key idea: new family of Reed-Solomon-like codes Given information symbols u 0 , u 1 , . . . , u k − 1 , form the correspon- 1 ding polynomial f ( X ) = u 0 + u 1 X + · · · + u k − 1 X k − 1 . � a mod e ( X ) , where e ( X ) is a fixed irre- � Compute g ( X ) : = f ( X ) 2 ducible polynomial of degree k and a is an integer parameter. Transmit the evaluation of f ( X ) + α g ( X ) , where { 1, α } is a basis 3 � � for F n f ( x 1 ) + α g ( x 1 ) , . . . , f ( x n ) + α g ( x n ) q 2 over F q , namely .
Geometric interpretation of decoding Given a received vector ( y 1 + α z 1 , y 2 + α z 2 , . . . , y n + α z n ) , we interpo- late through the points ( x 1 , y 1 , z 1 ) , ( x 2 , y 2 , z 2 ) , . . . , ( x n , y n , z n ) to obtain a trivariate interpolation polynomial Q ( X , Y , Z ) . Interpolation Polynomial Received word Transmitted codeword Interpolation polynomial � ≡ 0 � Q X , f ( X ) , g ( X )
Geometric interpretation of decoding Given a received vector ( y 1 + α z 1 , y 2 + α z 2 , . . . , y n + α z n ) , we interpo- late through the points ( x 1 , y 1 , z 1 ) , ( x 2 , y 2 , z 2 ) , . . . , ( x n , y n , z n ) to obtain a trivariate interpolation polynomial Q ( X , Y , Z ) . Encoder Polynomial Interpolation Polynomial Received word + Transmitted codeword Transmitted codeword Interpolation polynomial Encoder polynomial � ≡ 0 � � a � f ( X ) mod e ( X ) ≡ g ( X ) Q X , f ( X ) , g ( X )
Geometric interpretation of decoding Given a received vector ( y 1 + α z 1 , y 2 + α z 2 , . . . , y n + α z n ) , we interpo- late through the points ( x 1 , y 1 , z 1 ) , ( x 2 , y 2 , z 2 ) , . . . , ( x n , y n , z n ) to obtain a trivariate interpolation polynomial Q ( X , Y , Z ) . Encoder Polynomial Interpolation Polynomial Received word Transmitted codeword Recovery of information
Decoding radius of the new scheme 1 F. Parvaresh and A. Vardy , Correct- 1 0.9 Fraction of errors corrected Sudan ing errors beyond the Guruswami- 0.8 Sudan radius in polynomial time, 0.7 Guruswami−Sudan IEEE Symp. Foundations of Computer 0.6 Science (FOCS) , October 2005. Today: 1−R 0.5 0.4 V. Guruswami and A. Rudra , Expli- 2 0.3 cit capacity-achieving list-decodable 0.2 codes, ACM Symposium on Theory of Berlekamp−Massey 0.1 Computing (STOC) , May 2006. 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Rate The main problem of list-decoding solved: We can construct the best possible codes in polynomial time and decode them in polynomial time! Remaining problems: the alphabet-size is very large, the list-size in- creases (polynomially) with the length of the code. Conjecture: these methods achieve the capacity of the q -ary symmetric channel, with better complexity than anything known.
Decoding radius of the new scheme 1 F. Parvaresh and A. Vardy , Correct- 1 0.9 Fraction of errors corrected Sudan ing errors beyond the Guruswami- 0.8 Sudan radius in polynomial time, 0.7 Guruswami−Sudan IEEE Symp. Foundations of Computer 0.6 Science (FOCS) , October 2005. Today: 1−R 0.5 0.4 V. Guruswami and A. Rudra , Expli- 2 0.3 cit capacity-achieving list-decodable 0.2 codes, ACM Symposium on Theory of Berlekamp−Massey 0.1 Computing (STOC) , May 2006. 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Rate The main problem of list-decoding solved: We can construct the best possible codes in polynomial time and decode them in polynomial time! Remaining problems: the alphabet-size is very large, the list-size in- creases (polynomially) with the length of the code. Conjecture: these methods achieve the capacity of the q -ary symmetric channel, with better complexity than anything known.
Applications of Coding Theory in Theoretical Computer Science
Codes most useful in computer science Locally decodable codes Codes with sub-linear time error-correction algorithms Applications: private information retrieval, hardness amplification, hard-core predicates, generation of pseudo-random bits Locally testable codes Codes with sub-linear time error-detection algorithms Applications: at the core of probabilistically-checkable proofs theory
Codes most useful in computer science Locally decodable codes Codes with sub-linear time error-correction algorithms Applications: private information retrieval, hardness amplification, hard-core predicates, generation of pseudo-random bits Locally testable codes Codes with sub-linear time error-detection algorithms Applications: at the core of probabilistically-checkable proofs theory How is decoding in sub-linear time possible? Not enough time to read the input or write the output.
Codes most useful in computer science Locally decodable codes Codes with sub-linear time error-correction algorithms Applications: private information retrieval, hardness amplification, hard-core predicates, generation of pseudo-random bits Locally testable codes Codes with sub-linear time error-detection algorithms Applications: at the core of probabilistically-checkable proofs theory How is decoding in sub-linear time possible? Not enough time to read the input or write the output. We do the impossible for breakfast! — anonymous theoretical computer scientist , 2005
Hardness amplification problem How hard is it to compute a Boolean function f : { 0, 1 } n →{ 0, 1 } ? Worst-case hardness: no polynomial-time algorithm can com- pute f correctly on all possible inputs. Example: NP-complete problems are believed to be worst-case hard. Average-case hardness: no polynomial-time algorithm can compute f correctly on a small fraction δ of the inputs. Note: the smallest possible value is δ = 0.5 + ε , since random guess computes any f correctly on half the inputs in polynomial time. Average-case hard functions ( δ -hard for some δ < 1) are needed in cryp- tography, pseudo-random generators, and many other applications! Convert a worst-case hard function f into a δ -hard function g
Recommend
More recommend
