Locally recoverable codes In addition to correcting one erasure, sometimes the code is used to recover the data from a multi-node failure. Global correction: repairing a large number of failed nodes by contacting all the remaining coordinates Problem: What is largest possible minimum distance of an LRC code C ? G OPALAN ET AL . bound: Q k U d ď n ´ k ´ ` 2 r Alexander Barg, University of Maryland Facets of the repair problem 7 / 45
Variants of LRC codes • A linear code is LRC with locality r if every coordinate is involved in a parity check of weight r ` 1 For every i P t 1 , 2 , . . . , n u there exists a punctured code C i : “ C | t i uY I i such that • | I i | “ r • dim p C i q ď r • distance d p C i q “ 2 . • Local correction of s “ 2 , 3 , ... erasures: • A linear code C is LRC with locality p r , ρ q if every coordinate is contained in a “local code” of distance ρ This means that we can locally repair up to ρ ´ 1 erasures by contacting r helper nodes The distance of C is bounded above as follows: ´Q k U ¯ d ď n ´ k ` 1 ´ ´ 1 p ρ ´ 1 q r Alexander Barg, University of Maryland Facets of the repair problem 8 / 45
Codes with hierarchical locality An p n , k , d q linear code with two levels of locality Alexander Barg, University of Maryland Facets of the repair problem 9 / 45
Codes with hierarchical locality An p n , k , d q linear code with two levels of locality • Repair a single node by querying r 2 helper nodes; • Repair ρ ´ 1 nodes by querying r 1 helper nodes; • Repair d ´ 1 nodes by addressing all the remaining n ´ p d ´ 1 q nodes Alexander Barg, University of Maryland Facets of the repair problem 9 / 45
Codes with hierarchical locality An p n , k , d q linear code with two levels of locality • Repair a single node by querying r 2 helper nodes; • Repair ρ ´ 1 nodes by querying r 1 helper nodes; • Repair d ´ 1 nodes by addressing all the remaining n ´ p d ´ 1 q nodes Defintion: Let ρ 1 ą 2 and r 2 ď r 1 . A linear code C is H-LRC and parameters pp r 1 , ρ 1 q , p r 2 , 2 qq if for every i P t 1 , . . . , n u there is a punctured code C i such that 1. dim p C i q ď r 1 , 2. d p C i q ě ρ, and 3. C i is an p r 2 , 2 q LRC code. Alexander Barg, University of Maryland Facets of the repair problem 9 / 45
Reed-Solomon codes F “ F q the finite field of q elements Example: q “ 8 , α 3 “ α ` 1 F 8 “ t 0 , 1 , α, α 2 , α 3 “ α ` 1 , α 4 “ α 2 ` α, α 5 “ α 2 ` α ` 1 , α 6 “ α 2 ` 1 u The elements of F 8 can also be written as binary vectors p 000 q , p 001 q , p 010 q , p 101 q , p 110 q , p 111 q , p 101 q Encoding example: Let q “ 8 , p n , k q “ p 7 , 3 q Suppose that data symbols are 1 , α, α To encode, form a polynomial f p x q “ 1 ` α x ` α x 2 and compute the values of f p x q at the points 1 , α, α 2 , . . . , α 6 α 2 α 3 α 4 α 5 α 6 x 1 α α 4 α 6 α 4 α 6 f p x q 1 α α This encodes k “ 3 data symbols p 1 , α, α q into n “ 7 symbols of the codeword Alexander Barg, University of Maryland Facets of the repair problem 10 / 45
Evaluation codes Alexander Barg, University of Maryland Facets of the repair problem 11 / 45
Evaluation codes Given a polynomial f P F q r x s and a set A “ t P 1 , . . . , P n u Ă F q define the map ev A : f ÞÑ p f p P i q , i “ 1 , . . . , n q Alexander Barg, University of Maryland Facets of the repair problem 11 / 45
Evaluation codes Given a polynomial f P F q r x s and a set A “ t P 1 , . . . , P n u Ă F q define the map ev A : f ÞÑ p f p P i q , i “ 1 , . . . , n q Example: Let q “ 8 , f p x q “ 1 ` α x ` α x 2 f p x q ÞÑ p 1 , α 4 , α 6 , α 4 , α, α, α 6 q Alexander Barg, University of Maryland Facets of the repair problem 11 / 45
Evaluation codes Given a polynomial f P F q r x s and a set A “ t P 1 , . . . , P n u Ă F q define the map ev A : f ÞÑ p f p P i q , i “ 1 , . . . , n q Example: Let q “ 8 , f p x q “ 1 ` α x ` α x 2 f p x q ÞÑ p 1 , α 4 , α 6 , α 4 , α, α, α 6 q Evaluation code C p A q Let V “ t f P F q r x su be a set of polynomials, dim p V q “ k C : V Ñ F n q f ÞÑ ev A p f q “ p f p P i q , i “ 1 , . . . , n q Alexander Barg, University of Maryland Facets of the repair problem 11 / 45
Construction of p n , k , r q LRC codes: Example Parameters: n “ 12 , k “ 6 , r “ 3 , q “ 13 ; Set of points: A “ t 1 , . . . , 12 u Ă F 13 A “ t A 1 “ p 1 , 5 , 12 , 8 q , A 2 “ p 2 , 10 , 11 , 3 q , A 3 “ p 4 , 7 , 9 , 6 qu Basis of functions: Take g p x q constant on A i , i “ 1 , 2 , 3 , e.g., g p x q “ x 4 ´ 1 A E g p x q j x i , i “ 0 , 1 , 2 ; j “ 0 , 1 V “ ; dim p V q “ 6 Evaluation code C : V Ñ F n q is an LRC code of length | A | “ 12 with r “ 3 and d “ 6 Alexander Barg, University of Maryland Facets of the repair problem 12 / 45
Construction of p n , k , r q LRC codes: Example Parameters: n “ 12 , k “ 6 , r “ 3 , q “ 13 ; Set of points: A “ t 1 , . . . , 12 u Ă F 13 A “ t A 1 “ p 1 , 5 , 12 , 8 q , A 2 “ p 2 , 10 , 11 , 3 q , A 3 “ p 4 , 7 , 9 , 6 qu Basis of functions: Take g p x q constant on A i , i “ 1 , 2 , 3 , e.g., g p x q “ x 4 ´ 1 A E g p x q j x i , i “ 0 , 1 , 2 ; j “ 0 , 1 V “ ; dim p V q “ 6 Evaluation code C : V Ñ F n q is an LRC code of length | A | “ 12 with r “ 3 and d “ 6 ‚ This construction is general, and gives distance-optimal LRC codes which form certain subcodes of RS codes Alexander Barg, University of Maryland Facets of the repair problem 12 / 45
Geometric view of LRC codes A “ t 1 , . . . , 12 u Ă F 13 A “ A 1 Y A 2 Y A 3 A 1 “ p 1 , 5 , 12 , 8 q A 2 “ p 2 , 10 , 11 , 3 q A 3 “ p 4 , 7 , 9 , 6 q Alexander Barg, University of Maryland Facets of the repair problem 13 / 45
Geometric view of LRC codes A “ t 1 , . . . , 12 u Ă F 13 A “ A 1 Y A 2 Y A 3 A 1 “ p 1 , 5 , 12 , 8 q A 2 “ p 2 , 10 , 11 , 3 q A 3 “ p 4 , 7 , 9 , 6 q g : A Ñ F 13 x ÞÑ x 4 ´ 1 Alexander Barg, University of Maryland Facets of the repair problem 13 / 45
Geometric view of LRC codes A “ t 1 , . . . , 12 u Ă F 13 A “ A 1 Y A 2 Y A 3 A 1 “ p 1 , 5 , 12 , 8 q A 2 “ p 2 , 10 , 11 , 3 q A 3 “ p 4 , 7 , 9 , 6 q g : A Ñ F 13 x ÞÑ x 4 ´ 1 g : F 13 Ñ t 0 , 2 , 8 u Ă F 13 | g ´ 1 p y q| “ r ` 1 Alexander Barg, University of Maryland Facets of the repair problem 13 / 45
Geometric view of LRC codes A “ t 1 , . . . , 12 u Ă F 13 A “ A 1 Y A 2 Y A 3 A 1 “ p 1 , 5 , 12 , 8 q A 2 “ p 2 , 10 , 11 , 3 q A 3 “ p 4 , 7 , 9 , 6 q g : A Ñ F 13 x ÞÑ x 4 ´ 1 g : F 13 Ñ t 0 , 2 , 8 u Ă F 13 | g ´ 1 p y q| “ r ` 1 In the basic construction, X “ Y “ P 1 Alexander Barg, University of Maryland Facets of the repair problem 13 / 45
LRC codes on curves Consider the set of pairs p x , y q P F 9 that satisfy the equation x 3 ` x “ y 4 α 7 ‚ ‚ ‚ ‚ α 6 ‚ α 5 ‚ ‚ ‚ ‚ α 4 ‚ ‚ ‚ ‚ x α 3 ‚ ‚ ‚ ‚ α 2 ‚ α ‚ ‚ ‚ ‚ ‚ ‚ ‚ ‚ 1 0 ‚ 0 1 α α 2 α 3 α 4 α 5 α 6 α 7 y Alexander Barg, University of Maryland Facets of the repair problem 14 / 45
LRC codes on curves Consider the set of pairs p x , y q P F 9 that satisfy the equation x 3 ` x “ y 4 α 7 ‚ ‚ ‚ ‚ α 6 ‚ α 5 ‚ ‚ ‚ ‚ α 4 ‚ ‚ ‚ ‚ x α 3 ‚ ‚ ‚ ‚ α 2 ‚ α ‚ ‚ ‚ ‚ ‚ ‚ ‚ ‚ 1 0 ‚ 0 1 α α 2 α 3 α 4 α 5 α 6 α 7 y Affine points of the Hermitian curve X over F 9 ; α 2 “ α ` 1 Alexander Barg, University of Maryland Facets of the repair problem 14 / 45
Hermitian codes P 1 g : X Ñ p x , y q ÞÑ y Space of functions V : “ x 1 , y , y 2 , x , xy , xy 2 y A= { Affine points of the Hermitian curve over F 9 } ; n “ 27 , k “ 6 C : V Ñ F n 9 Alexander Barg, University of Maryland Facets of the repair problem 15 / 45
Hermitian codes P 1 g : X Ñ p x , y q ÞÑ y Space of functions V : “ x 1 , y , y 2 , x , xy , xy 2 y A= { Affine points of the Hermitian curve over F 9 } ; n “ 27 , k “ 6 C : V Ñ F n 9 E.g., message p 1 , α, α 2 , α 3 , α 4 , α 5 q F p x , y q “ 1 ` α y ` α 2 y 2 ` α 3 x ` α 4 xy ` α 5 xy 2 F p 0 , 0 q “ 1 etc. Alexander Barg, University of Maryland Facets of the repair problem 15 / 45
LRC codes on algebraic curves: General construction Alexander Barg, University of Maryland Facets of the repair problem 16 / 45
LRC codes on algebraic curves: General construction • φ : X Ñ Y degree- p r ` 1 q separable morphism of curves X , Y over K “ F q Alexander Barg, University of Maryland Facets of the repair problem 16 / 45
LRC codes on algebraic curves: General construction • φ : X Ñ Y degree- p r ` 1 q separable morphism of curves X , Y over K “ F q • Q 1 , Q 2 , . . . , Q s P Y p K q split completely in the cover X Ñ Y Alexander Barg, University of Maryland Facets of the repair problem 16 / 45
LRC codes on algebraic curves: General construction • φ : X Ñ Y degree- p r ` 1 q separable morphism of curves X , Y over K “ F q • Q 1 , Q 2 , . . . , Q s P Y p K q split completely in the cover X Ñ Y • Let P i , j , j “ 1 , . . . , r ` 1 be the set of points on X p K q above Q i P Y p K q Alexander Barg, University of Maryland Facets of the repair problem 16 / 45
LRC codes on algebraic curves: General construction • φ : X Ñ Y degree- p r ` 1 q separable morphism of curves X , Y over K “ F q • Q 1 , Q 2 , . . . , Q s P Y p K q split completely in the cover X Ñ Y • Let P i , j , j “ 1 , . . . , r ` 1 be the set of points on X p K q above Q i P Y p K q • φ ˚ : K p Y q ã Ñ K p X q Alexander Barg, University of Maryland Facets of the repair problem 16 / 45
LRC codes on algebraic curves: General construction • φ : X Ñ Y degree- p r ` 1 q separable morphism of curves X , Y over K “ F q • Q 1 , Q 2 , . . . , Q s P Y p K q split completely in the cover X Ñ Y • Let P i , j , j “ 1 , . . . , r ` 1 be the set of points on X p K q above Q i P Y p K q • φ ˚ : K p Y q ã Ñ K p X q • e 1 , . . . e r P K p X q linearly independent over K p Y q and such that P i , j R supp p e l q 8 Alexander Barg, University of Maryland Facets of the repair problem 16 / 45
LRC codes on algebraic curves: General construction • φ : X Ñ Y degree- p r ` 1 q separable morphism of curves X , Y over K “ F q • Q 1 , Q 2 , . . . , Q s P Y p K q split completely in the cover X Ñ Y • Let P i , j , j “ 1 , . . . , r ` 1 be the set of points on X p K q above Q i P Y p K q • φ ˚ : K p Y q ã Ñ K p X q • e 1 , . . . e r P K p X q linearly independent over K p Y q and such that P i , j R supp p e l q 8 • f 1 , . . . , f t P K p Y q l.i. over K and Q l R supp p f j q 8 Alexander Barg, University of Maryland Facets of the repair problem 16 / 45
LRC codes on algebraic curves: General construction • φ : X Ñ Y degree- p r ` 1 q separable morphism of curves X , Y over K “ F q • Q 1 , Q 2 , . . . , Q s P Y p K q split completely in the cover X Ñ Y • Let P i , j , j “ 1 , . . . , r ` 1 be the set of points on X p K q above Q i P Y p K q • φ ˚ : K p Y q ã Ñ K p X q • e 1 , . . . e r P K p X q linearly independent over K p Y q and such that P i , j R supp p e l q 8 • f 1 , . . . , f t P K p Y q l.i. over K and Q l R supp p f j q 8 • Given the data a “ p a u , v , 1 ď u ď s ; 0 ď v ď r q P K p r ` 1 q s , evaluate r ÿ ÿ a u , v φ ˚ f v f a : “ e u u “ 1 v “ 1 at each of the points P ij Alexander Barg, University of Maryland Facets of the repair problem 16 / 45
LRC codes on algebraic curves: Main results • A general construction of LRC codes from covering maps of curves (including quotient curves, fiber products) • Families of LRC codes of length n ě q • Infinite families of codes from the Garcia-Stichtenoth tower • Asymptotic tradeoff between the rate and distance better than the Gilbert-Varshamov bound • Codes with locality on algebraic surfaces Alexander Barg, University of Maryland Facets of the repair problem 17 / 45
Codes on curves with hierarchical locality (H-LRC codes) Consider a code C over the field F q with distance d Alexander Barg, University of Maryland Facets of the repair problem 18 / 45
Codes on curves with hierarchical locality (H-LRC codes) Consider a code C over the field F q with distance d Each coordinate i is included in a r ν, r 1 , ρ s code C 1 which is also an LRC code with locality r 2 , where 2 ă ρ ă d Alexander Barg, University of Maryland Facets of the repair problem 18 / 45
Codes on curves with hierarchical locality (H-LRC codes) Consider a code C over the field F q with distance d Each coordinate i is included in a r ν, r 1 , ρ s code C 1 which is also an LRC code with locality r 2 , where 2 ă ρ ă d Flexible functionality: • A single node failure is repaired by contacting r 2 nodes • Up to ρ ´ 1 failures are repaired by contacting ν helper nodes • Up to d ´ 1 failures can be corrected by contacting all the functional nodes in the encoding Alexander Barg, University of Maryland Facets of the repair problem 18 / 45
Codes on curves with hierarchical locality (H-LRC codes) Consider a code C over the field F q with distance d Each coordinate i is included in a r ν, r 1 , ρ s code C 1 which is also an LRC code with locality r 2 , where 2 ă ρ ă d Flexible functionality: • A single node failure is repaired by contacting r 2 nodes • Up to ρ ´ 1 failures are repaired by contacting ν helper nodes • Up to d ´ 1 failures can be corrected by contacting all the functional nodes in the encoding The Gopalan et al. bound extends as follows: ´Q k ´Q k U ¯ U ¯ d ď n ´ k ` 1 ´ ´ 1 ´ ´ 1 p ρ ´ 2 q r 2 r 1 (S ASIDHARAN -A GARWAL -K UMAR , ’15) Alexander Barg, University of Maryland Facets of the repair problem 18 / 45
Codes on curves with hierarchical locality (H-LRC codes) • Consider the following sequence of maps of algebraic curves: φ 2 φ 1 X Ý Ñ Y Ý Ñ Z where deg φ 1 “ r 2 ` 1 ; deg φ 2 “ s ` 1 ; define ψ “ φ 1 ˝ φ 2 • Let K p X q “ K p Y qp x q and K p Y q “ K p Z qp y q The covering map gives rise to the embedding of function fields: K p X q Ą K p Y q Ą K p Z q • Let S “ t P 1 , . . . , P m u be a collection of points in Z p K q that split completely on X , i.e., | ψ ´ 1 p P i q| “ p r 2 ` 1 qp s ` 1 q • The codes are defined by evaluating the functions t f i y j x k | 1 ď i ď t , 0 ď j ď s ´ 1 , 0 ď k ď r 2 ´ 1 u at the points P 1 , i “ 1 , . . . , m , where f 1 , . . . , f t form a basis for L p Q 8 q Alexander Barg, University of Maryland Facets of the repair problem 19 / 45
Example: Let m |p q ´ 1 q and consider X given by y m “ f p x q Alexander Barg, University of Maryland Facets of the repair problem 20 / 45
Example: Let m |p q ´ 1 q and consider X given by y m “ f p x q • L : “ F q p x , y q is a cyclic extension of K : “ F q p x q of degree m . G “ Gal p L { K q is cyclic of order m Alexander Barg, University of Maryland Facets of the repair problem 20 / 45
Example: Let m |p q ´ 1 q and consider X given by y m “ f p x q • L : “ F q p x , y q is a cyclic extension of K : “ F q p x q of degree m . G “ Gal p L { K q is cyclic of order m • Let m “ p a ` 1 qp b ` 1 q , let α be the generator of G Alexander Barg, University of Maryland Facets of the repair problem 20 / 45
Example: Let m |p q ´ 1 q and consider X given by y m “ f p x q • L : “ F q p x , y q is a cyclic extension of K : “ F q p x q of degree m . G “ Gal p L { K q is cyclic of order m • Let m “ p a ` 1 qp b ` 1 q , let α be the generator of G • Let H “ x α b ` 1 y , | H | “ a ` 1 Alexander Barg, University of Maryland Facets of the repair problem 20 / 45
Example: Let m |p q ´ 1 q and consider X given by y m “ f p x q • L : “ F q p x , y q is a cyclic extension of K : “ F q p x q of degree m . G “ Gal p L { K q is cyclic of order m • Let m “ p a ` 1 qp b ` 1 q , let α be the generator of G • Let H “ x α b ` 1 y , | H | “ a ` 1 • Invariants of H are generated by y a ` 1 Alexander Barg, University of Maryland Facets of the repair problem 20 / 45
Example: Let m |p q ´ 1 q and consider X given by y m “ f p x q • L : “ F q p x , y q is a cyclic extension of K : “ F q p x q of degree m . G “ Gal p L { K q is cyclic of order m • Let m “ p a ` 1 qp b ` 1 q , let α be the generator of G • Let H “ x α b ` 1 y , | H | “ a ` 1 • Invariants of H are generated by y a ` 1 â K p X q H “ K p x , y a ` 1 q Ð â K p X q G “ K p x , y p a ` 1 qp b ` 1 q q • K p X q “ K p x , y q Ð Alexander Barg, University of Maryland Facets of the repair problem 20 / 45
Example: Let m |p q ´ 1 q and consider X given by y m “ f p x q • L : “ F q p x , y q is a cyclic extension of K : “ F q p x q of degree m . G “ Gal p L { K q is cyclic of order m • Let m “ p a ` 1 qp b ` 1 q , let α be the generator of G • Let H “ x α b ` 1 y , | H | “ a ` 1 • Invariants of H are generated by y a ` 1 â K p X q H “ K p x , y a ` 1 q Ð â K p X q G “ K p x , y p a ` 1 qp b ` 1 q q • K p X q “ K p x , y q Ð • We obtain families of H-LRC codes by specializing this construction to various Kummer curves (Hermitian, Giulietti-Korchm´ aros, etc.) (work with S EAN B ALLENTINE and S ERGE V L ˘ ¸, arXiv.org:1807.05473) ADUT Alexander Barg, University of Maryland Facets of the repair problem 20 / 45
Regenerating Codes and the Repair Problem • The file of size M is divided into l -vectors over a finite field F “ F q (chunks of size l ). Alexander Barg, University of Maryland Facets of the repair problem 21 / 45
Regenerating Codes and the Repair Problem • The file of size M is divided into l -vectors over a finite field F “ F q (chunks of size l ). • Each chunk is placed on a separate storage node Alexander Barg, University of Maryland Facets of the repair problem 21 / 45
Regenerating Codes and the Repair Problem • The file of size M is divided into l -vectors over a finite field F “ F q (chunks of size l ). • Each chunk is placed on a separate storage node • The data is encoded using an erasure-correcting code of dimension k “ M { l Alexander Barg, University of Maryland Facets of the repair problem 21 / 45
Regenerating Codes and the Repair Problem • The file of size M is divided into l -vectors over a finite field F “ F q (chunks of size l ). • Each chunk is placed on a separate storage node • The data is encoded using an erasure-correcting code of dimension k “ M { l • Data encoding C 1 , C 2 , C 3 , . . . , C n ´ 1 , C n where each C i is located on its own node Another figure of merit: The repair bandwidth, i.e., the total amount of communication for repairing the failed node(s) Alexander Barg, University of Maryland Facets of the repair problem 21 / 45
Regenerating Codes and the Repair Problem • Let C be a code over F used for node repair (correcting one or several erasures) Alexander Barg, University of Maryland Facets of the repair problem 22 / 45
Regenerating Codes and the Repair Problem • Let C be a code over F used for node repair (correcting one or several erasures) • Let B be a subfield of F ; r F : B s “ l Alexander Barg, University of Maryland Facets of the repair problem 22 / 45
Regenerating Codes and the Repair Problem • Let C be a code over F used for node repair (correcting one or several erasures) • Let B be a subfield of F ; r F : B s “ l • Consider C as a code over B ; every coordinate is an l -vector over B Alexander Barg, University of Maryland Facets of the repair problem 22 / 45
Regenerating Codes and the Repair Problem • Let C be a code over F used for node repair (correcting one or several erasures) • Let B be a subfield of F ; r F : B s “ l • Consider C as a code over B ; every coordinate is an l -vector over B • Repair is performed by downloading symbols from helper nodes n l k Alexander Barg, University of Maryland Facets of the repair problem 22 / 45
Regenerating Codes and the Repair Problem • Let C be a code over F used for node repair (correcting one or several erasures) • Let B be a subfield of F ; r F : B s “ l • Consider C as a code over B ; every coordinate is an l -vector over B • Repair is performed by downloading symbols from helper nodes n l k Alexander Barg, University of Maryland Facets of the repair problem 22 / 45
Regenerating Codes and the Repair Problem • Let C be a code over F used for node repair (correcting one or several erasures) • Let B be a subfield of F ; r F : B s “ l • Consider C as a code over B ; every coordinate is an l -vector over B • Repair is performed by downloading symbols from helper nodes n l d Alexander Barg, University of Maryland Facets of the repair problem 22 / 45
Regenerating Codes and the Repair Problem • Let C be a code over F used for node repair (correcting one or several erasures) • Let B be a subfield of F ; r F : B s “ l • Consider C as a code over B ; every coordinate is an l -vector over B • Repair is performed by downloading symbols from helper nodes n l d=n−1 Repair Bandwidth – The number of symbols of B downloaded for node repair Alexander Barg, University of Maryland Facets of the repair problem 22 / 45
Vector (Array) codes • Let C be a code over the field F “ F q l . • Each coordinate can be considered as an l -vector over B “ F q . • A codeword C P C is an l ˆ n matrix over B . • The value of l is called sub-packetization of the code C • C is called a linear array code (or a vector code) if it is B -linear. It may not be F -linear; if it is, it is also called a scalar code. Alexander Barg, University of Maryland Facets of the repair problem 23 / 45
Formal definition of the (single-node) repair problem • Consider an p n , k , l q code C over B . Alexander Barg, University of Maryland Facets of the repair problem 24 / 45
Formal definition of the (single-node) repair problem • Consider an p n , k , l q code C over B . • A codeword C “ p C 1 , . . . , C n q , where C i “ p c i , 0 , c i , 1 , . . . , c i , l ´ 1 q T P B l , i “ 1 , . . . , n . Alexander Barg, University of Maryland Facets of the repair problem 24 / 45
Formal definition of the (single-node) repair problem • Consider an p n , k , l q code C over B . • A codeword C “ p C 1 , . . . , C n q , where C i “ p c i , 0 , c i , 1 , . . . , c i , l ´ 1 q T P B l , i “ 1 , . . . , n . • A node i P r n s can be repaired from a subset of d ě k helper nodes R i Ă r n szt i u , by downloading β i p R i q symbols of B if there are • numbers β i , j , j P R i and • d functions f i , j : B l Ñ B β i , j , j P R i and a function g i : B j β i , j Ñ B l ř such that C i “ g i p f i , j p C j q , j P R i q and ÿ β i , j “ β i p R i q . j P R i Alexander Barg, University of Maryland Facets of the repair problem 24 / 45
Formal definition of the (single-node) repair problem • Consider an p n , k , l q code C over B . • A codeword C “ p C 1 , . . . , C n q , where C i “ p c i , 0 , c i , 1 , . . . , c i , l ´ 1 q T P B l , i “ 1 , . . . , n . • A node i P r n s can be repaired from a subset of d ě k helper nodes R i Ă r n szt i u , by downloading β i p R i q symbols of B if there are • numbers β i , j , j P R i and • d functions f i , j : B l Ñ B β i , j , j P R i and a function g i : B j β i , j Ñ B l ř such that C i “ g i p f i , j p C j q , j P R i q and ÿ β i , j “ β i p R i q . j P R i The repair bandwidth of i from R i : β ˚ i p R i q “ min f i , j , g i β i p R i q Alexander Barg, University of Maryland Facets of the repair problem 24 / 45
Repair of several erasures Centralized and distributed (cooperative) models Suppose that nodes i and j are erased. Alexander Barg, University of Maryland Facets of the repair problem 25 / 45
Repair of several erasures Centralized and distributed (cooperative) models Suppose that nodes i and j are erased. Centralized repair: Download information from the set of helper nodes R , | R | “ d that is used for repair of both C i and C j Alexander Barg, University of Maryland Facets of the repair problem 25 / 45
Repair of several erasures Centralized and distributed (cooperative) models Suppose that nodes i and j are erased. Centralized repair: Download information from the set of helper nodes R , | R | “ d that is used for repair of both C i and C j Cooperative repair 1 q : • Round 1: Nodes C i and C j download (potentially, different) information from R • Round 2: Information exchange: C i Ô C j Both rounds of communication contribute to the repair bandwidth. 1 q Originally defined for T ě 2 communication rounds (S HUM -H U , ’13 ) Alexander Barg, University of Maryland Facets of the repair problem 25 / 45
Cut-set bound l β ě d ` 1 ´ k d ( D IMAKIS ET AL ., 2010 ) Alexander Barg, University of Maryland Facets of the repair problem 26 / 45
Cut-set bound l β ě d ` 1 ´ k d ( D IMAKIS ET AL ., 2010 ) The code meeting this bound with equality is said to afford optimal repair For d “ n ´ 1 , r “ n ´ k β ě l r p n ´ 1 q Alexander Barg, University of Maryland Facets of the repair problem 26 / 45
Cut-set bound l β ě d ` 1 ´ k d ( D IMAKIS ET AL ., 2010 ) The code meeting this bound with equality is said to afford optimal repair For d “ n ´ 1 , r “ n ´ k β ě l r p n ´ 1 q The cut-set bound extends to repair of h ě 1 erasures (failed nodes): hdl • Centralized model: β ě (V. C ADAMBE ET AL ., ’13 ) d ` h ´ k • Cooperative model: β ě h p d ` h ´ 1 q l (K. S HUM and Y. H U , ’13; M. Y E and A.B., ’17) d ` h ´ k Alexander Barg, University of Maryland Facets of the repair problem 26 / 45
Cut-set bound l β ě d ` 1 ´ k d ( D IMAKIS ET AL ., 2010 ) The code meeting this bound with equality is said to afford optimal repair For d “ n ´ 1 , r “ n ´ k β ě l r p n ´ 1 q The cut-set bound extends to repair of h ě 1 erasures (failed nodes): hdl • Centralized model: β ě (V. C ADAMBE ET AL ., ’13 ) d ` h ´ k • Cooperative model: β ě h p d ` h ´ 1 q l (K. S HUM and Y. H U , ’13; M. Y E and A.B., ’17) d ` h ´ k Codes that meet these bounds with equality are said to have p h , d q -optimal repair bandwidth Alexander Barg, University of Maryland Facets of the repair problem 26 / 45
Cooperative repair Cut-set bound for cooperative repair: β ě | F |p| R | ` | F | ´ 1 q l | F | ` | R | ´ k | R | l p| F | ´ 1 q l ´ ¯ “ | F | | F | ` | R | ´ k ` | F | ` | R | ´ k Alexander Barg, University of Maryland Facets of the repair problem 27 / 45
Cooperative repair Cut-set bound for cooperative repair: β ě | F |p| R | ` | F | ´ 1 q l | F | ` | R | ´ k | R | l p| F | ´ 1 q l ´ ¯ “ | F | | F | ` | R | ´ k ` | F | ` | R | ´ k Structure of optimal codes: l • Each failed node downloads | F | ` | R | ´ k from the helper nodes l • Each failed node downloads | F | ` | R | ´ k from each of the other nodes in F Alexander Barg, University of Maryland Facets of the repair problem 27 / 45
Cooperative repair model is stronger than the centralized model An MSD code that is cooperatively optimal-repair is also optimal-repair under the centralized model Alexander Barg, University of Maryland Facets of the repair problem 28 / 45
Cooperative repair model is stronger than the centralized model An MSD code that is cooperatively optimal-repair is also optimal-repair under the centralized model Theorem (Ye-B, ’18) Let C be an p n , k , l q MDS array code and let F , R Ď r n s be two disjoint subsets such that | F | ď r and | R | ě k . If β coop p C q “ | F |p| R | ` | F | ´ 1 q l , | F | ` | R | ´ k then | F || R | l β cent p C q “ | F | ` | R | ´ k . Alexander Barg, University of Maryland Facets of the repair problem 28 / 45
Cooperative repair of two nodes • Assume that nodes C 1 , C 2 are erased. Alexander Barg, University of Maryland Facets of the repair problem 29 / 45
Cooperative repair of two nodes • Assume that nodes C 1 , C 2 are erased. • We construct an p n , k , 3 q MDS array code, where k ă n ď | F | ´ 2 . Alexander Barg, University of Maryland Facets of the repair problem 29 / 45
Cooperative repair of two nodes • Assume that nodes C 1 , C 2 are erased. • We construct an p n , k , 3 q MDS array code, where k ă n ď | F | ´ 2 . • Let λ 1 , 0 , λ 1 , 1 , λ 2 , 0 , λ 2 , 1 , λ 3 , λ 4 , . . . , λ n P F Alexander Barg, University of Maryland Facets of the repair problem 29 / 45
Cooperative repair of two nodes • Assume that nodes C 1 , C 2 are erased. • We construct an p n , k , 3 q MDS array code, where k ă n ď | F | ´ 2 . • Let λ 1 , 0 , λ 1 , 1 , λ 2 , 0 , λ 2 , 1 , λ 3 , λ 4 , . . . , λ n P F • Parity-check equations: n λ t 1 , 0 c 1 , 0 ` λ t ÿ λ t 2 , 0 c 2 , 0 ` i c i , 0 “ 0 i “ 3 n ÿ λ t 1 , 1 c 1 , 1 ` λ t λ t 2 , 0 c 2 , 1 ` i c i , 1 “ 0 i “ 3 n ÿ λ t 1 , 0 c 1 , 2 ` λ t λ t 2 , 1 c 2 , 2 ` i c i , 2 “ 0 , t “ 0 , 1 , . . . , r ´ 1 i “ 3 Alexander Barg, University of Maryland Facets of the repair problem 29 / 45
Idea of the construction, I Take the first two groups of parities: n λ t 1 , 0 c 1 , 0 ` λ t ÿ λ t 2 , 0 c 2 , 0 ` i c i , 0 “ 0 i “ 3 n ÿ λ t 1 , 1 c 1 , 1 ` λ t λ t 2 , 0 c 2 , 1 ` i c i , 1 “ 0 , t “ 0 , 1 , . . . , r ´ 1 i “ 3 Alexander Barg, University of Maryland Facets of the repair problem 30 / 45
Idea of the construction, I Take the first two groups of parities: n λ t 1 , 0 c 1 , 0 ` λ t ÿ λ t 2 , 0 c 2 , 0 ` i c i , 0 “ 0 i “ 3 n ÿ λ t 1 , 1 c 1 , 1 ` λ t λ t 2 , 0 c 2 , 1 ` i c i , 1 “ 0 , t “ 0 , 1 , . . . , r ´ 1 i “ 3 Add them together: n λ t 1 , 0 c 1 , 0 ` λ t 1 , 1 c 1 , 1 ` λ t ÿ λ t 2 , 0 p c 2 , 0 ` c 2 , 1 q ` i p c i , 0 ` c i , 1 q “ 0 i “ 3 Alexander Barg, University of Maryland Facets of the repair problem 30 / 45
Idea of the construction, I Take the first two groups of parities: n λ t 1 , 0 c 1 , 0 ` λ t ÿ λ t 2 , 0 c 2 , 0 ` i c i , 0 “ 0 i “ 3 n ÿ λ t 1 , 1 c 1 , 1 ` λ t λ t 2 , 0 c 2 , 1 ` i c i , 1 “ 0 , t “ 0 , 1 , . . . , r ´ 1 i “ 3 Add them together: n λ t 1 , 0 c 1 , 0 ` λ t 1 , 1 c 1 , 1 ` λ t ÿ λ t 2 , 0 p c 2 , 0 ` c 2 , 1 q ` i p c i , 0 ` c i , 1 q “ 0 i “ 3 n ÿ λ t λ t 2 , 0 p c 2 , 0 ` c 2 , 1 q ` i p c i , 0 ` c i , 1 q “ 0 i “ 2 Alexander Barg, University of Maryland Facets of the repair problem 30 / 45
Recommend
More recommend
