Different facets of the repair problem Alexander Barg University of - - PowerPoint PPT Presentation

Different facets of the repair problem

Alexander Barg

University of Maryland, College Park LAWCI, July 2018

Overview

• 1. Motivation
• 2. Codes with hierarchical locality

LRC codes on curves Codes on curves with hierarchical locality

• 3. Regenerating codes

Regenerating codes and the repair problem

• 4. Cooperative repair

Centralized and cooperative models Cooperative repair of two nodes

• 5. Rack-aware storage model

(*) Repair of RS codes Repair bandwidth of RS codes

Alexander Barg, University of Maryland Facets of the repair problem 2 / 45

Motivation: Distributed Storage Systems (DSS)

• DSS spread data across thousands of storage nodes

Alexander Barg, University of Maryland Facets of the repair problem 3 / 45

Motivation: Distributed Storage Systems (DSS)

• DSS spread data across thousands of storage nodes
• Individual storage nodes fail frequently

Alexander Barg, University of Maryland Facets of the repair problem 3 / 45

Motivation: Distributed Storage Systems (DSS)

• DSS spread data across thousands of storage nodes
• Individual storage nodes fail frequently
• To protect the data we rely on erasure codes

Alexander Barg, University of Maryland Facets of the repair problem 3 / 45

Erasure codes for storage

• The file of size M is divided into l-vectors over a finite field Fq (chunks of size l).

Alexander Barg, University of Maryland Facets of the repair problem 4 / 45

Erasure codes for storage

• The file of size M is divided into l-vectors over a finite field Fq (chunks of size l).
• Each chunk is placed on a separate storage node

Alexander Barg, University of Maryland Facets of the repair problem 4 / 45

Erasure codes for storage

• The file of size M is divided into l-vectors over a finite field Fq (chunks of size l).
• Each chunk is placed on a separate storage node
• The data is encoded using an erasure-correcting code

Alexander Barg, University of Maryland Facets of the repair problem 4 / 45

Erasure codes for storage

• The file of size M is divided into l-vectors over a finite field Fq (chunks of size l).
• Each chunk is placed on a separate storage node
• The data is encoded using an erasure-correcting code
• Data encoding

C1, C2, C3, . . . , Cn´1, Cn where each Ci is located on its own node

Alexander Barg, University of Maryland Facets of the repair problem 4 / 45

Erasure codes for storage

• The file of size M is divided into l-vectors over a finite field Fq (chunks of size l).
• Each chunk is placed on a separate storage node
• The data is encoded using an erasure-correcting code
• Data encoding

C1, C2, C3, . . . , Cn´1, Cn where each Ci is located on its own node

• Inoperable (failed) node Ci in the encoding

C1 . . . Ci . . . Cn needs to be repaired to preserve the integrity of the data

Alexander Barg, University of Maryland Facets of the repair problem 4 / 45

Erasure codes for storage

• The file of size M is divided into l-vectors over a finite field Fq (chunks of size l).
• Each chunk is placed on a separate storage node
• The data is encoded using an erasure-correcting code
• Data encoding

C1, C2, C3, . . . , Cn´1, Cn where each Ci is located on its own node

• Inoperable (failed) node Ci in the encoding

C1 . . . Ci . . . Cn needs to be repaired to preserve the integrity of the data

• The repair task amounts to correcting a single erased coordinate in the encoding

Alexander Barg, University of Maryland Facets of the repair problem 4 / 45

Erasure codes for storage

• The file of size M is divided into l-vectors over a finite field Fq (chunks of size l).
• Each chunk is placed on a separate storage node
• The data is encoded using an erasure-correcting code
• Data encoding

C1, C2, C3, . . . , Cn´1, Cn where each Ci is located on its own node

• Inoperable (failed) node Ci in the encoding

C1 . . . Ci . . . Cn needs to be repaired to preserve the integrity of the data

• The repair task amounts to correcting a single erased coordinate in the encoding
• Repair involves transmitting the data between the nodes

Alexander Barg, University of Maryland Facets of the repair problem 4 / 45

Erasure codes for storage

• The file of size M is divided into l-vectors over a finite field Fq (chunks of size l).
• Each chunk is placed on a separate storage node
• The data is encoded using an erasure-correcting code
• Data encoding

C1, C2, C3, . . . , Cn´1, Cn where each Ci is located on its own node

• Inoperable (failed) node Ci in the encoding

C1 . . . Ci . . . Cn needs to be repaired to preserve the integrity of the data

• Efficient operation of the storage system depends on the amount of communication

between the nodes

Alexander Barg, University of Maryland Facets of the repair problem 4 / 45

Local Information Processing

Metric Operation

locality/ access bandwidth/ communication repair error correction

Alexander Barg, University of Maryland Facets of the repair problem 5 / 45

Local Information Processing

Metric Operation

locality/ access bandwidth/ communication repair error correction

1

1 Locally Recoverable codes (local recovery)

Alexander Barg, University of Maryland Facets of the repair problem 5 / 45

Local Information Processing

Metric Operation

locality/ access bandwidth/ communication repair error correction

1 2

1 Locally Recoverable codes (local recovery) 2 Regenerating codes (local recovery)

Alexander Barg, University of Maryland Facets of the repair problem 5 / 45

Local Information Processing

Metric Operation

locality/ access bandwidth/ communication repair error correction

1 2 3

1 Locally Recoverable codes (local recovery) 2 Regenerating codes (local recovery) 3 LDPC codes (global recovery)

Alexander Barg, University of Maryland Facets of the repair problem 5 / 45

Local Information Processing

Metric Operation

locality/ access bandwidth/ communication repair error correction

1 2 3 4

1 Locally Recoverable codes (local recovery) 2 Regenerating codes (local recovery) 3 LDPC codes (global recovery) 4 Fractional decoding (global recovery)

Alexander Barg, University of Maryland Facets of the repair problem 5 / 45

Local Information Processing

Metric Operation

locality/ access bandwidth/ communication repair error correction

1 2

In this talk we are interested in: 1 Locally Recoverable codes (local recovery) 2 Regenerating codes (local recovery)

Alexander Barg, University of Maryland Facets of the repair problem 5 / 45

Local erasure correction

• The encoding of k data symbols over a field Fq constitutes a codeword C of a linear

k-dimensional code C.

Alexander Barg, University of Maryland Facets of the repair problem 6 / 45

Local erasure correction

• The encoding of k data symbols over a field Fq constitutes a codeword C of a linear

k-dimensional code C.

• Coordinates Ci are written on different storage nodes

Alexander Barg, University of Maryland Facets of the repair problem 6 / 45

Local erasure correction

• The encoding of k data symbols over a field Fq constitutes a codeword C of a linear

k-dimensional code C.

• Coordinates Ci are written on different storage nodes
• Problem: Repair a failed coordinate by contacting as few, say r ! n, surviving nodes

as possible

Alexander Barg, University of Maryland Facets of the repair problem 6 / 45

Local erasure correction

• The encoding of k data symbols over a field Fq constitutes a codeword C of a linear

k-dimensional code C.

• Coordinates Ci are written on different storage nodes
• Problem: Repair a failed coordinate by contacting as few, say r ! n, surviving nodes

as possible

• Any parity-check equation that involves the failed coordinate can be used for repair

Alexander Barg, University of Maryland Facets of the repair problem 6 / 45

Local erasure correction

• The encoding of k data symbols over a field Fq constitutes a codeword C of a linear

k-dimensional code C.

• Coordinates Ci are written on different storage nodes
• Problem: Repair a failed coordinate by contacting as few, say r ! n, surviving nodes

as possible

• Any parity-check equation that involves the failed coordinate can be used for repair
• If every Ci, i “ 1, . . . , n is involved in a parity of weight r ` 1, the code has the locality

property

Alexander Barg, University of Maryland Facets of the repair problem 6 / 45

Local erasure correction

• The encoding of k data symbols over a field Fq constitutes a codeword C of a linear

k-dimensional code C.

• Coordinates Ci are written on different storage nodes
• Problem: Repair a failed coordinate by contacting as few, say r ! n, surviving nodes

as possible

• Any parity-check equation that involves the failed coordinate can be used for repair
• If every Ci, i “ 1, . . . , n is involved in a parity of weight r ` 1, the code has the locality

property What other features of the encoding are of interest?

Alexander Barg, University of Maryland Facets of the repair problem 6 / 45

Locally recoverable codes

In addition to correcting one erasure, sometimes the code is used to recover the data from a multi-node failure. Global correction: repairing a large number of failed nodes by contacting all the remaining coordinates Problem: What is largest possible minimum distance of an LRC code C?

GOPALAN ET AL. bound:

d ď n ´ k ´ Qk r U ` 2

Alexander Barg, University of Maryland Facets of the repair problem 7 / 45

Variants of LRC codes

• A linear code is LRC with locality r if every coordinate is involved in a parity check of

weight r ` 1 For every i P t1, 2, . . . , nu there exists a punctured code Ci :“ C|tiuYIi such that

• |Ii| “ r
• dimpCiq ď r
• distance dpCiq “ 2.
• Local correction of s “ 2, 3, ... erasures:
• A linear code C is LRC with locality pr, ρq if every coordinate is contained in a “local

code” of distance ρ This means that we can locally repair up to ρ ´ 1 erasures by contacting r helper nodes The distance of C is bounded above as follows: d ď n ´ k ` 1 ´ ´Qk r U ´ 1 ¯ pρ ´ 1q

Alexander Barg, University of Maryland Facets of the repair problem 8 / 45

Codes with hierarchical locality

An pn, k, dq linear code with two levels of locality

Alexander Barg, University of Maryland Facets of the repair problem 9 / 45

Codes with hierarchical locality

An pn, k, dq linear code with two levels of locality

• Repair a single node by querying r2 helper nodes;
• Repair ρ ´ 1 nodes by querying r1 helper nodes;
• Repair d ´ 1 nodes by addressing all the remaining n ´ pd ´ 1q nodes

Alexander Barg, University of Maryland Facets of the repair problem 9 / 45

Codes with hierarchical locality

An pn, k, dq linear code with two levels of locality

• Repair a single node by querying r2 helper nodes;
• Repair ρ ´ 1 nodes by querying r1 helper nodes;
• Repair d ´ 1 nodes by addressing all the remaining n ´ pd ´ 1q nodes

Defintion: Let ρ1 ą 2 and r2 ď r1. A linear code C is H-LRC and parameters ppr1, ρ1q, pr2, 2qq if for every i P t1, . . . , nu there is a punctured code Ci such that

• 1. dimpCiq ď r1,
• 2. dpCiq ě ρ, and
• 3. Ci is an pr2, 2q LRC code.

Alexander Barg, University of Maryland Facets of the repair problem 9 / 45

Reed-Solomon codes

F “ Fq the finite field of q elements Example: q “ 8, α3 “ α ` 1

F8 “ t0, 1, α, α2, α3 “ α ` 1, α4 “ α2 ` α, α5 “ α2 ` α ` 1, α6 “ α2 ` 1u

The elements of F8 can also be written as binary vectors p000q, p001q, p010q, p101q, p110q, p111q, p101q Encoding example: Let q “ 8, pn, kq “ p7, 3q Suppose that data symbols are 1, α, α To encode, form a polynomial fpxq “ 1 ` αx ` αx2 and compute the values of fpxq at the points 1, α, α2, . . . , α6 x 1 α α2 α3 α4 α5 α6 fpxq 1 α4 α6 α4 α α α6 This encodes k “ 3 data symbols p1, α, αq into n “ 7 symbols of the codeword

Alexander Barg, University of Maryland Facets of the repair problem 10 / 45

Evaluation codes

Alexander Barg, University of Maryland Facets of the repair problem 11 / 45

Evaluation codes

Given a polynomial f P Fqrxs and a set A “ tP1, . . . , Pnu Ă Fq define the map evA : f ÞÑ pfpPiq, i “ 1, . . . , nq

Alexander Barg, University of Maryland Facets of the repair problem 11 / 45

Evaluation codes

Given a polynomial f P Fqrxs and a set A “ tP1, . . . , Pnu Ă Fq define the map evA : f ÞÑ pfpPiq, i “ 1, . . . , nq Example: Let q “ 8, fpxq “ 1 ` αx ` αx2 fpxq ÞÑ p1, α4, α6, α4, α, α, α6q

Alexander Barg, University of Maryland Facets of the repair problem 11 / 45

Evaluation codes

Given a polynomial f P Fqrxs and a set A “ tP1, . . . , Pnu Ă Fq define the map evA : f ÞÑ pfpPiq, i “ 1, . . . , nq Example: Let q “ 8, fpxq “ 1 ` αx ` αx2 fpxq ÞÑ p1, α4, α6, α4, α, α, α6q Evaluation code CpAq Let V “ tf P Fqrxsu be a set of polynomials, dimpVq “ k C : V Ñ Fn

q

f ÞÑ evApfq “ pfpPiq, i “ 1, . . . , nq

Alexander Barg, University of Maryland Facets of the repair problem 11 / 45

Construction of pn, k, rq LRC codes: Example

Parameters: n “ 12, k “ 6, r “ 3, q “ 13; Set of points: A “ t1, . . . , 12u Ă F13 A “ tA1 “ p1, 5, 12, 8q, A2 “ p2, 10, 11, 3q, A3 “ p4, 7, 9, 6qu Basis of functions: Take gpxq constant on Ai, i “ 1, 2, 3, e.g., gpxq “ x4 ´ 1 V “ A gpxqjxi, i “ 0, 1, 2; j “ 0, 1 E ; dimpVq “ 6 Evaluation code C : V Ñ Fn

q is an LRC code of length |A| “ 12 with r “ 3 and d “ 6

Alexander Barg, University of Maryland Facets of the repair problem 12 / 45

Construction of pn, k, rq LRC codes: Example

Parameters: n “ 12, k “ 6, r “ 3, q “ 13; Set of points: A “ t1, . . . , 12u Ă F13 A “ tA1 “ p1, 5, 12, 8q, A2 “ p2, 10, 11, 3q, A3 “ p4, 7, 9, 6qu Basis of functions: Take gpxq constant on Ai, i “ 1, 2, 3, e.g., gpxq “ x4 ´ 1 V “ A gpxqjxi, i “ 0, 1, 2; j “ 0, 1 E ; dimpVq “ 6 Evaluation code C : V Ñ Fn

q is an LRC code of length |A| “ 12 with r “ 3 and d “ 6

‚ This construction is general, and gives distance-optimal LRC codes which form certain subcodes of RS codes

Alexander Barg, University of Maryland Facets of the repair problem 12 / 45

Geometric view of LRC codes

A “ t1, . . . , 12u Ă F13 A “ A1 Y A2 Y A3

A1 “ p1, 5, 12, 8q A2 “ p2, 10, 11, 3q A3 “ p4, 7, 9, 6q

Alexander Barg, University of Maryland Facets of the repair problem 13 / 45

Geometric view of LRC codes

A “ t1, . . . , 12u Ă F13 A “ A1 Y A2 Y A3

A1 “ p1, 5, 12, 8q A2 “ p2, 10, 11, 3q A3 “ p4, 7, 9, 6q

g :A Ñ F13 x ÞÑ x4 ´ 1

Alexander Barg, University of Maryland Facets of the repair problem 13 / 45

Geometric view of LRC codes

A “ t1, . . . , 12u Ă F13 A “ A1 Y A2 Y A3

A1 “ p1, 5, 12, 8q A2 “ p2, 10, 11, 3q A3 “ p4, 7, 9, 6q

g :A Ñ F13 x ÞÑ x4 ´ 1 g :F13 Ñ t0, 2, 8u Ă F13 |g´1pyq| “ r ` 1

Alexander Barg, University of Maryland Facets of the repair problem 13 / 45

Geometric view of LRC codes

A “ t1, . . . , 12u Ă F13 A “ A1 Y A2 Y A3

A1 “ p1, 5, 12, 8q A2 “ p2, 10, 11, 3q A3 “ p4, 7, 9, 6q

g :A Ñ F13 x ÞÑ x4 ´ 1 g :F13 Ñ t0, 2, 8u Ă F13 |g´1pyq| “ r ` 1

In the basic construction, X “ Y “ P1

Alexander Barg, University of Maryland Facets of the repair problem 13 / 45

LRC codes on curves

Consider the set of pairs px, yq P F9 that satisfy the equation x3 ` x “ y4 α7 ‚ ‚ ‚ ‚ α6 ‚ α5 ‚ ‚ ‚ ‚ α4 ‚ ‚ ‚ ‚ x α3 ‚ ‚ ‚ ‚ α2 ‚ α ‚ ‚ ‚ ‚ 1 ‚ ‚ ‚ ‚ 0 ‚ 0 1 α α2 α3 α4 α5 α6 α7 y

Alexander Barg, University of Maryland Facets of the repair problem 14 / 45

LRC codes on curves

Consider the set of pairs px, yq P F9 that satisfy the equation x3 ` x “ y4 α7 ‚ ‚ ‚ ‚ α6 ‚ α5 ‚ ‚ ‚ ‚ α4 ‚ ‚ ‚ ‚ x α3 ‚ ‚ ‚ ‚ α2 ‚ α ‚ ‚ ‚ ‚ 1 ‚ ‚ ‚ ‚ 0 ‚ 0 1 α α2 α3 α4 α5 α6 α7 y Affine points of the Hermitian curve X over F9; α2 “ α ` 1

Alexander Barg, University of Maryland Facets of the repair problem 14 / 45

Hermitian codes

g : X Ñ P1 px, yq ÞÑ y Space of functions V :“ x1, y, y2, x, xy, xy2y A={Affine points of the Hermitian curve over F9}; n “ 27, k “ 6 C : V Ñ Fn

9

Alexander Barg, University of Maryland Facets of the repair problem 15 / 45

Hermitian codes

g : X Ñ P1 px, yq ÞÑ y Space of functions V :“ x1, y, y2, x, xy, xy2y A={Affine points of the Hermitian curve over F9}; n “ 27, k “ 6 C : V Ñ Fn

9

E.g., message p1, α, α2, α3, α4, α5q Fpx, yq “ 1 ` αy ` α2y2 ` α3x ` α4xy ` α5xy2 Fp0, 0q “ 1 etc.

Alexander Barg, University of Maryland Facets of the repair problem 15 / 45

LRC codes on algebraic curves: General construction

Alexander Barg, University of Maryland Facets of the repair problem 16 / 45

LRC codes on algebraic curves: General construction

• φ : X Ñ Y degree-pr ` 1q separable morphism of curves X, Y over K “ Fq

Alexander Barg, University of Maryland Facets of the repair problem 16 / 45

LRC codes on algebraic curves: General construction

• φ : X Ñ Y degree-pr ` 1q separable morphism of curves X, Y over K “ Fq
• Q1, Q2, . . . , Qs P YpKq split completely in the cover X Ñ Y

Alexander Barg, University of Maryland Facets of the repair problem 16 / 45

LRC codes on algebraic curves: General construction

• φ : X Ñ Y degree-pr ` 1q separable morphism of curves X, Y over K “ Fq
• Q1, Q2, . . . , Qs P YpKq split completely in the cover X Ñ Y
• Let Pi,j, j “ 1, . . . , r ` 1 be the set of points on XpKq above Qi P YpKq

Alexander Barg, University of Maryland Facets of the repair problem 16 / 45

LRC codes on algebraic curves: General construction

• φ : X Ñ Y degree-pr ` 1q separable morphism of curves X, Y over K “ Fq
• Q1, Q2, . . . , Qs P YpKq split completely in the cover X Ñ Y
• Let Pi,j, j “ 1, . . . , r ` 1 be the set of points on XpKq above Qi P YpKq
• φ˚ : KpYq ã

Ñ KpXq

Alexander Barg, University of Maryland Facets of the repair problem 16 / 45

LRC codes on algebraic curves: General construction

• φ : X Ñ Y degree-pr ` 1q separable morphism of curves X, Y over K “ Fq
• Q1, Q2, . . . , Qs P YpKq split completely in the cover X Ñ Y
• Let Pi,j, j “ 1, . . . , r ` 1 be the set of points on XpKq above Qi P YpKq
• φ˚ : KpYq ã

Ñ KpXq

• e1, . . . er P KpXq linearly independent over KpYq and such that Pi,j R supppelq8

Alexander Barg, University of Maryland Facets of the repair problem 16 / 45

LRC codes on algebraic curves: General construction

• φ : X Ñ Y degree-pr ` 1q separable morphism of curves X, Y over K “ Fq
• Q1, Q2, . . . , Qs P YpKq split completely in the cover X Ñ Y
• Let Pi,j, j “ 1, . . . , r ` 1 be the set of points on XpKq above Qi P YpKq
• φ˚ : KpYq ã

Ñ KpXq

• e1, . . . er P KpXq linearly independent over KpYq and such that Pi,j R supppelq8
• f1, . . . , ft P KpYq l.i. over K and Ql R supppfjq8

Alexander Barg, University of Maryland Facets of the repair problem 16 / 45

LRC codes on algebraic curves: General construction

• φ : X Ñ Y degree-pr ` 1q separable morphism of curves X, Y over K “ Fq
• Q1, Q2, . . . , Qs P YpKq split completely in the cover X Ñ Y
• Let Pi,j, j “ 1, . . . , r ` 1 be the set of points on XpKq above Qi P YpKq
• φ˚ : KpYq ã

Ñ KpXq

• e1, . . . er P KpXq linearly independent over KpYq and such that Pi,j R supppelq8
• f1, . . . , ft P KpYq l.i. over K and Ql R supppfjq8
• Given the data a “ pau,v, 1 ď u ď s; 0 ď v ď rq P Kpr`1qs, evaluate

fa :“

r

ÿ

u“1

eu ÿ

v“1

au,vφ˚fv at each of the points Pij

Alexander Barg, University of Maryland Facets of the repair problem 16 / 45

LRC codes on algebraic curves: Main results

• A general construction of LRC codes from covering maps of curves (including quotient

curves, fiber products)

• Families of LRC codes of length n ě q
• Infinite families of codes from the Garcia-Stichtenoth tower
• Asymptotic tradeoff between the rate and distance better than the Gilbert-Varshamov

bound

• Codes with locality on algebraic surfaces

Alexander Barg, University of Maryland Facets of the repair problem 17 / 45

Codes on curves with hierarchical locality (H-LRC codes)

Consider a code C over the field Fq with distance d

Alexander Barg, University of Maryland Facets of the repair problem 18 / 45

Codes on curves with hierarchical locality (H-LRC codes)

Consider a code C over the field Fq with distance d Each coordinate i is included in a rν, r1, ρs code C1 which is also an LRC code with locality r2, where 2 ă ρ ă d

Alexander Barg, University of Maryland Facets of the repair problem 18 / 45

Codes on curves with hierarchical locality (H-LRC codes)

Consider a code C over the field Fq with distance d Each coordinate i is included in a rν, r1, ρs code C1 which is also an LRC code with locality r2, where 2 ă ρ ă d Flexible functionality:

• A single node failure is repaired by contacting r2 nodes
• Up to ρ ´ 1 failures are repaired by contacting ν helper nodes
• Up to d ´ 1 failures can be corrected by contacting all the functional nodes in the encoding

Alexander Barg, University of Maryland Facets of the repair problem 18 / 45

Codes on curves with hierarchical locality (H-LRC codes)

Consider a code C over the field Fq with distance d Each coordinate i is included in a rν, r1, ρs code C1 which is also an LRC code with locality r2, where 2 ă ρ ă d Flexible functionality:

• A single node failure is repaired by contacting r2 nodes
• Up to ρ ´ 1 failures are repaired by contacting ν helper nodes
• Up to d ´ 1 failures can be corrected by contacting all the functional nodes in the encoding

The Gopalan et al. bound extends as follows: d ď n ´ k ` 1 ´ ´Q k r2 U ´ 1 ¯ ´ ´Q k r1 U ´ 1 ¯ pρ ´ 2q

(SASIDHARAN-AGARWAL-KUMAR, ’15)

Alexander Barg, University of Maryland Facets of the repair problem 18 / 45

Codes on curves with hierarchical locality (H-LRC codes)

• Consider the following sequence of maps of algebraic curves:

X

φ2

Ý Ñ Y

φ1

Ý Ñ Z where deg φ1 “ r2 ` 1; deg φ2 “ s ` 1; define ψ “ φ1 ˝ φ2

• Let KpXq “ KpYqpxq and KpYq “ KpZqpyq The covering map gives rise to the

embedding of function fields: KpXq Ą KpYq Ą KpZq

• Let S “ tP1, . . . , Pmu be a collection of points in ZpKq that split completely on X, i.e.,

|ψ´1pPiq| “ pr2 ` 1qps ` 1q

• The codes are defined by evaluating the functions

tfiyjxk|1 ď i ď t, 0 ď j ď s ´ 1, 0 ď k ď r2 ´ 1u at the points P1, i “ 1, . . . , m, where f1, . . . , ft form a basis for LpQ8q

Alexander Barg, University of Maryland Facets of the repair problem 19 / 45

Example:

Let m|pq ´ 1q and consider X given by ym “ fpxq

Alexander Barg, University of Maryland Facets of the repair problem 20 / 45

Example:

Let m|pq ´ 1q and consider X given by ym “ fpxq

• L :“ Fqpx, yq is a cyclic extension of K :“ Fqpxq of degree m. G “ GalpL{Kq is cyclic of
• rder m

Alexander Barg, University of Maryland Facets of the repair problem 20 / 45

Example:

Let m|pq ´ 1q and consider X given by ym “ fpxq

• L :“ Fqpx, yq is a cyclic extension of K :“ Fqpxq of degree m. G “ GalpL{Kq is cyclic of
• rder m
• Let m “ pa ` 1qpb ` 1q, let α be the generator of G

Alexander Barg, University of Maryland Facets of the repair problem 20 / 45

Example:

Let m|pq ´ 1q and consider X given by ym “ fpxq

• L :“ Fqpx, yq is a cyclic extension of K :“ Fqpxq of degree m. G “ GalpL{Kq is cyclic of
• rder m
• Let m “ pa ` 1qpb ` 1q, let α be the generator of G
• Let H “ xαb`1y, |H| “ a ` 1

Alexander Barg, University of Maryland Facets of the repair problem 20 / 45

Example:

Let m|pq ´ 1q and consider X given by ym “ fpxq

• L :“ Fqpx, yq is a cyclic extension of K :“ Fqpxq of degree m. G “ GalpL{Kq is cyclic of
• rder m
• Let m “ pa ` 1qpb ` 1q, let α be the generator of G
• Let H “ xαb`1y, |H| “ a ` 1
• Invariants of H are generated by ya`1

Alexander Barg, University of Maryland Facets of the repair problem 20 / 45

Example:

Let m|pq ´ 1q and consider X given by ym “ fpxq

• L :“ Fqpx, yq is a cyclic extension of K :“ Fqpxq of degree m. G “ GalpL{Kq is cyclic of
• rder m
• Let m “ pa ` 1qpb ` 1q, let α be the generator of G
• Let H “ xαb`1y, |H| “ a ` 1
• Invariants of H are generated by ya`1
• KpXq “ Kpx, yq Ð

â KpXqH “ Kpx, ya`1q Ð â KpXqG “ Kpx, ypa`1qpb`1qq

Alexander Barg, University of Maryland Facets of the repair problem 20 / 45

Example:

Let m|pq ´ 1q and consider X given by ym “ fpxq

• L :“ Fqpx, yq is a cyclic extension of K :“ Fqpxq of degree m. G “ GalpL{Kq is cyclic of
• rder m
• Let m “ pa ` 1qpb ` 1q, let α be the generator of G
• Let H “ xαb`1y, |H| “ a ` 1
• Invariants of H are generated by ya`1
• KpXq “ Kpx, yq Ð

â KpXqH “ Kpx, ya`1q Ð â KpXqG “ Kpx, ypa`1qpb`1qq

• We obtain families of H-LRC codes by specializing this construction to various

Kummer curves (Hermitian, Giulietti-Korchm´ aros, etc.)

(work with SEAN BALLENTINE and SERGE VL˘

ADUT

¸, arXiv.org:1807.05473)

Alexander Barg, University of Maryland Facets of the repair problem 20 / 45

Regenerating Codes and the Repair Problem

• The file of size M is divided into l-vectors over a finite field F “ Fq (chunks of size l).

Alexander Barg, University of Maryland Facets of the repair problem 21 / 45

Regenerating Codes and the Repair Problem

• The file of size M is divided into l-vectors over a finite field F “ Fq (chunks of size l).
• Each chunk is placed on a separate storage node

Alexander Barg, University of Maryland Facets of the repair problem 21 / 45

Regenerating Codes and the Repair Problem

• The file of size M is divided into l-vectors over a finite field F “ Fq (chunks of size l).
• Each chunk is placed on a separate storage node
• The data is encoded using an erasure-correcting code of dimension k “ M{l

Alexander Barg, University of Maryland Facets of the repair problem 21 / 45

Regenerating Codes and the Repair Problem

• The file of size M is divided into l-vectors over a finite field F “ Fq (chunks of size l).
• Each chunk is placed on a separate storage node
• The data is encoded using an erasure-correcting code of dimension k “ M{l
• Data encoding

C1, C2, C3, . . . , Cn´1, Cn where each Ci is located on its own node Another figure of merit: The repair bandwidth, i.e., the total amount of communication for repairing the failed node(s)

Alexander Barg, University of Maryland Facets of the repair problem 21 / 45

Regenerating Codes and the Repair Problem

• Let C be a code over F used for node repair (correcting one or several erasures)

Alexander Barg, University of Maryland Facets of the repair problem 22 / 45

Regenerating Codes and the Repair Problem

• Let C be a code over F used for node repair (correcting one or several erasures)
• Let B be a subfield of F; rF : Bs “ l

Alexander Barg, University of Maryland Facets of the repair problem 22 / 45

Regenerating Codes and the Repair Problem

• Let C be a code over F used for node repair (correcting one or several erasures)
• Let B be a subfield of F; rF : Bs “ l
• Consider C as a code over B; every coordinate is an l-vector over B

Alexander Barg, University of Maryland Facets of the repair problem 22 / 45

Regenerating Codes and the Repair Problem

• Let C be a code over F used for node repair (correcting one or several erasures)
• Let B be a subfield of F; rF : Bs “ l
• Consider C as a code over B; every coordinate is an l-vector over B
• Repair is performed by downloading symbols from helper nodes

l n k

Alexander Barg, University of Maryland Facets of the repair problem 22 / 45

Regenerating Codes and the Repair Problem

• Let C be a code over F used for node repair (correcting one or several erasures)
• Let B be a subfield of F; rF : Bs “ l
• Consider C as a code over B; every coordinate is an l-vector over B
• Repair is performed by downloading symbols from helper nodes

l n k

Alexander Barg, University of Maryland Facets of the repair problem 22 / 45

Regenerating Codes and the Repair Problem

• Let C be a code over F used for node repair (correcting one or several erasures)
• Let B be a subfield of F; rF : Bs “ l
• Consider C as a code over B; every coordinate is an l-vector over B
• Repair is performed by downloading symbols from helper nodes

l n d

Alexander Barg, University of Maryland Facets of the repair problem 22 / 45

Regenerating Codes and the Repair Problem

• Let C be a code over F used for node repair (correcting one or several erasures)
• Let B be a subfield of F; rF : Bs “ l
• Consider C as a code over B; every coordinate is an l-vector over B
• Repair is performed by downloading symbols from helper nodes

l n d=n−1

Repair Bandwidth – The number of symbols of B downloaded for node repair

Alexander Barg, University of Maryland Facets of the repair problem 22 / 45

Vector (Array) codes

• Let C be a code over the field F “ Fql.
• Each coordinate can be considered as an l-vector over B “ Fq.
• A codeword C P C is an l ˆ n matrix over B.
• The value of l is called sub-packetization of the code C
• C is called a linear array code (or a vector code) if it is B-linear. It may not be F-linear;

if it is, it is also called a scalar code.

Alexander Barg, University of Maryland Facets of the repair problem 23 / 45

Formal definition of the (single-node) repair problem

• Consider an pn, k, lq code C over B.

Alexander Barg, University of Maryland Facets of the repair problem 24 / 45

Formal definition of the (single-node) repair problem

• Consider an pn, k, lq code C over B.
• A codeword C “ pC1, . . . , Cnq, where Ci “ pci,0, ci,1, . . . , ci,l´1qT P Bl, i “ 1, . . . , n.

Alexander Barg, University of Maryland Facets of the repair problem 24 / 45

Formal definition of the (single-node) repair problem

• Consider an pn, k, lq code C over B.
• A codeword C “ pC1, . . . , Cnq, where Ci “ pci,0, ci,1, . . . , ci,l´1qT P Bl, i “ 1, . . . , n.
• A node i P rns can be repaired from a subset of d ě k helper nodes Ri Ă rnsztiu,

by downloading βipRiq symbols of B if there are

• numbers βi,j, j P Ri and
• d functions fi,j : Bl Ñ Bβi,j, j P Ri and a function gi : B

ř

j βi,j Ñ Bl

such that Ci “ gipfi,jpCjq, j P Riq and ÿ

jPRi

βi,j “ βipRiq.

Alexander Barg, University of Maryland Facets of the repair problem 24 / 45

Formal definition of the (single-node) repair problem

• Consider an pn, k, lq code C over B.
• A codeword C “ pC1, . . . , Cnq, where Ci “ pci,0, ci,1, . . . , ci,l´1qT P Bl, i “ 1, . . . , n.
• A node i P rns can be repaired from a subset of d ě k helper nodes Ri Ă rnsztiu,

by downloading βipRiq symbols of B if there are

• numbers βi,j, j P Ri and
• d functions fi,j : Bl Ñ Bβi,j, j P Ri and a function gi : B

ř

j βi,j Ñ Bl

such that Ci “ gipfi,jpCjq, j P Riq and ÿ

jPRi

βi,j “ βipRiq. The repair bandwidth of i from Ri :

β˚

i pRiq “ min fi,j,gi βipRiq

Alexander Barg, University of Maryland Facets of the repair problem 24 / 45

Repair of several erasures

Centralized and distributed (cooperative) models

Suppose that nodes i and j are erased.

Alexander Barg, University of Maryland Facets of the repair problem 25 / 45

Repair of several erasures

Centralized and distributed (cooperative) models

Suppose that nodes i and j are erased. Centralized repair: Download information from the set of helper nodes R, |R| “ d that is used for repair of both Ci and Cj

Alexander Barg, University of Maryland Facets of the repair problem 25 / 45

Repair of several erasures

Centralized and distributed (cooperative) models

Suppose that nodes i and j are erased. Centralized repair: Download information from the set of helper nodes R, |R| “ d that is used for repair of both Ci and Cj Cooperative repair1q:

• Round 1: Nodes Ci and Cj download (potentially, different) information from R
• Round 2: Information exchange:

Ci Ô Cj Both rounds of communication contribute to the repair bandwidth.

1q Originally defined for T ě 2 communication rounds (SHUM-HU, ’13)

Alexander Barg, University of Maryland Facets of the repair problem 25 / 45

Cut-set bound

β ě l d ` 1 ´ k d (DIMAKIS ET AL., 2010)

Alexander Barg, University of Maryland Facets of the repair problem 26 / 45

Cut-set bound

β ě l d ` 1 ´ k d (DIMAKIS ET AL., 2010) The code meeting this bound with equality is said to afford optimal repair For d “ n ´ 1, r “ n ´ k β ě l r pn ´ 1q

Alexander Barg, University of Maryland Facets of the repair problem 26 / 45

Cut-set bound

β ě l d ` 1 ´ k d (DIMAKIS ET AL., 2010) The code meeting this bound with equality is said to afford optimal repair For d “ n ´ 1, r “ n ´ k β ě l r pn ´ 1q The cut-set bound extends to repair of h ě 1 erasures (failed nodes):

• Centralized model: β ě

hdl d ` h ´ k

(V. CADAMBE ET AL., ’13)

• Cooperative model: β ě hpd ` h ´ 1ql

d ` h ´ k

(K. SHUM and Y. HU, ’13; M. YE and A.B., ’17)

Alexander Barg, University of Maryland Facets of the repair problem 26 / 45

Cut-set bound

β ě l d ` 1 ´ k d (DIMAKIS ET AL., 2010) The code meeting this bound with equality is said to afford optimal repair For d “ n ´ 1, r “ n ´ k β ě l r pn ´ 1q The cut-set bound extends to repair of h ě 1 erasures (failed nodes):

• Centralized model: β ě

hdl d ` h ´ k

(V. CADAMBE ET AL., ’13)

• Cooperative model: β ě hpd ` h ´ 1ql

d ` h ´ k

(K. SHUM and Y. HU, ’13; M. YE and A.B., ’17)

Codes that meet these bounds with equality are said to have ph, dq-optimal repair bandwidth

Alexander Barg, University of Maryland Facets of the repair problem 26 / 45

Cooperative repair

Cut-set bound for cooperative repair: β ě |F|p|R| ` |F| ´ 1ql |F| ` |R| ´ k “ |F| ´ |R|l |F| ` |R| ´ k ` p|F| ´ 1ql |F| ` |R| ´ k ¯

Alexander Barg, University of Maryland Facets of the repair problem 27 / 45

Cooperative repair

Cut-set bound for cooperative repair: β ě |F|p|R| ` |F| ´ 1ql |F| ` |R| ´ k “ |F| ´ |R|l |F| ` |R| ´ k ` p|F| ´ 1ql |F| ` |R| ´ k ¯ Structure of optimal codes:

• Each failed node downloads

l |F| ` |R| ´ k from the helper nodes

• Each failed node downloads

l |F| ` |R| ´ k from each of the other nodes in F

Alexander Barg, University of Maryland Facets of the repair problem 27 / 45

Cooperative repair model is stronger than the centralized model

An MSD code that is cooperatively optimal-repair is also optimal-repair under the centralized model

Alexander Barg, University of Maryland Facets of the repair problem 28 / 45

Cooperative repair model is stronger than the centralized model

An MSD code that is cooperatively optimal-repair is also optimal-repair under the centralized model

Theorem (Ye-B, ’18)

Let C be an pn, k, lq MDS array code and let F, R Ď rns be two disjoint subsets such that |F| ď r and |R| ě k. If βcooppCq “ |F|p|R| ` |F| ´ 1ql |F| ` |R| ´ k , then βcentpCq “ |F||R|l |F| ` |R| ´ k .

Alexander Barg, University of Maryland Facets of the repair problem 28 / 45

Cooperative repair of two nodes

• Assume that nodes C1, C2 are erased.

Alexander Barg, University of Maryland Facets of the repair problem 29 / 45

Cooperative repair of two nodes

• Assume that nodes C1, C2 are erased.
• We construct an pn, k, 3q MDS array code, where k ă n ď |F| ´ 2.

Alexander Barg, University of Maryland Facets of the repair problem 29 / 45

Cooperative repair of two nodes

• Assume that nodes C1, C2 are erased.
• We construct an pn, k, 3q MDS array code, where k ă n ď |F| ´ 2.
• Let λ1,0, λ1,1, λ2,0, λ2,1, λ3, λ4, . . . , λn P F

Alexander Barg, University of Maryland Facets of the repair problem 29 / 45

Cooperative repair of two nodes

• Assume that nodes C1, C2 are erased.
• We construct an pn, k, 3q MDS array code, where k ă n ď |F| ´ 2.
• Let λ1,0, λ1,1, λ2,0, λ2,1, λ3, λ4, . . . , λn P F
• Parity-check equations:

λt

1,0c1,0 ` λt 2,0c2,0 ` n

ÿ

i“3

λt

ici,0 “ 0

λt

1,1c1,1 ` λt 2,0c2,1 ` n

ÿ

i“3

λt

ici,1 “ 0

λt

1,0c1,2 ` λt 2,1c2,2 ` n

ÿ

i“3

λt

ici,2 “ 0,

t “ 0, 1, . . . , r ´ 1

Alexander Barg, University of Maryland Facets of the repair problem 29 / 45

Idea of the construction, I

Take the first two groups of parities: λt

1,0c1,0 ` λt 2,0c2,0 ` n

ÿ

i“3

λt

ici,0 “ 0

λt

1,1c1,1 ` λt 2,0c2,1 ` n

ÿ

i“3

λt

ici,1 “ 0,

t “ 0, 1, . . . , r ´ 1

Alexander Barg, University of Maryland Facets of the repair problem 30 / 45

Idea of the construction, I

Take the first two groups of parities: λt

1,0c1,0 ` λt 2,0c2,0 ` n

ÿ

i“3

λt

ici,0 “ 0

λt

1,1c1,1 ` λt 2,0c2,1 ` n

ÿ

i“3

λt

ici,1 “ 0,

t “ 0, 1, . . . , r ´ 1 Add them together: λt

1,0c1,0 ` λt 1,1c1,1 ` λt 2,0pc2,0 ` c2,1q ` n

ÿ

i“3

λt

ipci,0 ` ci,1q “ 0

Alexander Barg, University of Maryland Facets of the repair problem 30 / 45

Idea of the construction, I

Take the first two groups of parities: λt

1,0c1,0 ` λt 2,0c2,0 ` n

ÿ

i“3

λt

ici,0 “ 0

λt

1,1c1,1 ` λt 2,0c2,1 ` n

ÿ

i“3

λt

ici,1 “ 0,

t “ 0, 1, . . . , r ´ 1 Add them together: λt

1,0c1,0 ` λt 1,1c1,1 ` λt 2,0pc2,0 ` c2,1q ` n

ÿ

i“3

λt

ipci,0 ` ci,1q “ 0

λt

2,0pc2,0 ` c2,1q ` n

ÿ

i“2

λt

ipci,0 ` ci,1q “ 0

Alexander Barg, University of Maryland Facets of the repair problem 30 / 45

Idea of the construction, II

λt

2,0pc2,0 ` c2,1q ` n

ÿ

i“2

λt

ipci,0 ` ci,1q “ 0

In matrix form: » — — — — — — — – 1 1 1 1 1 . . . 1 λ1,0 λ1,1 λ2,0 λ3 λ4 . . . λn λ2

1,0

λ2

1,1

λ2

2,0

λ2

3

λ2

4

. . . λ2

n

. . . . . . . . . . . . . . . . . . . . . λr´1

1,0

λr´1

1,1

λr´1

2,0

λr´1

3

λr´1

4

. . . λr´1

n

fi ffi ffi ffi ffi ffi ffi ffi fl » — — — — — — — — — – c1,0 c1,1 c2,0 ` c2,1 c3,0 ` c3,1 . . . cn`1,0 ` cn`1,1 fi ffi ffi ffi ffi ffi ffi ffi ffi ffi fl “ 0 The column vector is a codeword in a GRS code of length n, dimension n ´ r “ k ` 1. Thus, c1,0, c1,1 and c2,0 ` c2,1 can be found from any k ` 1 values out of c3,0 ` c3,1, . . . , cn`1,0 ` cn`1,1

Alexander Barg, University of Maryland Facets of the repair problem 31 / 45

Idea of the construction, II

c1,0, c1,1 and c2,0 ` c2,1 can be found from any k ` 1 values c3,0 ` c3,1, . . . , cn`1,0 ` cn`1,1

Alexander Barg, University of Maryland Facets of the repair problem 32 / 45

Idea of the construction, II

c1,0, c1,1 and c2,0 ` c2,1 can be found from any k ` 1 values c3,0 ` c3,1, . . . , cn`1,0 ` cn`1,1 Similarly, taking the first and the third groups of parity checks, we conclude that c2,0, c2,2 and c1,0 ` c1,2 can be found from any k ` 1 values out of c3,0 ` c3,2, . . . , cn`1,0 ` cn`1,2

Alexander Barg, University of Maryland Facets of the repair problem 32 / 45

Idea of the construction, II

c1,0, c1,1 and c2,0 ` c2,1 can be found from any k ` 1 values c3,0 ` c3,1, . . . , cn`1,0 ` cn`1,1 Similarly, taking the first and the third groups of parity checks, we conclude that c2,0, c2,2 and c1,0 ` c1,2 can be found from any k ` 1 values out of c3,0 ` c3,2, . . . , cn`1,0 ` cn`1,2 The repair protocol

• 1. Download c3,0 ` c3,1, . . . , ck`3,0 ` ck`3,1 to Node 1; find c1,0, c1,1 and c2,0 ` c2,1

Alexander Barg, University of Maryland Facets of the repair problem 32 / 45

Idea of the construction, II

c1,0, c1,1 and c2,0 ` c2,1 can be found from any k ` 1 values c3,0 ` c3,1, . . . , cn`1,0 ` cn`1,1 Similarly, taking the first and the third groups of parity checks, we conclude that c2,0, c2,2 and c1,0 ` c1,2 can be found from any k ` 1 values out of c3,0 ` c3,2, . . . , cn`1,0 ` cn`1,2 The repair protocol

• 1. Download c3,0 ` c3,1, . . . , ck`3,0 ` ck`3,1 to Node 1; find c1,0, c1,1 and c2,0 ` c2,1
• 2. Download c3,0 ` c3,2, . . . , ck`3,0 ` ck`3,2 to Node 2; find c2,0, c2,2 and c1,0 ` c1,2

Alexander Barg, University of Maryland Facets of the repair problem 32 / 45

Idea of the construction, II

c1,0, c1,1 and c2,0 ` c2,1 can be found from any k ` 1 values c3,0 ` c3,1, . . . , cn`1,0 ` cn`1,1 Similarly, taking the first and the third groups of parity checks, we conclude that c2,0, c2,2 and c1,0 ` c1,2 can be found from any k ` 1 values out of c3,0 ` c3,2, . . . , cn`1,0 ` cn`1,2 The repair protocol

• 1. Download c3,0 ` c3,1, . . . , ck`3,0 ` ck`3,1 to Node 1; find c1,0, c1,1 and c2,0 ` c2,1
• 2. Download c3,0 ` c3,2, . . . , ck`3,0 ` ck`3,2 to Node 2; find c2,0, c2,2 and c1,0 ` c1,2
• 3. Send c2,0 ` c2,1 from Node 1 to Node 2; Send c1,0 ` c1,2 from Node 2 to Node 1

Alexander Barg, University of Maryland Facets of the repair problem 32 / 45

Idea of the construction, II

c1,0, c1,1 and c2,0 ` c2,1 can be found from any k ` 1 values c3,0 ` c3,1, . . . , cn`1,0 ` cn`1,1 Similarly, taking the first and the third groups of parity checks, we conclude that c2,0, c2,2 and c1,0 ` c1,2 can be found from any k ` 1 values out of c3,0 ` c3,2, . . . , cn`1,0 ` cn`1,2 The repair protocol

• 1. Download c3,0 ` c3,1, . . . , ck`3,0 ` ck`3,1 to Node 1; find c1,0, c1,1 and c2,0 ` c2,1
• 2. Download c3,0 ` c3,2, . . . , ck`3,0 ` ck`3,2 to Node 2; find c2,0, c2,2 and c1,0 ` c1,2
• 3. Send c2,0 ` c2,1 from Node 1 to Node 2; Send c1,0 ` c1,2 from Node 2 to Node 1
• 4. Find the missing values c1,2 and c2,1

Alexander Barg, University of Maryland Facets of the repair problem 32 / 45

Parameters of the constructions

Repairing the first h nodes Repairing any h nodes Values of h “ |F|, d “ |R| |F| l |F| l h “ 2, d “ k ` 1 n ` 2 3 2n 3

´n 2 ¯

h “ 2, any d n ` 2ps ´ 1q s2 ´ 1 sn ps2 ´ 1q

´n 2 ¯

any h, d “ k ` 1 n ` h h ` 1 2n ph ` 1q

´n h ¯

any h, any d n ` hps ´ 1q ph ` d ´ kqps ´ 1qh´1 sn pph ` d ´ kqps ´ 1qh´1q

´n h ¯

(MIN YE AND A.B., T-IT ’18, ARXIV:1801.09665)

Alexander Barg, University of Maryland Facets of the repair problem 33 / 45

Rack-aware storage model

u nodes in each rack . . . Rack 1 Rack 2 Rack ¯ n

• Encoding of length n is stored in ¯

n racks, each containing u nodes

• Code length n “ ¯

nu

• Only communication between the racks counts toward repair bandwidth

Alexander Barg, University of Maryland Facets of the repair problem 34 / 45

Rack-aware storage model: Repairing single node

u nodes in each rack . . . Rack 1 Rack 2 Rack ¯ n

Alexander Barg, University of Maryland Facets of the repair problem 35 / 45

Rack-aware storage model: Repairing single node

u nodes in each rack . . . Rack 1 Rack 2 Rack ¯ n

Cut-set bound (HU, LEE, AND ZHANG, ISIT 2016): Let k “ ¯ ku ` v,¯ r :“ ¯ n ´ ¯ k, then β ě ¯ n ´ 1 ¯ r l This bound is better than the standard MSR cut-set bound if v ‰ 0

Alexander Barg, University of Maryland Facets of the repair problem 35 / 45

Code construction

• Given n, k, we construct codes with sub-packetization l “ ¯

r¯

n over F, |F| “ q ą ¯

rn

Alexander Barg, University of Maryland Facets of the repair problem 36 / 45

Code construction

• Given n, k, we construct codes with sub-packetization l “ ¯

r¯

n over F, |F| “ q ą ¯

rn

• Suppose that ¯

rn|pq ´ 1q, let λ P F : ordpλq “ ¯ rn.

Alexander Barg, University of Maryland Facets of the repair problem 36 / 45

Code construction

• Given n, k, we construct codes with sub-packetization l “ ¯

r¯

n over F, |F| “ q ą ¯

rn

• Suppose that ¯

rn|pq ´ 1q, let λ P F : ordpλq “ ¯ rn.

• Parity-check equations of the code C:

¯ n

ÿ

s“1

λtpps´1q¯

r`jsq u

ÿ

i“1

λtpi´1q¯

r ¯ ncps´1qu`i,j “ 0

for all t “ 0, . . . , r ´ 1; j “ 0, . . . , l ´ 1.

Alexander Barg, University of Maryland Facets of the repair problem 36 / 45

Code construction

• Given n, k, we construct codes with sub-packetization l “ ¯

r¯

n over F, |F| “ q ą ¯

rn

• Suppose that ¯

rn|pq ´ 1q, let λ P F : ordpλq “ ¯ rn.

• Parity-check equations of the code C:

¯ n

ÿ

s“1

λtpps´1q¯

r`jsq u

ÿ

i“1

λtpi´1q¯

r ¯ ncps´1qu`i,j “ 0

for all t “ 0, . . . , r ´ 1; j “ 0, . . . , l ´ 1.

• Suppose that a node in rack p has failed. Rearranging and putting α “ λu, we obtain

¯ r´1

ÿ

jp“0

αp¯

rpp´1q`jpqw u

ÿ

i“1

cpp´1qu`i,j “ ´ ÿ

s‰p

αp¯

rps´1q`jsqw ¯ r´1

ÿ

jp“0 u

ÿ

i“1

cps´1qu`i,j w “ 0, 1, . . . ,¯ r ´ 1 j¯

n, . . . , jp`1, jp´1, . . . , j1, where each jb “ 0, 1, . . . ,¯

r ´ 1.

Alexander Barg, University of Maryland Facets of the repair problem 36 / 45

Code construction

• Given n, k, we construct codes with sub-packetization l “ ¯

r¯

n over F, |F| “ q ą ¯

rn

• Suppose that ¯

rn|pq ´ 1q, let λ P F : ordpλq “ ¯ rn.

• Parity-check equations of the code C:

¯ n

ÿ

s“1

λtpps´1q¯

r`jsq u

ÿ

i“1

λtpi´1q¯

r ¯ ncps´1qu`i,j “ 0

for all t “ 0, . . . , r ´ 1; j “ 0, . . . , l ´ 1.

• Suppose that a node in rack p has failed. Rearranging and putting α “ λu, we obtain

¯ r´1

ÿ

jp“0

αp¯

rpp´1q`jpqw u

ÿ

i“1

cpp´1qu`i,j “ ´ ÿ

s‰p

αp¯

rps´1q`jsqw ¯ r´1

ÿ

jp“0 u

ÿ

i“1

cps´1qu`i,j w “ 0, 1, . . . ,¯ r ´ 1 j¯

n, . . . , jp`1, jp´1, . . . , j1, where each jb “ 0, 1, . . . ,¯

r ´ 1.

(work with ZITAN CHEN, ’18)

Alexander Barg, University of Maryland Facets of the repair problem 36 / 45

Repair of Reed-Solomon codes

Problem introduced by K. SHANMUGAM ET AL., 2014. It was developed by V. GURUSWAMI AND

• M. WOOTTERS (T-IT, Sept. 2017):

Alexander Barg, University of Maryland Facets of the repair problem 37 / 45

Repair of Reed-Solomon codes

Problem introduced by K. SHANMUGAM ET AL., 2014. It was developed by V. GURUSWAMI AND

• M. WOOTTERS (T-IT, Sept. 2017):
• Characterized repair schemes of RS codes
• Analyzed full-length RS codes for single-node repair

Alexander Barg, University of Maryland Facets of the repair problem 37 / 45

Repair of Reed-Solomon codes

Problem introduced by K. SHANMUGAM ET AL., 2014. It was developed by V. GURUSWAMI AND

• M. WOOTTERS (T-IT, Sept. 2017):
• Characterized repair schemes of RS codes
• Analyzed full-length RS codes for single-node repair

MIN YE AND A.B., RS codes with asymptotically optimal repair bandwidth, ISIT’16

• H. DAU AND O. MILENKOVIC, Optimal repair schemes of some families of full-length RS codes, ISIT’17
• A. CHOWDHURI AND A. VARDY, Schemes for asymptotically optimal repair of MDS codes, 2017

Alexander Barg, University of Maryland Facets of the repair problem 37 / 45

Repair of Reed-Solomon codes

Problem introduced by K. SHANMUGAM ET AL., 2014. It was developed by V. GURUSWAMI AND

• M. WOOTTERS (T-IT, Sept. 2017):
• Characterized repair schemes of RS codes
• Analyzed full-length RS codes for single-node repair

MIN YE AND A.B., RS codes with asymptotically optimal repair bandwidth, ISIT’16

• H. DAU AND O. MILENKOVIC, Optimal repair schemes of some families of full-length RS codes, ISIT’17
• A. CHOWDHURI AND A. VARDY, Schemes for asymptotically optimal repair of MDS codes, 2017

Optimal-repair (shortened) RS codes (work with I. TAMO AND MIN YE ’17):

Alexander Barg, University of Maryland Facets of the repair problem 37 / 45

Repair of Reed-Solomon codes

Problem introduced by K. SHANMUGAM ET AL., 2014. It was developed by V. GURUSWAMI AND

• M. WOOTTERS (T-IT, Sept. 2017):
• Characterized repair schemes of RS codes
• Analyzed full-length RS codes for single-node repair

MIN YE AND A.B., RS codes with asymptotically optimal repair bandwidth, ISIT’16

• H. DAU AND O. MILENKOVIC, Optimal repair schemes of some families of full-length RS codes, ISIT’17
• A. CHOWDHURI AND A. VARDY, Schemes for asymptotically optimal repair of MDS codes, 2017

Optimal-repair (shortened) RS codes (work with I. TAMO AND MIN YE ’17):

• Construction of RS codes for single-node repair with optimal repair bandwidth

Alexander Barg, University of Maryland Facets of the repair problem 37 / 45

Repair of Reed-Solomon codes

Problem introduced by K. SHANMUGAM ET AL., 2014. It was developed by V. GURUSWAMI AND

• M. WOOTTERS (T-IT, Sept. 2017):
• Characterized repair schemes of RS codes
• Analyzed full-length RS codes for single-node repair

MIN YE AND A.B., RS codes with asymptotically optimal repair bandwidth, ISIT’16

• H. DAU AND O. MILENKOVIC, Optimal repair schemes of some families of full-length RS codes, ISIT’17
• A. CHOWDHURI AND A. VARDY, Schemes for asymptotically optimal repair of MDS codes, 2017

Optimal-repair (shortened) RS codes (work with I. TAMO AND MIN YE ’17):

• Construction of RS codes for single-node repair with optimal repair bandwidth
• Lower bound on sub-packetization parameter l

Alexander Barg, University of Maryland Facets of the repair problem 37 / 45

Repair of Reed-Solomon codes

Problem introduced by K. SHANMUGAM ET AL., 2014. It was developed by V. GURUSWAMI AND

• M. WOOTTERS (T-IT, Sept. 2017):
• Characterized repair schemes of RS codes
• Analyzed full-length RS codes for single-node repair

MIN YE AND A.B., RS codes with asymptotically optimal repair bandwidth, ISIT’16

• H. DAU AND O. MILENKOVIC, Optimal repair schemes of some families of full-length RS codes, ISIT’17
• A. CHOWDHURI AND A. VARDY, Schemes for asymptotically optimal repair of MDS codes, 2017

Optimal-repair (shortened) RS codes (work with I. TAMO AND MIN YE ’17):

• Construction of RS codes for single-node repair with optimal repair bandwidth
• Lower bound on sub-packetization parameter l
• Construction of RS codes that universally achieve the cut-set bound for any number of

erasures

Alexander Barg, University of Maryland Facets of the repair problem 37 / 45

Use of RS codes for node repair

Alexander Barg, University of Maryland Facets of the repair problem 38 / 45

Use of RS codes for node repair

pa0, a1, a2, a3q

Alexander Barg, University of Maryland Facets of the repair problem 38 / 45

Use of RS codes for node repair

pa0, a1, a2, a3q Ñ fpxq “ a0 ` a1x ` a2x2 ` a3x3

Alexander Barg, University of Maryland Facets of the repair problem 38 / 45

Use of RS codes for node repair

pa0, a1, a2, a3q Ñ fpxq “ a0 ` a1x ` a2x2 ` a3x3

Alexander Barg, University of Maryland Facets of the repair problem 38 / 45

Use of RS codes for node repair

pa0, a1, a2, a3q Ñ fpxq “ a0 ` a1x ` a2x2 ` a3x3

Alexander Barg, University of Maryland Facets of the repair problem 38 / 45

Use of RS codes for node repair

pa0, a1, a2, a3q Ñ fpxq “ a0 ` a1x ` a2x2 ` a3x3 P1 fpP1q

Alexander Barg, University of Maryland Facets of the repair problem 38 / 45

Use of RS codes for node repair

pa0, a1, a2, a3q Ñ fpxq “ a0 ` a1x ` a2x2 ` a3x3 P1 fpP1q P2 fpP2q

Alexander Barg, University of Maryland Facets of the repair problem 38 / 45

Use of RS codes for node repair

pa0, a1, a2, a3q Ñ fpxq “ a0 ` a1x ` a2x2 ` a3x3 P1 fpP1q P2 fpP2q P3 fpP3q

Alexander Barg, University of Maryland Facets of the repair problem 38 / 45

Use of RS codes for node repair

pa0, a1, a2, a3q Ñ fpxq “ a0 ` a1x ` a2x2 ` a3x3 P1 fpP1q P2 fpP2q P3 fpP3q P4 fpP4q

Alexander Barg, University of Maryland Facets of the repair problem 38 / 45

Use of RS codes for node repair

pa0, a1, a2, a3q Ñ fpxq “ a0 ` a1x ` a2x2 ` a3x3 P1 fpP1q P2 fpP2q P3 fpP3q P4 fpP4q P5 fpP5q

Alexander Barg, University of Maryland Facets of the repair problem 38 / 45

Use of RS codes for node repair

pa0, a1, a2, a3q Ñ fpxq “ a0 ` a1x ` a2x2 ` a3x3 P1 fpP1q P2 fpP2q P3 fpP3q P4 fpP4q P5 fpP5q Pn fpPnq

Alexander Barg, University of Maryland Facets of the repair problem 38 / 45

Use of RS codes for node repair

pa0, a1, a2, a3q Ñ fpxq “ a0 ` a1x ` a2x2 ` a3x3 Ñ fpxq “ pfpP1q, fpP2q, ..., fpPnqq P1 fpP1q P2 fpP2q P3 fpP3q P4 fpP4q P5 fpP5q Pn fpPnq

Alexander Barg, University of Maryland Facets of the repair problem 38 / 45

Use of RS codes for node repair

pa0, a1, a2, a3q Ñ fpxq “ a0 ` a1x ` a2x2 ` a3x3 Ñ fpxq “ pfpP1q, fpP2q, ..., fpPnqq P1 fpP1q P2 fpP2q P3 fpP3q P4 fpP4q P5 fpP5q Pn fpPnq

Alexander Barg, University of Maryland Facets of the repair problem 38 / 45

Use of RS codes for node repair

pa0, a1, a2, a3q Ñ fpxq “ a0 ` a1x ` a2x2 ` a3x3 Ñ fpxq “ pfpP1q, fpP2q, ..., fpPnqq P1 fpP1q P2 fpP2q P3 fpP3q P4 fpP4q P5 fpP5q Pn fpPnq

Alexander Barg, University of Maryland Facets of the repair problem 38 / 45

Use of RS codes for node repair

pa0, a1, a2, a3q Ñ fpxq “ a0 ` a1x ` a2x2 ` a3x3 Ñ fpxq “ pfpP1q, fpP2q, ..., fpPnqq P1 fpP1q P2 fpP2q P3 fpP3q P4 fpP4q P5 fpP5q Pn fpPnq fpP2q fpP3q fpP4q fpP5q

Use of RS codes for node repair

pa0, a1, a2, a3q Ñ fpxq “ a0 ` a1x ` a2x2 ` a3x3 Ñ fpxq “ pfpP1q, fpP2q, ..., fpPnqq P1 fpP1q P2 fpP2q P3 fpP3q P4 fpP4q P5 fpP5q Pn fpPnq fpP2q fpP3q fpP4q fpP5q

Use of RS codes for node repair

pa0, a1, a2, a3q Ñ fpxq “ a0 ` a1x ` a2x2 ` a3x3 Ñ fpxq “ pfpP1q, fpP2q, ..., fpPnqq We can repair a failed node by downloading the contents of k “ 4 nodes P1 fpP1q P2 fpP2q P3 fpP3q P4 fpP4q P5 fpP5q Pn fpPnq fpP2q fpP3q fpP4q fpP5q

Alexander Barg, University of Maryland Facets of the repair problem 38 / 45

Repair bandwidth of Reed-Solomon codes

Idea: [SHANMUGAM-PAPAILIOPOULOS-DIMAKIS, ’14] Consider the RS code C over F as a code over a subfield B (“vectorize” C)

Alexander Barg, University of Maryland Facets of the repair problem 39 / 45

Repair bandwidth of Reed-Solomon codes

Idea: [SHANMUGAM-PAPAILIOPOULOS-DIMAKIS, ’14] Consider the RS code C over F as a code over a subfield B (“vectorize” C) Example:

Alexander Barg, University of Maryland Facets of the repair problem 39 / 45

Repair bandwidth of Reed-Solomon codes

Idea: [SHANMUGAM-PAPAILIOPOULOS-DIMAKIS, ’14] Consider the RS code C over F as a code over a subfield B (“vectorize” C) Example:

• Consider an RS code over F “ F16 as an array code over B “ F2, i.e., l “ 4

Alexander Barg, University of Maryland Facets of the repair problem 39 / 45

Repair bandwidth of Reed-Solomon codes

Idea: [SHANMUGAM-PAPAILIOPOULOS-DIMAKIS, ’14] Consider the RS code C over F as a code over a subfield B (“vectorize” C) Example:

• Consider an RS code over F “ F16 as an array code over B “ F2, i.e., l “ 4
• F can be represented as a 4-dimensional vector space over B “ t0, 1u

Alexander Barg, University of Maryland Facets of the repair problem 39 / 45

Repair bandwidth of Reed-Solomon codes

Idea: [SHANMUGAM-PAPAILIOPOULOS-DIMAKIS, ’14] Consider the RS code C over F as a code over a subfield B (“vectorize” C) Example:

• Consider an RS code over F “ F16 as an array code over B “ F2, i.e., l “ 4
• F can be represented as a 4-dimensional vector space over B “ t0, 1u
• To “compress” the values of the helper nodes we project them on a subfield of F

Alexander Barg, University of Maryland Facets of the repair problem 39 / 45

Repair bandwidth of Reed-Solomon codes

Idea: [SHANMUGAM-PAPAILIOPOULOS-DIMAKIS, ’14] Consider the RS code C over F as a code over a subfield B (“vectorize” C) Example:

• Consider an RS code over F “ F16 as an array code over B “ F2, i.e., l “ 4
• F can be represented as a 4-dimensional vector space over B “ t0, 1u
• To “compress” the values of the helper nodes we project them on a subfield of F
• Let α P F be such that α4 “ α ` 1, then p1, α, α2, α3q form a basis of F over B

Alexander Barg, University of Maryland Facets of the repair problem 39 / 45

Repair bandwidth of Reed-Solomon codes

Idea: [SHANMUGAM-PAPAILIOPOULOS-DIMAKIS, ’14] Consider the RS code C over F as a code over a subfield B (“vectorize” C) Example:

• Consider an RS code over F “ F16 as an array code over B “ F2, i.e., l “ 4
• F can be represented as a 4-dimensional vector space over B “ t0, 1u
• To “compress” the values of the helper nodes we project them on a subfield of F
• Let α P F be such that α4 “ α ` 1, then p1, α, α2, α3q form a basis of F over B
• Trace trpxq “ x ` x2 ` x22 ` x23 is a map from F to B:

trp0q “ 0, trp1q “ 0, trpαq “ 1, etc.

Alexander Barg, University of Maryland Facets of the repair problem 39 / 45

Repair bandwidth of Reed-Solomon codes

Idea: [SHANMUGAM-PAPAILIOPOULOS-DIMAKIS, ’14] Consider the RS code C over F as a code over a subfield B (“vectorize” C) Example:

• Consider an RS code over F “ F16 as an array code over B “ F2, i.e., l “ 4
• F can be represented as a 4-dimensional vector space over B “ t0, 1u
• To “compress” the values of the helper nodes we project them on a subfield of F
• Let α P F be such that α4 “ α ` 1, then p1, α, α2, α3q form a basis of F over B
• Trace trpxq “ x ` x2 ` x22 ` x23 is a map from F to B:

trp0q “ 0, trp1q “ 0, trpαq “ 1, etc.

• For any c P F the values trpcq, trpαcq, trpα2cq, trpα3cq suffice to recover c

Alexander Barg, University of Maryland Facets of the repair problem 39 / 45

General repair scheme

The repair scheme of GURUSWAMI-WOOTTERS ’16:

Alexander Barg, University of Maryland Facets of the repair problem 40 / 45

General repair scheme

The repair scheme of GURUSWAMI-WOOTTERS ’16:

• Let B Ă F be finite fields, rF : Bs “ l; Ω Ă F; |Ω| “ tP1, . . . , Pnu

Let C “ RSFpn, k, Ωq be the RS code; r “ n ´ k

Alexander Barg, University of Maryland Facets of the repair problem 40 / 45

General repair scheme

The repair scheme of GURUSWAMI-WOOTTERS ’16:

• Let B Ă F be finite fields, rF : Bs “ l; Ω Ă F; |Ω| “ tP1, . . . , Pnu

Let C “ RSFpn, k, Ωq be the RS code; r “ n ´ k

• Let ci be erased.

Alexander Barg, University of Maryland Facets of the repair problem 40 / 45

General repair scheme

The repair scheme of GURUSWAMI-WOOTTERS ’16:

• Let B Ă F be finite fields, rF : Bs “ l; Ω Ă F; |Ω| “ tP1, . . . , Pnu

Let C “ RSFpn, k, Ωq be the RS code; r “ n ´ k

• Let ci be erased.
• Let b1, b2, . . . , bl P CK be such that b1,i, . . . , bl,i form a basis of F over B. The values

trpbjiciq suffice to recover ci

Alexander Barg, University of Maryland Facets of the repair problem 40 / 45

General repair scheme

The repair scheme of GURUSWAMI-WOOTTERS ’16:

• Let B Ă F be finite fields, rF : Bs “ l; Ω Ă F; |Ω| “ tP1, . . . , Pnu

Let C “ RSFpn, k, Ωq be the RS code; r “ n ´ k

• Let ci be erased.
• Let b1, b2, . . . , bl P CK be such that b1,i, . . . , bl,i form a basis of F over B. The values

trpbjiciq suffice to recover ci

• We have cibj,i `

n

ÿ

m‰i

cmbj,m “ 0, j “ 1, . . . , l

Alexander Barg, University of Maryland Facets of the repair problem 40 / 45

General repair scheme

The repair scheme of GURUSWAMI-WOOTTERS ’16:

• Let B Ă F be finite fields, rF : Bs “ l; Ω Ă F; |Ω| “ tP1, . . . , Pnu

Let C “ RSFpn, k, Ωq be the RS code; r “ n ´ k

• Let ci be erased.
• Let b1, b2, . . . , bl P CK be such that b1,i, . . . , bl,i form a basis of F over B. The values

trpbjiciq suffice to recover ci

• We have cibj,i `

n

ÿ

m‰i

cmbj,m “ 0, j “ 1, . . . , l

• We have trpbjiciq “ ´ ř

t‰i trpbjtctq, j “ 1, . . . , l

Alexander Barg, University of Maryland Facets of the repair problem 40 / 45

General repair scheme

The repair scheme of GURUSWAMI-WOOTTERS ’16:

• Let B Ă F be finite fields, rF : Bs “ l; Ω Ă F; |Ω| “ tP1, . . . , Pnu

Let C “ RSFpn, k, Ωq be the RS code; r “ n ´ k

• Let ci be erased.
• Let b1, b2, . . . , bl P CK be such that b1,i, . . . , bl,i form a basis of F over B. The values

trpbjiciq suffice to recover ci

• We have cibj,i `

n

ÿ

m‰i

cmbj,m “ 0, j “ 1, . . . , l

• We have trpbjiciq “ ´ ř

t‰i trpbjtctq, j “ 1, . . . , l

• We need ttrpbjtctq, j “ 1, . . . , l; t ‰ iu

Alexander Barg, University of Maryland Facets of the repair problem 40 / 45

General repair scheme

The repair scheme of GURUSWAMI-WOOTTERS ’16:

• Let B Ă F be finite fields, rF : Bs “ l; Ω Ă F; |Ω| “ tP1, . . . , Pnu

Let C “ RSFpn, k, Ωq be the RS code; r “ n ´ k

• Let ci be erased.
• Let b1, b2, . . . , bl P CK be such that b1,i, . . . , bl,i form a basis of F over B. The values

trpbjiciq suffice to recover ci

• We have cibj,i `

n

ÿ

m‰i

cmbj,m “ 0, j “ 1, . . . , l

• We have trpbjiciq “ ´ ř

t‰i trpbjtctq, j “ 1, . . . , l

• We need ttrpbjtctq, j “ 1, . . . , l; t ‰ iu
• Let Bt be a maximum-size linearly independent subset of tbjt, j “ 1 . . . lu

We can find ci from Ť

t‰ittrpβctq, β P Btu

Alexander Barg, University of Maryland Facets of the repair problem 40 / 45

General repair scheme

The repair scheme of GURUSWAMI-WOOTTERS ’16:

• Let B Ă F be finite fields, rF : Bs “ l; Ω Ă F; |Ω| “ tP1, . . . , Pnu

Let C “ RSFpn, k, Ωq be the RS code; r “ n ´ k

• Let ci be erased.
• Let b1, b2, . . . , bl P CK be such that b1,i, . . . , bl,i form a basis of F over B. The values

trpbjiciq suffice to recover ci

• We have cibj,i `

n

ÿ

m‰i

cmbj,m “ 0, j “ 1, . . . , l

• We have trpbjiciq “ ´ ř

t‰i trpbjtctq, j “ 1, . . . , l

• We need ttrpbjtctq, j “ 1, . . . , l; t ‰ iu
• Let Bt be a maximum-size linearly independent subset of tbjt, j “ 1 . . . lu

We can find ci from Ť

t‰ittrpβctq, β P Btu

This is essentially the only possible linear repair scheme

Alexander Barg, University of Maryland Facets of the repair problem 40 / 45

RS codes for repair of a single node from d helper nodes

Alexander Barg, University of Maryland Facets of the repair problem 41 / 45

RS codes for repair of a single node from d helper nodes

• Let Ω “ tα1, . . . , αnu, where αi, i “ 1, . . . , n are algebraic elements over Fq;

Alexander Barg, University of Maryland Facets of the repair problem 41 / 45

RS codes for repair of a single node from d helper nodes

• Let Ω “ tα1, . . . , αnu, where αi, i “ 1, . . . , n are algebraic elements over Fq;
• Fi :“ Fqptαj, j ‰ iuq

Alexander Barg, University of Maryland Facets of the repair problem 41 / 45

RS codes for repair of a single node from d helper nodes

• Let Ω “ tα1, . . . , αnu, where αi, i “ 1, . . . , n are algebraic elements over Fq;
• Fi :“ Fqptαj, j ‰ iuq
• F :“ Fqpα1, . . . , αnq

Alexander Barg, University of Maryland Facets of the repair problem 41 / 45

RS codes for repair of a single node from d helper nodes

• Let Ω “ tα1, . . . , αnu, where αi, i “ 1, . . . , n are algebraic elements over Fq;
• Fi :“ Fqptαj, j ‰ iuq
• F :“ Fqpα1, . . . , αnq
• K :“ Fpβq, where degFpβq “ s :“ d ` k ´ 1

Alexander Barg, University of Maryland Facets of the repair problem 41 / 45

RS codes for repair of a single node from d helper nodes

• Let Ω “ tα1, . . . , αnu, where αi, i “ 1, . . . , n are algebraic elements over Fq;
• Fi :“ Fqptαj, j ‰ iuq
• F :“ Fqpα1, . . . , αnq
• K :“ Fpβq, where degFpβq “ s :“ d ` k ´ 1
• RSKpn, k, tα1, . . . , αnuq

Alexander Barg, University of Maryland Facets of the repair problem 41 / 45

RS codes for repair of a single node from d helper nodes

• Let Ω “ tα1, . . . , αnu, where αi, i “ 1, . . . , n are algebraic elements over Fq;
• Fi :“ Fqptαj, j ‰ iuq
• F :“ Fqpα1, . . . , αnq
• K :“ Fpβq, where degFpβq “ s :“ d ` k ´ 1
• RSKpn, k, tα1, . . . , αnuq
• Suppose that αi R Fqptαj, j ‰ iuq and degFipαiq ” 1 mod s

Fq F1 F2 . . . Fn

˜ α1 ˜ α2 ˜ αn

F

pα1, p1q pα2, p2q pαn, pnq

K

pβ, sq

Alexander Barg, University of Maryland Facets of the repair problem 41 / 45

Fq F1 F2 . . . Fn

˜ α1 ˜ α2 ˜ αn

F

pα1, p1q pα2, p2q pαn, pnq

K

pβ, sq

Fq F1 F2 . . . Fn

˜ α1 ˜ α2 ˜ αn

F

pα1, p1q pα2, p2q pαn, pnq

K

pβ, sq

Consider the RS code C :“ RSKpn, k, tα1, . . . , αnuq

Alexander Barg, University of Maryland Facets of the repair problem 42 / 45

Fq F1 F2 . . . Fn

˜ α1 ˜ α2 ˜ αn

F

pα1, p1q pα2, p2q pαn, pnq

K

pβ, sq

Consider the RS code C :“ RSKpn, k, tα1, . . . , αnuq Repair of the node i is performed over Fi

Alexander Barg, University of Maryland Facets of the repair problem 42 / 45

n, q, l

• Given n, we have

l :“ rK : Fqs “ s

n

ź

i“1 pi”1 mod s

pi

Alexander Barg, University of Maryland Facets of the repair problem 43 / 45

n, q, l

• Given n, we have

l :“ rK : Fqs “ s

n

ź

i“1 pi”1 mod s

pi

• Thus, C “ RSKpn, k, Ωq where

q “ pl, l « exppp1 ` op1qqn log nq

Alexander Barg, University of Maryland Facets of the repair problem 43 / 45

n, q, l

• Given n, we have

l :“ rK : Fqs “ s

n

ź

i“1 pi”1 mod s

pi

• Thus, C “ RSKpn, k, Ωq where

q “ pl, l « exppp1 ` op1qqn log nq

• Is l too large?

Alexander Barg, University of Maryland Facets of the repair problem 43 / 45

n, q, l

• Given n, we have

l :“ rK : Fqs “ s

n

ź

i“1 pi”1 mod s

pi

• Thus, C “ RSKpn, k, Ωq where

q “ pl, l « exppp1 ` op1qqn log nq

• Is l too large?

In fact l “ exppp1 ` op1qqk log kq is necessary!

Alexander Barg, University of Maryland Facets of the repair problem 43 / 45

n, q, l

Theorem

• Let B “ Fq and F “ Fql for a prime power q.

Alexander Barg, University of Maryland Facets of the repair problem 44 / 45

n, q, l

Theorem

• Let B “ Fq and F “ Fql for a prime power q.
• k ` 1 ď d ď n ´ 1

Alexander Barg, University of Maryland Facets of the repair problem 44 / 45

n, q, l

Theorem

• Let B “ Fq and F “ Fql for a prime power q.
• k ` 1 ď d ď n ´ 1
• C Ď Fn an pn, kq scalar linear MDS code with a linear repair scheme over F

Alexander Barg, University of Maryland Facets of the repair problem 44 / 45

n, q, l

Theorem

• Let B “ Fq and F “ Fql for a prime power q.
• k ` 1 ď d ď n ´ 1
• C Ď Fn an pn, kq scalar linear MDS code with a linear repair scheme over F
• Suppose that C supports optimal repair of a single node from d helper nodes

Alexander Barg, University of Maryland Facets of the repair problem 44 / 45

n, q, l

Theorem

• Let B “ Fq and F “ Fql for a prime power q.
• k ` 1 ď d ď n ´ 1
• C Ď Fn an pn, kq scalar linear MDS code with a linear repair scheme over F
• Suppose that C supports optimal repair of a single node from d helper nodes
• Then

l ě

k´1

ź

i“1

pi where pi is the i-th smallest prime.

Alexander Barg, University of Maryland Facets of the repair problem 44 / 45

n, q, l

Theorem

• Let B “ Fq and F “ Fql for a prime power q.
• k ` 1 ď d ď n ´ 1
• C Ď Fn an pn, kq scalar linear MDS code with a linear repair scheme over F
• Suppose that C supports optimal repair of a single node from d helper nodes
• Then

l ě

k´1

ź

i“1

pi where pi is the i-th smallest prime. To summarize: Sub-packetization for MDS codes with optimal repair satisfies

Alexander Barg, University of Maryland Facets of the repair problem 44 / 45

n, q, l

Theorem

• Let B “ Fq and F “ Fql for a prime power q.
• k ` 1 ď d ď n ´ 1
• C Ď Fn an pn, kq scalar linear MDS code with a linear repair scheme over F
• Suppose that C supports optimal repair of a single node from d helper nodes
• Then

l ě

k´1

ź

i“1

pi where pi is the i-th smallest prime. To summarize: Sub-packetization for MDS codes with optimal repair satisfies

• Scalar codes:

exppp1 ` op1qqk log kq ď l ď exppp1 ` op1qqn log nq

Alexander Barg, University of Maryland Facets of the repair problem 44 / 45

n, q, l

Theorem

• Let B “ Fq and F “ Fql for a prime power q.
• k ` 1 ď d ď n ´ 1
• C Ď Fn an pn, kq scalar linear MDS code with a linear repair scheme over F
• Suppose that C supports optimal repair of a single node from d helper nodes
• Then

l ě

k´1

ź

i“1

pi where pi is the i-th smallest prime. To summarize: Sub-packetization for MDS codes with optimal repair satisfies

• Scalar codes:

exppp1 ` op1qqk log kq ď l ď exppp1 ` op1qqn log nq

• Vector codes:

l “ rrn{rs

Alexander Barg, University of Maryland Facets of the repair problem 44 / 45

Quo vadis: Current work and open questions

(Difficult) open problems:

• Optimal-bandwidth cooperative repair of RS codes
• Lower bounds on the node size l for cooperative repair

Alexander Barg, University of Maryland Facets of the repair problem 45 / 45

Quo vadis: Current work and open questions

(Difficult) open problems:

• Optimal-bandwidth cooperative repair of RS codes
• Lower bounds on the node size l for cooperative repair

Research directions:

• Repair problem on graphs (with connectivity constraints)
• Random networks and node repair

Alexander Barg, University of Maryland Facets of the repair problem 45 / 45

Quo vadis: Current work and open questions

(Difficult) open problems:

• Optimal-bandwidth cooperative repair of RS codes
• Lower bounds on the node size l for cooperative repair

Research directions:

• Repair problem on graphs (with connectivity constraints)
• Random networks and node repair

Thank you!

Alexander Barg, University of Maryland Facets of the repair problem 45 / 45