Locally decodable codes: from computational complexity to cloud - - PowerPoint PPT Presentation

β–Ά
locally decodable codes
SMART_READER_LITE
LIVE PREVIEW

Locally decodable codes: from computational complexity to cloud - - PowerPoint PPT Presentation

Oct 22, 2022 149 likes β€’501 views

Locally decodable codes: from computational complexity to cloud computing Sergey Yekhanin Microsoft Research Error-correcting codes: paradigm 2 E() 2 E() +noise 0110001 011000100101 01*00*10010*

slide-1
SLIDE 1

Locally decodable codes:

from computational complexity to cloud computing

Sergey Yekhanin

Microsoft Research

slide-2
SLIDE 2

Error-correcting codes: paradigm

0110001 011000100101 0110001 01*00*10010*

Encoder Decoder Channel

π‘Œ ∈ 𝐺2

𝑙

E(π‘Œ) +noise π‘Œ E(π‘Œ) ∈ 𝐺2

π‘œ

The paradigm dates back to 1940s (Shannon / Hamming)

Corrupts up to 𝑓 coordinates.

slide-3
SLIDE 3

Local decoding: paradigm

0110001 011000100101 1 01*00*10010*

Encoder Local Decoder Channel

  • First account: Reed’s decoder for Muller’s codes (1954)
  • Implicit use: (1950s-1990s)
  • Formal definition and systematic study (late 1990s) [Levin’95, STV’98, KT’00]
  • Original applications in computational complexity theory
  • Cryptography
  • Most recently used in practice to provide reliability in distributed storage

Local decoder runs in time much smaller than the message length! π‘Œ ∈ 𝐺2

𝑙

E(π‘Œ) ∈ 𝐺2

π‘œ

E(π‘Œ) +noise π‘Œπ‘— Corrupts up to 𝑓 coordinates. Reads up to 𝑠 coordinates.

slide-4
SLIDE 4

Local decoding: example

X1 X E(X) X2 X3 X1 X1οƒ…X2 X2 X3 X1οƒ…X3 X2οƒ…X3 X1οƒ…X2οƒ…X3

Message length: k = 3 Codeword length: n = 7 Corrupted locations: 𝑓 = 3 Locality: 𝑠 = 2

slide-5
SLIDE 5

Local decoding: example

X1 X E(X) X2 X3 X1 X1οƒ…X2 X2 X3 X1οƒ…X3 X2οƒ…X3 X1οƒ…X2οƒ…X3

Message length: k = 3 Codeword length: n = 7 Corrupted locations: 𝑓 = 3 Locality: 𝑠 = 2

slide-6
SLIDE 6

Locally decodable codes

Definition: A code E: 𝐺

π‘Ÿ 𝑙 β†’ 𝐺 π‘Ÿ π‘œ is 𝑠-locally decodable, if for every message π‘Œ,

each π‘Œπ‘— can be recovered from reading some 𝑠 symbols of 𝐹(π‘Œ), even after up to 𝑓 coordinates of 𝐹(π‘Œ) are corrupted.

  • (Erasures.) Decoder is aware of erased locations. Output is always correct.
  • (Errors.) Decoder is randomized. Output is correct with probability 99%.

0 0 1 0 1 … 0 1 1 0 1 … 0 1 1 0 … 0 1

k symbol message n symbol codeword

Noise

Decoder reads only r symbols

slide-7
SLIDE 7

Locally decodable codes

Goal: Understand the true shape of the tradeoff between redundancy π‘œ βˆ’ 𝑙 and locality 𝑠, for different settings of 𝑓 (e.g., 𝑓 = πœ€π‘œ, π‘œπœ— , 𝑃 1 .)

𝑓 𝑠

𝑃(1) π‘œπœ πœ€π‘œ 𝑃(1) π‘™πœ (log 𝑙)𝑑 Taxonomy of known families of LDCs

Multiplicity codes Matching vector codes Reed Muller codes Local reconst- ruction codes Projective geometry codes

slide-8
SLIDE 8

Plan

  • Part I: (Computational complexity)
  • Average case hardness
  • An avg. case hard language in EXP (unless EXP βŠ† BPP)
  • Construction of LDCs
  • Open questions
  • Part II: (Distributed data storage)
  • Erasure coding for data storage
  • LDCs for data storage
  • Constructions and limitations
  • Open questions
slide-9
SLIDE 9

Part I: Computational complexity

slide-10
SLIDE 10

Average case complexity

  • A problem is hard-on-average if any efficient algorithm errs on 10% of the inputs.
  • Establishing hardness-on-average for a problem in NP is a major problem.
  • Below we establish hardness-on-average for a problem in EXP, assuming EXP ⊈ BPP.

Construction [STV]: E: 𝐺2

𝑙 β†’ 𝐺2 π‘œ

π‘œ = π‘žπ‘π‘šπ‘§ 𝑙 , 𝑠 = (log 𝑙)𝑑, 𝑓 = π‘œ/10. Theorem: If there is an efficient algorithm that errs on <10% of 𝑀′; then EXP βŠ† BPP.

𝑀 is EXP-complete 𝑀′ is in EXP

1 1 0 1 0 1 1 1 1 1 Level 𝑙 is a string π‘Œ of length 2𝑙

𝐹(π‘Œ) π‘Œ

slide-11
SLIDE 11

Average case complexity

E: 𝐺2

𝑙 β†’ 𝐺2 π‘œ

π‘œ = π‘žπ‘π‘šπ‘§ 𝑙 , 𝑠 = (log 𝑙)𝑑, 𝑓 = π‘œ/10. Theorem: If there is an efficient algorithm that errs on <10% of 𝑀′; then EXP βŠ† BPP.

𝑀 is EXP-complete 𝑀′ is in EXP

1 1 0 1 1 1 1 1 1

𝐹(π‘Œ) π‘Œ

Proof: We obtain a BPP algorithm for 𝑀:

  • Let A be the algorithm that errs on <10% of 𝑀′;

A gives us access to the corrupted encoding 𝐹(π‘Œ).

  • To decide if π‘Œπ‘— invoke the local decoder for 𝐹(π‘Œ).
  • Time complexity is (log 2𝑙)π‘‘βˆ— π‘žπ‘π‘šπ‘§ 𝑙 = π‘žπ‘π‘šπ‘§ 𝑙 .
  • Output is correct with probability 99%.
slide-12
SLIDE 12

Reed Muller codes

  • Parameters: π‘Ÿ, 𝑛, 𝑒 = 1 βˆ’ 4πœ€ π‘Ÿ.
  • Codewords: evaluations of degree 𝑒 polynomials in 𝑛 variables over 𝐺

π‘Ÿ.

  • Polynomial 𝑔 ∈ 𝐺

π‘Ÿ 𝑨1, … , 𝑨𝑛 , deg f < 𝑒 yields a codeword: 𝑔(𝑦 ) 𝑦 ∈𝐺

π‘Ÿ 𝑛

  • Parameters: π‘œ = π‘Ÿπ‘›, 𝑙 = 𝑛 + 𝑒

𝑛 , 𝑠 = π‘Ÿ βˆ’ 1, 𝑓 = πœ€π‘œ.

slide-13
SLIDE 13

Reed Muller codes: local decoding

. , , d m q

  • Key observation: Restriction of a codeword to an affine line yields an

evaluation of a univariate polynomial 𝑔 𝑀 of degree at most 𝑒.

  • To recover the value at 𝑦 :

– Pick a random affine line through 𝑦 . – Do noisy polynomial interpolation.

  • Locally decodable code: Decoder reads π‘Ÿ βˆ’ 1 random locations.

𝑦

𝐺

π‘Ÿ 𝑛

slide-14
SLIDE 14

Reed Muller codes: parameters

π‘œ = π‘Ÿπ‘›, 𝑙 = 𝑛 + 𝑒 𝑛 , 𝑒 = 1 βˆ’ 4πœ€ π‘Ÿ, 𝑠 = π‘Ÿ βˆ’ 1, 𝑓 = πœ€π‘œ.

Setting parameters:

  • q = 𝑃 1 , 𝑛 β†’ ∞: 𝑠 = 𝑃 1 , π‘œ = exp 𝑙

1 π‘ βˆ’1 .

  • q = 𝑛2 ∢ 𝑠 = (log 𝑙)2, π‘œ = π‘žπ‘π‘šπ‘§ 𝑙 .
  • q β†’ ∞, 𝑛 = 𝑃 1 : 𝑠 = π‘™πœ—, π‘œ = 𝑃 𝑙 .

Reducing codeword length is a major open question.

Better codes are known

slide-15
SLIDE 15

Part II: Distributed storage

slide-16
SLIDE 16

Data storage

  • Store data reliably
  • Keep it readily available for users
slide-17
SLIDE 17

Data storage: Replication

  • Store data reliably
  • Keep it readily available for users
  • Very large overhead
  • Moderate reliability
  • Local recovery:

Lose one machine, access one

slide-18
SLIDE 18

Data storage: Erasure coding

  • Store data reliably
  • Keep it readily available for users
  • Low overhead
  • High reliability
  • No local recovery:

Loose one machine, access k

…

… …

k data chunks n-k parity chunks

Need: Erasure codes with local decoding

slide-19
SLIDE 19

Codes for data storage

  • Goals:
  • (Cost) minimize the number of parities.
  • (Reliability) tolerate any pattern of h+1 simultaneous failures.
  • (Availability) recover any data symbol by accessing at most r other symbols
  • (Computational efficiency) use a small finite field to define parities.

X1 X2 Xk

…

P1

…

Pn-k

slide-20
SLIDE 20

Local reconstruction codes

  • Def: An (r,h) – Local Reconstruction Code (LRC) encodes k symbols to n symbols, and
  • Corrects any pattern of h+1 simultaneous failures;
  • Recovers any single erased data symbol by accessing at most r other symbols.
slide-21
SLIDE 21

Local reconstruction codes

  • Def: An (r,h) – Local Reconstruction Code (LRC) encodes k symbols to n symbols, and
  • Corrects any pattern of h+1 simultaneous failures;
  • Recovers any single erased data symbol by accessing at most r other symbols.
  • Theorem[GHSY]: In any (r,h) – (LRC), redundancy n-k satisfies π‘œ βˆ’ 𝑙 β‰₯

𝑙 𝑠 + β„Ž.

slide-22
SLIDE 22

Local reconstruction codes

  • Def: An (r,h) – Local Reconstruction Code (LRC) encodes k symbols to n symbols, and
  • Corrects any pattern of h+1 simultaneous failures;
  • Recovers any single erased data symbol by accessing at most r other symbols.
  • Theorem[GHSY]: In any (r,h) – (LRC), redundancy n-k satisfies π‘œ βˆ’ 𝑙 β‰₯

𝑙 𝑠 + β„Ž.

  • Theorem[GHSY]: If r | k and h<r+1; then any (r,h) – LRC has the following topology:

X1 Xr

…

Xk-r Xk

…

Hh H1 L1 Lg

…

… …

Light parities Heavy parities Data symbols Local group

slide-23
SLIDE 23

Local reconstruction codes

  • Def: An (r,h) – Local Reconstruction Code (LRC) encodes k symbols to n symbols, and
  • Corrects any pattern of h+1 simultaneous failures;
  • Recovers any single erased data symbol by accessing at most r other symbols.

X1 Xr

…

Xk-r Xk

…

Hh H1 L1 Lg

…

… …

  • Fact: There exist (r,h) – LRCs with optimal redundancy over a field of size k+h.

Light parities Heavy parities Data symbols Local group

  • Theorem[GHSY]: In any (r,h) – (LRC), redundancy n-k satisfies π‘œ βˆ’ 𝑙 β‰₯

𝑙 𝑠 + β„Ž.

  • Theorem[GHSY]: If r | k and h<r+1; then any (r,h) – LRC has the following topology:
slide-24
SLIDE 24

Reliability

Set k=8, r=4, and h=3.

X1 L1 X2 X3 X4 X5 X6 X7 X8 H3 H2 H1 L2

slide-25
SLIDE 25

Reliability

Set k=8, r=4, and h=3.

X1 L1

  • All 4-failure patterns are correctable.

X2 X3 X4 X5 X6 X7 X8 H3 H2 H1 L2

slide-26
SLIDE 26

Reliability

Set k=8, r=4, and h=3.

X1 L1

  • All 4-failure patterns are correctable.
  • Some 5-failure patterns are not correctable.

X2 X3 X4 X5 X6 X7 X8 H3 H2 H1 L2

slide-27
SLIDE 27

Reliability

Set k=8, r=4, and h=3.

X1 L1

  • All 4-failure patterns are correctable.
  • Some 5-failure patterns are not correctable.
  • Other 5-failure patterns might be correctable.

X2 X3 X4 X5 X6 X7 X8 H3 H2 H1 L2

slide-28
SLIDE 28

Reliability

Set k=8, r=4, and h=3.

X1 L1

  • All 4-failure patterns are correctable.
  • Some 5-failure patterns are not correctable.
  • Other 5-failure patterns might be correctable.

X2 X3 X4 X5 X6 X7 X8 H3 H2 H1 L2

slide-29
SLIDE 29

Combinatorics of correctable failure patterns

Def: A regular failure pattern for a (r,h)-LRC is a pattern that can be obtained by failing

  • ne symbol in each local group and h extra symbols.

X1 L1 X2 X3 X4 X5 X6 X7 X8 H3 H2 H1 L2 X1 L1 X2 X3 X4 X5 X6 X7 X8 H3 H2 H1 L2

Theorem:

  • Every failure pattern that is not dominated by a regular failure pattern is not

correctable by any LRC.

  • There exist LRCs that correct all regular failure patterns.
slide-30
SLIDE 30

Maximally recoverable codes

Def: An (r,h)-LRC is maximally recoverable if it corrects all regular failure patterns. Theorem: Maximally reliable (r,h)-LRCs exist. Proof sketch: Pick the coefficients in heavy parities at random from a large finite field. The tradeoff: Larger fields allow for more reliable codes up to maximal recoverability. We want both: small field size (efficiency) and maximal recoverability. Asymptotic setting: β„Ž = 𝑃 1 , 𝑠 = 𝑃 1 , 𝑙 β†’ ∞. Random choice needs a field of size at least: Ξ© π‘™β„Žβˆ’1 .

slide-31
SLIDE 31

Explicit maximally recoverable codes

Theorem[GHJY]: There exist maximally recoverable (r,h)-LRC over a field of size

𝑑𝑙

β„Žβˆ’1 1βˆ’ 1 2𝑠 .

Comparison:

  • Our alphabet grows as 𝑃 π‘™β„Žβˆ’1 or slower.
  • Beats random codes for small h and large h.
  • Our only lower bound for the alphabet size thus far is k+1 independent of h.
slide-32
SLIDE 32

Code construction

We use dual constraints to specify the code.

π’šπŸ π’šπŸ‘ … π’šπ’” π‘΄πŸ

…

π’šπ’βˆ’π’” π’šπ’βˆ’π’”+𝟐 … π’šπ’ 𝑴𝒍/𝒔 π‘°πŸ π‘°πŸ‘ … π‘°π’Š 1 1 … 1 1 1 1 … 1 1 𝑙 𝑠 h

π›½π‘—π‘˜ π›½π‘—π‘˜

2

… π›½π‘—π‘˜

2β„Žβˆ’1

Element π›½π‘—π‘˜ appears in the j-th column of the i-th group.

We consider a sequence field extensions 𝐺

2 βŠ† 𝐺2𝑏 βŠ† 𝐺2𝑐.

{πœŠπ‘˜} βŠ† 𝐺2𝑏 form a basis over 𝐺

2.

{πœ‡π‘—} βŠ† 𝐺2𝑐 are β„Ž-independent over 𝐺2𝑏.

π›½π‘—π‘˜=πœŠπ‘˜ Γ— πœ‡π‘—.

𝑙 𝑠 + 1 local groups.

slide-33
SLIDE 33

Erasure correction

k=8, r=4, h=2.

π’šπŸ π’šπŸ‘ π’šπŸ’ π’šπŸ“ π‘΄πŸ π’šπŸ” π’šπŸ• π’šπŸ– π’šπŸ— π‘΄πŸ‘ π‘°πŸ π‘°πŸ‘ π‘°πŸ’ 1 1 1 1 1 1 1 1 1 1 𝛽11 𝛽12 𝛽21 𝛽22 𝛽31 𝛽11

2 𝛽12 2

𝛽21

2 𝛽22 2

𝛽31

2

𝛽11

4 𝛽12 4

𝛽21

4 𝛽22 4

𝛽31

4

1 1 1 1 𝛽11 𝛽12 𝛽21 𝛽22 𝛽31 𝛽11

2 𝛽12 2 𝛽21 2 𝛽22 2 𝛽31 2

𝛽11

4 𝛽12 4 𝛽21 4 𝛽22 4 𝛽31 4

𝛽11+𝛽12 𝛽21+𝛽22 𝛽31 𝛽11

2 +𝛽12 2

𝛽21

2 +𝛽22 2

𝛽31

2

𝛽11

4 +𝛽12 4

𝛽21

4 +𝛽22 4

𝛽31

4

(𝛽11+𝛽12) (𝛽21+𝛽22) 𝛽31 (𝛽11+𝛽12) 2 (𝛽21+𝛽22 )2 𝛽31

2

(𝛽11+𝛽12) 4 (𝛽21+𝛽22 )4 𝛽31

4

(𝛽11+𝛽12) (𝛽21+𝛽22) 𝛽31 (𝜊1 + 𝜊2) Γ— πœ‡1 (𝜊1 + 𝜊2) Γ— πœ‡2 𝜊1 Γ— πœ‡3

slide-34
SLIDE 34

Looking forward

The main challenge in LRC design is to obtain maximally reliable codes over small finite

  • fields. Empirical evidence suggests that there is a tradeoff between reliability and

computational efficiency.

Open questions:

  • Study the tradeoff between redundancy and locality.
  • Develop tight bounds for redundancy when 𝑓 is a constant larger than 1.