Entangled Polynomial Codes for Secure, Private, and Batch - - PowerPoint PPT Presentation

▶
entangled polynomial codes for secure private and batch
SMART_READER_LITE
LIVE PREVIEW

Entangled Polynomial Codes for Secure, Private, and Batch - - PowerPoint PPT Presentation

Oct 03, 2023 198 likes •414 views

Entangled Polynomial Codes for Secure, Private, and Batch Distributed Matrix Multiplication: Breaking the "Cubic" Barrier QIAN YU (USC) joint work with Salman Avestimehr (USC) IEEE International Symposium on Information Theory (ISIT)

slide-1
SLIDE 1

Entangled Polynomial Codes for Secure, Private, and Batch Distributed Matrix Multiplication: Breaking the "Cubic" Barrier

QIAN YU (USC)

joint work with Salman Avestimehr (USC) IEEE International Symposium on Information Theory (ISIT) July 2020

slide-2
SLIDE 2

Backgrounds: Challenges in Modern Distributed Computing

Qian Yu - Entangled Polynomial Codes (ISIT) July 2020

Computational Scalability:

  • Node failure (stragglers)
  • Byzantine attack (adversaries)
  • Privacy breach (colluding workers)

…

adversaries stragglers colluding workers "The Tail at Scale" Google, 2013

slide-3
SLIDE 3

An Introduction to Coded Computing

Qian Yu - Entangled Polynomial Codes (ISIT) July 2020

f( ) f( )

Problem Parameters

  • Input: X
  • Compute: g(X)
  • N evaluations of f

f( )

…

N workers

Coded Computing Yu et al, NIPS 2017 Goal: minimize recovery threshold given N,g,f

Design Space

  • Encoding functions
  • Decoding functions

Recovery Threshold: # workers’ results needed for recovering final outputs (equivalent to straggler resiliency and computation security)

slide-4
SLIDE 4

An Introduction to Coded Computing

Qian Yu - Entangled Polynomial Codes (ISIT) July 2020 Computation f - Channel Coded variables – Coded symbols Worker – Channel usage Recovery threshold – Communication rate (tolerate errors) Coded Computing vs Classical Shannon Theory

Key Challenge: how to design codes so that

  • 1. Computation on coded data is meaningful?
  • 2. Encoding and Decoding has low complexity

Require New coding ideas! Differences:

  • Algebraic vs probabilistic or

combinatorial

  • Coding complexity

Linear codes as a natural assumption

slide-5
SLIDE 5

Solution: Polynomial Coded Computing (PCC)

Qian Yu - Entangled Polynomial Codes (ISIT) July 2020

Yu et al, NIPS 2017

  • Basic idea:
  • Design polynomials for input entries, s.t. the composition of f and polynomials recover the output.
  • Recovery threshold ≤ degree of composed polynomial + 1
  • Example: Matrix Multiplication (Column-wise partition)
  • Input: X = (𝑩 , 𝑪)
  • Compute: g(X)= 𝑩𝑼 ⋅ 𝑪
slide-6
SLIDE 6

Solution: Polynomial Coded Computing (PCC)

Qian Yu - Entangled Polynomial Codes (ISIT) July 2020

Yu et al, NIPS 2017

  • Example: Matrix Multiplication (Column-wise partition)

𝒏 submatrices 𝒐 submatrices

  • Input: X = (𝑩 , 𝑪)
  • Compute: g(X)= 𝑩𝑼 ⋅ 𝑪
  • N evaluations of f= ⋅
  • Basic idea:
  • Design polynomials for input entries, s.t. the composition of f and polynomials recover the output.
  • Recovery threshold ≤ degree of composed polynomial + 1
  • Goal: recover all 𝒏𝒐

pair-wise products Ai

TBj

Requires 𝒏𝒐 workers

slide-7
SLIDE 7

Solution: Polynomial Coded Computing (PCC)

Qian Yu - Entangled Polynomial Codes (ISIT) July 2020

First optimal code for non-linear operations!

Yu et al, NIPS 2017

  • Example: Matrix Multiplication (Column-wise partition)
  • Basic idea:
  • Design polynomials for input entries, s.t. the composition of f and polynomials recover the output.
  • Recovery threshold ≤ degree of composed polynomial + 1
  • Goal: recover all 𝒏𝒐

pair-wise products Ai

TBj

Requires 𝒏𝒐 workers

  • Design polynomials

Encoding: worker i obtains 𝑏 𝑗 , 𝑐(𝑗) Worker i essentially calculates:

Recovery threshold = 𝒏𝒐

slide-8
SLIDE 8

Solution: Polynomial Coded Computing (PCC)

Qian Yu - Entangled Polynomial Codes (ISIT) July 2020

First optimal code for non-linear operations!

…

Recovery threshold Recovery threshold Recovery threshold

slide-9
SLIDE 9

Qian Yu, Mohammad Ali Maddah-Ali, Salman Avestimehr, “Straggler Mitigation in Distributed Matrix Multiplication: Fundamental Limits and Optimal Coding”, Jan 2018, ISIT, TIT.

Problem Formulation: Block Matrix Multiplication

A better design for straggler mitigation (Entangled polynomial codes) Orderwise improvement for Secure, Private, Batch Matrix Multiplication

Qian Yu - Entangled Polynomial Codes (ISIT) July 2020

slide-10
SLIDE 10

Qian Yu (USC)

p × 𝑛 blocks p × 𝑜 blocks

Qian Yu, Mohammad Ali Maddah-Ali, Salman Avestimehr, “Straggler Mitigation in Distributed Matrix Multiplication: Fundamental Limits and Optimal Coding”, Jan 2018, ISIT, TIT.

each worker 𝑗 computes

෩ 𝑩𝒋

𝑼

෩ 𝑪𝒋 ⋅

Problem Formulation: Block Matrix Multiplication

  • Flexible tradeoff in computation,

storage, communication

  • Also studied in Fahim et al,

Allerton 2017

A better design for straggler mitigation (Entangled polynomial codes) Orderwise improvement for Secure, Private, Batch Matrix Multiplication

Qian Yu - Entangled Polynomial Codes (ISIT) July 2020

slide-11
SLIDE 11

Qian Yu, Mohammad Ali Maddah-Ali, Salman Avestimehr, “Straggler Mitigation in Distributed Matrix Multiplication: Fundamental Limits and Optimal Coding”, Jan 2018, ISIT, TIT.

Relate works

  • Polydot (Allerton 2017)

𝑛𝑜(2𝑞 − 1) for 𝑛 = 𝑜

  • Generalized Polydot (arxiv ver. May 2018)

𝑞𝑛𝑜 + 𝑞 − 1

Block Matrix Multiplication (Straggler Mitigation)

p × 𝑛 blocks p × 𝑜 blocks

Best known result

Entangled polynomial codes achieves Recovery threshold ≤ Theorem (Yu et al, Jan 2018)

Cubic ver. Sub-cubic ver.

Qian Yu - Entangled Polynomial Codes (ISIT) July 2020 All related works (Cubic interpretation, at least 𝒒𝒏𝒐 workers) Entangled Polynomial Codes: Order-wise coding gain for any large p,m,n

slide-12
SLIDE 12

Entangled Polynomial Codes

Qian Yu, Mohammad Ali Maddah-Ali, Salman Avestimehr, “Straggler Mitigation in Distributed Matrix Multiplication: Fundamental Limits and Optimal Coding”, Jan 2018, ISIT, TIT.

  • Breaking the “Cubic” Barrier

Qian Yu - Entangled Polynomial Codes (ISIT) July 2020

Entangled polynomial codes achieves Recovery threshold ≤ Theorem (Yu et al, Jan 2018)

Tensor decomposition since ~1960s Theorem: does not guarantee straggler resiliency What is missing?: Design codes for subcubic interpretation! All requiring at least 𝒒𝒏𝒐

  • recover linear combinations
  • f 𝒒𝒏𝒐 pair-wise products

෍

𝒋

Aij

TBik

Relate works

Cubic interpretation

  • 𝑆(𝑞, 𝑛, 𝑜)= # multiplications needed for multiplying an 𝑛-by-𝑞

matrix by a 𝑞-by-𝑜 matrix.

  • Bilinear complexity is sub-cubic: 𝑆 𝑞, 𝑛, 𝑜 = 𝑝 𝑞𝑛𝑜 for any

large p,m,n

  • Lazy proof (special case), 𝑆 2,2,2 = 7, 𝑆 2𝑦, 2𝑦, 2𝑦 ≤ 7𝑦

What if 𝑆 𝑞, 𝑛, 𝑜 not yet known? Sub-cubic upper bound exists for any large p,m,n

slide-13
SLIDE 13

Entangled Polynomial Codes

Qian Yu, Mohammad Ali Maddah-Ali, Salman Avestimehr, “Straggler Mitigation in Distributed Matrix Multiplication: Fundamental Limits and Optimal Coding”, Jan 2018, ISIT, TIT.

  • Breaking the “Cubic” Barrier
  • Not by just plugging in fast matrix multiplication

Qian Yu - Entangled Polynomial Codes (ISIT) July 2020

Entangled polynomial codes achieves Recovery threshold ≤ Theorem (Yu et al, Jan 2018)

Design codes for subcubic interpretation!

Cubic ver. Sub-cubic ver.

different coding structures

slide-14
SLIDE 14

Entangled Polynomial Codes

Coded Computing Matrix Multiplication Algorithms Fault-Tolerant Computing Secure Matrix Multiplication Private Matrix Multiplication Batch Matrix Multiplication

Unified framework with

  • rder-wise improvement

All related works (Cubic interpretation, at least 𝒒𝒏𝒐 workers) Entangled Polynomial Codes: Order-wise coding gain for any large p,m,n

Theorem (Entangled Polynomial Codes)

Entangled Polynomial Codes

  • Breaking the “Cubic” Barrier

Qian Yu - Entangled Polynomial Codes (ISIT) July 2020

slide-15
SLIDE 15

Secure, Private, Batch Matrix Multiplication

Qian Yu - Entangled Polynomial Codes (ISIT) July 2020

One/Two-sided secure:

One/Two of inputs information theoretically private against 𝑈 colluding workers

Secure Distributed Matrix Multiplication Private Distributed Matrix Multiplication Private:

Query D information theoretically private against any worker

Batch Distributed Matrix Multiplication Element-wise multiply two lists of matrices (Length: L) …and all possible mixtures

slide-16
SLIDE 16

Qian Yu - Entangled Polynomial Codes (ISIT) July 2020

Entangled Polynomial Codes

  • Breaking the “Cubic” Barrier Secure, Private, Batch Matrix Multiplication

all possible mixtures Secure Private Batch

Relate works

  • Nodehi et al, 2018
  • Aliasgari et al, 2019
  • Jia and Jafar, 2019

…

Entangled polynomial codes achieves Recovery threshold ≤ Θ(𝑆(𝑞, 𝑛, 𝑜)) for all setting Theorem Straggler resiliency comes for free Remark 1 (Polynomial Coded Computing) Optimal within a factor of 2 Remark 2 (Converse)

Requiring 𝐩(𝒒𝒏𝒐) workers for all settings All built upon cubic interpretation require at least 𝒒𝒏𝒐 workers

slide-17
SLIDE 17

Qian Yu - Entangled Polynomial Codes (ISIT) July 2020

Entangled Polynomial Codes

  • Breaking the “Cubic” Barrier Secure, Private, Batch Matrix Multiplication

Entangled polynomial codes achieves recovery threshold ≤ 2𝑆 𝑞, 𝑛, 𝑜 − 1

Theorem (Secure Distributed Matrix Multiplication) Example 1: Secure Distributed Matrix Multiplication

State of the arts

Require at least 𝒒𝒏𝒐 +𝑼 workers for one-sided security +𝟑𝑼 workers for one-sided security

One/Two-sided secure:

One/Two of inputs information theoretically private against 𝑈 colluding workers

Secure Distributed Matrix Multiplication

+𝑈 for one-sided security +2𝑈 for two-sided security

slide-18
SLIDE 18

Qian Yu - Entangled Polynomial Codes (ISIT) July 2020

Entangled Polynomial Codes

  • Breaking the “Cubic” Barrier Secure, Private, Batch Matrix Multiplication

Entangled polynomial codes achieves recovery threshold ≤ 2𝑆 𝑞, 𝑛, 𝑜 − 1

Theorem (Secure Distributed Matrix Multiplication) Example 1: Secure Distributed Matrix Multiplication

One/Two-sided secure:

One/Two of inputs information theoretically private against 𝑈 colluding workers

Secure Distributed Matrix Multiplication

+𝑈 for one-sided security +2𝑈 for two-sided security

Step 1: tensor decomposition Two vectors of length 𝑆(𝑞, 𝑛, 𝑜) suffice to compute their elementwise product

slide-19
SLIDE 19

Qian Yu - Entangled Polynomial Codes (ISIT) July 2020

Entangled Polynomial Codes

  • Breaking the “Cubic” Barrier Secure, Private, Batch Matrix Multiplication

Entangled polynomial codes achieves recovery threshold ≤ 2𝑆 𝑞, 𝑛, 𝑜 − 1

Theorem (Secure Distributed Matrix Multiplication) Example 1: Secure Distributed Matrix Multiplication

One/Two-sided secure:

One/Two of inputs information theoretically private against 𝑈 colluding workers

Secure Distributed Matrix Multiplication

+𝑈 for one-sided security +2𝑈 for two-sided security

Step 2: Optimal coding for elementwise product

No security requirement One-sided secure Pad by 𝑈 random keys Pad by 𝑈 random keys Pad by 𝑈 random keys Two-sided secure

Satisfies security requirement Does not interferer decodability Each padded key increases degree by 1 Recovery threshold +T Recovery threshold +2T QED

slide-20
SLIDE 20

Qian Yu - Entangled Polynomial Codes (ISIT) July 2020

Entangled Polynomial Codes

  • Breaking the “Cubic” Barrier Secure, Private, Batch Matrix Multiplication

State of the art

Require at least 𝐌𝒒𝒏𝒐 workers

Example 2: Batch Distributed Matrix Multiplication

  • View L-multiplications as one bilinear map
  • Define tensor rank 𝑺(𝑴, 𝒒, 𝒏, 𝒐)
  • Replace any 𝑺(𝒒, 𝒏, 𝒐) by 𝑺(𝑴, 𝒒, 𝒏, 𝒐)
  • Sub-additivity
  • 𝑺 𝑴, 𝒒, 𝒏, 𝒐 ≤ 𝐌𝑺 𝒒, 𝒏, 𝒐

QED 𝒑(𝑴𝒒𝒏𝒐)

slide-21
SLIDE 21

Thank you! Questions?