under Modular Updates Shachar Lovett (UCSD) Kaave Hosseini (UCSD - - PowerPoint PPT Presentation

Optimality of Linear Sketching under Modular Updates

Shachar Lovett (UCSD)

Kaave Hosseini (UCSD → CMU), Grigory Yaroslavtsev (Indiana)

Streaming and sketching

Streaming with binary updates

• Counters 𝑦1, … , 𝑦𝑜 ∈ 𝔾2
• Stream of updates: 𝑦𝑗 ← 𝑦𝑗 ⊕ 1
• At the end, want to compute function 𝑔(𝑦1, … , 𝑦𝑜)
• For which functions can we do it using ≪ 𝑜 memory?

Example

• Initially

000000

• Flip 𝑦1

100000

• Flip 𝑦5

100010

• Flip 𝑦2

110010

• Flip 𝑦5

100000

• …
• Compute

𝑔 𝑦1, … , 𝑦𝑜

Linear sketching

• Linear sketching is a useful primitive for streaming
• Let 𝑔: 𝔾2

𝑜 → {0,1}

• 𝑔 has a linear sketch of size k if it factors as 𝑔 𝑦 = 𝑞(𝑀 𝑦 ) where:

(i) 𝑀: 𝔾2

𝑜 → 𝔾2 𝑙 linear function

(ii) 𝑞: 𝔾2

𝑙 → {0,1} post-processing function

• Equivalently, the “Fourier dimension” of 𝑔 is 𝑙

Linear sketching implies streaming

• Assume 𝑔: 𝔾2

𝑜 → {0,1} factors as 𝑔 𝑦 = 𝑞(𝑀 𝑦 ) where

(i) 𝑀: 𝔾2

𝑜 → 𝔾2 𝑙 linear function

(ii) 𝑞: 𝔾2

𝑙 → {0,1} post-processing function

• To compute 𝑔 in the streaming model, maintain 𝑀 𝑦 ∈ 𝔾2

𝑙

• Easy to maintain under updates 𝑦𝑗 ← 𝑦𝑗 ⊕ 1
• Requires only k bits of memory

Randomized linear sketching

• Randomization makes linear sketching more powerful
• 𝑔: 𝔾2

𝑜 → {0,1} has a randomized linear sketch of size k if it can be

approximated by a distribution over linear sketches of size k

• That is, if exists a distribution over 𝑀, 𝑞 , where:

(i) 𝑀: 𝔾2

𝑜 → 𝔾2 𝑙 linear function

(ii) 𝑞: 𝔾2

𝑙 → {0,1} post-processing function

Such that Pr

L,p 𝑔 𝑦 = 𝑞(𝑀 𝑦 ) ≥ 1 − 𝜗

Randomized sketching gives additional power

• Consider the OR function: 𝑃𝑆 𝑦1, … , 𝑦𝑜 = 𝑦1 ∨ ⋯ ∨ 𝑦𝑜
• Deterministic sketching requires size n
• Randomized sketching can be done in size 𝑃 log 1/𝜗

(random parities)

Is linear sketching universal?

• Linear sketching seems like a very useful primitive for streaming
• Is it universal?
• That is: given a streaming algorithm that computes 𝑔 using 𝑙 bits of

memory, can we extract from it a linear sketch for 𝑔 of size ≈ 𝑙?

Universality of linear sketching

Universality of linear sketching

• Let 𝑔: 𝔾2

𝑜 → {0,1}

• Assume: randomized streaming algorithm supporting 𝑂 updates

and using 𝑙 bits of memory

• Goal: extract a randomized linear sketch of size ≈ 𝑙
• True if 𝑂 ≥ 222𝑜

[Li-Nguyen-Woordruff ‘14, Ai-Hu-Li-Woodruff ‘16]

• True if 𝑂 = Ω(𝑜) for random inputs [Kannan-Mossell-Sanyal-Yaroslavtsev ‘18]
• True if 𝑂 = Ω(𝑜2) [This work]

Main theorem: streaming

• Let 𝑔: 𝔾2

𝑜 → {0,1}

• Assume there exists a randomized streaming algorithm for 𝑔

supporting N = Ω 𝑜2 updates which uses 𝑙 bits of memory

• Then there exists a randomized linear sketch for 𝑔 of size 𝑃(𝑙)

Extensions (that I will not talk about)

• Extends to approximate real-valued functions 𝑔: 𝔾2

𝑜 → [0,1]

• Extends to functions over other fields
• Assuming only N = Ω(𝑜) updates are supported, we can still extract a

randomized linear sketch, but its size will be 𝑞𝑝𝑚𝑧(𝑙) instead of 𝑃(𝑙)

One-way communication complexity

One way communication complexity

• Model a streaming algorithm as a one-way communication protocol
• Break 𝑂 updates into 𝑁 = 𝑂/𝑜 chunks of size n each
• Setup: M players, holding inputs 𝑦1, … , 𝑦𝑁 ∈ 𝔾2

𝑜

(𝑦𝑗 is the aggregate of the n updates in the i-th chunk)

• Goal: compute 𝑔 𝑦1 + ⋯ + 𝑦𝑁
• Communication model: one-way

One way communication complexity

• M players, holding inputs 𝑦1, … , 𝑦𝑁 ∈ 𝔾2

𝑜

• Model: one-way communication with shared randomness
• Goal: output = 𝑔 𝑦1 + ⋯ + 𝑦𝑁

w.h.p over shared randomness

Player 1 Player 2 Player M

𝑦1 ∈ 𝔾2

𝑜

𝑦2 ∈ 𝔾2

𝑜

𝑦𝑁 ∈ 𝔾2

𝑜

Message 𝑛1 ∈ 0,1 𝑙 Message 𝑛2 ∈ 0,1 𝑙 Message 𝑛𝑁−1 ∈ 0,1 𝑙 Output 𝑝𝑣𝑢 ∈ {0,1}

…

Main theorem: one way communication

• Let 𝑔: 𝔾2

𝑜 → {0,1}

• Assume there exists a one-way communication protocol for computing

𝑔 𝑦1 + ⋯ + 𝑦𝑁 for 𝑁 = Ω(𝑜) players with k-bit messages (recall: this corresponds to 𝑂 = 𝑁𝑜 = Ω 𝑜2 binary updates)

• Then there exists a randomized linear sketch for f of size 𝑃 𝑙
• For 𝑁 = Ω 1 players, get linear sketch of size 𝑞𝑝𝑚𝑧(𝑙)

Proof

Proof

• The proof uses
• 1. Standard techniques in communication complexity
• 2. Additive combinatorics

Proof step 1: Yao’s minimax principle

• Let 𝑔: 𝔾2

𝑜 → {0,1}

• Fix a “hard distribution” 𝜈 over inputs
• Goal: linear sketch for 𝑔(𝑦) where 𝑦 ∼ 𝜈
• Embed hard distribution to the M players:
• First M-1 players inputs 𝑦1, … , 𝑦𝑁−1 are uniform in 𝔾2

𝑜

• Last player input 𝑦𝑁 is set so that 𝑦1 + ⋯ + 𝑦𝑁 = 𝑦
• Intuition: protocol has no information on x until the last player

Proof step 2: protocol structure

• Target: 𝑦 ∼ 𝜈
• Players inputs: 𝑦1, … , 𝑦𝑁−1 ∈ 𝔾2

𝑜 uniformly, 𝑦𝑁 = 𝑦1 + ⋯ + 𝑦𝑁−1 + 𝑦

• We may assume the protocol is deterministic
• Messages: 𝑛1 𝑦1 , 𝑛2 𝑛1, 𝑦2 , 𝑛3 𝑛1, 𝑛2, 𝑦3 , …
• Output: 𝑝𝑣𝑢 𝑛1, … , 𝑛𝑁−1, 𝑦𝑁
• With good probability out = 𝑔 𝑦1 + ⋯ + 𝑦𝑁 = 𝑔(𝑦)
• Can fix the messages (of the first M-1 players) to “typical messages”, without hurting

the success probability too much

Proof step 3: fixing to typical messages

• Fix typical messages 𝑛1

∗, 𝑛2 ∗, … , 𝑛𝑁−1 ∗

• Corresponds to the first M-1 players inputs:
• 𝐵1 = 𝑦1 ∈ 𝔾2

𝑜: 𝑛1 𝑦1 = 𝑛1 ∗

• 𝐵2 = 𝑦2 ∈ 𝔾2

𝑜: 𝑛2 𝑛1 ∗, 𝑦2 = 𝑛2 ∗

• …
• Sets are big: if the protocol uses k bits, then 𝐵𝑗 ≥ 2𝑜−𝑙
• After conditioning on 𝑦1 ∈ 𝐵1, … , 𝑦𝑁−1 ∈ 𝐵𝑁−1, protocol output is a

function of only 𝑦𝑁 = 𝑦1 + ⋯ + 𝑦𝑁−1 + 𝑦

Proof step 4: mixing

• Large sets 𝐵1, … , 𝐵𝑁−1 ⊂ 𝔾2

𝑜 of density 2−𝑙

• If we sample 𝑦1 ∈ 𝐵1, … , 𝑦𝑁−1 ∈ 𝐵𝑁−1 and 𝑦 ∼ 𝜈, then with high

probability 𝑝𝑣𝑢 𝑦1 + ⋯ + 𝑦𝑁−1 + 𝑦 = 𝑔 𝑦

• Technical lemma: for 𝑁 = Ω 𝑂 , the sum 𝑦1 + ⋯ + 𝑦𝑁−1 mixes in 𝔾2

𝑜

• More precisely, there exists a subspace 𝑊 ⊂ 𝔾2

𝑜 of co-dimension 𝑃(𝑙),

such that the sum is near invariant to a random shift from 𝑊

Proof step 5: extracting linear sketch

• We found a large subspace V of co-dimension O(k)
• If we sample 𝑦1 ∈ 𝐵1, … , 𝑦𝑁−1 ∈ 𝐵𝑁−1, 𝑦 ∼ 𝜈 and 𝑤 ∈ 𝑊, then with

high probability 𝑝𝑣𝑢 𝑦1 + ⋯ + 𝑦𝑁−1 + 𝑦 + 𝑤 = 𝑔 𝑦

• This allows to “factor out” V from the output function, and extract a

linear sketch for 𝑔 𝑦

Open problems

Linear sketching for modular updates

• For binary updates (or more general, modular updates), we prove that

linear sketching is universal

• Any streaming algorithm which supports 𝑂 = Ω 𝑜2 updates implies a

randomized linear sketch with similar guarantees

• Open problem 1: can this be improved to 𝑂 = Ω 𝑜 ?
• [Kannan-Mossell-Sanyal-Yaroslavtsev ‘18] proved a partial result in this

regime, giving a linear sketch for f on random inputs

• Our results in this regime incur a polynomial loss in the sketch size

Integer updates

• Streaming if often considered in the integer case
• Integer counters 𝑦1, … , 𝑦𝑜
• Updates 𝑦𝑗 += 1 or 𝑦𝑗 −= 1
• Sketching corresponds to linear functions over the integers
• The results of [Li-Nguyen-Woordruff ‘14, Ai-Hu-Li-Woodruff ‘16] work in

this regime as well, but require assuming 𝑂 ≥ 222𝑜

• Open problem 2: can our techniques be imported to this regime?
• Challenge: not clear what “mixing” should mean here