L ECTURE 10 Last time Multipurpose sketches Count-min and - - PowerPoint PPT Presentation

10/6/2020

Sublinear Algorithms

LECTURE 10

Last time

• Multipurpose sketches
• Count-min and count-sketch
• Range queries, heavy hitters, quantiles

Today

• Limitations of streaming algorithms
• Communication complexity

Sofya Raskhodnikova;Boston University

Recall: Frequency Moments Estimation

Input: a stream 𝑏1, 𝑏2, … , 𝑏𝑛 ∈ 𝑜 𝑛

• The frequency vector of the stream is 𝑔 = (𝑔

1, … , 𝑔 𝑜),

where 𝑔

𝑗 is the number of times 𝑗 appears in the stream

• The 𝑞-th frequency moment is 𝐺

𝑞 =

𝑔

𝑞 𝑞 = σ𝑗=1 𝑜

𝑔

𝑗 𝑞

𝐺0 is the number of nonzero entries of 𝑔 (# of distinct elements) 𝐺

1 = 𝑛 (# of elements in the stream)

𝐺2 = 𝑔

2 2 is a measure of non-uniformity

used e.g. for anomaly detection in network analysis 𝐺

∞ = max 𝑗

𝑔

𝑗 is the most frequent element

We obtained streaming algorithms for 𝐺

0, 𝐺 1, 𝐺 2.

What about 𝐺

3 to 𝐺 ∞?

2

Communication Complexity

A Method for Proving Lower Bounds

(Randomized) Communication Complexity

4

Compute 𝐷 𝑦, 𝑧 0100 11 001 ⋯ 0011 Bob Alice 𝐽𝑜𝑞𝑣𝑢: 𝑦 Input: 𝑧 1101000101110101110101010110… 𝑇ℎ𝑏𝑠𝑓𝑒 𝑠𝑏𝑜𝑒𝑝𝑛 𝑡𝑢𝑠𝑗𝑜𝑕 Goal: minimize the number of bits exchanged.

• Communication complexity of a protocol is the maximum number of bits

exchanged by the protocol.

• Communication complexity of a function 𝐷, denoted 𝑆(𝐷), is the

communication complexity of the best protocol for computing C.

Partially based on slides by Eric Blais

Example: Set Disjointness 𝐸𝐽𝑇𝐾𝒍

5

Theorem [Kalyanasundaram Schmitger 92, Razborov 92] 𝑆 DISJ𝑙 ≥ Ω 𝑙 for all 𝑙 ≤

𝑜 2.

Compute 𝐸𝐽𝑇𝐾𝑙 𝑇, 𝑈 = ቊ𝒃𝒅𝒅𝒇𝒒𝒖 if 𝑇 ∩ 𝑈 = ∅ 𝒔𝒇𝒌𝒇𝒅𝒖

• therwise

Bob Alice 𝐽𝑜𝑞𝑣𝑢: 𝑇 ⊆ [𝑜], 𝑇 = 𝑙. Input: 𝑈 ⊆ [𝑜], 𝑈 = 𝑙 1101000101110101110101010110…

One-Way Communication Complexity

6

Compute 𝐷 𝑦, 𝑧 𝑛1 Bob Alice 𝐽𝑜𝑞𝑣𝑢: 𝑦 Input: 𝑧 1101000101110101110101010110… 𝑇ℎ𝑏𝑠𝑓𝑒 𝑠𝑏𝑜𝑒𝑝𝑛 𝑡𝑢𝑠𝑗𝑜𝑕 Goal: minimize the number of bits Alice sends to Bob. One-way communication complexity of a function 𝐷, denoted 𝑆→(𝐷), is the communication complexity of the best one-way protocol for computing C.

3-Player One-Way Communication Complexity

7

𝑇ℎ𝑏𝑠𝑓𝑒 𝑠𝑏𝑜𝑒𝑝𝑛 𝑡𝑢𝑠𝑗𝑜𝑕 Goal: minimize 𝑛1 + |𝑛2|.

• Require correct output w.p. at least 2/3 over the random string

Carol Alice 𝐽𝑜𝑞𝑣𝑢: 𝑦 Input: 𝑨 Input: 𝑧 𝑛1 𝑛2 1101000101110101110101010110… Bob Compute 𝐷 𝑦, 𝑧, 𝑨

Converting Streaming Algorithm to CC Protocol

8

An 𝑡-bit algorithm 𝐵 for 𝓠 gives a 2𝑡-bit protocol for 𝐷

• Alice runs 𝐵 on 𝑡1 and sends memory state, 𝑛1, to Bob
• Bob instantiates 𝐵 with 𝑛1, runs 𝐵 on 𝑡2, sends memory state, 𝑛2, to Carol
• Carol instantiates 𝐵 with 𝑛2, runs 𝐵 on 𝑡3 to get 𝓠(𝑡1 ∘ 𝑡2 ∘ 𝑡3) and

computes 𝐷(𝑦, 𝑧, 𝑨) 𝐽𝑜𝑞𝑣𝑢: 𝑦 Input: 𝑨 Input: 𝑧 𝑛1 𝑛2 Let 𝓠 be a streaming problem.

• Suppose there is a transformation 𝑦 → 𝑡1, 𝑧 → 𝑡2, 𝑨 → 𝑡3 such that

𝓠(𝑡1 ∘ 𝑡2 ∘ 𝑡3) suffices to compute 𝐷(𝑦, 𝑧, 𝑨) Compute 𝐷 𝑦, 𝑧, 𝑨 𝑡1 𝑡2 𝑡3

Based on Andrew McGregor’s slides: https://people.cs.umass.edu/~mcgregor/711S18/lowerbounds-1.pdf

Converting Streaming Algorithm to CC Protocol

9

An 𝑡-bit algorithm 𝐵 for 𝓠 gives a 2𝑡-bit protocol for 𝐷

• If there are 𝑞 players than the protocol uses 𝑞 − 1 𝑡 bits
• A lower bound 𝑀 for computing 𝐷 implies 𝑐 = Ω

𝑀 𝑞

𝐽𝑜𝑞𝑣𝑢: 𝑦 Input: 𝑨 Input: 𝑧 𝑛1 𝑛2 Let 𝓠 be a streaming problem.

• Suppose there is a transformation 𝑦 → 𝑡1, 𝑧 → 𝑡2, 𝑨 → 𝑡3 such that

𝓠(𝑡1 ∘ 𝑡2 ∘ 𝑡3) suffices to compute 𝐷(𝑦, 𝑧, 𝑨) Compute 𝐷 𝑦, 𝑧, 𝑨 𝑡1 𝑡2 𝑡3

Based on Andrew McGregor’s slides: https://people.cs.umass.edu/~mcgregor/711S18/lowerbounds-1.pdf

A lower bound using CC method

Approximating 𝐺

∞

Application: Approximating 𝑮∞

Proof: Reduction from Set Disjointness On input 𝑦, 𝑧 ∈ 0,1 𝑜, players generate 𝑡1 = {𝑘: 𝑦𝑘 = 1} and 𝑡2 = {𝑘: 𝑧𝑘 = 1}

• Then 𝐺

∞ = 1 if 𝑦, 𝑧 represent disjoint sets, and 𝐺 ∞ = 2, otherwise.

• An 𝑡-space algorithm implies an 𝑡-bit protocol:

𝑡 = Ω 𝑜

11

Example: 0 0 1 1 0 0 (1 0 1 0 1 0) → 〈3,4; 1,3,5〉

by communication complexity of 𝑇𝑓𝑢 𝐸𝑗𝑡𝑘𝑝𝑗𝑜𝑢𝑜𝑓𝑡𝑡 Output ≤ 4/3

Based on Andrew McGregor’s slides: https://people.cs.umass.edu/~mcgregor/711S18/lowerbounds-1.pdf

Output ≥ 3/2

Theorem Every algorithm that computes 4/3-approximation of 𝐺

∞

(w.p. ≥2/3) needs Ω(𝑜) space.

A lower bound using CC method

Computing the median of a stream

Index

• Alice gets an 𝑜-bit string 𝑦, and Bob gets an index 𝑘 ∈ [𝑜].
• Define 𝐽𝑜𝑒𝑓𝑦(𝑦, 𝑘) = 𝑦𝑘.
• One-way communication complexity of 𝐽𝑜𝑒𝑓𝑦(𝑦, 𝑘) is Ω 𝑜

13

Application: Finding the Median of a Stream

14

Proof: Reduction from Index.

• On input 𝑦 ∈ 0,1 𝑜, Alice generates 𝑡1 = {2𝑗 + 𝑦𝑗: 𝑗 ∈ [𝑜]}
• On input 𝑘 ∈ [𝑜], Bob generates

𝑡2 = 𝑜 − 𝑘 copies of 0 and 𝑘 − 1 copies of 2𝑜 + 2

• Then 𝑛𝑓𝑒𝑗𝑏𝑜 𝑡1 ∘ 𝑡2 = 2𝑘 + 𝑦𝑘 and Index 𝑦, 𝑘 = 2𝑘 + 𝑦𝑘 𝑛𝑝𝑒 2
• An 𝑡-space algorithm implies an 𝑡-bit protocol:

𝑡 = Ω 𝑜

Theorem Every algorithm that computes the median of an (2𝑜 − 1)- element stream exactly (w.p. ≥2/3) needs Ω(𝑜) space.

Example: 0 0 1 1 0 1 1 → 〈2,4,7,9,10,13,15〉 Example: 𝑘 = 2 → 〈0,0,0,0,0,16〉

Based on Andrew McGregor’s slides: https://people.cs.umass.edu/~mcgregor/711S18/lowerbounds-1.pdf

by 1-way communication complexity of 𝐽𝑜𝑒𝑓𝑦

A lower bound using CC method

Approximating Frequency Moments

[Bar-Yossef, Jayram, Kumar, Sivakumar 04]

Multi-party Set Disjointness

• Consider a 𝑞 × 𝑜 binary matrix 𝑁 where each column has weight 0, 1 or 𝑞

1 1 1 1 1 1 1 1

• The input of player 𝑗 is row 𝑗 of 𝑁

𝐸𝐽𝑇𝐾 𝑞 𝑁 = ቊ0 if there is a column of 1s 1

• therwise
• Communication complexity of 𝐸𝐽𝑇𝐾 𝑞 𝑁 is Ω

𝑜 𝑞

16

1 4 3 5 6

Example:

Based on Andrew McGregor’s slides: https://people.cs.umass.edu/~mcgregor/711S18/lowerbounds-1.pdf

Application: Frequency Moments for 𝒍 > 𝟑

Proof: Reduction from multi-party Set Disjointness

• On input 𝑁 ∈ 0,1 𝑞×𝑜, player 𝑗 generates 𝑡𝑗 = {𝑘: 𝑁𝑗𝑘 = 1}
• If all columns have weight 0 or 1 then 𝐺𝑙 = σ𝑗=1

𝑜

𝑔

𝑗 𝑙 ≤ 𝑜

• If there is a column of weight 𝑞 then 𝐺𝑙 ≥ 𝑞𝑙
• A 2-approximation of 𝐺𝑙 distinguishes the cases if 𝑞𝑙 > 4𝑜 ⇔ 𝑞 > 4𝑜

1 𝑙

• An 𝑡-space algorithm implies 𝑡(𝑞 − 1)-bit protocol:

𝑡 = Ω 𝑜 𝑞2 = Ω 𝑜 4𝑜

2 𝑙

= Ω 𝑜1−2

𝑙

17

Every algorithm that 2-approximaes 𝐺𝑙 (w.p. ≥2/3) needs Ω 𝑜1−2

𝑙

space Thm.

1 4 3 5 6

Example: 1 1 1 1 1 1 1 1 → 〈3,4; 1,3,5; 3; 3,6〉

by communication complexity of 𝐸𝐽𝑇𝐾(𝑞) for constant 𝑙

Based on Andrew McGregor’s slides: https://people.cs.umass.edu/~mcgregor/711S18/lowerbounds-1.pdf

A lower bound using CC method

Distinct Elements

Gap Hamming

• Alice and Bob get 𝑜-bit strings 𝑦 and 𝑧, respectively.
• Hamming distance 𝐼𝑏𝑛(𝑦, 𝑧) is the number of positions on which 𝑦 and 𝑧

differ.

• Output: 𝐼𝑏𝑛(𝑦, 𝑧) with additive error 𝑜 w.p. ≥ 2/3
• Communication complexity of 𝐼𝑏𝑛(𝑦, 𝑧) is Ω 𝑜

even when |𝑦| and |𝑧| are known to both players

19

Based on Andrew McGregor’s slides: https://people.cs.umass.edu/~mcgregor/711S18/lowerbounds-1.pdf

Application: Distinct Elements

Proof: Reduction from Gap Hamming On input 𝑦, 𝑧 ∈ 0,1 𝑜, players generate 𝑡1 = {𝑘: 𝑦𝑘 = 1} and 𝑡2 = {𝑘: 𝑧𝑘 = 1}

• Then 2𝐺0 = 𝑦 + 𝑧 + 𝐼𝑏𝑛(𝑦, 𝑧)
• When |𝑦| is known to Bob,

(1 + 𝜁)-approximation of 𝐺0 gives an additive approximation to Ham 𝑦, 𝑧 𝜁 ⋅ 𝑦 + 𝑧 + 𝐼𝑏𝑛 𝑦, 𝑧 2 ≤ 𝜁𝑜 ≤ 𝑜

• An 𝑡-space algorithm implies an 𝑡-bit protocol:

𝑡 = Ω 𝑜 = Ω 1 𝜁2

20

Thm. Example: 0 0 1 1 0 0 (1 0 1 0 1 0) → 〈3,4; 1,3,5〉

by communication complexity of 𝐻𝑏𝑞 𝐼𝑏𝑛𝑛𝑗𝑜𝑕 for 𝜁 ≤ 1/ 𝑜

Based on Andrew McGregor’s slides: https://people.cs.umass.edu/~mcgregor/711S18/lowerbounds-1.pdf

Every algorithm (1 + 𝜁)-approximing 𝐺0 (w.p. ≥2/3) needs Ω 1/𝜁2 space