Sublinear Algorithms Lecture 6 Sofya Raskhodnikova Penn State - PowerPoint PPT Presentation

Sublinear Algorithms Lecture 6 Sofya Raskhodnikova Penn State University Thanks to Madhav Jha (Penn State) for help with creating these slides. 1

Communication Complexity A Method for Proving Lower Bounds [Blais Brody Matulef 11 ] Use known lower bounds for other models of computation Partially based on slides by Eric Blais

(Randomized) Communication Complexity 𝑇ℎ𝑏𝑠𝑓𝑒 𝑠𝑏𝑜𝑒𝑝𝑛 𝑡𝑢𝑠𝑗𝑜𝑕 1101000101110101110101010110 … Alice Bob 0100 11 001 ⋯ 0011 𝐽𝑜𝑞𝑣𝑢: 𝑦 Input: 𝑧 Compute 𝐷 𝑦, 𝑧 Goal: minimize the number of bits exchanged. Communication complexity of a protocol is t he maximum number of bits • exchanged by the protocol. Communication complexity of a function 𝐷 , denoted 𝑆(𝐷) , is the • communication complexity of the best protocol for computing C. 3

Example: Set Disjointness 𝐸𝐽𝑇𝐾 𝒍 1101000101110101110101010110 … Alice Bob 𝐽𝑜𝑞𝑣𝑢: 𝑇 ⊆ [𝑜] , 𝑇 = 𝑙 . Input: 𝑈 ⊆ [𝑜] , 𝑈 = 𝑙 Compute 𝐸𝐽𝑇𝐾 𝑙 𝑇, 𝑈 = 𝒃𝒅𝒅𝒇𝒒𝒖 if 𝑇 ∩ 𝑈 = ∅ 𝒔𝒇𝒌𝒇𝒅𝒖 otherwise Theorem [Hastad Wigderson 07 ] 𝑜 𝑆 DISJ 𝑙 ≥ Ω 𝑙 for all 𝑙 ≤ 2 . 4

A lower bound using CC method Testing if a Boolean function is a k-parity

Linear Functions Over Finite Field 𝔾 2 A Boolean function 𝑔: 0,1 𝑜 → {0,1} is linear (also called parity ) if 𝑔 𝑦 1 , … , 𝑦 𝑜 = 𝑏 1 𝑦 1 + ⋯ + 𝑏 𝑜 𝑦 𝑜 for some 𝑏 1 , … , 𝑏 𝑜 ∈ {0,1} no free term • Work in finite field 𝔾 2 – Other accepted notation for 𝔾 2 : 𝐻𝐺 example 2 and ℤ 2 – Addition and multiplication is mod 2 001001 – 𝒚 = 𝑦 1 , … , 𝑦 𝑜 , 𝒛 = 𝑧 1 , … , 𝑧 𝑜 , that is, 𝒚, 𝒛 ∈ 0,1 𝑜 + 011001 𝒚 + 𝒛 = 𝑦 1 + 𝑧 1 , … , 𝑦 𝑜 + 𝑧 𝑜 010000 6

Linear Functions Over Finite Field 𝔾 2 A Boolean function 𝑔: 0,1 𝑜 → {0,1} is linear (also called parity ) if 𝑔 𝑦 1 , … , 𝑦 𝑜 = 𝑏 1 𝑦 1 + ⋯ + 𝑏 𝑜 𝑦 𝑜 for some 𝑏 1 , … , 𝑏 𝑜 ∈ {0,1} ⇕ [𝑜] is a shorthand for {1, … 𝑜} 𝑔 𝑦 1 , … , 𝑦 𝑜 = 𝑦 𝑗 for some 𝑇 ⊆ 𝑜 . 𝑗∈S Notation: 𝜓 𝑇 𝑦 = 𝑦 𝑗 . 𝑗∈𝑇 7

Testing if a Boolean function is Linear Input: Boolean function 𝑔: 0,1 𝑜 → {0,1} Question: Is the function linear or 𝜁 -far from linear ( ≥ 𝜁2 𝑜 values need to be changed to make it linear)? Later in the course: 1 Famous BLR (Blum Lubi Rubinfeld 90) test runs in 𝑃 𝜁 time 8

k-Parity Functions 𝑙 -Parity Functions A function 𝑔 ∶ 0,1 𝑜 → {0,1} is a 𝑙 -parity if 𝑔 𝑦 = 𝜓 𝑇 𝑦 = 𝑦 𝑗 𝑗∈𝑇 for some set 𝑇 ⊆ 𝑜 of size 𝑇 = 𝑙 . 9

Testing if a Boolean Function is a k-Parity Input: Boolean function 𝑔: 0,1 𝑜 → {0,1} and an integer 𝑙 Question: Is the function a 𝑙 -parity or 𝜁 -far from a 𝑙 -parity ( ≥ 𝜁2 𝑜 values need to be changed to make it a 𝑙 -parity)? Time: O 𝑙 log 𝑙 [Chakraborty Garcia−Soriano Matsliah] W min (𝑙, 𝑜 − 𝑙 ) [Blais Brody Matulef 11] • Today: Ω(𝑙) for 𝑙 ≤ 𝑜/2 • Today’s bound implies W min (𝑙, 𝑜 − 𝑙 ) 10

Important Fact About Linear Functions Two different linear functions disagree on half of the values. Fact. • Consider functions 𝜓 𝑇 and 𝜓 𝑈 where 𝑇 ≠ 𝑈 . 0 0 – Let 𝑗 be an element on which 𝑇 and 𝑈 differ 1 1 (w.l.o.g. 𝑗 ∈ 𝑇 ∖ 𝑈 ) 1 0 – Pair up all 𝑜 -bit strings: (𝒚, 𝒚 𝑗 ) 𝒚 𝑏 𝑐 0 1 where 𝒚 𝑗 is 𝒚 with the 𝑗 th bit flipped. ⋅ ⋅ – For each such pair, 𝜓 𝑇 (𝒚) ≠ 𝜓 𝑇 (𝒚 𝑗 ) ⋅ ⋅ but 𝜓 𝑈 (𝒚) = 𝜓 𝑈 (𝒚 𝑗 ) ⋅ ⋅ 𝒚 𝑗 1 − 𝑏 So, 𝜓 𝑇 and 𝜓 𝑈 differ on exactly one of 𝒚, 𝒚 𝑗 . 𝑐 0 0 – Since all 𝒚 ’s are paired up, 1 0 𝜓 𝑇 and 𝜓 𝑈 differ on half of the values. 0 1 𝜓 𝑇 (x) 𝜓 𝑈 (x) A 𝑙 ′ - parity function, where 𝑙 ′ ≠ 𝑙, is ½-far from any k-parity. Corollary. 11

Reduction from 𝐸𝐽𝑇𝐾 𝒍/𝟑 to Testing k-Parity • Let 𝑈 be the best tester for the 𝑙 -parity property for 𝜁 = 1/2 – query complexity of T is 𝑟 testing 𝑙−parity . • We will construct a communication protocol for 𝐸𝐽𝑇𝐾 𝒍/𝟑 that runs 𝑈 and has communication complexity 2 ⋅ 𝑟 (testing 𝑙 −parity) . holds for CC of every protocol for 𝐸𝐽𝑇𝐾 𝒍 [Hastad Wigderson 07] • T hen 2 ⋅ 𝑟 (testing 𝑙 −parity) ≥ 𝑆 DISJ 𝑙/2 ≥ Ω 𝑙/2 for 𝑙 ≤ 𝑜/2 ⇓ 𝑟 (testing 𝑙 -parity) ≥ Ω 𝑙 for 𝑙 ≤ 𝑜/2 12

Reduction from 𝐸𝐽𝑇𝐾 𝒍/𝟑 to Testing k-Parity 1101000101110101110101010110 … ℎ = 𝑔 + 𝑕 (𝑛𝑝𝑒 2) ℎ 𝑦 ? 𝑔 𝑦 + 𝑕 𝑦 𝑛𝑝𝑒 2 Alice Bob T 𝒃𝒅𝒅𝒇𝒒𝒖/𝒔𝒇𝒌𝒇𝒅𝒖 𝑔(𝑦) 𝑕(𝑦) 𝐽𝑜𝑞𝑣𝑢: 𝑇 ⊆ [𝑜] , 𝑇 = 𝑙/2 . Input: 𝑈 ⊆ [𝑜] , 𝑈 = 𝑙/2 Compute: 𝑔 = 𝜓 𝑇 Compute: 𝑕 = 𝜓 𝑈 Output T’s answer • 𝑈 receives its random bits from the shared random string. 13

Analysis of the Reduction Queries: Alice and Bob exchange 2 bits for every bit queried by 𝑈 Correctness: • ℎ = 𝑔 + 𝑕 𝑛𝑝𝑒 2 = 𝜓 𝑇 + 𝜓 𝑈 𝑛𝑝𝑒 2 = 𝜓 𝑇Δ𝑈 • 𝑇Δ𝑈 = 𝑇 + 𝑈 − 2 𝑇 ∩ 𝑈 𝑙 if S∩T = ∅ • SΔ𝑈 = 𝑙 − 2 if S∩T ≠ ∅ ≤ ℎ is 𝑙−parity if S∩T = ∅ 𝑙 ′ −parity where 𝑙 ′ ≠ 𝑙 if S∩T ≠ ∅ 1/2-far from every 𝑙 -parity Summary: 𝑟 (testing 𝑙 -parity) ≥ Ω 𝑙 for 𝑙 ≤ 𝑜/2 14

Testing Lipschitz Property on Hypercube Lower Bound

Lipschitz Property of Functions f : 0,1 𝑜 → R 1 [Jha Raskhodnikova] 1 2 2 • A function 𝑔 ∶ 0,1 𝑜 → R is Lipschitz 1 1 0 if changing a bit of 𝑦 changes 𝑔(𝑦) by at most 1. 0 Lipschitz • Is 𝑔 Lipschitz or 𝜁 -far from Lipschitz 0 ( 𝑔 has to change on many points to become Lipschitz)? – Edge 𝑦 − 𝑧 is violated by 𝑔 if 𝑔 𝑦 − 𝑔(𝑧) > 1 . 0 0 2 2 0 2 Time: – 𝑃(𝑜 2 /𝜁) , logarithmic in the size of the input, 2 𝑜 1 2 2 -far from Lipschitz – Ω(𝑜) 16

Testing Lipschitz Property Theorem Testing Lipschitz property of functions f: 0,1 𝑜 → {0,1,2} requires Ω(𝑜) queries. Prove it. 17

Summary of Lower Bound Methods • Yao’s Principle – testing membership in 1*, sortedness of a list and monotonicity of Boolean functions • Reductions from communication complexity problems – testing if a Boolean function is a 𝑙 -parity 20

Other Models of Sublinear Computation

Tolerant Property Tester [Rubinfeld Parnas Ron] Randomized Algorithm Tolerant Property Tester YES YES Accept with Accept with probability ≥ 𝟑/𝟒 probability ≥ 𝟑/𝟒 𝜺 -close to YES Don’t care NO Reject with 𝜻 -far from Reject with  probability 2/3  probability 2/3 YES 22

Sublinear- Time “Restoration” Models Local Decoding Input: A slightly corrupted codeword Requirement: Recover individual bits of the closest codeword with a constant number of queries per recovered bit. Program Checking Input: A program 𝑄 computing 𝑔 correctly on most 𝑄 inputs. 𝑔 Requirement: Self-correct program 𝑄 : for a given input 𝑦 , compute 𝑔(𝑦) by making a few calls to P . Local Reconstruction Input: Function 𝑔 nearly satisfying some property 𝑄 Requirement: Reconstruct function 𝑔 to ensure that the reconstructed function 𝑕 satisfies 𝑄 , changing 𝑔 𝑔 only when necessary. For each input 𝑦 , compute 𝑕(𝑦) with a few queries to 𝑔. 23

Generalization: Local Computation [Rubinfeld Tamir Vardi Xie 2011] • Compute the 𝑗 -th character 𝑧 𝑗 of a legal output 𝑧 . • If there are several legal outputs for a given input, be consistent with one. • Example: maximal independent set in a graph. 24

Sublinear-Space Algorithms What if we cannot get a sublinear-time algorithm? Can we at least get sublinear space? Note: sublinear space is broader (for any algorithm, space complexity ≤ time complexity) 25

Data Stream Model B L A - B L A - B L A - B L A - B L A - B L A - B L A - Streaming (1) Quickly process each element Algorithm (2) Limited working memory (3) Quickly produce output Motivation: internet traffic analysis Model the stream as 𝑛 elements from [𝑜] , e.g., 𝑦 1 , 𝑦 2 , … , 𝑦 𝑛 = 3, 5, 3, 7, 5, 4, … Goal: Compute a function of the stream, e.g., median, number of distinct elements, longest increasing sequence. Based on Andrew McGregor’s slides: http://www.cs.umass.edu/~mcgregor/slides/10-jhu1.pdf 26

Streaming Puzzle A stream contains 𝑜 − 1 distinct elements from 𝑜 in arbitrary order. Problem: Find the missing element, using 𝑃(log 𝑜) space. 27