Sublinear Algorithms Lecture 26 Sofya Raskhodnikova Penn State - PowerPoint PPT Presentation

Sublinear Algorithms Lecture 26 Sofya Raskhodnikova Penn State University Thanks to Madhav Jha (Penn State) for help with creating these slides. 1

Testing Linearity

Linear Functions Over Finite Field 𝔾 2 A Boolean function 𝑔: 0,1 𝑜 → {0,1} is linear if 𝑔 𝑦 1 , … , 𝑦 𝑜 = 𝑏 1 𝑦 1 + ⋯ + 𝑏 𝑜 𝑦 𝑜 for some 𝑏 1 , … , 𝑏 𝑜 ∈ {0,1} no free term • Work in finite field 𝔾 2 – Other accepted notation for 𝔾 2 : 𝐻𝐺 example 2 and ℤ 2 – Addition and multiplication is mod 2 001001 – 𝒚 = 𝑦 1 , … , 𝑦 𝑜 , 𝒛 = 𝑧 1 , … , 𝑧 𝑜 , that is, 𝒚, 𝒛 ∈ 0,1 𝑜 + 011001 𝒚 + 𝒛 = 𝑦 1 + 𝑧 1 , … , 𝑦 𝑜 + 𝑧 𝑜 010000 Based on Ryan O’Donell’s lecture notes : http://www.cs.cmu.edu/~odonnell/boolean-analysis/ 3

Testing if a Boolean function is Linear Input: Boolean function 𝑔: 0,1 𝑜 → {0,1} Question: Is the function linear or 𝜁 -far from linear ( ≥ 𝜁2 𝑜 values need to be changed to make it linear)? 1 Today: can answer in 𝑃 𝜁 time 4

Motivation • Linearity test is one of the most celebrated testing algorithms – A special case of many important property tests – Computations over finite fields are used in • Cryptography • Coding Theory – Originally designed for program checkers and self-correctors – Low-degree testing is needed in constructions of Probabilistically Checkable Proofs (PCPs) • Used for proving inapproximability • Main tool in the correctness proof: Fourier analysis of Boolean functions – Powerful and widely used technique in understanding the structure of Boolean functions 5

Equivalent Definitions of Linear Functions Definition. 𝑔 is linear if 𝑔 𝑦 1 , … , 𝑦 𝑜 = 𝑏 1 𝑦 1 + ⋯ + 𝑏 𝑜 𝑦 𝑜 for some 𝑏 1 , … , 𝑏 𝑜 ∈ 𝔾 2 ⇕ [𝑜] is a shorthand for {1, … 𝑜} 𝑔 𝑦 1 , … , 𝑦 𝑜 = 𝑦 𝑗 for some 𝑇 ⊆ 𝑜 . 𝑗∈S Definition ′ . 𝑔 is linear if 𝑔 𝒚 + 𝒛 = 𝑔 𝒚 + 𝑔(𝒛) for all 𝒚, 𝒛 ∈ 0,1 𝑜 . Definition ⇒ Definition ′ • 𝑔 𝒚 + 𝒛 = 𝒚 + 𝒛 𝑗 = 𝑦 𝑗 + 𝑧 𝑗 = 𝑔 𝒚 + 𝑔 𝒛 . 𝑗∈𝑇 𝑗∈𝑇 𝑗∈𝑇 Definition ′ ⇒ Definition • 𝑓 𝑗 Let 𝛽 𝑗 = 𝑔((0, … , 0,1,0, … , 0 )) Repeatedly apply Definition ′ : 𝑔 𝑦 1 , … , 𝑦 𝑜 = 𝑔 𝑦 𝑗 𝑓 𝑗 = 𝑦 𝑗 𝑔 𝑓 𝑗 = 𝛽 𝑗 𝑦 𝑗 . Based on Ryan O’Donell’s lecture notes : http://www.cs.cmu.edu/~odonnell/boolean-analysis/ 6

Linearity Test [Blum Luby Rubinfeld 90] BLR Test (f, ε ) Pick 𝒚 and 𝒛 independently and uniformly at random from 0,1 𝑜 . 1. Set 𝒜 = 𝒚 + 𝒛 and query 𝑔 on 𝒚, 𝒛, and 𝒜 . Accept iff 𝑔 𝒜 = 𝑔 𝒚 + 𝑔 𝒛 . 2. Analysis If 𝑔 is linear, BLR always accepts. Correctness Theorem [Bellare Coppersmith Hastad Kiwi Sudan 95] If 𝑔 is 𝜁 -far from linear then > 𝜁 fraction of pairs 𝒚 and 𝒛 fail BLR test. Then, by Witness Lemma (Lecture 1), 2/𝜁 iterations suffice. • 7

Analysis Technique: Fourier Expansion

Representing Functions as Vectors Stack the 2 𝑜 values of 𝑔(𝒚) and treat it as a vector in {0,1} 2 𝑜 . 0 𝑔(0000) 1 𝑔(0001) 1 𝑔(0010) 0 𝑔(0011) 1 𝑔(0100) ⋅ 𝑔 = ⋅ ⋅ ⋅ ⋅ ⋅ 𝑔(1101) 1 𝑔(1110) 0 𝑔(1111) 0 9

Linear functions There are 2 𝑜 linear functions: one for each subset 𝑇 ⊆ [𝑜] . 0 0 0 0 1 1 0 0 1 0 1 0 0 0 1 ⋅ ⋅ ⋅ 𝜓 ∅ = , 𝜓 1 = , ⋯ ⋯ , 𝜓 𝑜 = ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ 0 1 1 0 0 0 0 1 0 Parity on the positions indexed by set 𝑇 is 𝜓 𝑇 𝑦 1 , … , 𝑦 𝑜 = 𝑦 𝑗 𝑗∈S 10

Great Notational Switch Idea: Change notation, so that we work over reals instead of a finite field. Vectors in 0,1 2 𝑜 ⟶ Vectors in ℝ 2 𝑜 . • 0/False ⟶ 1 1/True ⟶ -1. • Addition (mod 2) ⟶ Multiplication in ℝ . • Boolean function: 𝑔 ∶ −1, 1 𝑜 → {−1,1} . • Linear function 𝜓 𝑇 ∶ −1, 1 𝑜 → {−1,1} is given by 𝜓 𝑇 𝒚 = 𝑦 𝑗 • . 𝑗∈𝑇 11

Benefit 1 of New Notation The dot product of 𝑔 and 𝑕 as vectors in −1,1 2 𝑜 : • (# 𝒚 ’s such that 𝑔 𝒚 = 𝑕(𝒚) ) − (# 𝒚 ’s such that 𝑔 𝒚 ≠ 𝑕(𝒚) ) = 2 𝑜 − 2 ⋅ (# 𝒚 ’s such that 𝑔 𝒚 ≠ 𝑕(𝒚) ) disagreements between 𝑔 and 𝑕 Inner product of functions 𝑔, 𝑕 ∶ −1, 1 → {−1, 1} 𝑔, 𝑕 = 1 2 𝑜 dot product of 𝑔 and 𝑕 as vectors = 𝒚∈ −1,1 𝑜 𝑔 𝒚 𝑕 𝒚 avg = 𝒚∈ −1,1 𝑜 [ 𝑔 𝒚 𝑕 𝒚 ]. E 𝑔, 𝑕 = 1 − 2 ⋅ (fraction of disagreements between 𝑔 and 𝑕) 12

Benefit 2 of New Notation The functions 𝜓 𝑇 𝑇⊆ 𝑜 form an orthonormal basis for ℝ 2 𝑜 . Claim. • If 𝑇 ≠ 𝑈 then 𝜓 𝑇 and 𝜓 𝑈 are orthogonal: 𝜓 𝑇 , 𝜓 𝑈 = 0 . −1 +1 – Let 𝑗 be an element on which 𝑇 and 𝑈 differ −1 +1 (w.l.o.g. 𝑗 ∈ 𝑇 ∖ 𝑈 ) +1 +1 – Pair up all 𝑜 -bit strings: (𝒚, 𝒚 𝑗 ) 𝒚 +𝑏 𝑐 +1 +1 where 𝒚 𝑗 is 𝒚 with the 𝑗 th bit flipped. ⋅ ⋅ – Each such pair contributes 𝑏𝑐 − 𝑏𝑐 = 0 to 𝜓 𝑇 , 𝜓 𝑈 . ⋅ ⋅ – Since all 𝒚 ’s are paired up, 𝜓 𝑇 , 𝜓 𝑈 = 0 . ⋅ ⋅ • Recall that there are 2 𝑜 linear functions 𝜓 𝑇 . 𝒚 𝑗 −𝑏 𝑐 +1 −1 • 𝜓 𝑇 , 𝜓 𝑇 = 1 −1 +1 – In fact, 𝑔, 𝑔 = 1 for all 𝑔 ∶ −1, 1 𝑜 → −1, 1 . −1 +1 𝜓 𝑇 𝜓 𝑈 – (The norm of 𝑔, denoted 𝑔 , is 𝑔, 𝑔 ) 13

Fourier Expansion Theorem Idea: Work in the basis 𝜓 𝑇 𝑇⊆ 𝑜 , so it is easy to see how close a specific function 𝑔 is to each of the linear functions. Fourier Expansion Theorem Every function 𝑔 ∶ −1, 1 𝑜 → ℝ is uniquely expressible as a linear combination (over ℝ ) of the 2 𝑜 linear functions: 𝑇 𝜓 𝑇, 𝑔 = 𝑔 𝑇⊆ 𝑜 𝑇 = 𝑔, 𝜓 𝑇 is the Fourier Coefficient of 𝑔 on set 𝑇 . where 𝑔 Proof: 𝑔 can be written uniquely as a linear combination of basis vectors: 𝑔 = 𝑑 𝑇 ⋅ 𝜓 𝑇 𝑇⊆ 𝑜 𝑇 for all 𝑇 . It remains to prove that 𝑑 𝑇 = 𝑔 𝑇 = 𝑔, 𝜓 𝑇 = 𝑔 𝑑 𝑈 ⋅ 𝜓 𝑈 , 𝜓 𝑇 = 𝑑 𝑈 ⋅ 𝜓 𝑈 , 𝜓 𝑇 = 𝑑 𝑇 𝑈⊆[𝑜] 𝑈⊆[𝑜] 𝜓 𝑈 , 𝜓 𝑇 = 1 if 𝑈 = 𝑇 Linearity of ⋅,⋅ Definition of Fourier 0 otherwise coefficients 14

Examples: Fourier Expansion 𝒈 Fourier transform 𝑔 𝒚 = 1 1 𝑔 𝒚 = 𝑦 𝑗 𝑦 𝑗 2 + 1 1 2 𝑦 1 + 1 2 𝑦 2 − 1 AND (𝑦 1 , 𝑦 2 ) 2 𝑦 1 𝑦 2 1 2 𝑦 1 + 1 2 𝑦 2 + 1 2 𝑦 3 − 1 MAJORITY (𝑦 1 , 𝑦 2 , 𝑦 3 ) 2 𝑦 1 𝑦 2 𝑦 3 15

Parseval Equality Parseval Equality Let 𝑔: −1, 1 𝑜 → ℝ . Then 𝑇 2 𝑔, 𝑔 = 𝑔 𝑇⊆ 𝑜 Proof: By Fourier Expansion Theorem 𝑇 𝜓 𝑇 𝑈 𝜓 𝑈 𝑔, 𝑔 = 𝑔 , 𝑔 𝑇⊆ 𝑜 𝑈⊆ 𝑜 By linearity of inner product 𝑇 𝑈 𝜓 𝑇 , 𝜓 𝑈 = 𝑔 𝑔 𝑇 𝑈 By orthonormality of 𝜓 𝑇 ’s 𝑇 2 = 𝑔 𝑇 16

Parseval Equality Parseval Equality for Boolean Functions Let 𝑔: −1, 1 𝑜 → −1, 1 . Then 𝑇 2 𝑔, 𝑔 = 𝑔 = 1 𝑇⊆ 𝑜 Proof: By definition of inner product 𝒚∈ −1,1 𝑜 [𝑔 𝒚 2 ] 𝑔, 𝑔 = E Since 𝑔 is Boolean = 1 17

BLR Test in {-1,1} notation BLR Test (f, ε ) Pick 𝒚 and 𝒛 independently and uniformly at random from −1,1 𝑜 . 1. Set 𝒜 = 𝒚 ∘ 𝒛 and query 𝑔 on 𝒚, 𝒛, and 𝒜 . Accept iff 𝑔 𝒚 𝑔 𝒛 𝑔 𝒜 = 1 . 2. Vector product notation: 𝒚 ∘ 𝒛 = (𝑦 1 𝑧 1 , 𝑦 2 𝑧 2 , … , 𝑦 𝑜 𝑧 𝑜 ) 𝐲,𝐳∈ −1,1 𝑜 BLR 𝑔 accepts = 1 2 + 1 𝑇 3 Pr 2 𝑔 Sum-Of-Cubes Lemma. 𝑇⊆[𝑜] Proof: Indicator variable 𝟚 𝐶𝑀𝑆 = 1 if BLR accepts 1 1 ⇒ 𝟚 𝐶𝑀𝑆 = 2 + 2 𝑔 𝒚 𝑔 𝒛 𝑔 𝒜 . 0 otherwise 𝐲,𝐳∈ −1,1 𝑜 𝟚 𝐶𝑀𝑆 = 1 2 + 1 𝒚,𝒛∈ −1,1 𝑜 BLR 𝑔 accepts = Pr E 2 𝐲,𝐳∈ −1,1 𝑜 𝑔 𝒚 𝑔 𝒛 𝑔 𝒜 E By linearity of expectation 18

Proof of Sum-Of-Cubes Lemma 1 1 𝐲,𝐳∈ −1,1 𝑜 BLR 𝑔 accepts = Pr 2 + 𝐲,𝐳∈ −1,1 𝑜 𝑔 𝒚 𝑔 𝒛 𝑔 𝒜 E So far: 2 Next: 𝐲,𝐳∈ −1,1 𝑜 𝑔 𝒚 𝑔 𝒛 𝑔 𝒜 E By Fourier Expansion Theorem 𝑇 𝜓 𝑇 (𝒚) 𝑈 𝜓 𝑈 (𝒛) 𝑉 𝜓 𝑉 (𝒜) = E 𝑔 𝑔 𝑔 𝐲,𝐳∈ −1,1 𝑜 𝑇⊆[𝑜] 𝑈⊆[𝑜] 𝑉⊆[𝑜] Distributing out the product of sums 𝑇 𝑔 𝑈 𝑔 𝑉 𝜓 𝑇 (𝒚)𝜓 𝑈 (𝒛)𝜓 𝑉 (𝒜) = E 𝑔 𝐲,𝐳∈ −1,1 𝑜 𝑇,𝑈,𝑉⊆[𝑜] By linearity of expectation 𝑇 𝑔 𝑈 𝑔 𝑉 = 𝑔 𝐲,𝐳∈ −1,1 𝑜 [𝜓 𝑇 (𝒚)𝜓 𝑈 (𝒛)𝜓 𝑉 (𝒜) E ] 𝑇,𝑈,𝑉⊆[𝑜] 19