Sublinear Algorithms Lecture 26 Sofya Raskhodnikova Penn State - - PowerPoint PPT Presentation

1

Sublinear Algorithms

Lecture 26

Sofya Raskhodnikova

Penn State University

Thanks to Madhav Jha (Penn State) for help with creating these slides.

Testing Linearity

Linear Functions Over Finite Field 𝔾2

3

A Boolean function 𝑔: 0,1 𝑜 → {0,1} is linear if 𝑔 𝑦1, … , 𝑦𝑜 = 𝑏1𝑦1 + ⋯ + 𝑏𝑜𝑦𝑜 for some 𝑏1, … , 𝑏𝑜 ∈ {0,1}

• Work in finite field 𝔾2

– Other accepted notation for 𝔾2: 𝐻𝐺

2 and ℤ2

– Addition and multiplication is mod 2 – 𝒚= 𝑦1, … , 𝑦𝑜 , 𝒛= 𝑧1, … , 𝑧𝑜 , that is, 𝒚, 𝒛 ∈ 0,1 𝑜 𝒚 + 𝒛= 𝑦1 + 𝑧1, … , 𝑦𝑜 + 𝑧𝑜 no free term

Based on Ryan O’Donell’s lecture notes: http://www.cs.cmu.edu/~odonnell/boolean-analysis/

001001 011001 010000

+ example

Testing if a Boolean function is Linear

4

Input: Boolean function 𝑔: 0,1 𝑜 → {0,1} Question: Is the function linear or 𝜁-far from linear (≥ 𝜁2𝑜 values need to be changed to make it linear)? Today: can answer in 𝑃

1 𝜁 time

Motivation

• Linearity test is one of the most celebrated testing algorithms

– A special case of many important property tests – Computations over finite fields are used in

• Cryptography
• Coding Theory

– Originally designed for program checkers and self-correctors – Low-degree testing is needed in constructions of Probabilistically Checkable Proofs (PCPs)

• Used for proving inapproximability
• Main tool in the correctness proof: Fourier analysis of Boolean

functions

– Powerful and widely used technique in understanding the structure of Boolean functions

5

Equivalent Definitions of Linear Functions

6

• Definition. 𝑔 is linear if 𝑔 𝑦1, … , 𝑦𝑜 = 𝑏1𝑦1 + ⋯ + 𝑏𝑜𝑦𝑜 for some 𝑏1, … , 𝑏𝑜 ∈ 𝔾2

⇕ 𝑔 𝑦1, … , 𝑦𝑜 = 𝑦𝑗

𝑗∈S

for some 𝑇 ⊆ 𝑜 . Definition′. 𝑔 is linear if 𝑔 𝒚 + 𝒛 = 𝑔 𝒚 + 𝑔(𝒛) for all 𝒚, 𝒛 ∈ 0,1 𝑜.

• Definition ⇒ Definition′

𝑔 𝒚 + 𝒛 = 𝒚 + 𝒛 𝑗 = 𝑦𝑗 +

𝑗∈𝑇

𝑧𝑗 = 𝑔 𝒚 + 𝑔 𝒛 .

𝑗∈𝑇 𝑗∈𝑇

• Definition′ ⇒ Definition

Let 𝛽𝑗 = 𝑔((0, … , 0,1,0, … , 0

𝑓𝑗

)) Repeatedly apply Definition′: 𝑔 𝑦1, … , 𝑦𝑜 = 𝑔 𝑦𝑗𝑓𝑗 = 𝑦𝑗𝑔 𝑓𝑗 = 𝛽𝑗𝑦𝑗.

Based on Ryan O’Donell’s lecture notes: http://www.cs.cmu.edu/~odonnell/boolean-analysis/

[𝑜] is a shorthand for {1, … 𝑜}

Linearity Test [Blum Luby Rubinfeld 90]

7

1. Pick 𝒚 and 𝒛 independently and uniformly at random from 0,1 𝑜. 2. Set 𝒜 = 𝒚 + 𝒛 and query 𝑔on 𝒚, 𝒛, and 𝒜. Accept iff 𝑔 𝒜 = 𝑔 𝒚 + 𝑔 𝒛 . Analysis If 𝑔is linear, BLR always accepts. If 𝑔 is 𝜁-far from linear then > 𝜁 fraction of pairs 𝒚 and 𝒛 fail BLR test.

• Then, by Witness Lemma (Lecture 1), 2/𝜁 iterations suffice.

BLR Test (f, ε)

Correctness Theorem [Bellare Coppersmith Hastad Kiwi Sudan 95]

Analysis Technique: Fourier Expansion

Representing Functions as Vectors

Stack the 2𝑜 values of 𝑔(𝒚) and treat it as a vector in {0,1}2𝑜.

𝑔 = 1 1 1 ⋅ ⋅ ⋅ 1

𝑔(0000) 𝑔(0001) 𝑔(0010) 𝑔(0011) 𝑔(0100) ⋅ ⋅ ⋅ 𝑔(1101) 𝑔(1110) 𝑔(1111)

9

Linear functions

There are 2𝑜 linear functions: one for each subset 𝑇 ⊆ [𝑜]. 𝜓∅ = ⋅ ⋅ ⋅ , 𝜓 1 = 1 1 ⋅ ⋅ ⋅ 1 1 , ⋯ ⋯, 𝜓 𝑜 = 1 1 1 ⋅ ⋅ ⋅ 1

10

Parity on the positions indexed by set 𝑇 is 𝜓𝑇 𝑦1, … , 𝑦𝑜 = 𝑦𝑗

𝑗∈S

Great Notational Switch

Idea: Change notation, so that we work over reals instead of a finite field.

• Vectors in 0,1 2𝑜 ⟶ Vectors in ℝ2𝑜.
• 0/False ⟶ 1

1/True ⟶ -1.

• Addition (mod 2) ⟶ Multiplication in ℝ.
• Boolean function: 𝑔 ∶ −1, 1 𝑜 → {−1,1}.
• Linear function 𝜓𝑇∶ −1, 1 𝑜 → {−1,1} is given by 𝜓𝑇 𝒚 =

𝑦𝑗

𝑗∈𝑇

.

11

Benefit 1 of New Notation

• The dot product of 𝑔 and 𝑕 as vectors in −1,1 2𝑜:

(# 𝒚’s such that 𝑔 𝒚 = 𝑕(𝒚)) − (# 𝒚’s such that 𝑔 𝒚 ≠ 𝑕(𝒚)) = 2𝑜 − 2 ⋅ (# 𝒚’s such that 𝑔 𝒚 ≠ 𝑕(𝒚)) 𝑔, 𝑕 = 1 2𝑜 dot product of 𝑔 and 𝑕 as vectors = avg

𝒚∈ −1,1 𝑜 𝑔 𝒚 𝑕 𝒚

= E

𝒚∈ −1,1 𝑜[ 𝑔 𝒚 𝑕 𝒚 ].

𝑔, 𝑕 = 1 − 2 ⋅ (fraction of disagreements between 𝑔 and 𝑕)

12

Inner product of functions 𝑔, 𝑕 ∶ −1, 1 → {−1, 1}

disagreements between 𝑔 and 𝑕

Benefit 2 of New Notation

• If 𝑇 ≠ 𝑈 then 𝜓𝑇 and 𝜓𝑈 are orthogonal: 𝜓𝑇, 𝜓𝑈 = 0.

– Let 𝑗 be an element on which 𝑇 and 𝑈 differ (w.l.o.g. 𝑗 ∈ 𝑇 ∖ 𝑈) – Pair up all 𝑜-bit strings: (𝒚, 𝒚 𝑗 ) where 𝒚 𝑗 is 𝒚 with the 𝑗th bit flipped. – Each such pair contributes 𝑏𝑐 − 𝑏𝑐 = 0 to 𝜓𝑇, 𝜓𝑈 . – Since all 𝒚’s are paired up, 𝜓𝑇, 𝜓𝑈 = 0.

• Recall that there are 2𝑜 linear functions 𝜓𝑇 .
• 𝜓𝑇, 𝜓𝑇 = 1

– In fact, 𝑔, 𝑔 = 1 for all 𝑔 ∶ −1, 1 𝑜 → −1, 1 . – (The norm of 𝑔, denoted 𝑔 , is 𝑔, 𝑔 )

13

𝒚 𝒚 𝑗

𝜓𝑇 𝜓𝑈

+1 −1 +1 +𝑏 +1 ⋅ ⋅ ⋅ −𝑏 +1 −1 −1 −1 +1 +1 𝑐 +1 ⋅ ⋅ ⋅ 𝑐 −1 +1 +1 The functions 𝜓𝑇 𝑇⊆ 𝑜 form an orthonormal basis for ℝ2𝑜. Claim.

Idea: Work in the basis 𝜓𝑇 𝑇⊆ 𝑜 , so it is easy to see how close a specific function 𝑔 is to each of the linear functions.

Every function 𝑔 ∶ −1, 1 𝑜 → ℝ is uniquely expressible as a linear combination (over ℝ) of the 2𝑜 linear functions: where 𝑔 𝑇 = 𝑔, 𝜓𝑇 is the Fourier Coefficient of 𝑔 on set 𝑇. Proof: 𝑔 can be written uniquely as a linear combination of basis vectors: 𝑔 = 𝑑𝑇 ⋅ 𝜓𝑇

𝑇⊆ 𝑜

It remains to prove that 𝑑𝑇=𝑔 𝑇 for all 𝑇. 𝑔 𝑇 = 𝑔, 𝜓𝑇 = 𝑑𝑈 ⋅ 𝜓𝑈

𝑈⊆[𝑜]

, 𝜓𝑇 = 𝑑𝑈 ⋅ 𝜓𝑈, 𝜓𝑇

𝑈⊆[𝑜]

= 𝑑𝑇

Fourier Expansion Theorem

14

Fourier Expansion Theorem

𝑔 = 𝑔 𝑇 𝜓𝑇,

𝑇⊆ 𝑜

Linearity of ⋅,⋅

𝜓𝑈, 𝜓𝑇 =

1 if 𝑈 = 𝑇 0 otherwise Definition of Fourier coefficients

Examples: Fourier Expansion

𝒈 Fourier transform 𝑔 𝒚 = 1 1 𝑔 𝒚 = 𝑦𝑗 𝑦𝑗 AND(𝑦1, 𝑦2) 1 2 + 1 2 𝑦1 + 1 2 𝑦2 − 1 2 𝑦1𝑦2 MAJORITY(𝑦1, 𝑦2, 𝑦3) 1 2 𝑦1 + 1 2 𝑦2 + 1 2 𝑦3 − 1 2 𝑦1𝑦2𝑦3

15

Parseval Equality

Proof: 𝑔, 𝑔 = 𝑔 𝑇 𝜓𝑇

𝑇⊆ 𝑜

, 𝑔 𝑈 𝜓𝑈

𝑈⊆ 𝑜

= 𝑔 𝑇

𝑈 𝑇

𝑔 𝑈 𝜓𝑇, 𝜓𝑈 = 𝑔 𝑇 2

𝑇

16

By linearity of inner product By orthonormality of 𝜓𝑇’s

Parseval Equality

Let 𝑔: −1, 1 𝑜 → ℝ. Then 𝑔, 𝑔 = 𝑔 𝑇 2

𝑇⊆ 𝑜

By Fourier Expansion Theorem

Parseval Equality

Proof: 𝑔, 𝑔 = E

𝒚∈ −1,1 𝑜[𝑔 𝒚 2]

= 1

17

Parseval Equality for Boolean Functions

Let 𝑔: −1, 1 𝑜 → −1, 1 . Then 𝑔, 𝑔 = 𝑔 𝑇 2

𝑇⊆ 𝑜

= 1

By definition of inner product Since 𝑔 is Boolean

Vector product notation: 𝒚 ∘ 𝒛 = (𝑦1𝑧1, 𝑦2𝑧2, … , 𝑦𝑜𝑧𝑜) Proof: Indicator variable 𝟚𝐶𝑀𝑆 = 1 if BLR accepts 0 otherwise ⇒ 𝟚𝐶𝑀𝑆 =

1 2 + 1 2 𝑔 𝒚 𝑔 𝒛 𝑔 𝒜 .

Pr

𝒚,𝒛∈ −1,1 𝑜 BLR 𝑔 accepts =

E

𝐲,𝐳∈ −1,1 𝑜 𝟚𝐶𝑀𝑆 = 1

2 + 1 2 E

𝐲,𝐳∈ −1,1 𝑜 𝑔 𝒚 𝑔 𝒛 𝑔 𝒜

BLR Test in {-1,1} notation

18

BLR Test (f, ε)

1.

Pick 𝒚 and 𝒛 independently and uniformly at random from −1,1 𝑜. 2. Set 𝒜 = 𝒚 ∘ 𝒛 and query 𝑔 on 𝒚, 𝒛, and 𝒜. Accept iff 𝑔 𝒚 𝑔 𝒛 𝑔 𝒜 = 1. Pr

𝐲,𝐳∈ −1,1 𝑜 BLR 𝑔 accepts = 1

2 + 1 2 𝑔 𝑇 3

𝑇⊆[𝑜]

Sum-Of-Cubes Lemma.

By linearity of expectation

So far: Pr

𝐲,𝐳∈ −1,1 𝑜 BLR 𝑔 accepts = 1 2 + 1 2

E

𝐲,𝐳∈ −1,1 𝑜 𝑔 𝒚 𝑔 𝒛 𝑔 𝒜

Next:

Proof of Sum-Of-Cubes Lemma

19

= 𝑔 𝑇 𝑔 𝑈 𝑔 𝑉 E

𝐲,𝐳∈ −1,1 𝑜[𝜓𝑇(𝒚)𝜓𝑈(𝒛)𝜓𝑉(𝒜) 𝑇,𝑈,𝑉⊆[𝑜]

] = E

𝐲,𝐳∈ −1,1 𝑜

𝑔 𝑇 𝑔 𝑈 𝑔 𝑉 𝜓𝑇(𝒚)𝜓𝑈(𝒛)𝜓𝑉(𝒜)

𝑇,𝑈,𝑉⊆[𝑜]

E

𝐲,𝐳∈ −1,1 𝑜 𝑔 𝒚 𝑔 𝒛 𝑔 𝒜

= E

𝐲,𝐳∈ −1,1 𝑜

𝑔 𝑇 𝜓𝑇(𝒚)

𝑇⊆[𝑜]

𝑔 𝑈 𝜓𝑈(𝒛)

𝑈⊆[𝑜]

𝑔 𝑉 𝜓𝑉(𝒜)

𝑉⊆[𝑜]

By Fourier Expansion Theorem Distributing out the product of sums By linearity of expectation

Pr

𝐲,𝐳∈ −1,1 𝑜 BLR 𝑔 accepts

• Let 𝑇Δ𝑈denote symmetric difference of sets 𝑇 and 𝑈

Proof of Sum-Of-Cubes Lemma (Continued)

20

a= E

𝐲,𝐳∈ −1,1 𝑜

𝑦𝑗

𝑗∈𝑇

𝑧𝑗 𝑦𝑗𝑧𝑗

𝑗∈𝑉 𝑗∈𝑈

a= E

𝐲,𝐳∈ −1,1 𝑜

𝑦𝑗

𝑗∈𝑇Δ𝑉

𝑧𝑗

𝑗∈𝑈Δ𝑉

a= E

𝐲∈ −1,1 𝑜

𝑦𝑗

𝑗∈𝑇Δ𝑉

⋅ E

𝐳∈ −1,1 𝑜

𝑧𝑗

𝑗∈𝑇Δ𝑉

a= E

𝐲∈ −1,1 𝑜[𝑦𝑗] 𝑗∈𝑇Δ𝑉

⋅ E

𝐳∈ −1,1 𝑜[𝑧𝑗] 𝑗∈𝑈Δ𝑉

a E

𝐲,𝐳∈ −1,1 𝑜[𝜓𝑇(𝒚)𝜓𝑈(𝒛)𝜓𝑉(𝒜)]

= 1 when 𝑇Δ𝑉 = ∅ and 𝑈Δ𝑉 = ∅ ⇔ 𝑇 = 𝑈 = 𝑉 0 otherwise a= E

𝐲,𝐳∈ −1,1 𝑜

𝑦𝑗

𝑗∈𝑇

𝑧𝑗 𝑨𝑗

𝑗∈𝑉 𝑗∈𝑈

= 1 2 + 1 2

𝑔 𝑇 𝑔 𝑈 𝑔 𝑉 E

𝐲,𝐳∈ −1,1 𝑜[𝜓𝑇(𝒚)𝜓𝑈(𝒛)𝜓𝑉(𝒜) 𝑇,𝑈,𝑉⊆[𝑜]

]

Since 𝑦𝑗

2 = 𝑧𝑗 2 = 1

Since 𝐲 and 𝐳 are independent Since 𝐴 = 𝐲 ∘ 𝐳 Since 𝐲 and 𝐳′s coordinates are independent

a= E

𝑦𝑗∈{−1,1}[𝑦𝑗] 𝑗∈𝑇Δ𝑉

⋅ E

𝑧𝑗∈{−1,1}[𝑧𝑗] 𝑗∈𝑈Δ𝑉

E

𝐲,𝐳∈ −1,1 𝑜[𝜓𝑇(𝒚)𝜓𝑈(𝒛)𝜓𝑉(𝒜)] is 1 if 𝑇 = 𝑈 = 𝑉 and 0 otherwise.

Claim.

Proof of Sum-Of-Cubes Lemma (Done)

21

= 1 2 + 1 2 𝑔 𝑇 3

𝑇⊆[𝑜]

Pr

𝐲,𝐳∈ −1,1 𝑜 BLR 𝑔 accepts = 1

2 + 1 2 𝑔 𝑇 3

𝑇⊆[𝑜]

Sum-Of-Cubes Lemma.

Pr

𝐲,𝐳∈ −1,1 𝑜 BLR 𝑔 accepts

= 1 2 + 1 2

𝑔 𝑇 𝑔 𝑈 𝑔 𝑉 E

𝐲,𝐳∈ −1,1 𝑜[𝜓𝑇(𝒚)𝜓𝑈(𝒛)𝜓𝑉(𝒜) 𝑇,𝑈,𝑉⊆[𝑜]

]

Proof of Correctness Theorem

Proof: Suppose to the contrary that

• Then max

𝑇⊆ 𝑜 𝑔

𝑇 > 1 − 2𝜁. That is, 𝑔 𝑈 > 1 − 2𝜁 for some 𝑈 ⊆ 𝑜 .

• But 𝑔

𝑈 = 𝑔, 𝜓𝑈 = 1 − 2 ⋅ (fraction of disagreements between 𝑔 and 𝜓𝑈)

• 𝑔 disagrees with a linear function 𝜓𝑈 on < 𝜁 fraction of values.

22

By Sum-Of-Cubes Lemma Since 𝑔 𝑇 2 ≥ 0 Parseval Equality

Correctness Theorem (restated)

If 𝑔 is ε-far from linear then Pr BLR 𝑔 accepts ≤ 1 − 𝜁. = 1 2 + 1 2 𝑔 𝑇 3

𝑇⊆[𝑜]

≤ 1 2 + 1 2 ⋅ max

𝑇⊆ 𝑜 𝑔

𝑇 ⋅ 𝑔 𝑇 2

𝑇⊆ 𝑜

= 1 2 + 1 2 ⋅ max

𝑇⊆ 𝑜 𝑔

𝑇 1 − 𝜁 < Pr

𝐲,𝐳∈ −1,1 𝑜 BLR 𝑔 accepts

⨳

Summary

BLR tests whether a function 𝑔: 0,1 𝑜 → {0,1} is linear or 𝜁-far from linear (≥ 𝜁2𝑜 values need to be changed to make it linear) in 𝑃

1 𝜁 time.

23