Sublinear Algorithms Lecture 6 Sofya Raskhodnikova Penn State - - PowerPoint PPT Presentation

β–Ά
sublinear algorithms
SMART_READER_LITE
LIVE PREVIEW

Sublinear Algorithms Lecture 6 Sofya Raskhodnikova Penn State - - PowerPoint PPT Presentation

Jul 21, 2023 268 likes β€’555 views

Sublinear Algorithms Lecture 6 Sofya Raskhodnikova Penn State University Thanks to Madhav Jha (Penn State) for help with creating these slides. 1 Communication Complexity A Method for Proving Lower Bounds [Blais Brody Matulef 11 ] Use known

slide-1
SLIDE 1

1

Sublinear Algorithms

Lecture 6

Sofya Raskhodnikova

Penn State University

Thanks to Madhav Jha (Penn State) for help with creating these slides.

slide-2
SLIDE 2

Communication Complexity

A Method for Proving Lower Bounds

[Blais Brody Matulef 11]

Use known lower bounds for other models of computation

Partially based on slides by Eric Blais

slide-3
SLIDE 3

(Randomized) Communication Complexity

3

Compute 𝐷 𝑦, 𝑧 0100 11 001 β‹― 0011 Bob Alice π½π‘œπ‘žπ‘£π‘’: 𝑦 Input: 𝑧 1101000101110101110101010110… π‘‡β„Žπ‘π‘ π‘“π‘’ π‘ π‘π‘œπ‘’π‘π‘› π‘‘π‘’π‘ π‘—π‘œπ‘• Goal: minimize the number of bits exchanged.

  • Communication complexity of a protocol is the maximum number of bits

exchanged by the protocol.

  • Communication complexity of a function 𝐷, denoted 𝑆(𝐷), is the

communication complexity of the best protocol for computing C.

slide-4
SLIDE 4

Example: Set Disjointness 𝐸𝐽𝑇𝐾𝒍

4

Theorem [Hastad Wigderson 07] 𝑆 DISJ𝑙 β‰₯ Ξ© 𝑙 for all 𝑙 ≀

π‘œ 2.

Compute 𝐸𝐽𝑇𝐾𝑙 𝑇, π‘ˆ = 𝒃𝒅𝒅𝒇𝒒𝒖 if 𝑇 ∩ π‘ˆ = βˆ… π’”π’‡π’Œπ’‡π’…π’– otherwise Bob Alice π½π‘œπ‘žπ‘£π‘’: 𝑇 βŠ† [π‘œ], 𝑇 = 𝑙. Input: π‘ˆ βŠ† [π‘œ], π‘ˆ = 𝑙 1101000101110101110101010110…

slide-5
SLIDE 5

A lower bound using CC method

Testing if a Boolean function is a k-parity

slide-6
SLIDE 6

Linear Functions Over Finite Field 𝔾2

6

A Boolean function 𝑔: 0,1 π‘œ β†’ {0,1} is linear (also called parity) if 𝑔 𝑦1, … , π‘¦π‘œ = 𝑏1𝑦1 + β‹― + π‘π‘œπ‘¦π‘œ for some 𝑏1, … , π‘π‘œ ∈ {0,1}

  • Work in finite field 𝔾2

– Other accepted notation for 𝔾2: 𝐻𝐺

2 and β„€2

– Addition and multiplication is mod 2 – π’š= 𝑦1, … , π‘¦π‘œ , 𝒛= 𝑧1, … , π‘§π‘œ , that is, π’š, 𝒛 ∈ 0,1 π‘œ π’š + 𝒛= 𝑦1 + 𝑧1, … , π‘¦π‘œ + π‘§π‘œ no free term

001001 011001 010000

+ example

slide-7
SLIDE 7

Linear Functions Over Finite Field 𝔾2

7

A Boolean function 𝑔: 0,1 π‘œ β†’ {0,1} is linear (also called parity) if 𝑔 𝑦1, … , π‘¦π‘œ = 𝑏1𝑦1 + β‹― + π‘π‘œπ‘¦π‘œ for some 𝑏1, … , π‘π‘œ ∈ {0,1} ⇕ 𝑔 𝑦1, … , π‘¦π‘œ = 𝑦𝑗

π‘—βˆˆS

for some 𝑇 βŠ† π‘œ . Notation: πœ“π‘‡ 𝑦 = 𝑦𝑗

π‘—βˆˆπ‘‡

.

[π‘œ] is a shorthand for {1, … π‘œ}

slide-8
SLIDE 8

Testing if a Boolean function is Linear

8

Input: Boolean function 𝑔: 0,1 π‘œ β†’ {0,1} Question: Is the function linear or 𝜁-far from linear (β‰₯ 𝜁2π‘œ values need to be changed to make it linear)? Later in the course: Famous BLR (Blum Lubi Rubinfeld 90) test runs in 𝑃

1 𝜁 time

slide-9
SLIDE 9

k-Parity Functions

9

𝑙-Parity Functions

A function 𝑔 ∢ 0,1 π‘œ β†’ {0,1} is a 𝑙-parity if 𝑔 𝑦 = πœ“π‘‡ 𝑦 = 𝑦𝑗

π‘—βˆˆπ‘‡

for some set 𝑇 βŠ† π‘œ of size 𝑇 = 𝑙.

slide-10
SLIDE 10

Testing if a Boolean Function is a k-Parity

10

Input: Boolean function 𝑔: 0,1 π‘œ β†’ {0,1} and an integer 𝑙 Question: Is the function a 𝑙-parity or 𝜁-far from a 𝑙-parity (β‰₯ 𝜁2π‘œ values need to be changed to make it a 𝑙-parity)? Time: O 𝑙 log 𝑙 [Chakraborty Garciaβˆ’Soriano Matsliah]

W min (𝑙, π‘œ βˆ’ 𝑙 ) [Blais Brody Matulef 11]

  • Today: Ξ©(𝑙) for 𝑙 ≀ π‘œ/2
  • Today’s bound implies W min

(𝑙, π‘œ βˆ’ 𝑙 )

slide-11
SLIDE 11

Important Fact About Linear Functions

  • Consider functions πœ“π‘‡ and πœ“π‘ˆ where 𝑇 β‰  π‘ˆ.

– Let 𝑗 be an element on which 𝑇 and π‘ˆ differ (w.l.o.g. 𝑗 ∈ 𝑇 βˆ– π‘ˆ) – Pair up all π‘œ-bit strings: (π’š, π’š 𝑗 ) where π’š 𝑗 is π’š with the 𝑗th bit flipped. – For each such pair, πœ“π‘‡(π’š) β‰  πœ“π‘‡(π’š 𝑗 ) but πœ“π‘ˆ(π’š) = πœ“π‘ˆ(π’š 𝑗 ) So, πœ“π‘‡ and πœ“π‘ˆ differ on exactly one of π’š, π’š 𝑗 . – Since all π’šβ€™s are paired up,

πœ“π‘‡ and πœ“π‘ˆ differ on half of the values.

11

π’š π’š 𝑗

πœ“π‘‡(x) πœ“π‘ˆ(x)

1 1 𝑏 β‹… β‹… β‹… 1 βˆ’ 𝑏 1 1 𝑐 1 β‹… β‹… β‹… 𝑐 1 Two different linear functions disagree on half of the values. Fact. A 𝑙′-parity function, where 𝑙′ β‰  𝑙, is Β½-far from any k-parity. Corollary.

slide-12
SLIDE 12

Reduction from 𝐸𝐽𝑇𝐾𝒍/πŸ‘ to Testing k-Parity

  • Let π‘ˆ be the best tester for the 𝑙-parity property for 𝜁 = 1/2

– query complexity of T is π‘Ÿ testing π‘™βˆ’parity .

  • We will construct a communication protocol for 𝐸𝐽𝑇𝐾𝒍/πŸ‘ that runs

π‘ˆ and has communication complexity 2 β‹… π‘Ÿ(testing π‘™βˆ’parity).

  • Then 2 β‹… π‘Ÿ(testing π‘™βˆ’parity) β‰₯ 𝑆 DISJ𝑙/2 β‰₯ Ξ© 𝑙/2 for 𝑙 ≀ π‘œ/2

⇓ π‘Ÿ(testing 𝑙-parity) β‰₯ Ξ© 𝑙 for 𝑙 ≀ π‘œ/2

12

holds for CC of every protocol for 𝐸𝐽𝑇𝐾𝒍 [Hastad Wigderson 07]

slide-13
SLIDE 13

Reduction from 𝐸𝐽𝑇𝐾𝒍/πŸ‘ to Testing k-Parity

13

Bob Alice π½π‘œπ‘žπ‘£π‘’: 𝑇 βŠ† [π‘œ], 𝑇 = 𝑙/2. Compute: 𝑔 = πœ“π‘‡ Input: π‘ˆ βŠ† [π‘œ], π‘ˆ = 𝑙/2 Compute: 𝑕 = πœ“π‘ˆ 1101000101110101110101010110… Output T’s answer T

β„Ž = 𝑔 + 𝑕 (𝑛𝑝𝑒 2) 𝒃𝒅𝒅𝒇𝒒𝒖/π’”π’‡π’Œπ’‡π’…π’–

β„Ž 𝑦 ? 𝑔 𝑦 + 𝑕 𝑦 𝑛𝑝𝑒 2

𝑔(𝑦) 𝑕(𝑦)

  • π‘ˆ receives its random bits from the shared random string.
slide-14
SLIDE 14

Analysis of the Reduction

Queries: Alice and Bob exchange 2 bits for every bit queried by π‘ˆ Correctness:

  • β„Ž = 𝑔 + 𝑕 𝑛𝑝𝑒 2 = πœ“π‘‡ + πœ“π‘ˆ 𝑛𝑝𝑒 2 = πœ“π‘‡Ξ”π‘ˆ
  • π‘‡Ξ”π‘ˆ = 𝑇 + π‘ˆ βˆ’ 2 𝑇 ∩ π‘ˆ
  • SΞ”π‘ˆ =

𝑙 if S∩T = βˆ… ≀ 𝑙 βˆ’ 2 if S∩T β‰  βˆ… β„Ž is π‘™βˆ’parity if S∩T = βˆ… π‘™β€²βˆ’parity where 𝑙′ β‰  𝑙 if S∩T β‰  βˆ… Summary: π‘Ÿ(testing 𝑙-parity) β‰₯ Ξ© 𝑙 for 𝑙 ≀ π‘œ/2

14

1/2-far from every 𝑙-parity

slide-15
SLIDE 15

Testing Lipschitz Property

  • n Hypercube

Lower Bound

slide-16
SLIDE 16

Lipschitz Property of Functions f: 0,1 π‘œβ†’R

16

[Jha Raskhodnikova]

  • A function 𝑔 ∢ 0,1 π‘œ β†’ R is Lipschitz

if changing a bit of 𝑦 changes 𝑔(𝑦) by at most 1.

  • Is 𝑔 Lipschitz or 𝜁-far from Lipschitz

(𝑔 has to change on many points to become Lipschitz)? – Edge 𝑦 βˆ’ 𝑧 is violated by 𝑔 if 𝑔 𝑦 βˆ’ 𝑔(𝑧) > 1.

Time:

– 𝑃(π‘œ2/𝜁), logarithmic in the size of the input, 2π‘œ

– Ξ©(π‘œ)

1 1 2 1 2 1 2 2 2 2 Lipschitz

1 2-far from Lipschitz

slide-17
SLIDE 17

Testing Lipschitz Property

17

Prove it.

Theorem Testing Lipschitz property of functions f: 0,1 π‘œ β†’ {0,1,2} requires Ξ©(π‘œ) queries.

slide-18
SLIDE 18

Summary of Lower Bound Methods

  • Yao’s Principle

– testing membership in 1*, sortedness of a list and monotonicity

  • f Boolean functions
  • Reductions from communication complexity problems

– testing if a Boolean function is a 𝑙-parity

20

slide-19
SLIDE 19

Other Models of Sublinear Computation

slide-20
SLIDE 20

22

Tolerant Property Tester

𝜺-close to YES

𝜻-far from

YES

YES

Reject with probability 2/3 Don’t care Accept with probability β‰₯ πŸ‘/πŸ’

ο‚³

Tolerant Property Tester [Rubinfeld Parnas Ron]

Randomized Algorithm

YES

Accept with probability β‰₯ πŸ‘/πŸ’ Reject with probability 2/3

NO ο‚³

slide-21
SLIDE 21

Sublinear-Time β€œRestoration” Models

Local Decoding Program Checking Local Reconstruction

23

Input: Function 𝑔 nearly satisfying some property 𝑄 Requirement: Reconstruct function 𝑔 to ensure that the reconstructed function 𝑕 satisfies 𝑄, changing 𝑔 only when necessary. For each input 𝑦, compute 𝑕(𝑦) with a few queries to 𝑔. 𝑔 𝑄 Input: A program 𝑄 computing 𝑔 correctly on most inputs. Requirement: Self-correct program 𝑄: for a given input 𝑦, compute 𝑔(𝑦) by making a few calls to P. Input: A slightly corrupted codeword Requirement: Recover individual bits of the closest codeword with a constant number of queries per recovered bit. 𝑔

slide-22
SLIDE 22

Generalization: Local Computation

[Rubinfeld Tamir Vardi Xie 2011]

  • Compute the 𝑗-th character 𝑧𝑗 of a legal output 𝑧.
  • If there are several legal outputs for a given input, be

consistent with one.

  • Example: maximal independent set in a graph.

24

slide-23
SLIDE 23

Sublinear-Space Algorithms

What if we cannot get a sublinear-time algorithm? Can we at least get sublinear space?

Note: sublinear space is broader (for any algorithm,

space complexity ≀ time complexity)

25

slide-24
SLIDE 24

Data Stream Model

Motivation: internet traffic analysis Model the stream as 𝑛 elements from [π‘œ], e.g., 𝑦1, 𝑦2, … , 𝑦𝑛 = 3, 5, 3, 7, 5, 4, … Goal: Compute a function of the stream, e.g., median, number of distinct elements, longest increasing sequence.

26

B L A - B L A - B L A - B L A - B L A - B L A - B L A -

(2) Limited working memory (3) Quickly produce output (1) Quickly process each element

Streaming Algorithm

Based on Andrew McGregor’s slides: http://www.cs.umass.edu/~mcgregor/slides/10-jhu1.pdf

slide-25
SLIDE 25

Streaming Puzzle

A stream contains π‘œ βˆ’ 1 distinct elements from π‘œ in arbitrary order. Problem: Find the missing element, using 𝑃(log π‘œ) space.

27

slide-26
SLIDE 26

Sampling from a Stream of Unknown Length

Problem: Find a uniform sample 𝑑 from a stream 𝑦1, 𝑦2, … , 𝑦𝑛

  • f unknown length 𝑛

Analysis: What is the probability that 𝑑 = 𝑦𝑗 at some time 𝑒 β‰₯ 𝑗? Pr 𝑑 = 𝑦𝑗 = 1 𝑗 β‹… 1 βˆ’ 1 𝑗 + 1 β‹… … β‹… 1 βˆ’ 1 𝑒 = 1 𝑗 β‹… 𝑗 𝑗 + 1 β‹… … β‹… 𝑒 βˆ’ 1 𝑒 = 1 𝑒 Space: 𝑃(𝑙 log π‘œ) bits to get 𝑙 samples.

28

Algorithm

  • 1. Initially, 𝑑 ← 𝑦1
  • 2. On seeing the 𝑒th element, 𝑑 ← 𝑦𝑒 with probability 1/𝑒
slide-27
SLIDE 27

Conclusion

Sublinear algorithms are possible in many settings

  • simple algorithms, more involved analysis
  • nice combinatorial problems
  • unexpected connections to other areas
  • many open questions

In the remainder of the course, we will cover research papers in the area.

29