L ECTURE 6 Last time Limitations of sublinear-time algorithms - - PowerPoint PPT Presentation

β–Ά
l ecture 6
SMART_READER_LITE
LIVE PREVIEW

L ECTURE 6 Last time Limitations of sublinear-time algorithms - - PowerPoint PPT Presentation

Jun 11, 2023 175 likes β€’567 views

Sublinear Algorithms L ECTURE 6 Last time Limitations of sublinear-time algorithms Yaos Minimax Principle Example: testing monotonicity Communication complexity Today Communication complexity Other models of computation

slide-1
SLIDE 1

10/5/2020

Sublinear Algorithms

LECTURE 6

Last time

  • Limitations of sublinear-time algorithms
  • Yao’s Minimax Principle
  • Example: testing monotonicity
  • Communication complexity

Today

  • Communication complexity
  • Other models of computation

Sofya Raskhodnikova;Boston University

slide-2
SLIDE 2

Communication Complexity

A Method for Proving Lower Bounds

[Blais Brody Matulef 11]

Use known lower bounds for other models of computation

Partially based on slides by Eric Blais

slide-3
SLIDE 3

(Randomized) Communication Complexity

3

Compute 𝐷 𝑦, 𝑧 0100 11 001 β‹― 0011 Bob Alice π½π‘œπ‘žπ‘£π‘’: 𝑦 Input: 𝑧 1101000101110101110101010110… π‘‡β„Žπ‘π‘ π‘“π‘’ π‘ π‘π‘œπ‘’π‘π‘› π‘‘π‘’π‘ π‘—π‘œπ‘• Goal: minimize the number of bits exchanged.

  • Communication complexity of a protocol is the maximum number of bits

exchanged by the protocol.

  • Communication complexity of a function 𝐷, denoted 𝑆(𝐷), is the

communication complexity of the best protocol for computing C.

slide-4
SLIDE 4

Example: Set Disjointness 𝐸𝐽𝑇𝐾𝒍

4

Theorem [Kalyanasundaram Schmitger 92, Razborov 92] 𝑆 DISJ𝑙 β‰₯ Ξ© 𝑙 for all 𝑙 ≀

π‘œ 2.

Compute 𝐸𝐽𝑇𝐾𝑙 𝑇, π‘ˆ = α‰Šπ’ƒπ’…π’…π’‡π’’π’– if 𝑇 ∩ π‘ˆ = βˆ… π’”π’‡π’Œπ’‡π’…π’–

  • therwise

Bob Alice π½π‘œπ‘žπ‘£π‘’: 𝑇 βŠ† [π‘œ], 𝑇 = 𝑙. Input: π‘ˆ βŠ† [π‘œ], π‘ˆ = 𝑙 1101000101110101110101010110…

slide-5
SLIDE 5

A lower bound using CC method

Testing if a Boolean function is a k-parity

slide-6
SLIDE 6

Linear Functions Over Finite Field 𝔾2

6

A Boolean function 𝑔: 0,1 π‘œ β†’ {0,1} is linear (also called parity) if 𝑔 𝑦1, … , π‘¦π‘œ = 𝑏1𝑦1 + β‹― + π‘π‘œπ‘¦π‘œ for some 𝑏1, … , π‘π‘œ ∈ {0,1}

  • Work in finite field 𝔾2

– Other accepted notation for 𝔾2: 𝐻𝐺

2 and β„€2

– Addition and multiplication is mod 2 – π’š= 𝑦1, … , π‘¦π‘œ , 𝒛= 𝑧1, … , π‘§π‘œ , that is, π’š, 𝒛 ∈ 0,1 π‘œ π’š + 𝒛= 𝑦1 + 𝑧1, … , π‘¦π‘œ + π‘§π‘œ no free term

001001 011001 010000

+ example

slide-7
SLIDE 7

Linear Functions Over Finite Field 𝔾2

7

A Boolean function 𝑔: 0,1 π‘œ β†’ {0,1} is linear (also called parity) if 𝑔 𝑦1, … , π‘¦π‘œ = 𝑏1𝑦1 + β‹― + π‘π‘œπ‘¦π‘œ for some 𝑏1, … , π‘π‘œ ∈ {0,1} ⇕ 𝑔 𝑦1, … , π‘¦π‘œ = Οƒπ‘—βˆˆS 𝑦𝑗 for some 𝑇 βŠ† π‘œ . Notation: πœ“π‘‡ 𝑦 = Οƒπ‘—βˆˆπ‘‡ 𝑦𝑗.

[π‘œ] is a shorthand for {1, … π‘œ}

slide-8
SLIDE 8

k-Parity Functions

9

𝑙-Parity Functions

A function 𝑔 ∢ 0,1 π‘œ β†’ {0,1} is a 𝑙-parity if 𝑔 𝑦 = πœ“π‘‡ 𝑦 = Οƒπ‘—βˆˆπ‘‡ 𝑦𝑗 for some set 𝑇 βŠ† π‘œ of size 𝑇 = 𝑙.

slide-9
SLIDE 9

Testing if a Boolean Function is a k-Parity

10

Input: Boolean function 𝑔: 0,1 π‘œ β†’ {0,1} and an integer 𝑙 Question: Is the function a 𝑙-parity or 𝜁-far from a 𝑙-parity (β‰₯ 𝜁2π‘œ values need to be changed to make it a 𝑙-parity)? Time: O 𝑙 log 𝑙

[Buhrman, Carciaβˆ’Soriano, Matsliah, de Wolf 13]

W min(𝑙, π‘œ βˆ’ 𝑙 ) [Blais Brody Matulef 12]

  • Today: Ξ©(𝑙) for 𝑙 ≀ π‘œ/2
  • Today’s bound implies W min(𝑙, π‘œ βˆ’ 𝑙 )
slide-10
SLIDE 10

Important Fact About Linear Functions

  • Consider functions πœ“π‘‡ and πœ“π‘ˆ where 𝑇 β‰  π‘ˆ.

– Let 𝑗 be an element on which 𝑇 and π‘ˆ differ (w.l.o.g. 𝑗 ∈ 𝑇 βˆ– π‘ˆ) – Pair up all π‘œ-bit strings: (π’š, π’š 𝑗 ) where π’š 𝑗 is π’š with the 𝑗th bit flipped. – For each such pair, πœ“π‘‡(π’š) β‰  πœ“π‘‡(π’š 𝑗 ) but πœ“π‘ˆ(π’š) = πœ“π‘ˆ(π’š 𝑗 ) So, πœ“π‘‡ and πœ“π‘ˆ differ on exactly one of π’š, π’š 𝑗 . – Since all π’šβ€™s are paired up,

πœ“π‘‡ and πœ“π‘ˆ differ on half of the values.

11

π’š π’š 𝑗

πœ“π‘‡(x) πœ“π‘ˆ(x)

1 1 𝑏 β‹… β‹… β‹… 1 βˆ’ 𝑏 1 1 𝑐 1 β‹… β‹… β‹… 𝑐 1 Two different linear functions disagree on half of the values. Fact. A π‘™β€²βˆ’parity function, where 𝑙′ β‰  𝑙, is Β½-far from any k-parity. Corollary.

slide-11
SLIDE 11

Reduction from 𝐸𝐽𝑇𝐾𝒍/πŸ‘ to Testing k-Parity

  • Let π‘ˆ be the best tester for the 𝑙-parity property for 𝜁 = 1/2

– query complexity of T is π‘Ÿ testing π‘™βˆ’parity .

  • We will construct a communication protocol for 𝐸𝐽𝑇𝐾𝒍/πŸ‘ that runs

π‘ˆ and has communication complexity 2 β‹… π‘Ÿ(testing π‘™βˆ’parity).

  • Then 2 β‹… π‘Ÿ(testing π‘™βˆ’parity) β‰₯ 𝑆 DISJ𝑙/2 β‰₯ Ξ© 𝑙/2 for 𝑙 ≀ π‘œ/2

⇓ π‘Ÿ(testing 𝑙-parity) β‰₯ Ξ© 𝑙 for 𝑙 ≀ π‘œ/2

12

holds for CC of every protocol for 𝐸𝐽𝑇𝐾𝒍 [Hastad Wigderson 07]

slide-12
SLIDE 12

Reduction from 𝐸𝐽𝑇𝐾𝒍/πŸ‘ to Testing k-Parity

13

Bob Alice π½π‘œπ‘žπ‘£π‘’: 𝑇 βŠ† [π‘œ], 𝑇 = 𝑙/2. Compute: 𝑔 = πœ“π‘‡ Input: π‘ˆ βŠ† [π‘œ], π‘ˆ = 𝑙/2 Compute: 𝑕 = πœ“π‘ˆ 1101000101110101110101010110… Output T’s answer T

β„Ž = 𝑔 + 𝑕 (𝑛𝑝𝑒 2) 𝒃𝒅𝒅𝒇𝒒𝒖/π’”π’‡π’Œπ’‡π’…π’–

β„Ž 𝑦 ? 𝑔 𝑦 + 𝑕 𝑦 𝑛𝑝𝑒 2

𝑔(𝑦) 𝑕(𝑦)

  • π‘ˆ receives its random bits from the shared random string.
slide-13
SLIDE 13

Analysis of the Reduction

Queries: Alice and Bob exchange 2 bits for every bit queried by π‘ˆ Correctness:

  • β„Ž = 𝑔 + 𝑕 𝑛𝑝𝑒 2 = πœ“π‘‡ + πœ“π‘ˆ 𝑛𝑝𝑒 2 = πœ“π‘‡Ξ”π‘ˆ
  • π‘‡Ξ”π‘ˆ = 𝑇 + π‘ˆ βˆ’ 2 𝑇 ∩ π‘ˆ
  • SΞ”π‘ˆ = α‰Š

𝑙 if S∩T = βˆ… ≀ 𝑙 βˆ’ 2 if S∩T β‰  βˆ… β„Ž is α‰Šπ‘™βˆ’parity if S∩T = βˆ… π‘™β€²βˆ’parity where 𝑙′ β‰  𝑙 if S∩T β‰  βˆ… Summary: π‘Ÿ(testing 𝑙-parity) β‰₯ Ξ© 𝑙 for 𝑙 ≀ π‘œ/2

14

1/2-far from every 𝑙-parity

slide-14
SLIDE 14

Testing Lipschitz Property

  • n Hypercube

Lower Bound

slide-15
SLIDE 15

Lipschitz Property of Functions f: 0,1 π‘œβ†’R

16

  • A function 𝑔 ∢ 0,1 π‘œ β†’ R is Lipschitz

if changing a bit of 𝑦 changes 𝑔(𝑦) by at most 1.

  • Is 𝑔 Lipschitz or 𝜁-far from Lipschitz

(𝑔 has to change on many points to become Lipschitz)? – Edge 𝑦 βˆ’ 𝑧 is violated by 𝑔 if 𝑔 𝑦 βˆ’ 𝑔(𝑧) > 1.

Time:

– 𝑃(π‘œ/𝜁), logarithmic in the size of the input, 2π‘œ [Chakrabarty Seshadhri] – Ξ©(π‘œ) [Jha Raskhodnikova] 1 1 2 1 2 1 2 2 2 2 Lipschitz

1 2-far from Lipschitz

slide-16
SLIDE 16

Testing Lipschitz Property

17

Prove it.

Theorem Testing Lipschitz property of functions f: 0,1 π‘œ β†’ {0,1,2} requires Ξ©(π‘œ) queries.

slide-17
SLIDE 17

Summary of Lower Bound Methods

  • Yao’s Principle

– testing membership in 1*, sortedness of a list and monotonicity

  • f Boolean functions
  • Reductions from communication complexity problems

– testing if a Boolean function is a 𝑙-parity

18

slide-18
SLIDE 18

Other Models of Sublinear Computation

slide-19
SLIDE 19

Algorithms Resilient to Erasures (or Errors)

20

βŠ₯ βŠ₯ A - B L βŠ₯ βŠ₯ B L A - B L A - βŠ₯ L A - B L A - B L βŠ₯ - B L A

? L ? B ? L ?

  • ≀ 𝜷 fraction of the input is erased (or modified)

adversarially before algorithm runs

  • Algorithm does not know in advance what’s erased

(or modified)

  • Can we still perform computational tasks?

randomized algorithm

slide-20
SLIDE 20

Property Tester [Rubinfeld Sudan 96,

Goldreich Goldwasser Ron 98]

randomized algorithm

21

Property Testing

Two objects are at distance 𝜁 = they differ in an 𝜁 fraction of places

Don’t care Accept with probability β‰₯ πŸ‘/πŸ’ Reject with probability β‰₯ πŸ‘/πŸ’

YES NO far from YES

𝜁

slide-21
SLIDE 21

Property Tester [Rubinfeld Sudan 96,

Goldreich Goldwasser Ron 98]

randomized algorithm

22

Property Testing with Erasures

Two objects are at distance 𝜁 = they differ in an 𝜁 fraction of places

Don’t care Accept with probability β‰₯ πŸ‘/πŸ’ Reject with probability β‰₯ πŸ‘/πŸ’

YES NO far from YES

𝜁

Erasure-Resilient Property Tester [Dixit

Raskhodnikova Thakurta Varma 16]

  • ≀ 𝛽 fraction of the input is erased

adversarially Don’t care Accept with probability β‰₯ πŸ‘/πŸ’ Reject with probability β‰₯ πŸ‘/πŸ’

Can be completed to YES

NO

Any completion is far from

YES

𝜁

slide-22
SLIDE 22

Can We Make Testers Ξ±-Erasure-Resilient?

It is easy if a tester makes only uniform queries (and the property is extendable).

  • Use the original tester as black box and ignore erasures:
  • Applies to many properties

– Monotonicity over poset domains

[Fischer Lehman Newman Raskhodnikova Rubinfeld Samorodnitsky 02]

– Convexity of black and white images

[Berman Murzabulatov Raskhodnikova 16]

– Boolean arrays having at most 𝑙 alternations in values – …

23

O

1 1βˆ’Ξ± factor query complexity overhead for all Ξ± ∈ 0,1 .

slide-23
SLIDE 23

Erasure-Resilient Sortedness Tester?

Example: Testing sortedness of π‘œ-element arrays

  • Every uniform tester requires Ξ©

π‘œ queries.

  • [EKKRV00] (optimal) tester that makes 𝑃(log π‘œ) queries
  • Can we make it erasure-resilient O

1 1βˆ’Ξ± factor overhead?

  • All known optimal sortedness testers [EKKRV00, BGJRW09, CS13a] break

with just one erasure.

24

Known optimal testers for monotonicity, Lipschitz property and convexity of functions [GGLRS00, DGLRRS99, EKKRV00, F04, CS13a, CS13b,

CST14, BRY14, BRY14, CDST15, KMS15, BB16, JR13, CS13a, BRY14, BRY14, CDJS15, PRR03, BRY14] break on a constant number of erasures.

1 2 π‘œ

random search element

βŠ₯ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

slide-24
SLIDE 24

Erasure-Resilient Sortedness Tester

25

Input: Ρ,α ∈ (0,1); query access to an array

1.

Repeat Θ(1/Ξ΅) times: a. Sample uniformly until you get a nonerased search point 𝒕. b. Binary search for 𝒕 with uniform nonerased split points. c. Reject if there are violations along the search path. 2. Accept if no violations were found.

1 2 π‘œ 𝒕

βŠ₯

37 π’’πŸ‘ π’’πŸ

βŠ₯ βŠ₯

9

βŠ₯

88 61 37

slide-25
SLIDE 25

Analysis of the Sortedness Tester

  • 1. Array is sorted ⟹ tester accepts
  • 2. Array is Ξ΅-far from sorted ⟹ one iteration rejects with

probability β‰₯ Ξ΅

– Need to repeat only Θ(1/Ξ΅) times to get error probability 2/3

3. Want to show: expected # of queries per iteration is 𝑃

log π‘œ 1βˆ’π›½

– Tester traverses a uniformly random search path in a random binary search tree. – The # of levels in a random binary search is 𝑃 log π‘œ w.h.p.

26

  • Claim. Expected # of queries to one level of binary search is

𝑃

1 1βˆ’π›½

slide-26
SLIDE 26

Expected Number of Queries in One Iteration

At level 𝑙

27

1 2 π‘œ Interval I 𝛽𝐽 = fraction of erasures in I

Pr [search point 𝒕 is in 𝐽] = # nonerased points in I total # nonerased points ≀ |I|(1 βˆ’Ξ±π½) n(1 βˆ’ Ξ±) E [Q] = = ෍

𝐽

1 1 βˆ’ α𝐽 β‹… |I| (1 βˆ’Ξ±π½) n(1 βˆ’ Ξ±) ෍

intervals 𝐽 in level 𝑙

𝐹 𝑅 𝑑 ∈ 𝐽] β‹… Pr[𝑑 ∈ 𝐽] 𝑹 = # of queries ≀ 1 1 βˆ’ Ξ±

slide-27
SLIDE 27

What We Proved

  • [Dixit Raskhodnikova Thakurta Varma 16]

28

Our Ξ±-erasure-resilient Ξ΅-tester for sortedness of 𝒐-element arrays makes O

𝐦𝐩𝐑 𝒐

Ξ΅ πŸβˆ’Ξ± queries for all Ξ±, Ξ΅ ∈ 0,1 .

Theorem

slide-28
SLIDE 28

Property Tester [Rubinfeld Sudan 96,

Goldreich Goldwasser Ron 98]

randomized algorithm

29

Property Testing with Erasures

Two objects are at distance 𝜁 = they differ in an 𝜁 fraction of places

Don’t care Accept with probability β‰₯ πŸ‘/πŸ’ Reject with probability β‰₯ πŸ‘/πŸ’

YES NO far from YES

𝜁

Erasure-Resilient Property Tester [Dixit

Raskhodnikova Thakurta Varma 16]

  • ≀ 𝛽 fraction of the input is erased

adversarially Don’t care Accept with probability β‰₯ πŸ‘/πŸ’ Reject with probability β‰₯ πŸ‘/πŸ’

Can be completed to YES

NO

Any completion is far from

YES

𝜁

slide-29
SLIDE 29

Property Tester [Rubinfeld Sudan 96,

Goldreich Goldwasser Ron 98]

randomized algorithm

30

Property Testing with Errors

Two objects are at distance 𝜁 = they differ in an 𝜁 fraction of places

Don’t care Accept with probability β‰₯ πŸ‘/πŸ’ Reject with probability β‰₯ πŸ‘/πŸ’

YES NO far from YES

𝜁

Tolerant Property Tester

[Parnas Ron Rubinfeld 06]

  • ≀ 𝛽 fraction of the input is wrong

Don’t care Accept with probability β‰₯ πŸ‘/πŸ’ Reject with probability β‰₯ πŸ‘/πŸ’

YES NO far from YES

𝜁 𝛽

slide-30
SLIDE 30

Property Tester [Rubinfeld Sudan 96,

Goldreich Goldwasser Ron 98]

randomized algorithm

31

Property Testing with Errors

Two objects are at distance 𝜁 = they differ in an 𝜁 fraction of places

Don’t care Accept with probability β‰₯ πŸ‘/πŸ’ Reject with probability β‰₯ πŸ‘/πŸ’

YES NO far from YES

𝜁

Tolerant Property Tester

[Parnas Ron Rubinfeld 06]

  • ≀ 𝛽 fraction of the input is wrong

Don’t care Accept with probability β‰₯ πŸ‘/πŸ’ Reject with probability β‰₯ πŸ‘/πŸ’

YES NO far from YES

𝜁 𝛽

slide-31
SLIDE 31

Relationships Between Models

Containments are strict:

  • [Fischer Fortnow 05]: standard vs. tolerant
  • [Dixit R Thakurta Varma 16]: standard vs. erasure-resilient
  • [R Ron-Zewi Varma 19]: erasure-resilient vs. tolerant

32

Ξ΅-testable 𝛃-erasure-resiliently Ξ΅-testable (𝛃, Ξ΅)-tolerantly testable

slide-32
SLIDE 32

Distance Approximation for Boolean Functions

33

[Parnas Ron Rubinfeld 06] Goal: Output 𝑒𝑗𝑑𝑒 𝑔, 𝒬 Β± 𝜁 in sublinear time

ො 𝜻

Algorithm

π’š π’ˆ(π’š) 𝜻 ∈ (𝟏, 𝟐)

π’ˆ

slide-33
SLIDE 33

Sublinear-Time β€œRestoration” Models

Local Decoding Program Checking Local Reconstruction

34

Input: Function 𝑔 nearly satisfying some property 𝑄 Requirement: Reconstruct function 𝑔 to ensure that the reconstructed function 𝑕 satisfies 𝑄, changing 𝑔 only when necessary. For each input 𝑦, compute 𝑕(𝑦) with a few queries to 𝑔. 𝑔 𝑄 Input: A program 𝑄 computing 𝑔 correctly on most inputs. Requirement: Self-correct program 𝑄: for a given input 𝑦, compute 𝑔(𝑦) by making a few calls to P. Input: A slightly corrupted codeword Requirement: Recover individual bits of the closest codeword with a constant number of queries per recovered bit. 𝑔

slide-34
SLIDE 34

Generalization: Local Computation

[Rubinfeld Tamir Vardi Xie 2011]

  • Compute the 𝑗-th character 𝑧𝑗 of a legal output 𝑧.
  • If there are several legal outputs for a given input, be

consistent with one.

  • Example: maximal independent set in a graph.

35

slide-35
SLIDE 35

Sublinear-Space Algorithms

What if we cannot get a sublinear-time algorithm? Can we at least get sublinear space?

Note: sublinear space is broader (for any algorithm,

space complexity ≀ time complexity)

36

slide-36
SLIDE 36

Data Stream Model

Motivation: internet traffic analysis Model the stream as 𝑛 elements from [π‘œ], e.g., 𝑦1, 𝑦2, … , 𝑦𝑛 = 3, 5, 3, 7, 5, 4, … Goal: Compute a function of the stream, e.g., median, number of distinct elements, longest increasing sequence.

37

B L A - B L A - B L A - B L A - B L A - B L A - B L A -

(2) Limited working memory (3) Quickly produce output (1) Quickly process each element

Streaming Algorithm

Based on Andrew McGregor’s slides: http://www.cs.umass.edu/~mcgregor/slides/10-jhu1.pdf

slide-37
SLIDE 37

Streaming Puzzle

A stream contains π‘œ βˆ’ 1 distinct elements from π‘œ in arbitrary order. Problem: Find the missing element, using 𝑃(log π‘œ) space.

38

slide-38
SLIDE 38

Conclusion

Sublinear algorithms are possible in many settings

  • simple algorithms, more involved analysis
  • nice combinatorial problems
  • unexpected connections to other areas
  • many open questions

In the remainder of the course, we will cover research papers in the area.

42