Complexity Theory of Polynomial-Time Problems Lecture 7: 3SUM II - - PowerPoint PPT Presentation

Complexity Theory of Polynomial-Time Problems

Sebastian Krinninger

Lecture 7: 3SUM II

Reminder: 3SUM

June 16, 2016 2/19

given sets 𝐵, 𝐶, 𝐷 of 𝑜 integers are there 𝑏 ∈ 𝐵, 𝑐 ∈ 𝐶, 𝑑 ∈ 𝐷 such that 𝑏 + 𝑐 + 𝑑 = 0? well-known: 𝑃(𝑜2) Conjecture: no 𝑃 𝑜2−𝜁 algorithm → 3SUM-Hardness Alternative algorithm: 𝑃( 𝐵 ⋅ 𝐶 + |𝐷|) (store negated pairwise sums in hashmap)

Reminder: Hashing

June 16, 2016 3/19

Hash function ℎ: 𝑉 → [𝑆]

1 2 𝑆 ⋯ 𝑦 ℎ(𝑦)

Goal: Distribute uniformly, avoid collisions, etc.

Magical hash functions

June 16, 2016 4/19

Uniform difference: Pr ℎ 𝑦 − ℎ(𝑧) = 𝑗 = 1/𝑆 Balanced: 𝑦 ∈ 𝑇 ∶ ℎ 𝑦 = 𝑗 ≤ 3𝑜/𝑆 Linear: ℎ 𝑦 + ℎ 𝑧 = ℎ 𝑦 + 𝑧 (mod 𝑆) (for any set 𝑇 = 𝑦1, … , 𝑦𝑜 ⊆ 𝑉 and any 𝑗 ∈ 𝑆 ) Desired properties for family of hash functions from 𝑉 → 𝑆 (i.e., for every ℎ chosen from family) (for any 𝑦, 𝑧 ∈ 𝑉 ) (for any 𝑦, 𝑧 ∈ 𝑉 s.t. 𝑦 ≠ 𝑧 and 𝑗 ∈ 𝑆 )

But: We do not know such a family…

June 16, 2016 5/19

Almost magical hash functions

June 16, 2016 6/19

Uniform difference: Pr ℎ 𝑦 − ℎ(𝑧) = 𝑗 = 1/𝑆 Almost balanced: Expected number of elements from S hashed to heavy values is 𝑃(𝑆), where value 𝑗 ∈ 𝑆 is heavy if 𝑦 ∈ 𝑇 ∶ ℎ 𝑦 = 𝑗 > 3𝑜/𝑆 Almost linear: ℎ 𝑦 + ℎ 𝑧 ∈ ℎ 𝑦 + 𝑧 + 𝑑ℎ + 0,1 (mod 𝑆) (for any set 𝑇 = 𝑦1, … , 𝑦𝑜 ⊆ 𝑉 and any 𝑗 ∈ 𝑆 ) Desired properties for family of hash functions from 𝑉 → 𝑆 (i.e., for every ℎ chosen from family) (for any 𝑦, 𝑧 ∈ 𝑉 and some integer 𝑑ℎ depending only on ℎ) (for any 𝑦, 𝑧 ∈ 𝑉 s.t. 𝑦 ≠ 𝑧 and 𝑗 ∈ 𝑆 )

Definition of hash function

June 16, 2016 7/19

Rest of this lecture: ℎ picked randomly from this family ℋ𝑉,𝑆,𝑠 = ℎ𝑏,𝑐: 𝑉 → 𝑆 ∣ 𝑏 ∈ 𝑠 odd integer and 𝑐 ∈ 𝑠 ℎ𝑏,𝑐 𝑦 = 𝑏𝑦 + 𝑐 mod 𝑠 div 𝑠/𝑆 Thm: Family ℋ𝑉,𝑆,𝑠 is has the uniform difference property, is almost balanced and almost linear with 𝑑ℎ𝑏,𝑐 = (𝑐 − 1 mod 𝑠) div 𝑠/𝑆 . Set 𝑠 = 𝑙𝑛 for some 𝑙 ≥ 𝑉/2 and 𝑉, 𝑆, 𝑠 powers of 2 (Pairwise independence [Dietzfelbinger ’96] implies uniform difference (easy to check) and almost balanced [Baran et al. ‘08]. Almost linear: easy to check.)

Hashing down the universe

June 16, 2016 8/19

Lem: If 3SUM on universe of size 𝑃(𝑜3) solvable in exp. time 𝑃(𝑜2−𝜗), then 3SUM on arbitrary universe solvable in expect. time 𝑃(𝑜2−𝜗). Algorithm: Repeat until output:

• Pick hash function ℎ: 1 … 𝑉 → 1 … 6𝑜3 at random
• 𝐵′ =

ℎ 𝑏 𝑏 ∈ 𝐵 , 𝐶′ = {ℎ 𝑐 ∣ 𝑐 ∈ 𝐶}, 𝐷′ = ℎ 𝑑 + 𝑑ℎ 𝑑 ∈ 𝐷

• 𝐵′′ =

ℎ 𝑏 𝑏 ∈ 𝐵 , 𝐶′′ = {ℎ 𝑐 ∣ 𝑐 ∈ 𝐶}, 𝐷′′ = ℎ 𝑑 + 𝑑ℎ + 1 𝑑 ∈ 𝐷

• Solve two 3SUM instances 𝐵′, 𝐶′, 𝐷′ and 𝐵′′, 𝐶′′, 𝐷′′
• If algorithm reports no 3SUM witness: output ‘no 3SUM’
• Consider first reported 3SUM witness 𝑦′, 𝑧′, 𝑨′ for 𝐵′, 𝐶′, 𝐷′ :
• If ℎ−1 𝑦′ , ℎ−1 𝑧′ , ℎ−1 𝑨′ − 𝑑ℎ contains witness 𝑦, 𝑧, 𝑨: output

𝑦, 𝑧, 𝑨

• Consider first reported 3SUM witness 𝑦′′, 𝑧′′, 𝑨′′ for 𝐵′′, 𝐶′′, 𝐷′′ :
• If ℎ−1 𝑦′′ , ℎ−1 𝑧′′ , ℎ−1 𝑨′′ − 𝑑ℎ − 1 contains witness 𝑦, 𝑧, 𝑨:
• utput 𝑦, 𝑧, 𝑨

No false negatives: If 𝑦 + 𝑧 = 𝑨, then ℎ 𝑦 + ℎ 𝑧 ∈ ℎ 𝑨 + 𝑑ℎ + 0,1

Follows from [Baran et al. ‘08]

Running Time

June 16, 2016 9/19

Number of iterations: Triple 𝑦, 𝑧, 𝑨 gives false positive if 𝑦 + 𝑧 ≠ 𝑨 and one of ℎ 𝑦 + ℎ 𝑧 = ℎ 𝑨 + 𝑑ℎ or ℎ 𝑦 + ℎ 𝑧 = ℎ 𝑨 + 𝑑ℎ + 1 Linearity: ℎ 𝑦 + ℎ 𝑧 = ℎ 𝑦 + 𝑧 + 𝑑ℎ or ℎ 𝑦 + ℎ 𝑧 = ℎ 𝑦 + 𝑧 + 𝑑ℎ + 1 Thus, probability that fixed 𝑦, 𝑧, 𝑨 (with 𝑦 + 𝑧 ≠ 𝑨) gives false positive is: Pr ℎ 𝑦 + 𝑧 − ℎ 𝑨 ∈ {−1,0,1} ≤

3 6𝑜3 = 1 2𝑜3

(uniform difference) Overall probability of false positive: ≤ 𝑜3 ⋅

1 2𝑜3 = 1 2

In expectation: 2 iterations until no false positive (waiting time bound) (If no false positive, then algorithm certainly stops) We need to bound:

• Number of iterations 𝑃(1)
• Number of candidate witnesses 𝑃(1)

Then: number of calls to 3SUM algorithm: 𝑃(1)

Running Time

June 16, 2016 10/19

Number of candidate witnesses: Fix 3SUM witness 𝑦′, 𝑧′, 𝑨′ of instance (𝐵′, 𝐶′, 𝐷′) Let 𝑦∗ ∈ ℎ−1 𝑦′ For every 𝑦 ≠ 𝑦∗: Pr ℎ 𝑦 = ℎ 𝑦∗ =

1 6𝑜3

(uniform difference) 𝐹 ℎ−1 𝑦′ ≤ 1 +

𝑜 4𝑜3 ≤ 2

Similarly: 𝐹 ℎ−1 𝑧′ ≤ 2, 𝐹 ℎ−1 𝑨′ ≤ 2 𝐹 ℎ−1 𝑦′ ∪ ℎ−1 𝑧′ ∪ ℎ−1 𝑨′ ≤ 𝑃(1) (linearity of expectation) In expectation, algorithm manually checks constant number of candidate witnesses per iteration We need to bound:

• Number of iterations 𝑃(1)
• Number of candidate witnesses 𝑃(1)

Then: number of calls to 3SUM algorithm: 𝑃(1)

Convolution 3SUM

June 16, 2016 11/19

Given array 𝐵 1 … 𝑜 of integers are there 𝑗, 𝑘 such that 𝐵 𝑗 + 𝐵 𝑘 = 𝐵[𝑗 + 𝑘]? trivial algorithm: 𝑃(𝑜2) Thm: There is no 𝑃(𝑜2−𝜗) algorithm for Convolution 3SUM unless the 3SUM Conjecture fails.

𝑗 𝑦 𝑧 𝑦 + 𝑧 𝑘 𝑗 + 𝑘 [Pătrașcu 2010]

Stepping stone towards hardness of other “structured” problems

Reduction from 3SUM

June 16, 2016 12/19

Given set 𝑌 ⊆ [𝑉] of integers are there 𝑦, 𝑧, 𝑨 ∈ 𝑌 such that 𝑦 + 𝑧 = 𝑨? Preprocessing: Check if there is a solution 2𝑦 = 𝑨 𝑃 𝑜 log 𝑜 Pick random hash function ℎ: 𝑉 → 𝑆 (almost linear, etc.) 1 2 𝑆 ⋯ In expectation: 𝑃 𝑆 elements in buckets with load > 3𝑜/𝑆 (almost bal.) For each such 𝑦: check for 3SUM triple involving 𝑦 𝑃 𝑆𝑜 (in exp.) 1 3𝑜/𝑆 ⋮ For this proof: assume ℎ is almost balanced and linear (magically…)

Convolution 3SUM instance

June 16, 2016 13/19

1 𝑢 𝑆 ⋯

𝑙 𝑗 𝑘

Number elements in each bucket from 0 to

3𝑜 𝑆 − 1

Iterate over all triples 𝑗, 𝑘, 𝑙 ∈ 3𝑜/𝑆 ⋯ ⋯ ⋯ For every bucket 𝑢:

• Put 𝑗-th element to 𝐵[8𝑢 + 1]
• Put 𝑘-th element to 𝐵[8𝑢 + 3]
• Put 𝑙-th element to 𝐵[8𝑢 + 4]

Set all other array entries to ∞ (sufficiently large number)

3𝑜 𝑆 3

instances of Convolution 3SUM

Correctness

Assume 𝑦 + 𝑧 = 𝑨 Then ℎ 𝑦 + ℎ 𝑧 = ℎ 𝑨 (mod 𝑆) (linearity) Duplicate array for Convolution 3SUM instance If 𝑦 = 𝑧, triple found in preprocessing If 𝑦, 𝑧, or 𝑨 hashed to heavy bucket: triple found in second step Either ℎ 𝑦 + ℎ 𝑧 = ℎ 𝑨 or ℎ 𝑦 + ℎ 𝑧 = ℎ 𝑨 + 𝑆 Observation: 𝐵 𝑗 + 𝐵 𝑘 = 𝐵 𝑗 + 𝑘 only if 𝑗 = 8𝑢1 + 1 and 𝑘 = 8𝑢2 + 3 𝐵 8ℎ 𝑦 + 1 + 𝐵 8ℎ 𝑧 + 3 = 𝐵 8ℎ 𝑨 + 4 or 𝐵 8ℎ 𝑦 + 1 + 𝐵 8ℎ 𝑧 + 3 = 𝐵 8(ℎ 𝑨 + 𝑆) + 4 Thus, no false negatives. Also no false positives: (𝑦 + 𝑧 = 𝑨 mod 8 has unique solution over 1,3,4 and 𝐵 𝑗 ≠ 𝐵[𝑘])

Running Time

Assumption: Convolution 3SUM in time 𝑃(𝑜2−𝜗) Total expected running time: 𝑃 𝑜 log 𝑜 + 𝑜𝑆 +

𝑜 𝑆 3

𝑜2−𝜗 Set 𝑆 = 𝑜1−𝜗/4 Total time: 𝑃 𝑜2−𝜗/4 Contradicts 3SUM Conjecture

Set Disjointness Problem

• 1. Preprocess subsets 𝒝, ℬ ⊆ 𝑉 over universe 𝑉
• 2. Answer queries: Given 𝐵 ∈ 𝒝, 𝐶 ∈ ℬ, is 𝐵 ∩ 𝐶 ≠ ∅?

Repeated queries (Static) data structure Goal: Lower bound on preprocessing and query time Queries not known in advance Offline Set Disjointness: 𝑟 queries known in advance (part of input)

Reduction to 3SUM [Kopelowitz et al]

Thm: Let 𝑔 𝑜 be such that 3SUM requires expected time Ω 𝑜2/𝑔(𝑜) . For any constant 0 ≤ 𝛿 < 1, let ALG be an algorithm for offline Set Disjointness where 𝒝 = ℬ = Θ(𝑜 log 𝑜), 𝑉 = Θ 𝑜2−2𝛿 , each set in 𝒝 ∪ ℬ has at most 𝑃(𝑜1−𝛿) elements from 𝑉, and 𝑟 = Θ 𝑜1+𝛿 log 𝑜 . Then ALG requires expected time Ω 𝑜2/𝑔(𝑜) . Cor: Assuming the 3SUM conjecture, for any 0 < 𝛿 < 1, any data structure for Set Disjointness has 𝑢𝑞 + 𝑂

1+𝛿 2−𝛿𝑢𝑟 = Ω 𝑂 2 2−𝛿−𝑝(1)

where 𝑂 is the sum of the set sizes, 𝑢𝑞 is the preprocessing time, and 𝑢𝑟 is the time per query. Example: Data structures with constant query time Make 𝛿 tend to 1, need 𝑢𝑞 = Ω 𝑂2−𝑝(1) Evidence that trivial preprocessing algorithm is optimal (for constant query) (From Thm: 𝑂 = Θ 𝑜2−𝛿 log 𝑜 )

3SUM version

In the following proof we use a balanced, linear hash function with uniform difference property. (magically…) This can be modified for almost balanced, almost linear hash function with uniform difference property. Given set 𝑌 ⊆ [𝑉] of integers are there 𝑦, 𝑧, 𝑨 ∈ 𝑌 such that 𝑦 − 𝑧 = 𝑨?

Crucial insight

June 16, 2016 19/19

⇔

𝑏 𝑐 + = 𝑑 𝑏 𝑐↑ + = 𝑑 − 𝑐↓

higher order bits lower order bits

Algorithm Overview

June 16, 2016 20/19

Set 𝑆 = 𝑜𝛿, 𝑅 =

5𝑜 𝑆 2

Pick random hash functions ℎ: 𝑉 → [𝑆] and 𝑕𝑙: 𝑉 → [𝑅] for 𝑙 = 1 to 10 log 𝑜 Initialize buckets 𝐶 1 , … , 𝐶[𝑆] s.t. 𝐶 𝑗 = {𝑦 ∶ ℎ 𝑦 = 𝑗} For all 𝑗 ∈ 𝑆 , 𝑘 ∈ 𝑅 , initialize buckets 𝐶𝑙

↑ 𝑗, 𝑘 and 𝐶𝑙 ↓[𝑗, 𝑘] s.t.

𝐶𝑙

↑ 𝑗, 𝑘 = 𝑕𝑙 𝑦 + 𝑘 ⋅

𝑅 mod 𝑅 ∣ 𝑦 ∈ 𝐶 𝑗 𝐶𝑙

↓ 𝑗, 𝑘 = 𝑕𝑙 𝑦 − 𝑘 (mod 𝑅) ∣ 𝑦 ∈ 𝐶 𝑗

Initialize 𝑙 set intersection problems with 𝐶𝑙

↑ 𝑗, 𝑘 ’s and 𝐶𝑙 ↓[𝑗, 𝑘]’s

For every 𝑨 ∈ 𝑌 and every 𝑗 = 1 to 𝑆 Check if 𝐶𝑙

↑ 𝑗, 𝑕𝑙 ↑(𝑨) and 𝐶𝑙 ↓ 𝑗 − ℎ 𝑨

(mod 𝑆), 𝑕𝑙

↓(𝑨) intersect

If intersection for all 𝑙: Search for 𝑦 ∈ 𝐶 𝑗 and 𝑧 ∈ 𝐶[𝑗 − ℎ 𝑨 (mod 𝑆)] s.t. 𝑦 − 𝑧 = 𝑨 and output it if found If nothing found: output ‘no 3SUM’

𝑕𝑙

↑ 𝑨 : higher order bits of 𝑕𝑙(𝑨)

𝑕𝑙

↓ 𝑨 : higher order bits of 𝑕𝑙(𝑨)

Correctness I

June 16, 2016 21/19

Algorithm verifies every triple before stopping Need to show: if 𝑦 − 𝑧 = 𝑨, then algorithm finds it Claim 1: If 𝑦 − 𝑧 = 𝑨, then 𝐶 𝑗 ∩ (𝐶 𝑘 + 𝑨) ≠ ∅ where 𝑗 = ℎ 𝑦 , 𝑘 = 𝑗 − ℎ 𝑨 mod 𝑆 Linear hash function: ℎ 𝑦 − ℎ 𝑧 = ℎ 𝑦 − 𝑧 = ℎ 𝑨 mod 𝑆 Thus: 𝑘 = ℎ 𝑦 − ℎ 𝑨 = 𝑗 − ℎ 𝑨 mod 𝑆 𝑧 ∈ 𝐶 𝑘 ⇒ 𝑦 = 𝑧 + 𝑨 ∈ 𝐶 𝑘 + 𝑨

𝑦

𝐶[𝑗]

𝑧

𝐶[𝑘]

𝑧 + 𝑨

𝐶 𝑘 + 𝑨 +𝑨 𝑦 𝑧

Correctness II

June 16, 2016 22/19

Claim 2: If 𝐶 𝑗 ∩ 𝐶 𝑘 + 𝑨 ≠ ∅, then 𝐶↑ 𝑗, 𝑕𝑙

↑(𝑨) ∩ 𝐶↓ 𝑘, 𝑕𝑙 ↓ 𝑨

≠ ∅ ∀𝑙. 𝐶 𝑗 ∩ 𝐶 𝑘 + 𝑨 ≠ ∅ ⇓ 𝑕𝑙 𝐶 𝑗 ∩ 𝑕𝑙 𝐶 𝑘 + 𝑨 ≠ ∅ ⇕ 𝑕𝑙 𝐶 𝑗 ∩ 𝑕𝑙 𝐶 𝑘 + 𝑕𝑙 𝑨 ≠ ∅ ⇕ 𝑕𝑙 𝐶 𝑗 ∩ 𝑕𝑙 𝐶 𝑘 + 𝑕𝑙

↑ 𝑨 + 𝑕𝑙 ↓ 𝑨

≠ ∅ ⇕ 𝑕𝑙 𝐶 𝑗 − 𝑕𝑙

↑ 𝑨

∩ 𝑕𝑙 𝐶 𝑘 + 𝑕𝑙

↓ 𝑨

≠ ∅ ⇕ 𝐶↑ 𝑗, 𝑕𝑙

↑(𝑨) ∩ 𝐶↓ 𝑘, 𝑕𝑙 ↓ 𝑨

≠ ∅ Claim 1: If 𝑦 − 𝑧 = 𝑨, then 𝐶 𝑗 ∩ (𝐶 𝑘 + 𝑨) ≠ ∅ where 𝑗 = ℎ 𝑦 , 𝑘 = 𝑗 − ℎ 𝑨 mod 𝑆 Conclusion: If 𝑦 − 𝑧 = 𝑨, then 𝐶↑ 𝑗, 𝑕𝑙

↑(𝑨) ∩ 𝐶↓ 𝑘, 𝑕𝑙 ↓ 𝑨

≠ ∅ ∀𝑙.

Running time

June 16, 2016 23/19

Set intersection instance:

• Number of sets: 𝑃 𝑆 𝑅𝑙 = 𝑃(𝑜 log 𝑜)
• Number of elements in each set: 𝑃

𝑅 = 𝑃 𝑜1−𝛿

• Size of universe: 𝑃 𝑅 = 𝑃 𝑜2−2𝛿
• Number of set intersection queries: 𝑃 𝑜𝑆𝑙 = 𝑃 𝑜1+𝛿 log 𝑜

Finding witnesses:

• If 𝐶𝑙

↑ 𝑗, 𝑕𝑙 ↑(𝑨) and 𝐶𝑙 ↓ 𝑘, 𝑕𝑙 ↓(𝑨) intersect, try to find 𝑦 ∈ 𝐶 𝑗 , 𝑧 ∈ 𝐶[𝑘]

s.t. 𝑦 − 𝑧 = 𝑨

• Time 𝑃

𝑜 𝑆 per witness check

• But: pair 𝑗, 𝑘 could be false positive with no such 𝑦 ∈ 𝐶 𝑗 , 𝑧 ∈ 𝐶[𝑘]
• Probability of false positive is small
• In expectation: 𝑃(1) false positives (next slide)
• Total time: 𝑃

𝑜 𝑆 + #false positives 𝑜 𝑆 = 𝑃 𝑜 𝑆

Bounding number of false positive

June 16, 2016 24/19

For a fixed 𝑨 and any pair 𝑦, 𝑧 ∈ 𝑉 s.t. 𝑦 − 𝑧 ≠ 𝑨: Pr 𝑕𝑙 𝑦 = 𝑕𝑙 𝑧 + 𝑕𝑙(𝑨) = Pr 𝑕𝑙 𝑦 − 𝑧 = 𝑕𝑙(𝑨) = 1 𝑅 (linear and uniform difference) Remember: Every bucket has size ≤

3𝑜 𝑆

(balanced)

• Prob. of false positive in buckets 𝐶[𝑗] and 𝐶[𝑘] for hash function 𝑕𝑙:

Pr 𝑕𝑙 𝐶[𝑗] = 𝑕𝑙 𝐶[𝑘] + 𝑕𝑙(𝑨) ≤ 3𝑜 𝑆

2 1

𝑅 = 9 25

• Prob. of false positive in buckets 𝐶[𝑗] and 𝐶[𝑘] for all hash functions 𝑕𝑙:

≤ 1 𝑜𝑑 In expectation: total number of false positives is a constant.