Strong Direct Sum for Randomized Query Complexity Eric Blais Joshua - - PowerPoint PPT Presentation

Strong Direct Sum for Randomized Query Complexity

Conference on Computational Complexity New Brunswick, New Jersey July 18, 2019

Eric Blais Joshua Brody University of Waterloo Swarthmore College

Outline

• Introduction
• Strong Direct Sum
• Query Resistance
• Separation Theorem
• Open Problems

Direct Sum Theorems

Does computing f(x) on k copies scale with k?

Direct Sum Theorems

Direct Sum Theorem: Computing k copies of f requires k times the resources Direct Product Theorem: Success prob. of computing k copies of f with << k resources is 2-Ω(k)

Does computing f(x) on k copies scale with k?

Direct Sum Theorems

Direct Sum Theorem: Computing k copies of f requires k times the resources Direct Product Theorem: Success prob. of computing k copies of f with << k resources is 2-Ω(k)

Does computing f(x) on k copies scale with k? Strong Direct Sum: computing k copies of f w/error ε requires >> k times the resources

Our Main Results

Corollary: There is f such that Rε(fk) = ϴ(klog(k)Rε(f)) Strong Direct Sum for average query complexity: For any f and any k, computing fk satisfies: Rε(fk) = ϴ(kRε/k(f)) Separation Theorem: for all ε > 2-n^1/3 , there is total function f : {0,1}N → {0,1} such that Rε(f) = ϴ(R(f)log(1/ε))

Query Complexity

aka Decision Tree Complexity

x1 x3 x8 1

abort

Query Complexity

aka Decision Tree Complexity Decision Tree for f: {0,1}n → {0,1}:

• internal nodes labeled w/input bits xi
• leaves labeled w/output or ABORT
• cost(T,x): depth of T on input x

Randomized DT: distribution A on decision trees

• cost(A) = maxT,x cost(T,x)
• acost(A) = maxxET~A [cost(T, x)]

x1 x3 x8 1

abort

Query Complexity

aka Decision Tree Complexity Decision Tree for f: {0,1}n → {0,1}:

• internal nodes labeled w/input bits xi
• leaves labeled w/output or ABORT
• cost(T,x): depth of T on input x

Randomized DT: distribution A on decision trees

• cost(A) = maxT,x cost(T,x)
• acost(A) = maxxET~A [cost(T, x)]

x1 x3 x8 1

abort

Distributional QC : min Ex[cost(T,x)] s.t. Pr[abort] ≤ 훿 and Pr[error] ≤ ε

D훿,ε(f)

μ

Randomized QC : minimum cost of randomized algorithm s.t. Pr[abort] ≤ 훿 and Pr[error] ≤ ε R훿,ε(f) Average case Randomized QC : minimum acost of randomized algorithm s.t. Pr[error] ≤ ε Rε(f)

Basic Results

Minimax Lemma: max D2훿,2ε(f)

μ μ

max D훿/2,ε/2(f)

μ μ

≤ R훿,ε(f) ≤ Error Reduction: R (f) ≤ O(log(t)R1/2, 1/3(f))

O(1/t), O(1/t)

Average QC vs Aborts: ≤ Rε(f) ≤

훿R훿,ε(f)

R훿,(1-훿)ε(f)/(1-훿)

Basic Results

Average QC vs Aborts: R훿,(1-훿)ε(f)/(1-훿) ≤ Rε(f) ≤

훿R훿,ε(f)

First inequality: Algorithm A: ε-error, acost(A)= q Second inequality: Algorithm B’: (1-훿)ε-error, 훿-abort, q queries.

Basic Results

Average QC vs Aborts: R훿,(1-훿)ε(f)/(1-훿) ≤ Rε(f) ≤

훿R훿,ε(f)

Algorithm B(x) { emulate A(x) abort if > q/훿 queries } Algorithm A’(x) { repeat: emulate B’(x) until no aborts }

First inequality: Algorithm A: ε-error, acost(A)= q Second inequality: Algorithm B’: (1-훿)ε-error, 훿-abort, q queries.

Previous Work

Information Complexity: [MWY13, MWY15]

• strong direct sum for information complexity w/aborts + error
• applications for streaming/sketching algorithms

Direct Product Theorem: [Drucker 12]

• direct product theorems for randomized query complexity

Separation Theorems: [GPW15, ABBLSS17]

• query complexity separations based on pointer functions
• polynomial separation R0(f) vs Rε(f)

Direct Sum Theorems:

• [Jain Klauck Santha 10]: Rε(fk) ≥ 훿2kRε/(1-훿)+훿(f)
• [Ben-David Kothari 18]: Rε(fk) ≥ kRε(f)

Our Results

Strong Direct Sum Theorem: D0,ε(fk) = Ω(kD1/5,40ε/k(f))

μk μ

Separation Theorem: There is f : {0,1}N → {0,1} such that for all ε > 2-N^1/3 , we have R = Ω(R1/3(f)log(1/ε))

훿,ε(f)

Corollary: There is f such that R1/3(fk) = Ω(klog(k)Rε(f))

Our Results

Strong Direct Sum Theorem: D0,ε(fk) = Ω(kD1/5,40ε/k(f))

μk μ

Separation Theorem: There is f : {0,1}N → {0,1} such that for all ε > 2-N^1/3 , we have R = Ω(R1/3(f)log(1/ε))

훿,ε(f)

Corollary: There is f such that R1/3(fk) = Ω(klog(k)Rε(f)) proof: R1/3(fk) ≥ R0,1/3(fk) = Ω(kR1/5,40/3k(f)) = Ω(klog(k)R1/3(f))

Our Results

Strong Direct Sum Theorem: D0,ε(fk) = Ω(kD1/5,40ε/k(f))

μk μ

Separation Theorem: There is f : {0,1}N → {0,1} such that for all ε > 2-N^1/3 , we have R = Ω(R1/3(f)log(1/ε))

훿,ε(f)

Corollary: There is f such that R1/3(fk) = Ω(klog(k)Rε(f)) Query-resistant codes: probabilistic encoding G: Σ →{0,1}N such that N/3 bits of G(x) needed to learn anything about x Key Technical result: proof: R1/3(fk) ≥ R0,1/3(fk) = Ω(kR1/5,40/3k(f)) = Ω(klog(k)R1/3(f))

Outline

• Introduction
• Strong Direct Sum
• Query Resistance
• Separation Theorem
• Open Problems

Strong Direct Sum Theorem: D0,ε(fk) = 훀(kD1/5,40ε/k(f))

μk μ

Let A be an ε-error algorithm for fk with q queries. Goal: (ε/k)-error algorithm B for f with q/k queries. Let y = (y1,…, yk). Embed(y,i,x) := y, w/i-th coord replaced by x.

Strong Direct Sum Theorem: D0,ε(fk) = 훀(kD1/5,40ε/k(f))

μk μ

Let A be an ε-error algorithm for fk with q queries. Goal: (ε/k)-error algorithm B for f with q/k queries. Let y = (y1,…, yk). Embed(y,i,x) := y, w/i-th coord replaced by x.

Algorithm B(x) { carefully select y,i emulate A(EMBED(y,i,x)) abort if problems found }

Strong Direct Sum Theorem: D0,ε(fk) = 훀(kD1/5,40ε/k(f))

μk μ

Let A be an ε-error algorithm for fk with q queries. Goal: (ε/k)-error algorithm B for f with q/k queries. Let y = (y1,…, yk). Embed(y,i,x) := y, w/i-th coord replaced by x.

Algorithm B(x) { carefully select y,i emulate A(EMBED(y,i,x)) abort if problems found }

Intuition: success on typical coordinate ≥ 1- 10ε/k else overall success < (1- 10ε/k)k < 1-ε

Strong Direct Sum Theorem: D0,ε(fk) = 훀(kD1/5,40ε/k(f))

μk μ

1-ε ≤ Pr[A(Y) = fk(Y)] = ∏ Pr[A(Y)i = fk(Y)i | A(Y)<i = fk(Y)<i]

Y~μk Y~μk i=1 k

Strong Direct Sum Theorem: D0,ε(fk) = 훀(kD1/5,40ε/k(f))

μk μ

1-ε ≤ Pr[A(Y) = fk(Y)] = ∏ Pr[A(Y)i = fk(Y)i | A(Y)<i = fk(Y)<i]

Y~μk Y~μk i=1 k

Want: i such that (1) conditional error very low: Pr[A err. on i-th coord. | correct on < i] ≤ 10 ε/k (2) Expected # queries on i-th coord not too high: E[queries on i-th coord.] ≤ 3q/k

Strong Direct Sum Theorem: D0,ε(fk) = 훀(kD1/5,40ε/k(f))

μk μ

1-ε ≤ Pr[A(Y) = fk(Y)] = ∏ Pr[A(Y)i = fk(Y)i | A(Y)<i = fk(Y)<i]

Y~μk Y~μk i=1 k

Want: i such that (1) conditional error very low: Pr[A err. on i-th coord. | correct on < i] ≤ 10 ε/k (2) Expected # queries on i-th coord not too high: E[queries on i-th coord.] ≤ 3q/k Fact: at least 2k/3 coords. satisfy (1) Fact: at least 2k/3 coords. satisfy (2) ⟹There is i* satisfying (1) and (2). Y* := Embed(Y,i*,x).

Strong Direct Sum Theorem: D0,ε(fk) = 훀(kD1/5,40ε/k(f))

μk μ

This i* satisfies:

• 1. EY~μk[ Prx~μ[A(Y*)<i* ≠ fk(Y*)<i*] ] ≤ ε
• 2. EY~μk[Prx~μ[A(Y*)i* ≠ fk(Y*)i* | A(Y*)<i* = fk(Y*)<i*] ≤ 10 ε/k
• 3. EY~μk[ EX [qi*(Y*)] ] ≤ 3q/k

Strong Direct Sum Theorem: D0,ε(fk) = 훀(kD1/5,40ε/k(f))

μk μ

This i* satisfies:

• 1. EY~μk[ Prx~μ[A(Y*)<i* ≠ fk(Y*)<i*] ] ≤ ε
• 2. EY~μk[Prx~μ[A(Y*)i* ≠ fk(Y*)i* | A(Y*)<i* = fk(Y*)<i*] ≤ 10 ε/k
• 3. EY~μk[ EX [qi*(Y*)] ] ≤ 3q/k

Markov Inequality: there is y* such that

• 1. Prx~μ[A(Y*)<i* ≠ fk(Y*)<i*] ≤ 4ε
• 2. Prx~μ[A(Y*)i* ≠ fk(Y*)i* | A(Y*)<i* = fk(Y*)<i*] ≤ 40 ε/k
• 3. EX [qi*(Y*)] ≤ 12q/k

Algorithm B(x) { z := EMBED(y*,i*,x) emulate A(z) abort if qi*(z) > 120q/k abort if A(z)<i* ≠ fk(z)<i* }

Strong Direct Sum Theorem: D0,ε(fk) = 훀(kD1/5,40ε/k(f))

μk μ

This i* satisfies:

• 1. EY~μk[ Prx~μ[A(Y*)<i* ≠ fk(Y*)<i*] ] ≤ ε
• 2. EY~μk[Prx~μ[A(Y*)i* ≠ fk(Y*)i* | A(Y*)<i* = fk(Y*)<i*] ≤ 10 ε/k
• 3. EY~μk[ EX [qi*(Y*)] ] ≤ 3q/k

Markov Inequality: there is y* such that

• 1. Prx~μ[A(Y*)<i* ≠ fk(Y*)<i*] ≤ 4ε
• 2. Prx~μ[A(Y*)i* ≠ fk(Y*)i* | A(Y*)<i* = fk(Y*)<i*] ≤ 40 ε/k
• 3. EX [qi*(Y*)] ≤ 12q/k

Algorithm B(x) { z := EMBED(y*,i*,x) emulate A(z) abort if qi*(z) > 120q/k abort if A(z)<i* ≠ fk(z)<i* }

Strong Direct Sum Theorem: D0,ε(fk) = 훀(kD1/5,40ε/k(f))

μk μ

This i* satisfies:

• 1. EY~μk[ Prx~μ[A(Y*)<i* ≠ fk(Y*)<i*] ] ≤ ε
• 2. EY~μk[Prx~μ[A(Y*)i* ≠ fk(Y*)i* | A(Y*)<i* = fk(Y*)<i*] ≤ 10 ε/k
• 3. EY~μk[ EX [qi*(Y*)] ] ≤ 3q/k

Markov Inequality: there is y* such that

• 1. Prx~μ[A(Y*)<i* ≠ fk(Y*)<i*] ≤ 4ε
• 2. Prx~μ[A(Y*)i* ≠ fk(Y*)i* | A(Y*)<i* = fk(Y*)<i*] ≤ 40 ε/k
• 3. EX [qi*(Y*)] ≤ 12q/k

abort probability: 1/10 + 4ε < 1/5 error probability: 40ε/k

Outline

• Introduction
• Strong Direct Sum
• Query Resistance
• Separation Theorem
• Open Problems

Pointer Function

PtrFcn: Σn*n →{0,1}. each cell z ϵ Σ has:

• value b ϵ {0,1}
• n row ptrs row1(z), …, rown(z)
• back ptr back(z)

PtrFcn(X) := 1 iff

• ∃ unique col j*: val(zi,j*) = 1 for all i.
• ∃ special cell zi*,j*. all ptrs NULL in col j* except for special cell
• special cell pts to 0-value linked cells in each other col
• exactly half of linked cells point back to special cell

1,[⊥,…,⊥] 1,[⊥,…,⊥] 1,[⊥,…,⊥] 1,[⊥,…,⊥] 1,[⊥,…,⊥] 1,[ ] 0,[⊥,…, ] 0,[⊥,…,⊥] 0,[⊥,…,⊥] 0,[⊥,…, ]

[GPW15, ABBLSS17,BB19]

Query Resistant Codes

Definition: a 훿N-query resistant code of Σ is a set of distribs {G(x)}

• For each x ϵ Σ, G(x) is a distribution on {0,1}N
• { support(G(x)) : x ϵ Σ} partition {0,1}N
• For all S ⊆ [N] with |S| ≤ 훿N, distributions G(x)|S = G(x’)|S
• “decoding function” h(y) := x iff y ϵ support(G(x))

Query Resistant Codes

Definition: a 훿N-query resistant code of Σ is a set of distribs {G(x)}

• For each x ϵ Σ, G(x) is a distribution on {0,1}N
• { support(G(x)) : x ϵ Σ} partition {0,1}N
• For all S ⊆ [N] with |S| ≤ 훿N, distributions G(x)|S = G(x’)|S
• “decoding function” h(y) := x iff y ϵ support(G(x))

Theorem: [Chor er al. 85] For any Σ, there is a (N/3)-query resistant code with N = 12.5log(|Σ|). Furthermore, conditional distributions G(x)|S are uniform.

Query Resistance

For f : Σn →{0,1}, define F : {0,1}nN →{0,1} as: F(y1,…,yn) := f(h(y1),…, h(yn))

Theorem: R훿,ε(f) ≤ (3/N)R훿,ε(F)

cell

Query Resistance

For f : Σn →{0,1}, define F : {0,1}nN →{0,1} as: F(y1,…,yn) := f(h(y1),…, h(yn))

Theorem: R훿,ε(f) ≤ (3/N)R훿,ε(F)

cell

Proof: Let A be a (q, 훿, ε)-algorithm for F.

Algorithm B(x1,…,xn) { emulate A(G(x1),…,G(xn)) when A queries G(xi) for k-th time: if k < N/3, sample G(xi) cond. on prev. queries if k = N/3, sample xi if k ≥ N/3, sample G(xi) cond. on prev. history. }

Open Problems

• 1. Characterize functions robust to aborts
• 2. Strong Direct Sum for Composed Functions

(a) XOR Lemma (b)Strong Direct Sum for MAJ

• 3. How does R훿,ε(f) compare to other QC measures?

Thanks!

NOTE: Swarthmore has a tenure-track opening for fall 2020!