Preserving Randomness for Adaptive Algorithms William M. Hoza 1 Adam - - PowerPoint PPT Presentation

Preserving Randomness for Adaptive Algorithms

William M. Hoza1 Adam R. Klivans August 20, 2018 RANDOM

1Supported by the NSF GRFP under Grant DGE-1610403 and by a

Harrington Fellowship from UT Austin

1 / 13

Randomized estimation algorithms

◮ Algorithm Est(C) estimates some value µ(C) ∈ Rd

2 / 13

Randomized estimation algorithms

◮ Algorithm Est(C) estimates some value µ(C) ∈ Rd Pr[Est(C) − µ(C)∞ > ε] ≤ δ

2 / 13

Randomized estimation algorithms

◮ Algorithm Est(C) estimates some value µ(C) ∈ Rd Pr[Est(C) − µ(C)∞ > ε] ≤ δ ◮ Canonical example:

2 / 13

Randomized estimation algorithms

◮ Algorithm Est(C) estimates some value µ(C) ∈ Rd Pr[Est(C) − µ(C)∞ > ε] ≤ δ ◮ Canonical example:

◮ C is a Boolean circuit

2 / 13

Randomized estimation algorithms

◮ Algorithm Est(C) estimates some value µ(C) ∈ Rd Pr[Est(C) − µ(C)∞ > ε] ≤ δ ◮ Canonical example:

◮ C is a Boolean circuit ◮ µ(C)

def

= Prx[C(x) = 1] (d = 1)

2 / 13

Randomized estimation algorithms

◮ Algorithm Est(C) estimates some value µ(C) ∈ Rd Pr[Est(C) − µ(C)∞ > ε] ≤ δ ◮ Canonical example:

◮ C is a Boolean circuit ◮ µ(C)

def

= Prx[C(x) = 1] (d = 1) ◮ Est(C) evaluates C at several randomly chosen points

2 / 13

Using Est as a subroutine

Owner Steward

3 / 13

Using Est as a subroutine

Owner Steward C1

3 / 13

Using Est as a subroutine

Owner Steward C1 Y1 ≈ µ(C1)

3 / 13

Using Est as a subroutine

Owner Steward C1 Y1 ≈ µ(C1) C2

3 / 13

Using Est as a subroutine

Owner Steward C1 Y1 ≈ µ(C1) C2 Y2 ≈ µ(C2)

3 / 13

Using Est as a subroutine

Owner Steward C1 Y1 ≈ µ(C1) C2 Y2 ≈ µ(C2) . . . Ck Yk ≈ µ(Ck)

3 / 13

Using Est as a subroutine

Owner Steward C1 Y1 ≈ µ(C1) C2 Y2 ≈ µ(C2) . . . Ck Yk ≈ µ(Ck) ◮ Suppose Est uses n random bits

3 / 13

Using Est as a subroutine

Owner Steward C1 Y1 ≈ µ(C1) C2 Y2 ≈ µ(C2) . . . Ck Yk ≈ µ(Ck) ◮ Suppose Est uses n random bits ◮ Na¨ ıvely, total number of random bits = nk

3 / 13

Using Est as a subroutine

Owner Steward C1 Y1 ≈ µ(C1) C2 Y2 ≈ µ(C2) . . . Ck Yk ≈ µ(Ck) ◮ Suppose Est uses n random bits ◮ Na¨ ıvely, total number of random bits = nk ◮ Can we do better?

3 / 13

Main result

◮ Theorem (informal): There is a steward that uses just n + O(k log(d + 1)) random bits!

4 / 13

Main result

◮ Theorem (informal): There is a steward that uses just n + O(k log(d + 1)) random bits! ◮ Mild increases in both error and failure probability

4 / 13

Main result

◮ Theorem (informal): There is a steward that uses just n + O(k log(d + 1)) random bits! ◮ Mild increases in both error and failure probability ◮ Prior work:

4 / 13

Main result

◮ Theorem (informal): There is a steward that uses just n + O(k log(d + 1)) random bits! ◮ Mild increases in both error and failure probability ◮ Prior work:

◮ [Saks, Zhou ’99], [Impagliazzo, Zuckerman ’89] both imply stewards

4 / 13

Main result

◮ Theorem (informal): There is a steward that uses just n + O(k log(d + 1)) random bits! ◮ Mild increases in both error and failure probability ◮ Prior work:

◮ [Saks, Zhou ’99], [Impagliazzo, Zuckerman ’89] both imply stewards ◮ Our steward has better parameters

4 / 13

Outline of our steward

• 1. Compute pseudorandom bits Xi ∈ {0, 1}n

5 / 13

Outline of our steward

• 1. Compute pseudorandom bits Xi ∈ {0, 1}n
• 2. Compute Wi := Est(Ci, Xi)

5 / 13

Outline of our steward

• 1. Compute pseudorandom bits Xi ∈ {0, 1}n
• 2. Compute Wi := Est(Ci, Xi)
• 3. Compute Yi by carefully modifying Wi

5 / 13

Pseudorandom bits

◮ Gen : {0, 1}s → {0, 1}nk: Variant of INW pseudorandom generator

6 / 13

Pseudorandom bits

◮ Gen : {0, 1}s → {0, 1}nk: Variant of INW pseudorandom generator ◮ Before first round, steward computes (X1, X2, . . . , Xk) = Gen(Us)

6 / 13

Pseudorandom bits

◮ Gen : {0, 1}s → {0, 1}nk: Variant of INW pseudorandom generator ◮ Before first round, steward computes (X1, X2, . . . , Xk) = Gen(Us) ◮ In round i, steward runs Est(Ci, Xi)

6 / 13

Shifting and rounding

7 / 13

(d + 1) · 2ε

Shifting and rounding

7 / 13

(d + 1) · 2ε Wi

Shifting and rounding

7 / 13

(d + 1) · 2ε Wi µ(Ci)

Shifting and rounding

7 / 13

(d + 1) · 2ε Wi µ(Ci)

Shifting and rounding

7 / 13

(d + 1) · 2ε Wi µ(Ci)

Shifting and rounding

7 / 13

(d + 1) · 2ε Wi µ(Ci)

Shifting and rounding

7 / 13

(d + 1) · 2ε Wi µ(Ci)

Shifting and rounding

7 / 13

(d + 1) · 2ε Wi µ(Ci)

Shifting and rounding

7 / 13

(d + 1) · 2ε Wi µ(Ci)

Shifting and rounding

7 / 13

(d + 1) · 2ε Yi µ(Ci)

Analysis

◮ Theorem (informal): With high probability, for every i, Yi − µ(Ci)∞ ≤ O(εd).

8 / 13

Analysis

◮ Theorem (informal): With high probability, for every i, Yi − µ(Ci)∞ ≤ O(εd). ◮ Notation: For W ∈ Rd, ∆ ∈ [d + 1], define ⌊W ⌉∆ ∈ Rd by rounding each coordinate to nearest value y such that y ≡ 2ε∆ mod (d + 1) · 2ε

8 / 13

Analysis

◮ Theorem (informal): With high probability, for every i, Yi − µ(Ci)∞ ≤ O(εd). ◮ Notation: For W ∈ Rd, ∆ ∈ [d + 1], define ⌊W ⌉∆ ∈ Rd by rounding each coordinate to nearest value y such that y ≡ 2ε∆ mod (d + 1) · 2ε ◮ In this notation, Yi = ⌊Wi⌉∆ for a suitable ∆ ∈ [d + 1]

8 / 13

Analysis (continued)

◮ Yi = ⌊Wi⌉∆

9 / 13

Analysis (continued)

◮ Yi = ⌊Wi⌉∆ ◮ If Xi = fresh randomness, then w.h.p., ⌊Wi⌉∆ = ⌊µ(Ci)⌉∆

9 / 13

Analysis (continued)

◮ Yi = ⌊Wi⌉∆ ◮ If Xi = fresh randomness, then w.h.p., ⌊Wi⌉∆ = ⌊µ(Ci)⌉∆ ◮ Consider d + 2 cases:

9 / 13

Analysis (continued)

◮ Yi = ⌊Wi⌉∆ ◮ If Xi = fresh randomness, then w.h.p., ⌊Wi⌉∆ = ⌊µ(Ci)⌉∆ ◮ Consider d + 2 cases:

◮ Yi = ⌊µ(Ci)⌉1, or

9 / 13

Analysis (continued)

◮ Yi = ⌊Wi⌉∆ ◮ If Xi = fresh randomness, then w.h.p., ⌊Wi⌉∆ = ⌊µ(Ci)⌉∆ ◮ Consider d + 2 cases:

◮ Yi = ⌊µ(Ci)⌉1, or ◮ Yi = ⌊µ(Ci)⌉2, or

9 / 13

Analysis (continued)

◮ Yi = ⌊Wi⌉∆ ◮ If Xi = fresh randomness, then w.h.p., ⌊Wi⌉∆ = ⌊µ(Ci)⌉∆ ◮ Consider d + 2 cases:

◮ Yi = ⌊µ(Ci)⌉1, or ◮ Yi = ⌊µ(Ci)⌉2, or . . .

9 / 13

Analysis (continued)

◮ Yi = ⌊Wi⌉∆ ◮ If Xi = fresh randomness, then w.h.p., ⌊Wi⌉∆ = ⌊µ(Ci)⌉∆ ◮ Consider d + 2 cases:

◮ Yi = ⌊µ(Ci)⌉1, or ◮ Yi = ⌊µ(Ci)⌉2, or . . . ◮ Yi = ⌊µ(Ci)⌉d+1, or

9 / 13

Analysis (continued)

◮ Yi = ⌊Wi⌉∆ ◮ If Xi = fresh randomness, then w.h.p., ⌊Wi⌉∆ = ⌊µ(Ci)⌉∆ ◮ Consider d + 2 cases:

◮ Yi = ⌊µ(Ci)⌉1, or ◮ Yi = ⌊µ(Ci)⌉2, or . . . ◮ Yi = ⌊µ(Ci)⌉d+1, or ◮ Yi = something else.

9 / 13

Block decision tree

C1

10 / 13

Block decision tree

C1 C (1)

2

C (2)

2

C (3)

2

⊥

10 / 13

Block decision tree

C1 C (1)

2

C (2)

2

C (3)

2

⊥ C (2,1)

3

C (2,2)

3

C (2,3)

3

⊥

10 / 13

Block decision tree

C1 C (1)

2

C (2)

2

C (3)

2

⊥ C (2,1)

3

C (2,2)

3

C (2,3)

3

⊥ ◮ A sequence (X1, . . . , Xk) determines:

10 / 13

Block decision tree

C1 C (1)

2

C (2)

2

C (3)

2

⊥ C (2,1)

3

C (2,2)

3

C (2,3)

3

⊥ ◮ A sequence (X1, . . . , Xk) determines:

◮ A transcript (C1, Y1, C2, Y2, . . . , Ck, Yk)

10 / 13

Block decision tree

C1 C (1)

2

C (2)

2

C (3)

2

⊥ C (2,1)

3

C (2,2)

3

C (2,3)

3

⊥ ◮ A sequence (X1, . . . , Xk) determines:

◮ A transcript (C1, Y1, C2, Y2, . . . , Ck, Yk) ◮ A path P through tree

10 / 13

Block decision tree

C1 C (1)

2

C (2)

2

C (3)

2

⊥ C (2,1)

3

C (2,2)

3

C (2,3)

3

⊥ ◮ A sequence (X1, . . . , Xk) determines:

◮ A transcript (C1, Y1, C2, Y2, . . . , Ck, Yk) ◮ A path P through tree

◮ If we pick X1, . . . , Xk independently and u.a.r., Pr

(X1,...,Xk)[P has a ⊥ node] ≤ kδ

10 / 13

Fooling the tree

C1 C (1)

2

C (2)

2

C (3)

2

⊥ C (2,1)

3

C (2,2)

3

C (2,3)

3

⊥ ◮ Tree has low memory

11 / 13

Fooling the tree

C1 C (1)

2

C (2)

2

C (3)

2

⊥ C (2,1)

3

C (2,2)

3

C (2,3)

3

⊥ ◮ Tree has low memory ◮ So when X1, . . . , Xk are pseudorandom, Pr

(X1,...,Xk)[P has a ⊥ node] ≤ kδ + γ

11 / 13

The tree certifies correctness

C1 C (1)

2

C (2)

2

C (3)

2

⊥ C (2,1)

3

C (2,2)

3

C (2,3)

3

⊥ ◮ (Certification) No ⊥ nodes in P = ⇒ every Yi has error O(εd)

12 / 13

Application: Randomness-efficient Goldreich-Levin

13 / 13

Application: Randomness-efficient Goldreich-Levin

◮ Oracle access to x ∈ {0, 1}2n

13 / 13

Application: Randomness-efficient Goldreich-Levin

◮ Oracle access to x ∈ {0, 1}2n ◮ Theorem: Can find all Hadamard codewords that agree with x in ( 1

2 + θ)-fraction of positions

13 / 13

Application: Randomness-efficient Goldreich-Levin

◮ Oracle access to x ∈ {0, 1}2n ◮ Theorem: Can find all Hadamard codewords that agree with x in ( 1

2 + θ)-fraction of positions

◮ Runtime poly(n, 1/θ, log(1/δ)) (δ = failure prob)

13 / 13

Application: Randomness-efficient Goldreich-Levin

◮ Oracle access to x ∈ {0, 1}2n ◮ Theorem: Can find all Hadamard codewords that agree with x in ( 1

2 + θ)-fraction of positions

◮ Runtime poly(n, 1/θ, log(1/δ)) (δ = failure prob) ◮ O(n + log n log(1/δ)) random bits (independent of θ!)

13 / 13

Application: Randomness-efficient Goldreich-Levin

◮ Oracle access to x ∈ {0, 1}2n ◮ Theorem: Can find all Hadamard codewords that agree with x in ( 1

2 + θ)-fraction of positions

◮ Runtime poly(n, 1/θ, log(1/δ)) (δ = failure prob) ◮ O(n + log n log(1/δ)) random bits (independent of θ!)

◮ Previous best: O(n log(n/θ) log(1/(δθ))) random bits (Bshouty et al. ’04)

13 / 13

Application: Randomness-efficient Goldreich-Levin

◮ Oracle access to x ∈ {0, 1}2n ◮ Theorem: Can find all Hadamard codewords that agree with x in ( 1

2 + θ)-fraction of positions

◮ Runtime poly(n, 1/θ, log(1/δ)) (δ = failure prob) ◮ O(n + log n log(1/δ)) random bits (independent of θ!)

◮ Previous best: O(n log(n/θ) log(1/(δθ))) random bits (Bshouty et al. ’04) ◮ Proof ingredients:

13 / 13

Application: Randomness-efficient Goldreich-Levin

◮ Oracle access to x ∈ {0, 1}2n ◮ Theorem: Can find all Hadamard codewords that agree with x in ( 1

2 + θ)-fraction of positions

◮ Runtime poly(n, 1/θ, log(1/δ)) (δ = failure prob) ◮ O(n + log n log(1/δ)) random bits (independent of θ!)

◮ Previous best: O(n log(n/θ) log(1/(δθ))) random bits (Bshouty et al. ’04) ◮ Proof ingredients:

◮ Standard Goldreich-Levin algorithm

13 / 13

Application: Randomness-efficient Goldreich-Levin

◮ Oracle access to x ∈ {0, 1}2n ◮ Theorem: Can find all Hadamard codewords that agree with x in ( 1

2 + θ)-fraction of positions

◮ Runtime poly(n, 1/θ, log(1/δ)) (δ = failure prob) ◮ O(n + log n log(1/δ)) random bits (independent of θ!)

◮ Previous best: O(n log(n/θ) log(1/(δθ))) random bits (Bshouty et al. ’04) ◮ Proof ingredients:

◮ Standard Goldreich-Levin algorithm ◮ Our steward with d = poly(1/θ)

13 / 13

Application: Randomness-efficient Goldreich-Levin

◮ Oracle access to x ∈ {0, 1}2n ◮ Theorem: Can find all Hadamard codewords that agree with x in ( 1

2 + θ)-fraction of positions

◮ Runtime poly(n, 1/θ, log(1/δ)) (δ = failure prob) ◮ O(n + log n log(1/δ)) random bits (independent of θ!)

◮ Previous best: O(n log(n/θ) log(1/(δθ))) random bits (Bshouty et al. ’04) ◮ Proof ingredients:

◮ Standard Goldreich-Levin algorithm ◮ Our steward with d = poly(1/θ) ◮ Goldreich-Wigderson sampler

13 / 13

Open questions

◮ Optimal randomness complexity when d is large?

14 / 13

Open questions

◮ Optimal randomness complexity when d is large? ◮ Avoid error blowup ε → O(εd)?

14 / 13

Open questions

◮ Optimal randomness complexity when d is large? ◮ Avoid error blowup ε → O(εd)? ◮ More applications?

14 / 13

Open questions

◮ Optimal randomness complexity when d is large? ◮ Avoid error blowup ε → O(εd)? ◮ More applications? ◮ Thanks! Questions?

14 / 13