Computational Barriers to Estimation from Low-Degree Polynomials - - PowerPoint PPT Presentation

Computational Barriers to Estimation from Low-Degree Polynomials Alex Wein

Courant Institute, New York University Joint work with: Tselil Schramm

Stanford

1 / 23

Part I: Why Low-Degree Polynomials?

2 / 23

Problems in High-Dimensional Statistics

Example: planted k-clique in a random graph G(n, 1/2)

3 / 23

Problems in High-Dimensional Statistics

Example: planted k-clique in a random graph G(n, 1/2) ◮ Detection/testing: distinguish between a random graph and a graph with a planted clique

3 / 23

Problems in High-Dimensional Statistics

Example: planted k-clique in a random graph G(n, 1/2) ◮ Detection/testing: distinguish between a random graph and a graph with a planted clique ◮ Recovery/estimation: given a graph with a planted clique, find the clique

3 / 23

Problems in High-Dimensional Statistics

Example: planted k-clique in a random graph G(n, 1/2) ◮ Detection/testing: distinguish between a random graph and a graph with a planted clique ◮ Recovery/estimation: given a graph with a planted clique, find the clique Both problems have an information-computation gap

3 / 23

Problems in High-Dimensional Statistics

Example: planted k-clique in a random graph G(n, 1/2) ◮ Detection/testing: distinguish between a random graph and a graph with a planted clique ◮ Recovery/estimation: given a graph with a planted clique, find the clique Both problems have an information-computation gap

3 / 23

Problems in High-Dimensional Statistics

Example: planted k-clique in a random graph G(n, 1/2) ◮ Detection/testing: distinguish between a random graph and a graph with a planted clique ◮ Recovery/estimation: given a graph with a planted clique, find the clique Both problems have an information-computation gap What makes problems easy vs hard?

3 / 23

The Low-Degree Polynomial Method

A framework for predicting/explaining average-case computational complexity

4 / 23

The Low-Degree Polynomial Method

A framework for predicting/explaining average-case computational complexity Originated from sum-of-squares literature (for detection)

[Barak, Hopkins, Kelner, Kothari, Moitra, Potechin ’16] [Hopkins, Steurer ’17] [Hopkins, Kothari, Potechin, Raghavendra, Schramm, Steurer ’17] [Hopkins ’18 (PhD thesis)]

4 / 23

The Low-Degree Polynomial Method

A framework for predicting/explaining average-case computational complexity Originated from sum-of-squares literature (for detection)

[Barak, Hopkins, Kelner, Kothari, Moitra, Potechin ’16] [Hopkins, Steurer ’17] [Hopkins, Kothari, Potechin, Raghavendra, Schramm, Steurer ’17] [Hopkins ’18 (PhD thesis)]

Today: self-contained motivation (without SoS)

4 / 23

The Low-Degree Polynomial Method

Study a restricted class of algorithms: low-degree polynomials

5 / 23

The Low-Degree Polynomial Method

Study a restricted class of algorithms: low-degree polynomials ◮ Multivariate polynomial f : RN → RM

5 / 23

The Low-Degree Polynomial Method

Study a restricted class of algorithms: low-degree polynomials ◮ Multivariate polynomial f : RN → RM

◮ Input: e.g. graph Y ∈ {0, 1}(

n 2)

5 / 23

The Low-Degree Polynomial Method

Study a restricted class of algorithms: low-degree polynomials ◮ Multivariate polynomial f : RN → RM

◮ Input: e.g. graph Y ∈ {0, 1}(

n 2)

◮ Output: b ∈ {0, 1} (detection)

5 / 23

The Low-Degree Polynomial Method

Study a restricted class of algorithms: low-degree polynomials ◮ Multivariate polynomial f : RN → RM

◮ Input: e.g. graph Y ∈ {0, 1}(

n 2)

◮ Output: b ∈ {0, 1} (detection) or v ∈ Rn (recovery)

5 / 23

The Low-Degree Polynomial Method

Study a restricted class of algorithms: low-degree polynomials ◮ Multivariate polynomial f : RN → RM

◮ Input: e.g. graph Y ∈ {0, 1}(

n 2)

◮ Output: b ∈ {0, 1} (detection) or v ∈ Rn (recovery)

◮ “Low” means O(log n) where n is dimension

5 / 23

The Low-Degree Polynomial Method

Study a restricted class of algorithms: low-degree polynomials ◮ Multivariate polynomial f : RN → RM

◮ Input: e.g. graph Y ∈ {0, 1}(

n 2)

◮ Output: b ∈ {0, 1} (detection) or v ∈ Rn (recovery)

◮ “Low” means O(log n) where n is dimension Examples of low-degree algorithms:

5 / 23

The Low-Degree Polynomial Method

Study a restricted class of algorithms: low-degree polynomials ◮ Multivariate polynomial f : RN → RM

◮ Input: e.g. graph Y ∈ {0, 1}(

n 2)

◮ Output: b ∈ {0, 1} (detection) or v ∈ Rn (recovery)

◮ “Low” means O(log n) where n is dimension Examples of low-degree algorithms: input Y ∈ Rn×n

5 / 23

The Low-Degree Polynomial Method

Study a restricted class of algorithms: low-degree polynomials ◮ Multivariate polynomial f : RN → RM

◮ Input: e.g. graph Y ∈ {0, 1}(

n 2)

◮ Output: b ∈ {0, 1} (detection) or v ∈ Rn (recovery)

◮ “Low” means O(log n) where n is dimension Examples of low-degree algorithms: input Y ∈ Rn×n ◮ Power iteration: Y k1 or Tr(Y k) k = O(log n)

5 / 23

The Low-Degree Polynomial Method

Study a restricted class of algorithms: low-degree polynomials ◮ Multivariate polynomial f : RN → RM

◮ Input: e.g. graph Y ∈ {0, 1}(

n 2)

◮ Output: b ∈ {0, 1} (detection) or v ∈ Rn (recovery)

◮ “Low” means O(log n) where n is dimension Examples of low-degree algorithms: input Y ∈ Rn×n ◮ Power iteration: Y k1 or Tr(Y k) k = O(log n) ◮ Approximate message passing: v ← Y h(v) O(1) rounds

5 / 23

The Low-Degree Polynomial Method

Study a restricted class of algorithms: low-degree polynomials ◮ Multivariate polynomial f : RN → RM

◮ Input: e.g. graph Y ∈ {0, 1}(

n 2)

◮ Output: b ∈ {0, 1} (detection) or v ∈ Rn (recovery)

◮ “Low” means O(log n) where n is dimension Examples of low-degree algorithms: input Y ∈ Rn×n ◮ Power iteration: Y k1 or Tr(Y k) k = O(log n) ◮ Approximate message passing: v ← Y h(v) O(1) rounds ◮ Local algorithms on sparse graphs radius O(1)

5 / 23

The Low-Degree Polynomial Method

Study a restricted class of algorithms: low-degree polynomials ◮ Multivariate polynomial f : RN → RM

◮ Input: e.g. graph Y ∈ {0, 1}(

n 2)

◮ Output: b ∈ {0, 1} (detection) or v ∈ Rn (recovery)

◮ “Low” means O(log n) where n is dimension Examples of low-degree algorithms: input Y ∈ Rn×n ◮ Power iteration: Y k1 or Tr(Y k) k = O(log n) ◮ Approximate message passing: v ← Y h(v) O(1) rounds ◮ Local algorithms on sparse graphs radius O(1) ◮ Or any of the above applied to ˜ Y = g(Y ) deg g = O(1)

5 / 23

Optimality of Low-Degree Polynomials?

Low-degree polynomials seem to be optimal for many problems!

6 / 23

Optimality of Low-Degree Polynomials?

Low-degree polynomials seem to be optimal for many problems! For all of these problems...

6 / 23

Optimality of Low-Degree Polynomials?

Low-degree polynomials seem to be optimal for many problems! For all of these problems...

planted clique, sparse PCA, community detection, tensor PCA, spiked Wigner/Wishart, planted submatrix, planted dense subgraph, ...

6 / 23

Optimality of Low-Degree Polynomials?

Low-degree polynomials seem to be optimal for many problems! For all of these problems...

planted clique, sparse PCA, community detection, tensor PCA, spiked Wigner/Wishart, planted submatrix, planted dense subgraph, ...

...it is the case that

6 / 23

Optimality of Low-Degree Polynomials?

Low-degree polynomials seem to be optimal for many problems! For all of these problems...

planted clique, sparse PCA, community detection, tensor PCA, spiked Wigner/Wishart, planted submatrix, planted dense subgraph, ...

...it is the case that ◮ the best known poly-time algorithms are captured by O(log n)-degree polynomials (spectral/AMP)

6 / 23

Optimality of Low-Degree Polynomials?

Low-degree polynomials seem to be optimal for many problems! For all of these problems...

planted clique, sparse PCA, community detection, tensor PCA, spiked Wigner/Wishart, planted submatrix, planted dense subgraph, ...

...it is the case that ◮ the best known poly-time algorithms are captured by O(log n)-degree polynomials (spectral/AMP) ◮ low-degree polynomials fail in the “hard” regime

6 / 23

Optimality of Low-Degree Polynomials?

Low-degree polynomials seem to be optimal for many problems! For all of these problems...

planted clique, sparse PCA, community detection, tensor PCA, spiked Wigner/Wishart, planted submatrix, planted dense subgraph, ...

...it is the case that ◮ the best known poly-time algorithms are captured by O(log n)-degree polynomials (spectral/AMP) ◮ low-degree polynomials fail in the “hard” regime “Low-degree conjecture” (informal): for “natural” problems, if low-degree polynomials fail then all poly-time algorithms fail

[Hopkins ’18]

6 / 23

Optimality of Low-Degree Polynomials?

Low-degree polynomials seem to be optimal for many problems! For all of these problems...

planted clique, sparse PCA, community detection, tensor PCA, spiked Wigner/Wishart, planted submatrix, planted dense subgraph, ...

...it is the case that ◮ the best known poly-time algorithms are captured by O(log n)-degree polynomials (spectral/AMP) ◮ low-degree polynomials fail in the “hard” regime “Low-degree conjecture” (informal): for “natural” problems, if low-degree polynomials fail then all poly-time algorithms fail

[Hopkins ’18]

Caveat: Gaussian elimination for planted XOR-SAT

6 / 23

Overview

This talk: techniques to prove that all low-degree polynomials fail

7 / 23

Overview

This talk: techniques to prove that all low-degree polynomials fail ◮ Gives evidence for computational hardness

7 / 23

Overview

This talk: techniques to prove that all low-degree polynomials fail ◮ Gives evidence for computational hardness Settings: ◮ Detection (prior work)

[Hopkins, Steurer ’17] [Hopkins, Kothari, Potechin, Raghavendra, Schramm, Steurer ’17] [Hopkins ’18] (PhD thesis) [Kunisky, W., Bandeira ’19] (survey)

7 / 23

Overview

This talk: techniques to prove that all low-degree polynomials fail ◮ Gives evidence for computational hardness Settings: ◮ Detection (prior work)

[Hopkins, Steurer ’17] [Hopkins, Kothari, Potechin, Raghavendra, Schramm, Steurer ’17] [Hopkins ’18] (PhD thesis) [Kunisky, W., Bandeira ’19] (survey)

◮ Recovery (this work)

[Schramm, W. ’20]

7 / 23

Overview

This talk: techniques to prove that all low-degree polynomials fail ◮ Gives evidence for computational hardness Settings: ◮ Detection (prior work)

[Hopkins, Steurer ’17] [Hopkins, Kothari, Potechin, Raghavendra, Schramm, Steurer ’17] [Hopkins ’18] (PhD thesis) [Kunisky, W., Bandeira ’19] (survey)

◮ Recovery (this work)

[Schramm, W. ’20]

◮ Optimization

[Gamarnik, Jagannath, W. ’20]

7 / 23

Relation to Other Frameworks

8 / 23

Relation to Other Frameworks

◮ Sum-of-squares lower bounds [BHKKMP16,...]

8 / 23

Relation to Other Frameworks

◮ Sum-of-squares lower bounds [BHKKMP16,...]

◮ Actually for certification

8 / 23

Relation to Other Frameworks

◮ Sum-of-squares lower bounds [BHKKMP16,...]

◮ Actually for certification ◮ Connected to low-degree [HKPRSS17]

8 / 23

Relation to Other Frameworks

◮ Sum-of-squares lower bounds [BHKKMP16,...]

◮ Actually for certification ◮ Connected to low-degree [HKPRSS17]

◮ Statistical query lower bounds [FGRVX12,...]

8 / 23

Relation to Other Frameworks

◮ Sum-of-squares lower bounds [BHKKMP16,...]

◮ Actually for certification ◮ Connected to low-degree [HKPRSS17]

◮ Statistical query lower bounds [FGRVX12,...]

◮ Need i.i.d. samples

8 / 23

Relation to Other Frameworks

◮ Sum-of-squares lower bounds [BHKKMP16,...]

◮ Actually for certification ◮ Connected to low-degree [HKPRSS17]

◮ Statistical query lower bounds [FGRVX12,...]

◮ Need i.i.d. samples ◮ Equivalent to low-degree [BBHLS20]

8 / 23

Relation to Other Frameworks

◮ Sum-of-squares lower bounds [BHKKMP16,...]

◮ Actually for certification ◮ Connected to low-degree [HKPRSS17]

◮ Statistical query lower bounds [FGRVX12,...]

◮ Need i.i.d. samples ◮ Equivalent to low-degree [BBHLS20]

◮ Approximate message passing (AMP) [DMM09, LKZ15,...]

8 / 23

Relation to Other Frameworks

◮ Sum-of-squares lower bounds [BHKKMP16,...]

◮ Actually for certification ◮ Connected to low-degree [HKPRSS17]

◮ Statistical query lower bounds [FGRVX12,...]

◮ Need i.i.d. samples ◮ Equivalent to low-degree [BBHLS20]

◮ Approximate message passing (AMP) [DMM09, LKZ15,...]

◮ AMP algorithms are low-degree

8 / 23

Relation to Other Frameworks

◮ Sum-of-squares lower bounds [BHKKMP16,...]

◮ Actually for certification ◮ Connected to low-degree [HKPRSS17]

◮ Statistical query lower bounds [FGRVX12,...]

◮ Need i.i.d. samples ◮ Equivalent to low-degree [BBHLS20]

◮ Approximate message passing (AMP) [DMM09, LKZ15,...]

◮ AMP algorithms are low-degree ◮ AMP can be sub-optimal (e.g. tensor PCA) [MR14]

8 / 23

Relation to Other Frameworks

◮ Sum-of-squares lower bounds [BHKKMP16,...]

◮ Actually for certification ◮ Connected to low-degree [HKPRSS17]

◮ Statistical query lower bounds [FGRVX12,...]

◮ Need i.i.d. samples ◮ Equivalent to low-degree [BBHLS20]

◮ Approximate message passing (AMP) [DMM09, LKZ15,...]

◮ AMP algorithms are low-degree ◮ AMP can be sub-optimal (e.g. tensor PCA) [MR14]

◮ Overlap gap property / MCMC lower bounds [GS13, GZ17,...]

8 / 23

Relation to Other Frameworks

◮ Sum-of-squares lower bounds [BHKKMP16,...]

◮ Actually for certification ◮ Connected to low-degree [HKPRSS17]

◮ Statistical query lower bounds [FGRVX12,...]

◮ Need i.i.d. samples ◮ Equivalent to low-degree [BBHLS20]

◮ Approximate message passing (AMP) [DMM09, LKZ15,...]

◮ AMP algorithms are low-degree ◮ AMP can be sub-optimal (e.g. tensor PCA) [MR14]

◮ Overlap gap property / MCMC lower bounds [GS13, GZ17,...]

◮ MCMC algorithms are not low-degree (?)

8 / 23

Relation to Other Frameworks

◮ Sum-of-squares lower bounds [BHKKMP16,...]

◮ Actually for certification ◮ Connected to low-degree [HKPRSS17]

◮ Statistical query lower bounds [FGRVX12,...]

◮ Need i.i.d. samples ◮ Equivalent to low-degree [BBHLS20]

◮ Approximate message passing (AMP) [DMM09, LKZ15,...]

◮ AMP algorithms are low-degree ◮ AMP can be sub-optimal (e.g. tensor PCA) [MR14]

◮ Overlap gap property / MCMC lower bounds [GS13, GZ17,...]

◮ MCMC algorithms are not low-degree (?) ◮ MCMC can be sub-optimal (e.g. tensor PCA) [BGJ18]

8 / 23

Relation to Other Frameworks

◮ Sum-of-squares lower bounds [BHKKMP16,...]

◮ Actually for certification ◮ Connected to low-degree [HKPRSS17]

◮ Statistical query lower bounds [FGRVX12,...]

◮ Need i.i.d. samples ◮ Equivalent to low-degree [BBHLS20]

◮ Approximate message passing (AMP) [DMM09, LKZ15,...]

◮ AMP algorithms are low-degree ◮ AMP can be sub-optimal (e.g. tensor PCA) [MR14]

◮ Overlap gap property / MCMC lower bounds [GS13, GZ17,...]

◮ MCMC algorithms are not low-degree (?) ◮ MCMC can be sub-optimal (e.g. tensor PCA) [BGJ18]

◮ Average-case reductions [BR13,...]

8 / 23

Relation to Other Frameworks

◮ Sum-of-squares lower bounds [BHKKMP16,...]

◮ Actually for certification ◮ Connected to low-degree [HKPRSS17]

◮ Statistical query lower bounds [FGRVX12,...]

◮ Need i.i.d. samples ◮ Equivalent to low-degree [BBHLS20]

◮ Approximate message passing (AMP) [DMM09, LKZ15,...]

◮ AMP algorithms are low-degree ◮ AMP can be sub-optimal (e.g. tensor PCA) [MR14]

◮ Overlap gap property / MCMC lower bounds [GS13, GZ17,...]

◮ MCMC algorithms are not low-degree (?) ◮ MCMC can be sub-optimal (e.g. tensor PCA) [BGJ18]

◮ Average-case reductions [BR13,...]

◮ Need to argue that starting problem is hard [BB20]

8 / 23

Part II: Detection

9 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Goal: hypothesis test with error probability o(1) between: ◮ Null model Y ∼ Qn

e.g. G(n, 1/2)

◮ Planted model Y ∼ Pn

e.g. G(n, 1/2) ∪ {random k-clique}

10 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Goal: hypothesis test with error probability o(1) between: ◮ Null model Y ∼ Qn

e.g. G(n, 1/2)

◮ Planted model Y ∼ Pn

e.g. G(n, 1/2) ∪ {random k-clique}

Look for a degree-D polynomial f : Rn×n → R that distinguishes P from Q

10 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Goal: hypothesis test with error probability o(1) between: ◮ Null model Y ∼ Qn

e.g. G(n, 1/2)

◮ Planted model Y ∼ Pn

e.g. G(n, 1/2) ∪ {random k-clique}

Look for a degree-D polynomial f : Rn×n → R that distinguishes P from Q ◮ f (Y ) is “big” when Y ∼ P and “small” when Y ∼ Q

10 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Goal: hypothesis test with error probability o(1) between: ◮ Null model Y ∼ Qn

e.g. G(n, 1/2)

◮ Planted model Y ∼ Pn

e.g. G(n, 1/2) ∪ {random k-clique}

Look for a degree-D polynomial f : Rn×n → R that distinguishes P from Q ◮ f (Y ) is “big” when Y ∼ P and “small” when Y ∼ Q Compute “advantage”: Adv≤D := max

f deg D

EY ∼P[f (Y )]

• EY ∼Q[f (Y )2]

mean in P fluctuations in Q

10 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Goal: hypothesis test with error probability o(1) between: ◮ Null model Y ∼ Qn

e.g. G(n, 1/2)

◮ Planted model Y ∼ Pn

e.g. G(n, 1/2) ∪ {random k-clique}

Look for a degree-D polynomial f : Rn×n → R that distinguishes P from Q ◮ f (Y ) is “big” when Y ∼ P and “small” when Y ∼ Q Compute “advantage”: Adv≤D := max

f deg D

EY ∼P[f (Y )]

• EY ∼Q[f (Y )2]

mean in P fluctuations in Q = ω(1) “degree-D polynomial succeed” O(1) “degree-D polynomials fail”

10 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Prototypical result (planted clique):

11 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Prototypical result (planted clique): Theorem [BHKKMP16,Hop18]: For a planted k-clique in G(n, 1/2),

11 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Prototypical result (planted clique): Theorem [BHKKMP16,Hop18]: For a planted k-clique in G(n, 1/2), ◮ if k = Ω(√n) then Adv≤D = ω(1) for some D = O(log n)

low-degree polynomials succeed when k √n

11 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Prototypical result (planted clique): Theorem [BHKKMP16,Hop18]: For a planted k-clique in G(n, 1/2), ◮ if k = Ω(√n) then Adv≤D = ω(1) for some D = O(log n)

low-degree polynomials succeed when k √n

◮ if k = O(n1/2−ǫ) then Adv≤D = O(1) for any D = O(log n)

low-degree polynomials fail when k ≪ √n

11 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Prototypical result (planted clique): Theorem [BHKKMP16,Hop18]: For a planted k-clique in G(n, 1/2), ◮ if k = Ω(√n) then Adv≤D = ω(1) for some D = O(log n)

low-degree polynomials succeed when k √n

◮ if k = O(n1/2−ǫ) then Adv≤D = O(1) for any D = O(log n)

low-degree polynomials fail when k ≪ √n

Sometimes can rule out polynomials of degree D = nδ

11 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Prototypical result (planted clique): Theorem [BHKKMP16,Hop18]: For a planted k-clique in G(n, 1/2), ◮ if k = Ω(√n) then Adv≤D = ω(1) for some D = O(log n)

low-degree polynomials succeed when k √n

◮ if k = O(n1/2−ǫ) then Adv≤D = O(1) for any D = O(log n)

low-degree polynomials fail when k ≪ √n

Sometimes can rule out polynomials of degree D = nδ Extended low-degree conjecture [Hopkins ’18]: degree-D polynomials ⇔ n ˜

Θ(D)-time algorithms

D = nδ ⇔ exp(nδ±o(1)) time

11 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Goal: compute Adv≤D := max

f deg D

EY ∼P[f (Y )]

• EY ∼Q[f (Y )2]

✶

12 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Goal: compute Adv≤D := max

f deg D

EY ∼P[f (Y )]

• EY ∼Q[f (Y )2]

Suppose Q is i.i.d. Unif(±1) ✶

12 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Goal: compute Adv≤D := max

f deg D

EY ∼P[f (Y )]

• EY ∼Q[f (Y )2]

Suppose Q is i.i.d. Unif(±1) Write f (Y ) =

|S|≤D ˆ

fSY S Y S :=

i∈S Yi

S ⊆ [m] ✶

12 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Goal: compute Adv≤D := max

f deg D

EY ∼P[f (Y )]

• EY ∼Q[f (Y )2]

Suppose Q is i.i.d. Unif(±1) Write f (Y ) =

|S|≤D ˆ

fSY S Y S :=

i∈S Yi

S ⊆ [m] {Y S}S⊆[m] are orthonormal: EY ∼Q[Y SY T] = ✶S=T

12 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Goal: compute Adv≤D := max

f deg D

EY ∼P[f (Y )]

• EY ∼Q[f (Y )2]

Suppose Q is i.i.d. Unif(±1) Write f (Y ) =

|S|≤D ˆ

fSY S Y S :=

i∈S Yi

S ⊆ [m] {Y S}S⊆[m] are orthonormal: EY ∼Q[Y SY T] = ✶S=T Numerator: E

Y ∼P[f (Y )]

12 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Goal: compute Adv≤D := max

f deg D

EY ∼P[f (Y )]

• EY ∼Q[f (Y )2]

Suppose Q is i.i.d. Unif(±1) Write f (Y ) =

|S|≤D ˆ

fSY S Y S :=

i∈S Yi

S ⊆ [m] {Y S}S⊆[m] are orthonormal: EY ∼Q[Y SY T] = ✶S=T Numerator: E

Y ∼P[f (Y )] =

• |S|≤D

ˆ fS E

Y ∼P[Y S]

12 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Goal: compute Adv≤D := max

f deg D

EY ∼P[f (Y )]

• EY ∼Q[f (Y )2]

Suppose Q is i.i.d. Unif(±1) Write f (Y ) =

|S|≤D ˆ

fSY S Y S :=

i∈S Yi

S ⊆ [m] {Y S}S⊆[m] are orthonormal: EY ∼Q[Y SY T] = ✶S=T Numerator: E

Y ∼P[f (Y )] =

• |S|≤D

ˆ fS E

Y ∼P[Y S] =: ˆ

f , c

12 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Goal: compute Adv≤D := max

f deg D

EY ∼P[f (Y )]

• EY ∼Q[f (Y )2]

Suppose Q is i.i.d. Unif(±1) Write f (Y ) =

|S|≤D ˆ

fSY S Y S :=

i∈S Yi

S ⊆ [m] {Y S}S⊆[m] are orthonormal: EY ∼Q[Y SY T] = ✶S=T Numerator: E

Y ∼P[f (Y )] =

• |S|≤D

ˆ fS E

Y ∼P[Y S] =: ˆ

f , c Denominator: E

Y ∼Q[f (Y )2]

12 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Goal: compute Adv≤D := max

f deg D

EY ∼P[f (Y )]

• EY ∼Q[f (Y )2]

Suppose Q is i.i.d. Unif(±1) Write f (Y ) =

|S|≤D ˆ

fSY S Y S :=

i∈S Yi

S ⊆ [m] {Y S}S⊆[m] are orthonormal: EY ∼Q[Y SY T] = ✶S=T Numerator: E

Y ∼P[f (Y )] =

• |S|≤D

ˆ fS E

Y ∼P[Y S] =: ˆ

f , c Denominator: E

Y ∼Q[f (Y )2] =

• |S|≤D

ˆ f 2

S

(orthonormality)

12 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Goal: compute Adv≤D := max

f deg D

EY ∼P[f (Y )]

• EY ∼Q[f (Y )2]

Suppose Q is i.i.d. Unif(±1) Write f (Y ) =

|S|≤D ˆ

fSY S Y S :=

i∈S Yi

S ⊆ [m] {Y S}S⊆[m] are orthonormal: EY ∼Q[Y SY T] = ✶S=T Numerator: E

Y ∼P[f (Y )] =

• |S|≤D

ˆ fS E

Y ∼P[Y S] =: ˆ

f , c Denominator: E

Y ∼Q[f (Y )2] =

• |S|≤D

ˆ f 2

S = ˆ

f 2

(orthonormality)

12 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Goal: compute Adv≤D := max

f deg D

EY ∼P[f (Y )]

• EY ∼Q[f (Y )2]

Suppose Q is i.i.d. Unif(±1) Write f (Y ) =

|S|≤D ˆ

fSY S Y S :=

i∈S Yi

S ⊆ [m] {Y S}S⊆[m] are orthonormal: EY ∼Q[Y SY T] = ✶S=T Numerator: E

Y ∼P[f (Y )] =

• |S|≤D

ˆ fS E

Y ∼P[Y S] =: ˆ

f , c Denominator: E

Y ∼Q[f (Y )2] =

• |S|≤D

ˆ f 2

S = ˆ

f 2

(orthonormality)

Adv≤D = max

ˆ f

ˆ f , c ˆ f

12 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Goal: compute Adv≤D := max

f deg D

EY ∼P[f (Y )]

• EY ∼Q[f (Y )2]

Suppose Q is i.i.d. Unif(±1) Write f (Y ) =

|S|≤D ˆ

fSY S Y S :=

i∈S Yi

S ⊆ [m] {Y S}S⊆[m] are orthonormal: EY ∼Q[Y SY T] = ✶S=T Numerator: E

Y ∼P[f (Y )] =

• |S|≤D

ˆ fS E

Y ∼P[Y S] =: ˆ

f , c Denominator: E

Y ∼Q[f (Y )2] =

• |S|≤D

ˆ f 2

S = ˆ

f 2

(orthonormality)

Adv≤D = max

ˆ f

ˆ f , c ˆ f Optimizer: ˆ f ∗ = c

12 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Goal: compute Adv≤D := max

f deg D

EY ∼P[f (Y )]

• EY ∼Q[f (Y )2]

Suppose Q is i.i.d. Unif(±1) Write f (Y ) =

|S|≤D ˆ

fSY S Y S :=

i∈S Yi

S ⊆ [m] {Y S}S⊆[m] are orthonormal: EY ∼Q[Y SY T] = ✶S=T Numerator: E

Y ∼P[f (Y )] =

• |S|≤D

ˆ fS E

Y ∼P[Y S] =: ˆ

f , c Denominator: E

Y ∼Q[f (Y )2] =

• |S|≤D

ˆ f 2

S = ˆ

f 2

(orthonormality)

Adv≤D = max

ˆ f

ˆ f , c ˆ f = c, c c Optimizer: ˆ f ∗ = c

12 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Goal: compute Adv≤D := max

f deg D

EY ∼P[f (Y )]

• EY ∼Q[f (Y )2]

Suppose Q is i.i.d. Unif(±1) Write f (Y ) =

|S|≤D ˆ

fSY S Y S :=

i∈S Yi

S ⊆ [m] {Y S}S⊆[m] are orthonormal: EY ∼Q[Y SY T] = ✶S=T Numerator: E

Y ∼P[f (Y )] =

• |S|≤D

ˆ fS E

Y ∼P[Y S] =: ˆ

f , c Denominator: E

Y ∼Q[f (Y )2] =

• |S|≤D

ˆ f 2

S = ˆ

f 2

(orthonormality)

Adv≤D = max

ˆ f

ˆ f , c ˆ f = c, c c = c Optimizer: ˆ f ∗ = c

12 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Goal: compute Adv≤D := max

f deg D

EY ∼P[f (Y )]

• EY ∼Q[f (Y )2]

Suppose Q is i.i.d. Unif(±1) Write f (Y ) =

|S|≤D ˆ

fSY S Y S :=

i∈S Yi

S ⊆ [m] {Y S}S⊆[m] are orthonormal: EY ∼Q[Y SY T] = ✶S=T Numerator: E

Y ∼P[f (Y )] =

• |S|≤D

ˆ fS E

Y ∼P[Y S] =: ˆ

f , c Denominator: E

Y ∼Q[f (Y )2] =

• |S|≤D

ˆ f 2

S = ˆ

f 2

(orthonormality)

Adv≤D = max

ˆ f

ˆ f , c ˆ f = c, c c = c =

• |S|≤D
• E

Y ∼P[Y S]

2 Optimizer: ˆ f ∗ = c

12 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Remarks: ✶

13 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Remarks: ◮ Best test is likelihood ratio (Neyman-Pearson lemma) L(Y ) = dP dQ(Y ) ✶

13 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Remarks: ◮ Best test is likelihood ratio (Neyman-Pearson lemma) L(Y ) = dP dQ(Y ) ◮ Best degree-D test (maximizer of Adv≤D) is f ∗ = L≤D := projection of L onto deg-D subspace ✶

13 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Remarks: ◮ Best test is likelihood ratio (Neyman-Pearson lemma) L(Y ) = dP dQ(Y ) ◮ Best degree-D test (maximizer of Adv≤D) is f ∗ = L≤D := projection of L onto deg-D subspace

• rthogonal projection w.r.t. f , g :=

E

Y ∼Q[f (Y )g(Y )]

✶

13 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Remarks: ◮ Best test is likelihood ratio (Neyman-Pearson lemma) L(Y ) = dP dQ(Y ) ◮ Best degree-D test (maximizer of Adv≤D) is f ∗ = L≤D := projection of L onto deg-D subspace

• rthogonal projection w.r.t. f , g :=

E

Y ∼Q[f (Y )g(Y )]

“low-degree likelihood ratio” ✶

13 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Remarks: ◮ Best test is likelihood ratio (Neyman-Pearson lemma) L(Y ) = dP dQ(Y ) ◮ Best degree-D test (maximizer of Adv≤D) is f ∗ = L≤D := projection of L onto deg-D subspace

• rthogonal projection w.r.t. f , g :=

E

Y ∼Q[f (Y )g(Y )]

“low-degree likelihood ratio” ◮ Adv≤D = L≤D

f :=

• f , f =

E

Y ∼Q[f (Y )2]

✶

13 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Remarks: ◮ Best test is likelihood ratio (Neyman-Pearson lemma) L(Y ) = dP dQ(Y ) ◮ Best degree-D test (maximizer of Adv≤D) is f ∗ = L≤D := projection of L onto deg-D subspace

• rthogonal projection w.r.t. f , g :=

E

Y ∼Q[f (Y )g(Y )]

“low-degree likelihood ratio” ◮ Adv≤D = L≤D

f :=

• f , f =

E

Y ∼Q[f (Y )2]

“norm of low-degree likelihood ratio” ✶

13 / 23

Detection (e.g. [Hopkins, Steurer ’17])

Remarks: ◮ Best test is likelihood ratio (Neyman-Pearson lemma) L(Y ) = dP dQ(Y ) ◮ Best degree-D test (maximizer of Adv≤D) is f ∗ = L≤D := projection of L onto deg-D subspace

• rthogonal projection w.r.t. f , g :=

E

Y ∼Q[f (Y )g(Y )]

“low-degree likelihood ratio” ◮ Adv≤D = L≤D

f :=

• f , f =

E

Y ∼Q[f (Y )2]

“norm of low-degree likelihood ratio” Proof: ˆ LS = E

Y ∼Q[L(Y )Y S] = E Y ∼P[Y S]

ˆ f ∗

S = E Y ∼P[Y S] ✶|S|≤D

13 / 23

Part III: Recovery

14 / 23

Planted Submatrix

Example (planted submatrix): observe n × n matrix Y = X + Z ◮ Signal: X = λvv⊤ λ > 0 vi ∼ Bernoulli(ρ) ◮ Noise: Z i.i.d. N(0, 1)

15 / 23

Planted Submatrix

Example (planted submatrix): observe n × n matrix Y = X + Z ◮ Signal: X = λvv⊤ λ > 0 vi ∼ Bernoulli(ρ) ◮ Noise: Z i.i.d. N(0, 1) Regime: 1/√n ≪ ρ ≪ 1

15 / 23

Planted Submatrix

Example (planted submatrix): observe n × n matrix Y = X + Z ◮ Signal: X = λvv⊤ λ > 0 vi ∼ Bernoulli(ρ) ◮ Noise: Z i.i.d. N(0, 1) Regime: 1/√n ≪ ρ ≪ 1 Detection: distinguish P : Y = X + Z vs Q : Y = Z w.h.p.

15 / 23

Planted Submatrix

Example (planted submatrix): observe n × n matrix Y = X + Z ◮ Signal: X = λvv⊤ λ > 0 vi ∼ Bernoulli(ρ) ◮ Noise: Z i.i.d. N(0, 1) Regime: 1/√n ≪ ρ ≪ 1 Detection: distinguish P : Y = X + Z vs Q : Y = Z w.h.p. ◮ Sum of all entries succeeds when λ ≫ (ρ√n)−2

15 / 23

Planted Submatrix

Example (planted submatrix): observe n × n matrix Y = X + Z ◮ Signal: X = λvv⊤ λ > 0 vi ∼ Bernoulli(ρ) ◮ Noise: Z i.i.d. N(0, 1) Regime: 1/√n ≪ ρ ≪ 1 Detection: distinguish P : Y = X + Z vs Q : Y = Z w.h.p. ◮ Sum of all entries succeeds when λ ≫ (ρ√n)−2 Recovery: given Y ∼ P, recover v

15 / 23

Planted Submatrix

Example (planted submatrix): observe n × n matrix Y = X + Z ◮ Signal: X = λvv⊤ λ > 0 vi ∼ Bernoulli(ρ) ◮ Noise: Z i.i.d. N(0, 1) Regime: 1/√n ≪ ρ ≪ 1 Detection: distinguish P : Y = X + Z vs Q : Y = Z w.h.p. ◮ Sum of all entries succeeds when λ ≫ (ρ√n)−2 Recovery: given Y ∼ P, recover v ◮ Leading eigenvector succeeds when λ ≫ (ρ√n)−1

15 / 23

Planted Submatrix

Example (planted submatrix): observe n × n matrix Y = X + Z ◮ Signal: X = λvv⊤ λ > 0 vi ∼ Bernoulli(ρ) ◮ Noise: Z i.i.d. N(0, 1) Regime: 1/√n ≪ ρ ≪ 1 Detection: distinguish P : Y = X + Z vs Q : Y = Z w.h.p. ◮ Sum of all entries succeeds when λ ≫ (ρ√n)−2 Recovery: given Y ∼ P, recover v ◮ Leading eigenvector succeeds when λ ≫ (ρ√n)−1 ◮ Exhaustive search succeeds when λ ≫ (ρn)−1/2

15 / 23

Planted Submatrix

Example (planted submatrix): observe n × n matrix Y = X + Z ◮ Signal: X = λvv⊤ λ > 0 vi ∼ Bernoulli(ρ) ◮ Noise: Z i.i.d. N(0, 1) Regime: 1/√n ≪ ρ ≪ 1 Detection: distinguish P : Y = X + Z vs Q : Y = Z w.h.p. ◮ Sum of all entries succeeds when λ ≫ (ρ√n)−2 Recovery: given Y ∼ P, recover v ◮ Leading eigenvector succeeds when λ ≫ (ρ√n)−1 ◮ Exhaustive search succeeds when λ ≫ (ρn)−1/2 Detection-recovery gap

15 / 23

Recovery Hardness from Detection Hardness?

16 / 23

Recovery Hardness from Detection Hardness?

If you can recover then you can detect (poly-time reduction)

16 / 23

Recovery Hardness from Detection Hardness?

If you can recover then you can detect (poly-time reduction) ◮ How: run recovery algorithm to get ˆ v ∈ {0, 1}n; check ˆ v⊤Y ˆ v

16 / 23

Recovery Hardness from Detection Hardness?

If you can recover then you can detect (poly-time reduction) ◮ How: run recovery algorithm to get ˆ v ∈ {0, 1}n; check ˆ v⊤Y ˆ v So if Adv≤D = O(1), this suggests recovery is hard

16 / 23

Recovery Hardness from Detection Hardness?

If you can recover then you can detect (poly-time reduction) ◮ How: run recovery algorithm to get ˆ v ∈ {0, 1}n; check ˆ v⊤Y ˆ v So if Adv≤D = O(1), this suggests recovery is hard But how to show hardness of recovery when detection is easy?

16 / 23

Recovery Hardness from Detection Hardness?

If you can recover then you can detect (poly-time reduction) ◮ How: run recovery algorithm to get ˆ v ∈ {0, 1}n; check ˆ v⊤Y ˆ v So if Adv≤D = O(1), this suggests recovery is hard But how to show hardness of recovery when detection is easy? Attempt: choose a better null distribution?

16 / 23

Recovery Hardness from Detection Hardness?

If you can recover then you can detect (poly-time reduction) ◮ How: run recovery algorithm to get ˆ v ∈ {0, 1}n; check ˆ v⊤Y ˆ v So if Adv≤D = O(1), this suggests recovery is hard But how to show hardness of recovery when detection is easy? Attempt: choose a better null distribution? ◮ Match mean of planted distribution?

16 / 23

Recovery Hardness from Detection Hardness?

If you can recover then you can detect (poly-time reduction) ◮ How: run recovery algorithm to get ˆ v ∈ {0, 1}n; check ˆ v⊤Y ˆ v So if Adv≤D = O(1), this suggests recovery is hard But how to show hardness of recovery when detection is easy? Attempt: choose a better null distribution? ◮ Match mean of planted distribution? ◮ Gaussian matching first 2 moments of planted distribution?

16 / 23

Recovery Hardness from Detection Hardness?

If you can recover then you can detect (poly-time reduction) ◮ How: run recovery algorithm to get ˆ v ∈ {0, 1}n; check ˆ v⊤Y ˆ v So if Adv≤D = O(1), this suggests recovery is hard But how to show hardness of recovery when detection is easy? Attempt: choose a better null distribution? ◮ Match mean of planted distribution? ◮ Gaussian matching first 2 moments of planted distribution? This closes detection-recovery gap partially but not all the way

16 / 23

Low-Degree Recovery

Example (planted submatrix): observe n × n matrix Y = X + Z ◮ Signal: X = λvv⊤ λ > 0 vi ∼ Bernoulli(ρ) ◮ Noise: Z i.i.d. N(0, 1)

17 / 23

Low-Degree Recovery

Example (planted submatrix): observe n × n matrix Y = X + Z ◮ Signal: X = λvv⊤ λ > 0 vi ∼ Bernoulli(ρ) ◮ Noise: Z i.i.d. N(0, 1) Goal: given Y , estimate v1 via polynomial f : Rn×n → R

17 / 23

Low-Degree Recovery

Example (planted submatrix): observe n × n matrix Y = X + Z ◮ Signal: X = λvv⊤ λ > 0 vi ∼ Bernoulli(ρ) ◮ Noise: Z i.i.d. N(0, 1) Goal: given Y , estimate v1 via polynomial f : Rn×n → R Low-degree minimum mean squared error: MMSE≤D = min

f deg D E(f (Y ) − v1)2

17 / 23

Low-Degree Recovery

Example (planted submatrix): observe n × n matrix Y = X + Z ◮ Signal: X = λvv⊤ λ > 0 vi ∼ Bernoulli(ρ) ◮ Noise: Z i.i.d. N(0, 1) Goal: given Y , estimate v1 via polynomial f : Rn×n → R Low-degree minimum mean squared error: MMSE≤D = min

f deg D E(f (Y ) − v1)2

Equivalent to low-degree maximum correlation: Corr≤D = max

f deg D

E[f (Y ) · v1]

• E[f (Y )2]

Fact: MMSE≤D = E[v2

1 ] − Corr2 ≤D

17 / 23

Hardness of Recovery

For hardness, want upper bound on Corr≤D = max

f deg D

E[f (Y ) · v1]

• E[f (Y )2]

18 / 23

Hardness of Recovery

For hardness, want upper bound on Corr≤D = max

f deg D

E[f (Y ) · v1]

• E[f (Y )2]

Same proof as detection?

18 / 23

Hardness of Recovery

For hardness, want upper bound on Corr≤D = max

f deg D

E[f (Y ) · v1]

• E[f (Y )2]

Same proof as detection? f =

• |S|≤D

ˆ fSY S

18 / 23

Hardness of Recovery

For hardness, want upper bound on Corr≤D = max

f deg D

E[f (Y ) · v1]

• E[f (Y )2]

Same proof as detection? f =

• |S|≤D

ˆ fSY S Numerator: E[f (Y ) · v1]

18 / 23

Hardness of Recovery

For hardness, want upper bound on Corr≤D = max

f deg D

E[f (Y ) · v1]

• E[f (Y )2]

Same proof as detection? f =

• |S|≤D

ˆ fSY S Numerator: E[f (Y ) · v1] =

• |S|≤D

ˆ fS E[Y S · v1]

18 / 23

Hardness of Recovery

For hardness, want upper bound on Corr≤D = max

f deg D

E[f (Y ) · v1]

• E[f (Y )2]

Same proof as detection? f =

• |S|≤D

ˆ fSY S Numerator: E[f (Y ) · v1] =

• |S|≤D

ˆ fS E[Y S · v1] =: ˆ f , c

18 / 23

Hardness of Recovery

For hardness, want upper bound on Corr≤D = max

f deg D

E[f (Y ) · v1]

• E[f (Y )2]

Same proof as detection? f =

• |S|≤D

ˆ fSY S Numerator: E[f (Y ) · v1] =

• |S|≤D

ˆ fS E[Y S · v1] =: ˆ f , c Denominator: E[f (Y )2]

18 / 23

Hardness of Recovery

For hardness, want upper bound on Corr≤D = max

f deg D

E[f (Y ) · v1]

• E[f (Y )2]

Same proof as detection? f =

• |S|≤D

ˆ fSY S Numerator: E[f (Y ) · v1] =

• |S|≤D

ˆ fS E[Y S · v1] =: ˆ f , c Denominator: E[f (Y )2] =

• S,T

ˆ fS ˆ fT E[Y S · Y T]

18 / 23

Hardness of Recovery

For hardness, want upper bound on Corr≤D = max

f deg D

E[f (Y ) · v1]

• E[f (Y )2]

Same proof as detection? f =

• |S|≤D

ˆ fSY S Numerator: E[f (Y ) · v1] =

• |S|≤D

ˆ fS E[Y S · v1] =: ˆ f , c Denominator: E[f (Y )2] =

• S,T

ˆ fS ˆ fT E[Y S · Y T] = ˆ f ⊤M ˆ f

18 / 23

Hardness of Recovery

For hardness, want upper bound on Corr≤D = max

f deg D

E[f (Y ) · v1]

• E[f (Y )2]

Same proof as detection? f =

• |S|≤D

ˆ fSY S Numerator: E[f (Y ) · v1] =

• |S|≤D

ˆ fS E[Y S · v1] =: ˆ f , c Denominator: E[f (Y )2] =

• S,T

ˆ fS ˆ fT E[Y S · Y T] = ˆ f ⊤M ˆ f Corr≤D = max

ˆ f

ˆ f , c

• ˆ

f ⊤M ˆ f

18 / 23

Hardness of Recovery

For hardness, want upper bound on Corr≤D = max

f deg D

E[f (Y ) · v1]

• E[f (Y )2]

Same proof as detection? f =

• |S|≤D

ˆ fSY S Numerator: E[f (Y ) · v1] =

• |S|≤D

ˆ fS E[Y S · v1] =: ˆ f , c Denominator: E[f (Y )2] =

• S,T

ˆ fS ˆ fT E[Y S · Y T] = ˆ f ⊤M ˆ f Corr≤D = max

ˆ f

ˆ f , c

• ˆ

f ⊤M ˆ f = √ c⊤M−1c

18 / 23

Hardness of Recovery

For hardness, want upper bound on Corr≤D = max

f deg D

E[f (Y ) · v1]

• E[f (Y )2]

19 / 23

Hardness of Recovery

For hardness, want upper bound on Corr≤D = max

f deg D

E[f (Y ) · v1]

• E[f (Y )2]

Trick: bound denominator via Jensen’s inequality on “signal” X E[f (Y )2] = E

Z E X[f (X + Z)2] ≥ E Z

• E

X f (X + Z)

2

19 / 23

Hardness of Recovery

For hardness, want upper bound on Corr≤D = max

f deg D

E[f (Y ) · v1]

• E[f (Y )2]

Trick: bound denominator via Jensen’s inequality on “signal” X E[f (Y )2] = E

Z E X[f (X + Z)2] ≥ E Z

• E

X f (X + Z)

2 Why is this tight?

19 / 23

Hardness of Recovery

For hardness, want upper bound on Corr≤D = max

f deg D

E[f (Y ) · v1]

• E[f (Y )2]

Trick: bound denominator via Jensen’s inequality on “signal” X E[f (Y )2] = E

Z E X[f (X + Z)2] ≥ E Z

• E

X f (X + Z)

2 Why is this tight? In hard regime, f depends mostly on Z

19 / 23

Hardness of Recovery

For hardness, want upper bound on Corr≤D = max

f deg D

E[f (Y ) · v1]

• E[f (Y )2]

Trick: bound denominator via Jensen’s inequality on “signal” X E[f (Y )2] = E

Z E X[f (X + Z)2] ≥ E Z

• E

X f (X + Z)

2 Why is this tight? In hard regime, f depends mostly on Z This simplifies expression enough to find a closed form:

19 / 23

Hardness of Recovery

For hardness, want upper bound on Corr≤D = max

f deg D

E[f (Y ) · v1]

• E[f (Y )2]

Trick: bound denominator via Jensen’s inequality on “signal” X E[f (Y )2] = E

Z E X[f (X + Z)2] ≥ E Z

• E

X f (X + Z)

2 Why is this tight? In hard regime, f depends mostly on Z This simplifies expression enough to find a closed form: Corr≤D ≤ max

ˆ f

ˆ f , c M ˆ f where M is upper triangular

19 / 23

Hardness of Recovery

For hardness, want upper bound on Corr≤D = max

f deg D

E[f (Y ) · v1]

• E[f (Y )2]

Trick: bound denominator via Jensen’s inequality on “signal” X E[f (Y )2] = E

Z E X[f (X + Z)2] ≥ E Z

• E

X f (X + Z)

2 Why is this tight? In hard regime, f depends mostly on Z This simplifies expression enough to find a closed form: Corr≤D ≤ max

ˆ f

ˆ f , c M ˆ f = c⊤M−1 where M is upper triangular (can invert)

19 / 23

Main Result

20 / 23

Main Result

Theorem [Schramm, W. ’20] Additive Gaussian model Y = X + Z Scalar value to recover: x

20 / 23

Main Result

Theorem [Schramm, W. ’20] Additive Gaussian model Y = X + Z Scalar value to recover: x Corr2

≤D ≤

• |S|≤D

κ2

S

where κS is the joint cumulant of {x} ∪ {Yi : i ∈ S}

20 / 23

Main Result

Theorem [Schramm, W. ’20] Additive Gaussian model Y = X + Z Scalar value to recover: x Corr2

≤D ≤

• |S|≤D

κ2

S

where κS is the joint cumulant of {x} ∪ {Yi : i ∈ S} Corollary (tight bounds for planted submatrix recovery)

20 / 23

Main Result

Theorem [Schramm, W. ’20] Additive Gaussian model Y = X + Z Scalar value to recover: x Corr2

≤D ≤

• |S|≤D

κ2

S

where κS is the joint cumulant of {x} ∪ {Yi : i ∈ S} Corollary (tight bounds for planted submatrix recovery) ◮ if λ ≪ min{1,

1 ρ√n} then MMSE≤nΩ(1) ≈ ρ(1 − ρ)

low-degree polynomials have trivial MSE in the “hard” regime

20 / 23

Main Result

Theorem [Schramm, W. ’20] Additive Gaussian model Y = X + Z Scalar value to recover: x Corr2

≤D ≤

• |S|≤D

κ2

S

where κS is the joint cumulant of {x} ∪ {Yi : i ∈ S} Corollary (tight bounds for planted submatrix recovery) ◮ if λ ≪ min{1,

1 ρ√n} then MMSE≤nΩ(1) ≈ ρ(1 − ρ)

low-degree polynomials have trivial MSE in the “hard” regime

◮ if λ ≫ min{1,

1 ρ√n} then MMSE≤O(log n) = o(ρ)

low-degree polynomials succeed in the “easy” regime

20 / 23

Future Directions?

21 / 23

Future Directions?

◮ (Detection) bound Adv≤D when Q is not a product measure

◮ E.g. random regular graphs

21 / 23

Future Directions?

◮ (Detection) bound Adv≤D when Q is not a product measure

◮ E.g. random regular graphs

◮ (Recovery) bound MMSE≤D when not “signal + noise”

◮ E.g. sparse regression, phase retrieval

21 / 23

Future Directions?

◮ (Detection) bound Adv≤D when Q is not a product measure

◮ E.g. random regular graphs

◮ (Recovery) bound MMSE≤D when not “signal + noise”

◮ E.g. sparse regression, phase retrieval

◮ (Recovery) sharp threshold for planted submatrix

◮ AMP succeeds when λ > (ρ√en)−1 [Hajek, Wu, Xu ’15]

21 / 23

Future Directions?

◮ (Detection) bound Adv≤D when Q is not a product measure

◮ E.g. random regular graphs

◮ (Recovery) bound MMSE≤D when not “signal + noise”

◮ E.g. sparse regression, phase retrieval

◮ (Recovery) sharp threshold for planted submatrix

◮ AMP succeeds when λ > (ρ√en)−1 [Hajek, Wu, Xu ’15]

◮ Implications for other algorithms?

◮ E.g. convex programming, MCMC

21 / 23

References

◮ Detection (survey article)

Notes on Computational Hardness of Hypothesis Testing: Predictions using the Low-Degree Likelihood Ratio Kunisky, W., Bandeira arXiv:1907.11636

◮ Recovery

Computational Barriers to Estimation from Low-Degree Polynomials Schramm, W. arXiv:2008.02269

◮ Optimization

Low-Degree Hardness of Random Optimization Problems Gamarnik, Jagannath, W. arXiv:2004.12063

22 / 23

(extra scratch paper)

23 / 23