Fourier and Circulant Matrices are Not Rigid Allen Liu (MIT) Joint - - PowerPoint PPT Presentation

Preliminaries Results Proof Overview Conclusion

Fourier and Circulant Matrices are Not Rigid

Allen Liu (MIT) Joint work with Zeev Dvir (Princeton) July 18, 2019

Preliminaries Results Proof Overview Conclusion

Matrix Rigidity

Preliminaries Results Proof Overview Conclusion

Matrix Rigidity

A matrix M is rigid if it cannot be written as A + E where A is low-rank and E is sparse

Preliminaries Results Proof Overview Conclusion

Matrix Rigidity

A matrix M is rigid if it cannot be written as A + E where A is low-rank and E is sparse Definition RM(r) is the smallest number s for which M = A + E rank(A) ≤ r E is s-sparse

Preliminaries Results Proof Overview Conclusion

Motivation

Preliminaries Results Proof Overview Conclusion

Motivation

Matrix rigidity was introduced as a method for proving circuit lower bounds

Preliminaries Results Proof Overview Conclusion

Motivation

Matrix rigidity was introduced as a method for proving circuit lower bounds Theorem [Valiant 77] Let M be an N × N matrix over a field F. If for some constant ǫ > 0 RF

M

• N

log log N

• ≥ Ω
• N1+ǫ

then M cannot be computed by circuits of size O(N) and depth O(log N).

Preliminaries Results Proof Overview Conclusion

Motivation

Matrix rigidity was introduced as a method for proving circuit lower bounds Theorem [Valiant 77] Let M be an N × N matrix over a field F. If for some constant ǫ > 0 RF

M

• N

log log N

• ≥ Ω
• N1+ǫ

then M cannot be computed by circuits of size O(N) and depth O(log N). A random matrix has RF

M

• N

log log N

• ≥ Ω
• N2−ǫ

Preliminaries Results Proof Overview Conclusion

Motivation

Matrix rigidity was introduced as a method for proving circuit lower bounds Theorem [Valiant 77] Let M be an N × N matrix over a field F. If for some constant ǫ > 0 RF

M

• N

log log N

• ≥ Ω
• N1+ǫ

then M cannot be computed by circuits of size O(N) and depth O(log N). A random matrix has RF

M

• N

log log N

• ≥ Ω
• N2−ǫ

It is a long standing open problem to give an explicit construction of a rigid matrix

Preliminaries Results Proof Overview Conclusion

Previous Work on Rigid Matrices

Preliminaries Results Proof Overview Conclusion

Previous Work on Rigid Matrices

Theorem [Shokrollahi et. al., 1997] For any N × N matrix M for which all minors are nonsingular RM(r) ≥ Ω N2 r log N r

Preliminaries Results Proof Overview Conclusion

Previous Work on Rigid Matrices

Theorem [Shokrollahi et. al., 1997] For any N × N matrix M for which all minors are nonsingular RM(r) ≥ Ω N2 r log N r

• Theorem [Goldreich, Tal 2016]

Let A ∈ FN×N

2

be a (uniformly) random circulant matrix. Then for every r ∈ [ √ N, N

32], with 1 − o(1) probability

RF2

A (r) = Ω

• N3

r2 log N

Preliminaries Results Proof Overview Conclusion

Previous Work on Rigid Matrices

Theorem [Shokrollahi et. al., 1997] For any N × N matrix M for which all minors are nonsingular RM(r) ≥ Ω N2 r log N r

• Theorem [Goldreich, Tal 2016]

Let A ∈ FN×N

2

be a (uniformly) random circulant matrix. Then for every r ∈ [ √ N, N

32], with 1 − o(1) probability

RF2

A (r) = Ω

• N3

r2 log N

• Neither of these is strong enough for Valiant’s method

Preliminaries Results Proof Overview Conclusion

Special Families of Matrices

Preliminaries Results Proof Overview Conclusion

Special Families of Matrices

(Generalized) Hadamard Matrix Hd,n Hxy = ωx,y where ω = e

2πi d

and x, y ∈ Zn

d

Preliminaries Results Proof Overview Conclusion

Special Families of Matrices

(Generalized) Hadamard Matrix Hd,n Hxy = ωx,y where ω = e

2πi d

and x, y ∈ Zn

d

Fourier Matrix FN Fij = ζx·y where ζ = e

2πi N and x, y ∈ ZN

Preliminaries Results Proof Overview Conclusion

Special Families of Matrices

(Generalized) Hadamard Matrix Hd,n Hxy = ωx,y where ω = e

2πi d

and x, y ∈ Zn

d

Fourier Matrix FN Fij = ζx·y where ζ = e

2πi N and x, y ∈ ZN

Circulant Matrix MN Mij = f (i − j mod N) for i, j ∈ ZN

Preliminaries Results Proof Overview Conclusion

Special Families of Matrices

(Generalized) Hadamard Matrix Hd,n Hxy = ωx,y where ω = e

2πi d

and x, y ∈ Zn

d

Fourier Matrix FN Fij = ζx·y where ζ = e

2πi N and x, y ∈ ZN

Circulant Matrix MN Mij = f (i − j mod N) for i, j ∈ ZN Group Algebra Matrix MG Mab = f (ab−1) for a, b ∈ G

Preliminaries Results Proof Overview Conclusion

Special Families of Matrices

(Generalized) Hadamard Matrix Hd,n Hxy = ωx,y where ω = e

2πi d

and x, y ∈ Zn

d

Fourier Matrix FN Fij = ζx·y where ζ = e

2πi N and x, y ∈ ZN

Circulant Matrix MN Mij = f (i − j mod N) for i, j ∈ ZN Group Algebra Matrix MG Mab = f (ab−1) for a, b ∈ G Do any of these families contain rigid matrices?

Preliminaries Results Proof Overview Conclusion

Special Families of Matrices

(Generalized) Hadamard Matrix Hd,n Hxy = ωx,y where ω = e

2πi d

and x, y ∈ Zn

d

Fourier Matrix FN Fij = ζx·y where ζ = e

2πi N and x, y ∈ ZN

Circulant Matrix MN Mij = f (i − j mod N) for i, j ∈ ZN Group Algebra Matrix MG Mab = f (ab−1) for a, b ∈ G Do any of these families contain rigid matrices? Showing that any of these families contains some rigid matrix would still imply circuit lower bounds

Preliminaries Results Proof Overview Conclusion

Previous Work on Non-rigid Matrices

Preliminaries Results Proof Overview Conclusion

Previous Work on Non-rigid Matrices

The Hadamard matrix is not rigid [Alman, Williams 2016] For every ǫ > 0, there exists ǫ′ > 0 such that for all sufficiently large n, RQ

H2,n

• 2n(1−ǫ′)

≤ 2n(1+ǫ)

Preliminaries Results Proof Overview Conclusion

Previous Work on Non-rigid Matrices

The Hadamard matrix is not rigid [Alman, Williams 2016] For every ǫ > 0, there exists ǫ′ > 0 such that for all sufficiently large n, RQ

H2,n

• 2n(1−ǫ′)

≤ 2n(1+ǫ) Group algebra matrices for Zn

q are not rigid [Dvir, Edelman 2017]

Fix ǫ > 0. There is ǫ′ > 0 such that group algebra matrices for Zn

q

with entries over Fq satisfy RFq

M

• qn(1−ǫ′)

≤ qn(1+ǫ) for fixed q and n sufficiently large

Preliminaries Results Proof Overview Conclusion

Previous Work on Non-rigid Matrices

The Hadamard matrix is not rigid [Alman, Williams 2016] For every ǫ > 0, there exists ǫ′ > 0 such that for all sufficiently large n, RQ

H2,n

• 2n(1−ǫ′)

≤ 2n(1+ǫ) Group algebra matrices for Zn

q are not rigid [Dvir, Edelman 2017]

Fix ǫ > 0. There is ǫ′ > 0 such that group algebra matrices for Zn

q

with entries over Fq satisfy RFq

M

• qn(1−ǫ′)

≤ qn(1+ǫ) for fixed q and n sufficiently large Neither of these matrices is rigid enough to carry out Valiant’s method for proving circuit lower bounds.

Preliminaries Results Proof Overview Conclusion

Our results

Preliminaries Results Proof Overview Conclusion

Our results

Theorem (informal) Generalized Hadamard, Fourier, circulant, and group algebra matrices for abelian groups are all not Valiant-rigid.

Preliminaries Results Proof Overview Conclusion

Our results

Theorem (informal) Generalized Hadamard, Fourier, circulant, and group algebra matrices for abelian groups are all not Valiant-rigid. Theorem For all sufficiently large N, if M is an N × N circulant matrix over C or some fixed finite field Fq, RM

• N

2ǫ6(log N)0.35

• ≤ N1+15ǫ

Preliminaries Results Proof Overview Conclusion

Our results

Theorem (informal) Generalized Hadamard, Fourier, circulant, and group algebra matrices for abelian groups are all not Valiant-rigid. Theorem For all sufficiently large N, if M is an N × N circulant matrix over C or some fixed finite field Fq, RM

• N

2ǫ6(log N)0.35

• ≤ N1+15ǫ

Previous results [AW16, DE17] only work with matrices whose symmetry group has small characteristic

Preliminaries Results Proof Overview Conclusion

Our results

Theorem (informal) Generalized Hadamard, Fourier, circulant, and group algebra matrices for abelian groups are all not Valiant-rigid. Theorem For all sufficiently large N, if M is an N × N circulant matrix over C or some fixed finite field Fq, RM

• N

2ǫ6(log N)0.35

• ≤ N1+15ǫ

Previous results [AW16, DE17] only work with matrices whose symmetry group has small characteristic In this presentation treat everything over C

Preliminaries Results Proof Overview Conclusion

Proof Overview

Preliminaries Results Proof Overview Conclusion

Diagonalization Trick

Preliminaries Results Proof Overview Conclusion

Diagonalization Trick

Observation: FN diagonalizes any N × N circulant matrix M

Preliminaries Results Proof Overview Conclusion

Diagonalization Trick

Observation: FN diagonalizes any N × N circulant matrix M M = F ∗

NDFN = (FN − E)∗DFN + E ∗D(FN − E) + E ∗DE

Preliminaries Results Proof Overview Conclusion

Diagonalization Trick

Observation: FN diagonalizes any N × N circulant matrix M M = F ∗

NDFN = (FN − E)∗DFN + E ∗D(FN − E)

• Low Rank

+ E ∗DE

Sparse

FN not rigid → all circulant matrices are not rigid

Preliminaries Results Proof Overview Conclusion

Diagonalization Trick

Observation: FN diagonalizes any N × N circulant matrix M M = F ∗

NDFN = (FN − E)∗DFN + E ∗D(FN − E)

• Low Rank

+ E ∗DE

Sparse

FN not rigid → all circulant matrices are not rigid Similar conclusion holds for Hadamard matrix Hd,n and group algebra matrix MZn

d

Preliminaries Results Proof Overview Conclusion

Proof Overview

Preliminaries Results Proof Overview Conclusion

Proof Overview

1 Show generalized Hadamard matrices Hd,n are not rigid for

fixed d and n ≫ d2

Preliminaries Results Proof Overview Conclusion

Proof Overview

1 Show generalized Hadamard matrices Hd,n are not rigid for

fixed d and n ≫ d2

2 Show N × N Fourier matrices are not rigid for some N with a

nice prime factorization

Preliminaries Results Proof Overview Conclusion

Proof Overview

1 Show generalized Hadamard matrices Hd,n are not rigid for

fixed d and n ≫ d2

2 Show N × N Fourier matrices are not rigid for some N with a

nice prime factorization

3 Use diagonalization lemma to deduce nonrigidity of all N × N

circulant matrices for these N

Preliminaries Results Proof Overview Conclusion

Proof Overview

1 Show generalized Hadamard matrices Hd,n are not rigid for

fixed d and n ≫ d2

2 Show N × N Fourier matrices are not rigid for some N with a

nice prime factorization

3 Use diagonalization lemma to deduce nonrigidity of all N × N

circulant matrices for these N

4 Show N × N Fourier and circulant matrices are not rigid for

all N

Preliminaries Results Proof Overview Conclusion

Embedding Additive structure in Fourier matrices

Preliminaries Results Proof Overview Conclusion

Embedding Additive structure in Fourier matrices

Consider the Fourier matrix Fp for a prime p

Preliminaries Results Proof Overview Conclusion

Embedding Additive structure in Fourier matrices

Consider the Fourier matrix Fp for a prime p Remove the first row and column (i.e. x ≡ 0 mod p or y ≡ 0 mod p) M′

xy = ζx·y ∀x, y ∈ Z∗ p

Preliminaries Results Proof Overview Conclusion

Embedding Additive structure in Fourier matrices

Consider the Fourier matrix Fp for a prime p Remove the first row and column (i.e. x ≡ 0 mod p or y ≡ 0 mod p) M′

xy = ζx·y ∀x, y ∈ Z∗ p

(Z∗

p, ×) ∼

= (Zp−1, +) so M′ is a group algebra matrix for Zp−1 (as an additive group)

Preliminaries Results Proof Overview Conclusion

Embedding Additive structure in Fourier matrices

Consider the Fourier matrix Fp for a prime p Remove the first row and column (i.e. x ≡ 0 mod p or y ≡ 0 mod p) M′

xy = ζx·y ∀x, y ∈ Z∗ p

(Z∗

p, ×) ∼

= (Zp−1, +) so M′ is a group algebra matrix for Zp−1 (as an additive group)       1 1 1 1 1 1 ω ω2 ω3 ω4 1 ω2 ω4 ω ω3 1 ω3 ω ω4 ω2 1 ω4 ω3 ω2 ω       →       1 1 1 1 1 1 ω ω2 ω4 ω3 1 ω2 ω4 ω3 ω 1 ω4 ω3 ω ω2 1 ω3 ω ω2 ω4      

Preliminaries Results Proof Overview Conclusion

Embedding Additive structure in Fourier matrices (cont.)

Preliminaries Results Proof Overview Conclusion

Embedding Additive structure in Fourier matrices (cont.)

For an integer N = p1p2 . . . pn, the multiplicative subgroup Z∗

N has the same structure as the additive group

Zp1−1 ⊗ · · · ⊗ Zpn−1

Preliminaries Results Proof Overview Conclusion

Embedding Additive structure in Fourier matrices (cont.)

For an integer N = p1p2 . . . pn, the multiplicative subgroup Z∗

N has the same structure as the additive group

Zp1−1 ⊗ · · · ⊗ Zpn−1 Decompose Zpi−1 into a direct product of groups of prime power order

Preliminaries Results Proof Overview Conclusion

Embedding Additive structure in Fourier matrices (cont.)

For an integer N = p1p2 . . . pn, the multiplicative subgroup Z∗

N has the same structure as the additive group

Zp1−1 ⊗ · · · ⊗ Zpn−1 Decompose Zpi−1 into a direct product of groups of prime power order (Z25 ⊗ Z32 ⊗ . . . )

• Zp1−1

⊗ (Z24 ⊗ Z32 ⊗ Z5 . . . )

• Zp2−1

· · · ⊗ (Z5 ⊗ . . . )

• Zpn−1

Preliminaries Results Proof Overview Conclusion

Embedding Additive structure in Fourier matrices (cont.)

For an integer N = p1p2 . . . pn, the multiplicative subgroup Z∗

N has the same structure as the additive group

Zp1−1 ⊗ · · · ⊗ Zpn−1 Decompose Zpi−1 into a direct product of groups of prime power order (Z25 ⊗ Z32 ⊗ . . . )

• Zp1−1

⊗ (Z24 ⊗ Z32 ⊗ Z5 . . . )

• Zp2−1

· · · ⊗ (Z5 ⊗ . . . )

• Zpn−1

Preliminaries Results Proof Overview Conclusion

Embedding Additive structure in Fourier matrices (cont.)

For an integer N = p1p2 . . . pn, the multiplicative subgroup Z∗

N has the same structure as the additive group

Zp1−1 ⊗ · · · ⊗ Zpn−1 Decompose Zpi−1 into a direct product of groups of prime power order (Z25 ⊗ Z32 ⊗ . . . )

• Zp1−1

⊗ (Z24 ⊗ Z32 ⊗ Z5 . . . )

• Zp2−1

· · · ⊗ (Z5 ⊗ . . . )

• Zpn−1

Collect like terms (Z2 ⊗ · · · ⊗ Z2)

• t2

⊗ (Z3 ⊗ · · · ⊗ Z3)

• t3

⊗ . . .

Preliminaries Results Proof Overview Conclusion

Embedding Additive structure in Fourier matrices (cont.)

For an integer N = p1p2 . . . pn, the multiplicative subgroup Z∗

N has the same structure as the additive group

Zp1−1 ⊗ · · · ⊗ Zpn−1 Decompose Zpi−1 into a direct product of groups of prime power order (Z25 ⊗ Z32 ⊗ . . . )

• Zp1−1

⊗ (Z24 ⊗ Z32 ⊗ Z5 . . . )

• Zp2−1

· · · ⊗ (Z5 ⊗ . . . )

• Zpn−1

Collect like terms (Z2 ⊗ · · · ⊗ Z2)

• t2

⊗ (Z3 ⊗ · · · ⊗ Z3)

• t3

⊗ . . . If p1 − 1, . . . , pn − 1 factor smoothly, then t2, t3, . . . will be large

Preliminaries Results Proof Overview Conclusion

Hadamard-type Structure

Preliminaries Results Proof Overview Conclusion

Hadamard-type Structure

We have a group algebra matrix M for (Zd1 ⊗ · · · ⊗ Zd1)

• t1

⊗ (Zd2 ⊗ · · · ⊗ Zd2)

• t2

⊗ . . .

Preliminaries Results Proof Overview Conclusion

Hadamard-type Structure

We have a group algebra matrix M for (Zd1 ⊗ · · · ⊗ Zd1)

• t1

⊗ (Zd2 ⊗ · · · ⊗ Zd2)

• t2

⊗ . . . M is diagonalized by Hd1,t1 ⊗ Hd2,t2 ⊗ . . .

Preliminaries Results Proof Overview Conclusion

Hadamard-type Structure

We have a group algebra matrix M for (Zd1 ⊗ · · · ⊗ Zd1)

• t1

⊗ (Zd2 ⊗ · · · ⊗ Zd2)

• t2

⊗ . . . M is diagonalized by Hd1,t1 ⊗ Hd2,t2 ⊗ . . . If t1 ≫ d2

1, t2 ≫ d2 2, . . . we can use the nonrigidity of

Hadamard matrices and the diagonalization trick to show M is not rigid

Preliminaries Results Proof Overview Conclusion

Nonrigidity for some Fourier matrices

Preliminaries Results Proof Overview Conclusion

Nonrigidity for some Fourier matrices

Definition (informal) An integer N is well-factorable if N = p1 . . . pn p1, . . . , pn are distinct primes that are roughly the same size as n For all i, the largest prime power divisor of pi − 1 is at most p0.3

i

Preliminaries Results Proof Overview Conclusion

Nonrigidity for some Fourier matrices

Definition (informal) An integer N is well-factorable if N = p1 . . . pn p1, . . . , pn are distinct primes that are roughly the same size as n For all i, the largest prime power divisor of pi − 1 is at most p0.3

i

Fourier matrices of well-factorable size are not rigid For any fixed 0 < ǫ < 0.1 and well-factorable integer N, we have rFN

• N

2ǫ6(log N)0.36

• ≤ N7ǫ

Preliminaries Results Proof Overview Conclusion

Nonrigidity for some Fourier matrices

Definition (informal) An integer N is well-factorable if N = p1 . . . pn p1, . . . , pn are distinct primes that are roughly the same size as n For all i, the largest prime power divisor of pi − 1 is at most p0.3

i

Fourier matrices of well-factorable size are not rigid For any fixed 0 < ǫ < 0.1 and well-factorable integer N, we have rFN

• N

2ǫ6(log N)0.36

• ≤ N7ǫ

Using the diagonalization trick, we can show that N × N circulant matrices with N well-factorable are not rigid

Preliminaries Results Proof Overview Conclusion

Nonrigidity of all Fourier matrices

Preliminaries Results Proof Overview Conclusion

Nonrigidity of all Fourier matrices

For a given N, we can embed any N × N circulant matrix in an N′ × N′ circulant matrix for N′ ≥ 2N − 1

Preliminaries Results Proof Overview Conclusion

Nonrigidity of all Fourier matrices

For a given N, we can embed any N × N circulant matrix in an N′ × N′ circulant matrix for N′ ≥ 2N − 1       a b c d e b c d e a c d e a b d e a b c e a b c d      

Preliminaries Results Proof Overview Conclusion

Nonrigidity of all Fourier matrices

For a given N, we can embed any N × N circulant matrix in an N′ × N′ circulant matrix for N′ ≥ 2N − 1       a b c d e b c d e a c d e a b d e a b c e a b c d       Suffices to show that we can find an N′ that is well factorable and not too much larger than N i.e. 2N ≤ N′ ≤ N1+o(1)

Preliminaries Results Proof Overview Conclusion

Nonrigidity of all Fourier matrices

For a given N, we can embed any N × N circulant matrix in an N′ × N′ circulant matrix for N′ ≥ 2N − 1       a b c d e b c d e a c d e a b d e a b c e a b c d       Suffices to show that we can find an N′ that is well factorable and not too much larger than N i.e. 2N ≤ N′ ≤ N1+o(1) Finish using a result from analytic number theory [Baker, Harman 1998]

Preliminaries Results Proof Overview Conclusion

Conclusion

Preliminaries Results Proof Overview Conclusion

Directions for Future Work

Preliminaries Results Proof Overview Conclusion

Directions for Future Work

Group algebra matrices for nonabelian groups

Preliminaries Results Proof Overview Conclusion

Directions for Future Work

Group algebra matrices for nonabelian groups

Cannot be completely diagonalized

Preliminaries Results Proof Overview Conclusion

Directions for Future Work

Group algebra matrices for nonabelian groups

Cannot be completely diagonalized Can be block diagonalized i.e. MG = F ∗DF where D is block-diagonal Sizes of the blocks correspond to the sizes of the irreducible reprsentations of G.

Preliminaries Results Proof Overview Conclusion

Directions for Future Work

Group algebra matrices for nonabelian groups

Cannot be completely diagonalized Can be block diagonalized i.e. MG = F ∗DF where D is block-diagonal Sizes of the blocks correspond to the sizes of the irreducible reprsentations of G. Natural candidates are groups whose irreducible representations are large

Preliminaries Results Proof Overview Conclusion

Directions for Future Work

Preliminaries Results Proof Overview Conclusion

Directions for Future Work

Conjecture Group algebra matrices for SL2(p) are rigid

Preliminaries Results Proof Overview Conclusion

Directions for Future Work

Conjecture Group algebra matrices for SL2(p) are rigid SL2(p) consists of 2 × 2 matrices a b c d

• with determinant 1

Preliminaries Results Proof Overview Conclusion

Directions for Future Work

Conjecture Group algebra matrices for SL2(p) are rigid SL2(p) consists of 2 × 2 matrices a b c d

• with determinant 1

|SL2(p)| = p3 − p

Preliminaries Results Proof Overview Conclusion

Directions for Future Work

Conjecture Group algebra matrices for SL2(p) are rigid SL2(p) consists of 2 × 2 matrices a b c d

• with determinant 1

|SL2(p)| = p3 − p All nontrivial irreducible representations have size O(p)

Preliminaries Results Proof Overview Conclusion

Directions for Future Work

Conjecture Group algebra matrices for SL2(p) are rigid SL2(p) consists of 2 × 2 matrices a b c d

• with determinant 1

|SL2(p)| = p3 − p All nontrivial irreducible representations have size O(p) Want to show that for some function f : SL2(p) → F the (p3 − p) × (p3 − p) matrix given by MAB = f (AB−1) ∀A, B ∈ SL2(p) is rigid

Preliminaries Results Proof Overview Conclusion

Thanks!