Quantitative invertibility of random matrices: a combinatorial - - PowerPoint PPT Presentation

Quantitative invertibility of random matrices: a combinatorial perspective

Vishesh Jain Massachusetts Institute of Technology Stanford Combinatorics Seminar

October 31, 2019

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 1 / 36

The quantitative invertibility problem

Definition (Least singular value) The least singular value of an n × n matrix Mn is defined by sn(Mn) := inf

v∈Sn−1 Mnv2.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 2 / 36

The quantitative invertibility problem

Definition (Least singular value) The least singular value of an n × n matrix Mn is defined by sn(Mn) := inf

v∈Sn−1 Mnv2.

Quantitative invertibility problem What is the probability that sn(Mn) is smaller than η ≥ 0?

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 2 / 36

Regime I: Invertibility of random discrete matrices

Suppose that each entry of Mn is an independent Rademacher random variable i.e. +1 or −1 with probability 1/2 each. Estimate Pr(sn(Mn) = 0).

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 3 / 36

Regime I: Invertibility of random discrete matrices

Suppose that each entry of Mn is an independent Rademacher random variable i.e. +1 or −1 with probability 1/2 each. Estimate Pr(sn(Mn) = 0). Folklore Conjecture: Pr(sn(Mn) = 0) n22−n.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 3 / 36

Regime I: Invertibility of random discrete matrices

Suppose that each entry of Mn is an independent Rademacher random variable i.e. +1 or −1 with probability 1/2 each. Estimate Pr(sn(Mn) = 0). Folklore Conjecture: Pr(sn(Mn) = 0) n22−n. Koml´

• s (1967): Pr(sn(Mn) = 0) = on(1).

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 3 / 36

Regime I: Invertibility of random discrete matrices

Suppose that each entry of Mn is an independent Rademacher random variable i.e. +1 or −1 with probability 1/2 each. Estimate Pr(sn(Mn) = 0). Folklore Conjecture: Pr(sn(Mn) = 0) n22−n. Koml´

• s (1967): Pr(sn(Mn) = 0) = on(1).

Kahn, Koml´

• s, and Szemer´

edi (1995): Pr(sn(Mn) = 0) 0.999n.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 3 / 36

Regime I: Invertibility of random discrete matrices

Suppose that each entry of Mn is an independent Rademacher random variable i.e. +1 or −1 with probability 1/2 each. Estimate Pr(sn(Mn) = 0). Folklore Conjecture: Pr(sn(Mn) = 0) n22−n. Koml´

• s (1967): Pr(sn(Mn) = 0) = on(1).

Kahn, Koml´

• s, and Szemer´

edi (1995): Pr(sn(Mn) = 0) 0.999n. Tao and Vu (2006, 2007): Pr(sn(Mn) = 0) 0.75n.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 3 / 36

Regime I: Invertibility of random discrete matrices

Suppose that each entry of Mn is an independent Rademacher random variable i.e. +1 or −1 with probability 1/2 each. Estimate Pr(sn(Mn) = 0). Folklore Conjecture: Pr(sn(Mn) = 0) n22−n. Koml´

• s (1967): Pr(sn(Mn) = 0) = on(1).

Kahn, Koml´

• s, and Szemer´

edi (1995): Pr(sn(Mn) = 0) 0.999n. Tao and Vu (2006, 2007): Pr(sn(Mn) = 0) 0.75n. Bourgain, Vu, and Wood (2010): Pr(sn(Mn) = 0) (1/ √ 2)n.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 3 / 36

Regime I: Invertibility of random discrete matrices

Suppose that each entry of Mn is an independent Rademacher random variable i.e. +1 or −1 with probability 1/2 each. Estimate Pr(sn(Mn) = 0). Folklore Conjecture: Pr(sn(Mn) = 0) n22−n. Koml´

• s (1967): Pr(sn(Mn) = 0) = on(1).

Kahn, Koml´

• s, and Szemer´

edi (1995): Pr(sn(Mn) = 0) 0.999n. Tao and Vu (2006, 2007): Pr(sn(Mn) = 0) 0.75n. Bourgain, Vu, and Wood (2010): Pr(sn(Mn) = 0) (1/ √ 2)n. Tikhomirov (2018): Pr(sn(Mn) = 0) (0.5 + on(1))n.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 3 / 36

Regime II: Least singular value of ‘typical’ random matrices

Suppose that each entry of Mn is an independent copy of the standard

• Gaussian. Estimate sn(Mn) for a ‘typical’ such matrix.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 4 / 36

Regime II: Least singular value of ‘typical’ random matrices

Suppose that each entry of Mn is an independent copy of the standard

• Gaussian. Estimate sn(Mn) for a ‘typical’ such matrix.

Edelman (1988), Szarek (1991): Pr(sn(Mn) ≤ ǫn−1/2) ≤ ǫ. Hence, for ‘most’ such matrices, sn(Mn) = Ω(n−1/2).

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 4 / 36

Regime II: Least singular value of ‘typical’ random matrices

Suppose that each entry of Mn is an independent copy of the standard

• Gaussian. Estimate sn(Mn) for a ‘typical’ such matrix.

Edelman (1988), Szarek (1991): Pr(sn(Mn) ≤ ǫn−1/2) ≤ ǫ. Hence, for ‘most’ such matrices, sn(Mn) = Ω(n−1/2). Sankar, Spielman, and Teng (2006): Pr(sn(An + Mn) ≤ ǫn−1/2) ǫ. Here, An is an arbitrary square matrix.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 4 / 36

The Spielman-Teng conjecture

Conjecture (Spielman and Teng, ICM 2002) Suppose that the entries of Mn are independent Rademacher random

• variables. There exists some constant c ∈ (0, 1) such that for all η ≥ 0,

Pr (sn(Mn) ≤ η) ≤ √nη + cn. This combines ‘Gaussian behavior’ with the added possibility of singularity.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 5 / 36

Resolution of the Spielman-Teng conjecture

Theorem (Rudelson and Vershynin, 2007) Suppose that the entries of Mn are i.i.d. subgaussian random variables with mean 0 and variance 1. Then, there exists c ∈ (0, 1) such that for all η ≥ 0, Pr(sn(Mn) ≤ η) √nη + cn.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 6 / 36

Resolution of the Spielman-Teng conjecture

Theorem (Rudelson and Vershynin, 2007) Suppose that the entries of Mn are i.i.d. subgaussian random variables with mean 0 and variance 1. Then, there exists c ∈ (0, 1) such that for all η ≥ 0, Pr(sn(Mn) ≤ η) √nη + cn. Rebrova and Tikhomirov (2015): Removed subgaussian assumption.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 6 / 36

Resolution of the Spielman-Teng conjecture

Theorem (Rudelson and Vershynin, 2007) Suppose that the entries of Mn are i.i.d. subgaussian random variables with mean 0 and variance 1. Then, there exists c ∈ (0, 1) such that for all η ≥ 0, Pr(sn(Mn) ≤ η) √nη + cn. Rebrova and Tikhomirov (2015): Removed subgaussian assumption. Livshyts, Tikhomirov, and Vershynin (2019): Removed identically distributed assumption.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 6 / 36

Least singular value of shifted i.i.d. matrices

Thus far, the high-dimensional geometric methods used in the proofs of the previous results have failed to address the following important model of random matrices:

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 7 / 36

Least singular value of shifted i.i.d. matrices

Thus far, the high-dimensional geometric methods used in the proofs of the previous results have failed to address the following important model of random matrices: Mn := An + Nn, where An is a ‘large’ fixed complex matrix, and Nn is a random matrix, each of whose entries is an independent copy of a complex random variable of mean 0 and variance 1.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 7 / 36

Least singular value of shifted i.i.d. matrices

Thus far, the high-dimensional geometric methods used in the proofs of the previous results have failed to address the following important model of random matrices: Mn := An + Nn, where An is a ‘large’ fixed complex matrix, and Nn is a random matrix, each of whose entries is an independent copy of a complex random variable of mean 0 and variance 1. Why is this important?

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 7 / 36

Least singular value of shifted i.i.d. matrices

For the strong circular law, known reductions (Girko, 1984; Bai, 1997; Tao and Vu, 2008) show that we need to study sn(Mn) for Mn = z · Idn + Nn √n, with z ∈ C fixed.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 8 / 36

Least singular value of shifted i.i.d. matrices

For the strong circular law, known reductions (Girko, 1984; Bai, 1997; Tao and Vu, 2008) show that we need to study sn(Mn) for Mn = z · Idn + Nn √n, with z ∈ C fixed. For numerical linear algebra, the smoothed analysis program of Spielman and Teng (2001) considers Mn = An + Nn, where Nn represents the random ‘noise’ in the system.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 8 / 36

Least singular value of shifted i.i.d. matrices

For the strong circular law, known reductions (Girko, 1984; Bai, 1997; Tao and Vu, 2008) show that we need to study sn(Mn) for Mn = z · Idn + Nn √n, with z ∈ C fixed. For numerical linear algebra, the smoothed analysis program of Spielman and Teng (2001) considers Mn = An + Nn, where Nn represents the random ‘noise’ in the system. Moreover, for these applications, sharp results are not necessary.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 8 / 36

Least singular value of shifted i.i.d. matrices

Prior to our work, the best known result in this setting is due to Tao and Vu, based on deep ideas from additive combinatorics.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 9 / 36

Least singular value of shifted i.i.d. matrices

Prior to our work, the best known result in this setting is due to Tao and Vu, based on deep ideas from additive combinatorics. Theorem (Tao and Vu, 2007, 2008, 2010) Let Fn be a fixed n × n complex matrix, each of whose entries is O(nB). Suppose that the entries of Nn are i.i.d. complex random variables with mean 0 and variance 1. Then, for any A > 0, there exists C > 0 such that: Pr

• sn(Fn + Nn) ≤ n−C

n−A.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 9 / 36

Random matrices with dependent entries

There are significant additional challenges in dealing with random matrices with dependent entries.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 10 / 36

Random matrices with dependent entries

There are significant additional challenges in dealing with random matrices with dependent entries. As a result, there are big gaps even in our knowledge of the singularity probability for most discrete random matrix models.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 10 / 36

Random matrices with dependent entries

There are significant additional challenges in dealing with random matrices with dependent entries. As a result, there are big gaps even in our knowledge of the singularity probability for most discrete random matrix models. For instance, in the next simplest model of symmetric random Rademacher matrices, the best known upper bound is only of the form (1/2)

√n (Campos, Mattos, Morris, and Morrison, 2019), as

compared to the conjectured bound of (1/2 + on(1))n.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 10 / 36

Random matrices with dependent entries

For combinatorial models, such as adjacency matrices of regular (di)graphs, the situation is much more dire – bounds of the form n−1 are not known except in special cases.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 11 / 36

Random matrices with dependent entries

For combinatorial models, such as adjacency matrices of regular (di)graphs, the situation is much more dire – bounds of the form n−1 are not known except in special cases. In fact, even on(1) bounds were only very recently obtained e.g. Huang (2018), Landon, Sosoe, and Yau (2016), Litvak, Lytova, Tikhomirov, Tomczak-Jaegermann, and Youssef (2015), Cook (2014)...

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 11 / 36

A motivating result

Theorem (Vershynin, 2011) Suppose that Mn is a symmetric matrix, each of whose above diagonal entries is an independent copy of a subgaussian random variable with mean 0 and variance 1. Then, Pr(sn(Mn) ≤ η) (√nη)1/9 + exp(−nc).

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 12 / 36

A motivating result

Theorem (Vershynin, 2011) Suppose that Mn is a symmetric matrix, each of whose above diagonal entries is an independent copy of a subgaussian random variable with mean 0 and variance 1. Then, Pr(sn(Mn) ≤ η) (√nη)1/9 + exp(−nc). As discussed, for many applications, it suffices to have a result with the (optimal) √n replaced by a larger power of n.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 12 / 36

Our goals and results

Summary We prove bounds of the form Pr (sn(Mn) ≤ η) nCηδ + exp(−nc) in a simple and unified way for quite general random matrix models

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 13 / 36

Our goals and results

Summary We prove bounds of the form Pr (sn(Mn) ≤ η) nCηδ + exp(−nc) in a simple and unified way for quite general random matrix models To this end, we: Introduce new tools, in particular for the so-called ‘counting problem in inverse Littlewood–Offord theory’.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 13 / 36

Our goals and results

Summary We prove bounds of the form Pr (sn(Mn) ≤ η) nCηδ + exp(−nc) in a simple and unified way for quite general random matrix models To this end, we: Introduce new tools, in particular for the so-called ‘counting problem in inverse Littlewood–Offord theory’. Introduce new reductions, some of which can even be used in combination with previously known tools.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 13 / 36

Our results - Non centered complex matrices

Theorem (Tao and Vu, 2007, 2008, 2010) Let Fn be a fixed n × n complex matrix with operator norm O(nB). If the entries of Nn are i.i.d. complex random variables with mean 0 and variance 1, then for all η ≥ 0, Pr (sn(Fn + Nn) ≤ η) nCηδη + n−ω(1).

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 14 / 36

Our results - Non centered complex matrices

Theorem (Tao and Vu, 2007, 2008, 2010) Let Fn be a fixed n × n complex matrix with operator norm O(nB). If the entries of Nn are i.i.d. complex random variables with mean 0 and variance 1, then for all η ≥ 0, Pr (sn(Fn + Nn) ≤ η) nCηδη + n−ω(1). Theorem (J., 2019+) Under the same assumptions, Pr (sn(Fn + Nn) ≤ η) nCηδη + exp(−nc).

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 14 / 36

Our results - dependent entries

Theorem (J., 2019+) Let Mn be an n × n random matrix with independent rows in {0, 1}n, each

• f which sums to n/2. Then, for any η ≥ 0,

Pr (sn(Mn) ≤ η) n2η + exp(−nc).

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 15 / 36

Our results - dependent entries

Theorem (J., 2019+) Let Mn be an n × n random matrix with independent rows in {0, 1}n, each

• f which sums to n/2. Then, for any η ≥ 0,

Pr (sn(Mn) ≤ η) n2η + exp(−nc). Theorem (Ferber and J., 2018) Let Mn be an n × n symmetric matrix whose above diagonal entries are independent Rademacher random variables. Then, Pr (sn(Mn) = 0) exp(−n1/4).

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 15 / 36

A proof template

In the remainder of this talk, I will sketch some of the ideas and techniques that go into the proofs of our results. In order to motivate them, I will first present a high-level ‘proof template’.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 16 / 36

A proof template

In the remainder of this talk, I will sketch some of the ideas and techniques that go into the proofs of our results. In order to motivate them, I will first present a high-level ‘proof template’. To keep technicalities to a minimum, our goal in the next few slides will be to discuss how to obtain upper bounds on the probability that an i.i.d matrix is singular in the ‘light-tailed’ setting.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 16 / 36

A proof template: the anti-concentration phenomenon

Definition (Small ball probability) The r-ball probability of a vector v := (v1, . . . , vn) ∈ Rn with respect to a random variable ξ is defined by ρr,ξ(v) := sup

x∈R

Pr (|v1ξ1 + · · · + vnξn − x| ≤ r) , where ξ1, . . . , ξn are independent copies of ξ.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 17 / 36

A proof template: the anti-concentration phenomenon

Definition (Small ball probability) The r-ball probability of a vector v := (v1, . . . , vn) ∈ Rn with respect to a random variable ξ is defined by ρr,ξ(v) := sup

x∈R

Pr (|v1ξ1 + · · · + vnξn − x| ≤ r) , where ξ1, . . . , ξn are independent copies of ξ. Examples (when ξ is Rademacher): If v = (10, 100, 1000, . . . , 10n), then ρ1/4,ξ(v) = 2−n.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 17 / 36

A proof template: the anti-concentration phenomenon

Definition (Small ball probability) The r-ball probability of a vector v := (v1, . . . , vn) ∈ Rn with respect to a random variable ξ is defined by ρr,ξ(v) := sup

x∈R

Pr (|v1ξ1 + · · · + vnξn − x| ≤ r) , where ξ1, . . . , ξn are independent copies of ξ. Examples (when ξ is Rademacher): If v = (10, 100, 1000, . . . , 10n), then ρ1/4,ξ(v) = 2−n. If v = (1, . . . , 1), then ρ1/4,ξ(v) = 2−n

n ⌊n/2⌋

• = Θ
• 1

√n

• .

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 17 / 36

A proof template: reduction to anti-concentrating normals

Let X1, . . . , Xn denote the rows of Mn. Let S denote the event that Mn is singular. Let Si denote the event that Xi lies in the span of the other rows. Since 1S ≤ 1S1 + · · · + 1Sn, Pr(S) ≤ n Pr(Sn).

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 18 / 36

A proof template: reduction to anti-concentrating normals

Suppose we could prove the following: Random normals anti-concentrate Except with probability exp(−n), any v ∈ Sn−1 which is orthogonal to X1, . . . , Xn−1 satisfies ρ0,ξ(v) ≤ ρ.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 19 / 36

A proof template: reduction to anti-concentrating normals

Suppose we could prove the following: Random normals anti-concentrate Except with probability exp(−n), any v ∈ Sn−1 which is orthogonal to X1, . . . , Xn−1 satisfies ρ0,ξ(v) ≤ ρ. Then, Pr(Sn) ≤ ρ + exp(−n), and we would be done.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 19 / 36

Proof template: Random normals anti-concentrate

Random normals anti-concentrate Except with probability exp(−n), any v ∈ Sn−1 which is orthogonal to X1, . . . , Xn−1 satisfies ρ0,ξ(v) ≤ ρ.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 20 / 36

Proof template: Random normals anti-concentrate

Random normals anti-concentrate Except with probability exp(−n), any v ∈ Sn−1 which is orthogonal to X1, . . . , Xn−1 satisfies ρ0,ξ(v) ≤ ρ. Proof: Union bound!

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 20 / 36

Proof template: Random normals anti-concentrate

Random normals anti-concentrate Except with probability exp(−n), any v ∈ Sn−1 which is orthogonal to X1, . . . , Xn−1 satisfies ρ0,ξ(v) ≤ ρ. Proof: Union bound! Let v ∈ Sn−1 have ρ0,ξ(v) ∈ (λ/2, λ].

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 20 / 36

Proof template: Random normals anti-concentrate

Random normals anti-concentrate Except with probability exp(−n), any v ∈ Sn−1 which is orthogonal to X1, . . . , Xn−1 satisfies ρ0,ξ(v) ≤ ρ. Proof: Union bound! Let v ∈ Sn−1 have ρ0,ξ(v) ∈ (λ/2, λ]. By the independence of X1, . . . , Xn−1, Pr (X1 · v = 0 ∧ · · · ∧ Xn−1 · v = 0) ≤ λn−1.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 20 / 36

Proof template: Random normals anti-concentrate

Random normals anti-concentrate Except with probability exp(−n), any v ∈ Sn−1 which is orthogonal to X1, . . . , Xn−1 satisfies ρ0,ξ(v) ≤ ρ. Proof: Union bound! Let v ∈ Sn−1 have ρ0,ξ(v) ∈ (λ/2, λ]. By the independence of X1, . . . , Xn−1, Pr (X1 · v = 0 ∧ · · · ∧ Xn−1 · v = 0) ≤ λn−1. Suppose we could show that the ‘number’ of v ∈ Sn−1 with ρ0,ξ(v) ∈ (λ/2, λ] is at most (λ−1/nγ)n.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 20 / 36

Proof template: Random normals anti-concentrate

Random normals anti-concentrate Except with probability exp(−n), any v ∈ Sn−1 which is orthogonal to X1, . . . , Xn−1 satisfies ρ0,ξ(v) ≤ ρ. Proof: Union bound! Let v ∈ Sn−1 have ρ0,ξ(v) ∈ (λ/2, λ]. By the independence of X1, . . . , Xn−1, Pr (X1 · v = 0 ∧ · · · ∧ Xn−1 · v = 0) ≤ λn−1. Suppose we could show that the ‘number’ of v ∈ Sn−1 with ρ0,ξ(v) ∈ (λ/2, λ] is at most (λ−1/nγ)n. Then, by a union bound, and ranging over λ = 1, 2−1, 2−2, . . . , ρ,

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 20 / 36

Proof template: Random normals anti-concentrate

Random normals anti-concentrate Except with probability exp(−n), any v ∈ Sn−1 which is orthogonal to X1, . . . , Xn−1 satisfies ρ0,ξ(v) ≤ ρ. Proof: Union bound! Let v ∈ Sn−1 have ρ0,ξ(v) ∈ (λ/2, λ]. By the independence of X1, . . . , Xn−1, Pr (X1 · v = 0 ∧ · · · ∧ Xn−1 · v = 0) ≤ λn−1. Suppose we could show that the ‘number’ of v ∈ Sn−1 with ρ0,ξ(v) ∈ (λ/2, λ] is at most (λ−1/nγ)n. Then, by a union bound, and ranging over λ = 1, 2−1, 2−2, . . . , ρ, Pr (BADρ) ≤ λ−1 nγ n · (λ)n−1 log(1/ρ)ρn−γn.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 20 / 36

Can we count?

Not directly, since Sn−1 has uncountably many points! Overcoming this

• bstacle is the heart of the matter.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 21 / 36

Can we count?

Not directly, since Sn−1 has uncountably many points! Overcoming this

• bstacle is the heart of the matter.

High-dimensional geometric framework

Discretize the unit sphere using a net based on a very exact relation between Diophantine approximation and anti-concentration.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 21 / 36

Can we count?

Not directly, since Sn−1 has uncountably many points! Overcoming this

• bstacle is the heart of the matter.

High-dimensional geometric framework

Discretize the unit sphere using a net based on a very exact relation between Diophantine approximation and anti-concentration. Requires strong control on the operator norm, which is often not available when dealing with non-centered entries.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 21 / 36

Can we count?

Not directly, since Sn−1 has uncountably many points! Overcoming this

• bstacle is the heart of the matter.

High-dimensional geometric framework

Discretize the unit sphere using a net based on a very exact relation between Diophantine approximation and anti-concentration. Requires strong control on the operator norm, which is often not available when dealing with non-centered entries.

Additive combinatorial framework

Uses deep structure vs. randomness ideas from additive combinatorics to build a net on the sphere.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 21 / 36

Can we count?

Not directly, since Sn−1 has uncountably many points! Overcoming this

• bstacle is the heart of the matter.

High-dimensional geometric framework

Discretize the unit sphere using a net based on a very exact relation between Diophantine approximation and anti-concentration. Requires strong control on the operator norm, which is often not available when dealing with non-centered entries.

Additive combinatorial framework

Uses deep structure vs. randomness ideas from additive combinatorics to build a net on the sphere. Only effective for ρ ≥ n−C.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 21 / 36

Can we count?

Not directly, since Sn−1 has uncountably many points! Overcoming this

• bstacle is the heart of the matter.

High-dimensional geometric framework

Discretize the unit sphere using a net based on a very exact relation between Diophantine approximation and anti-concentration. Requires strong control on the operator norm, which is often not available when dealing with non-centered entries.

Additive combinatorial framework

Uses deep structure vs. randomness ideas from additive combinatorics to build a net on the sphere. Only effective for ρ ≥ n−C. Extensions to dependent models require much more work e.g. quadratic inverse Littlewood–Offord theory of Nguyen (2011).

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 21 / 36

We can count something!

Theorem (Ferber, J., Luh, and Samotij, 2018+; J., 2019+) For all ρ ≥ exp(−nc1), the number of vectors v ∈ Zn with v∞ ≤ exp(nc2); ρ1,ξ(v) ≥ ρ is at most (ρ−1/n0.5−ǫ)n. Proved by a (perhaps surprisingly!) short and elementary double counting argument!

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 22 / 36

What can we do with the counting theorem?

Recall that we wanted to prove: Random normals anti-concentrate Except with probability exp(−n), any v ∈ Sn−1 which is orthogonal to X1, . . . , Xn−1 i.e. for which n−1

i=1 |Xi · v|2 = 0 satisfies ρ0,ξ(v) ≤ ρ.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 23 / 36

What can we do with the counting theorem?

Recall that we wanted to prove: Random normals anti-concentrate Except with probability exp(−n), any v ∈ Sn−1 which is orthogonal to X1, . . . , Xn−1 i.e. for which n−1

i=1 |Xi · v|2 = 0 satisfies ρ0,ξ(v) ≤ ρ.

Let us show how to use the counting theorem to prove: Random integer ‘approximate normals’ anti-concentrate Except with probability exp(−n), any non-zero z ∈ Zn, z∞ ≤ exp(nc) for which

• n−1
• i=1

|Xi · z|2 ≤ n1−2ǫ satisfies ρ1,ξ(z) ≤ ρ.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 23 / 36

What can we do with the counting theorem?

Random integer ‘approximate normals’ anti-concentrate Except with probability exp(−n), any non-zero z ∈ Zn, z∞ ≤ exp(nc) for which n−1

i=1 |Xi · z|2 ≤ n1−2ǫ satisfies ρ1,ξ(z) ≤ ρ.

Proof: By a similar union bound to what we have seen.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 24 / 36

What can we do with the counting theorem?

Random integer ‘approximate normals’ anti-concentrate Except with probability exp(−n), any non-zero z ∈ Zn, z∞ ≤ exp(nc) for which n−1

i=1 |Xi · z|2 ≤ n1−2ǫ satisfies ρ1,ξ(z) ≤ ρ.

Proof: By a similar union bound to what we have seen. Let z be an integer vector with z∞ ≤ exp(nc) and ρ1,ξ(z) ∈ (λ/2, λ]. By independence, the probability that the vector (X1 · z, . . . Xn−1 · z) lies in any fixed hypercube with side length 1 is at most λn−1.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 24 / 36

What can we do with the counting theorem?

Key point: Since the volume of the n1−2ǫ-ball in Rn−1 is at most (n1−2ǫ/√n)(n−1),

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 25 / 36

What can we do with the counting theorem?

Key point: Since the volume of the n1−2ǫ-ball in Rn−1 is at most (n1−2ǫ/√n)(n−1), the probability that n−1

i=1 |Xi · z|2 ≤ n1−2ǫ is at

most λn−1 · n(0.5−2ǫ)n.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 25 / 36

What can we do with the counting theorem?

Key point: Since the volume of the n1−2ǫ-ball in Rn−1 is at most (n1−2ǫ/√n)(n−1), the probability that n−1

i=1 |Xi · z|2 ≤ n1−2ǫ is at

most λn−1 · n(0.5−2ǫ)n. On the other hand, by the counting theorem, the number of such z is at most

• λ−1n · n(−0.5+ǫ)n.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 25 / 36

What can we do with the counting theorem?

Key point: Since the volume of the n1−2ǫ-ball in Rn−1 is at most (n1−2ǫ/√n)(n−1), the probability that n−1

i=1 |Xi · z|2 ≤ n1−2ǫ is at

most λn−1 · n(0.5−2ǫ)n. On the other hand, by the counting theorem, the number of such z is at most

• λ−1n · n(−0.5+ǫ)n.

Therefore, the contribution of such z to the union bound is at most

• λ−1n n(−0.5+ǫ)n · λn−1n(0.5−2ǫ)n = λ−1 · n−ǫn.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 25 / 36

Getting around by rounding?

Recall that we wanted to prove: Random normals anti-concentrate Except with probability exp(−n), any v ∈ Sn−1 which is orthogonal to X1, . . . , Xn−1 satisfies ρ0,ξ(v) ≤ ρ.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 26 / 36

Getting around by rounding?

Recall that we wanted to prove: Random normals anti-concentrate Except with probability exp(−n), any v ∈ Sn−1 which is orthogonal to X1, . . . , Xn−1 satisfies ρ0,ξ(v) ≤ ρ. Can we reduce it to what we can prove? Random integer ‘approximate normals’ anti-concentrate Except with probability exp(−n), any non-zero z ∈ Zn, z∞ ≤ exp(nc) for which

• n−1
• i=1

|Xi · z|2 ≤ n1−2ǫ satisfies ρ1,ξ(z) ≤ ρ.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 26 / 36

Failed attempt: n¨ aive rounding

By rounding v ∈ Sn−1 to the nearest integer multiple of 1/√n, we obtain some z ∈ Zn such that v − (z/√n)2 ≤ 1/2 i.e. √nv − z2 ≤ √n/2.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 27 / 36

Failed attempt: n¨ aive rounding

By rounding v ∈ Sn−1 to the nearest integer multiple of 1/√n, we obtain some z ∈ Zn such that v − (z/√n)2 ≤ 1/2 i.e. √nv − z2 ≤ √n/2. Let ˜ Mn−1 denote the matrix consisting of the first n − 1 rows of Mn. So, if ˜ Mn−1v = 0 and ˜ Mn−1 ≤ √n, we get

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 27 / 36

Failed attempt: n¨ aive rounding

By rounding v ∈ Sn−1 to the nearest integer multiple of 1/√n, we obtain some z ∈ Zn such that v − (z/√n)2 ≤ 1/2 i.e. √nv − z2 ≤ √n/2. Let ˜ Mn−1 denote the matrix consisting of the first n − 1 rows of Mn. So, if ˜ Mn−1v = 0 and ˜ Mn−1 ≤ √n, we get

• n−1
• i=1

|Xi · z|2 = ˜ Mn−1z2 = ˜ Mn−1(z − √nv)2 ≤ ˜ Mn−1 · √nv − z2 ≤ √n · 1 2 √n = n 2.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 27 / 36

Failed attempt: n¨ aive rounding

By rounding v ∈ Sn−1 to the nearest integer multiple of 1/√n, we obtain some z ∈ Zn such that v − (z/√n)2 ≤ 1/2 i.e. √nv − z2 ≤ √n/2. Let ˜ Mn−1 denote the matrix consisting of the first n − 1 rows of Mn. So, if ˜ Mn−1v = 0 and ˜ Mn−1 ≤ √n, we get

• n−1
• i=1

|Xi · z|2 = ˜ Mn−1z2 = ˜ Mn−1(z − √nv)2 ≤ ˜ Mn−1 · √nv − z2 ≤ √n · 1 2 √n = n 2. But we wanted something of the form n1−2ǫ on the right hand side...

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 27 / 36

Successful attempt: non-trivial rounding available!

We saw that n¨ aive rounding ‘just’ fails. However, we are not trying to round any v ∈ Sn−1 but only those with ρ0,ξ(v) ≥ ρ.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 28 / 36

Successful attempt: non-trivial rounding available!

We saw that n¨ aive rounding ‘just’ fails. However, we are not trying to round any v ∈ Sn−1 but only those with ρ0,ξ(v) ≥ ρ. But such vectors are already special and have a better-than-trivial integer approximation! Proposition (Diophantine approximation vs. small-ball probability) For v ∈ Sn−1 with ρ0,ξ(v) ≥ ρ ≥ exp(−nc), there exists some γ ∈ [1, exp(nc)] and non-zero z ∈ Zn such that γv − z2 ≤ nδ.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 28 / 36

Diophantine approximation vs. small-ball probability

Proposition (Diophantine approximation vs. small-ball probability) For v ∈ Sn−1 with ρ0,ξ(v) ≥ ρ ≥ exp(−nc), there exists some γ ∈ [1, exp(nc)] and non-zero z ∈ Zn such that γv − z2 ≤ nδ. Key point: Don’t require any specific dependence of γ on ρ.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 29 / 36

Diophantine approximation vs. small-ball probability

Proposition (Diophantine approximation vs. small-ball probability) For v ∈ Sn−1 with ρ0,ξ(v) ≥ ρ ≥ exp(−nc), there exists some γ ∈ [1, exp(nc)] and non-zero z ∈ Zn such that γv − z2 ≤ nδ. Key point: Don’t require any specific dependence of γ on ρ. Direct consequence of the following Fourier analytic bound: ρ1,ξ(v)

• R

exp

• −dist (λv, Zn)2 − λ2

dλ, which essentially appears in a classical work of Hal´ asz (1977).

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 29 / 36

Putting everything together

Proposition (Diophantine approximation vs. small-ball probability) For v ∈ Sn−1 with ρ0,ξ(v) ≥ ρ ≥ exp(−nc), there exists some γ ∈ [1, exp(nc)] and non-zero z ∈ Zn such that γv − z2 ≤ nδ. Suppose ˜ Mn−1v = 0 for v ∈ Sn−1 satisfying ρ0,ξ(v) ≥ ρ.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 30 / 36

Putting everything together

Proposition (Diophantine approximation vs. small-ball probability) For v ∈ Sn−1 with ρ0,ξ(v) ≥ ρ ≥ exp(−nc), there exists some γ ∈ [1, exp(nc)] and non-zero z ∈ Zn such that γv − z2 ≤ nδ. Suppose ˜ Mn−1v = 0 for v ∈ Sn−1 satisfying ρ0,ξ(v) ≥ ρ. Then, by the theorem, we can find a non-zero z ∈ Zn with γv − z2 ≤ nδ and z∞ ≤ exp(nc).

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 30 / 36

Putting everything together

Proposition (Diophantine approximation vs. small-ball probability) For v ∈ Sn−1 with ρ0,ξ(v) ≥ ρ ≥ exp(−nc), there exists some γ ∈ [1, exp(nc)] and non-zero z ∈ Zn such that γv − z2 ≤ nδ. Suppose ˜ Mn−1v = 0 for v ∈ Sn−1 satisfying ρ0,ξ(v) ≥ ρ. Then, by the theorem, we can find a non-zero z ∈ Zn with γv − z2 ≤ nδ and z∞ ≤ exp(nc). Therefore, if ˜ Mn−1 ≤ √n, we get

• n−1
• i=1

|Xi · z|2 = ˜ Mn−1z2 ≤ ˜ Mn−1 · γv − z2 ≤ √n · nδ ≪ n1−2ǫ, so that z is an integer ‘approximate normal’.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 30 / 36

Putting everything together

Proposition (Diophantine approximation vs. small-ball probability) For v ∈ Sn−1 with ρ0,ξ(v) ≥ ρ ≥ exp(−nc), there exists some γ ∈ [1, exp(nc)] and non-zero z ∈ Zn such that γv − z2 ≤ nδ. Suppose ˜ Mn−1v = 0 for v ∈ Sn−1 satisfying ρ0,ξ(v) ≥ ρ. Then, by the theorem, we can find a non-zero z ∈ Zn with γv − z2 ≤ nδ and z∞ ≤ exp(nc). Therefore, if ˜ Mn−1 ≤ √n, we get

• n−1
• i=1

|Xi · z|2 = ˜ Mn−1z2 ≤ ˜ Mn−1 · γv − z2 ≤ √n · nδ ≪ n1−2ǫ, so that z is an integer ‘approximate normal’. This is exactly what we have ruled out with high probability!

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 30 / 36

Rounding in the presence of heavy tails

How can we round if the entries are only assumed to have finite second moment?

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 31 / 36

Rounding in the presence of heavy tails

How can we round if the entries are only assumed to have finite second moment? Control on ∞-to-2 norm Let An×m be an n × m ‘tall’ random matrix, each of whose entries is an independent copy of a random variable with mean 0 and variance 1. Then, except with probability exp(−n1−c), An′×m∞→2 nǫ · √n · √m. Proved using Chernoff bound + standard concentration inequalities on the symmetric group.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 31 / 36

Rounding in the presence of heavy tails

Let err := γv − z. Recall that err2 ≤ nδ and we want to show that ˜ Mn−1err2 ≤ n1−2ǫ.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 32 / 36

Rounding in the presence of heavy tails

Let err := γv − z. Recall that err2 ≤ nδ and we want to show that ˜ Mn−1err2 ≤ n1−2ǫ. Decompose err = err sp + err sm, where err sp consists of the largest n0.8 coordinates in absolute value.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 32 / 36

Rounding in the presence of heavy tails

Let err := γv − z. Recall that err2 ≤ nδ and we want to show that ˜ Mn−1err2 ≤ n1−2ǫ. Decompose err = err sp + err sm, where err sp consists of the largest n0.8 coordinates in absolute value. Since n0.8err sm2

∞ ≤ err sp2 2 ≤ err2 2,

err sm∞ ≤ nδ/n0.4.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 32 / 36

Rounding in the presence of heavy tails

Let err := γv − z. Recall that err2 ≤ nδ and we want to show that ˜ Mn−1err2 ≤ n1−2ǫ. Decompose err = err sp + err sm, where err sp consists of the largest n0.8 coordinates in absolute value. Since n0.8err sm2

∞ ≤ err sp2 2 ≤ err2 2,

err sm∞ ≤ nδ/n0.4. Therefore, ˜ Mn−1err sm2 ≤ ˜ Mn−1∞→2err sm∞ ≤ n1+ǫ+δ n0.4 ≪ n1−2ǫ.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 32 / 36

Rounding in the presence of heavy tails

It remains to show that ˜ Mn−1err sp2 ≤ n1−2ǫ.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 33 / 36

Rounding in the presence of heavy tails

It remains to show that ˜ Mn−1err sp2 ≤ n1−2ǫ. Note that ˜ Mn−1err sp = ( ˜ Mn−1ProjJ)err sp, where |J| = n0.8.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 33 / 36

Rounding in the presence of heavy tails

It remains to show that ˜ Mn−1err sp2 ≤ n1−2ǫ. Note that ˜ Mn−1err sp = ( ˜ Mn−1ProjJ)err sp, where |J| = n0.8. Except with probability at most exp(−n1−c), ˜ Mn−1ProjJ∞→2 ≤ nǫ · √n ·

• |J| = n0.9+ǫ.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 33 / 36

Rounding in the presence of heavy tails

It remains to show that ˜ Mn−1err sp2 ≤ n1−2ǫ. Note that ˜ Mn−1err sp = ( ˜ Mn−1ProjJ)err sp, where |J| = n0.8. Except with probability at most exp(−n1−c), ˜ Mn−1ProjJ∞→2 ≤ nǫ · √n ·

• |J| = n0.9+ǫ.

Hence, ˜ Mn−1err sp2 ≤ ˜ Mn−1ProjJ∞→2err sp∞ ≤ n0.9+ǫ ≪ n1−2ǫ.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 33 / 36

References

The strong circular law: a combinatorial view. Preprint available at http://arxiv.org/abs/1904.11108. Smoothed analysis of the least singular value without inverse Littlewood-Offord theory. Preprint available at https://arxiv.org/abs/1809.04718 Approximate Spielman-Teng theorems for the least singular value of random combinatorial matrices. Preprint available at https://arxiv.org/abs/1904.10592 On the counting problem in inverse Littlewood–Offord theory, joint with Asaf Ferber, Kyle Luh, and Wojciech Samotij. Preprint available at https://arxiv.org/abs/1904.10425 Singularity of random symmetric matrices – a combinatorial approach to improved bounds, joint with Asaf Ferber. Forum of Mathematics, Sigma, vol. 7, e22, 29 pages (2019).

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 34 / 36

References

For an introduction to the geometric framework of Rudelson and Vershynin, see their survey Non-asymptotic theory of random matrices: extreme singular values from the 2010 ICM Proceedings. For an introduction to inverse Littlewood-Offord theory and its applications to random matrix theory, see Small Ball Probability, Inverse Theorems, and Applications, Nguyen and Vu. Erd˝

• s

Centennial pp 409-463.

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 35 / 36

THANK YOU!

References available at math.mit.edu/∼ visheshj For any questions or comments: visheshj@mit.edu

Vishesh Jain (MIT) Quantitative invertibility of random matrices October 31, 2019 36 / 36