Rates of f Estimation for Dis iscrete Determinantal Point - PowerPoint PPT Presentation

Rates of f Estimation for Dis iscrete Determinantal Point Processes V.-E. Brunel, A. Moitra, P. Rigollet, J. Urschel COLT 2017, Amsterdam

Discrete DPPs Random variables on the hypercube 𝟏, 𝟐 𝑶 , represented as subsets of [𝑶] . ↔ {1,4,5,7,9,10,12,15,19} 1 0 0 1 1 0 1 0 1 1 0 1 0 0 1 0 0 0 1 0 0 0 1 1 0 1 0 1 1 0 0 1 0 0 1 0 0 0 1 0 ↔ {3,4,6,8,9, 12,15,19} ↔ {1,4,8,12,14,17,18,20} 1 0 0 1 0 0 0 1 0 0 0 1 0 1 0 0 1 1 0 1 … ↔ {3,6,8,9, 15,16,18} 0 0 1 0 0 1 0 1 1 0 0 0 0 0 1 1 0 1 0 0

Discrete DPPs • Probabilistic model for correlated Bernoulli r.v. • Feature repulsion (negative association) Random subset 𝑍 ⊆ [𝑂] , Definition 𝐿 ∈ ℝ 𝑂×𝑂 , symmetric, 0 ≼ 𝐿 ≼ 𝐽 ℙ 𝐾 ⊆ 𝑍 = det 𝐿 𝐾 , ∀𝐾 • 𝐿 𝑗,𝑘 ↬ repulsion between items 𝑗 and 𝑘 . • PMF: ℙ 𝑍 = 𝐾 = det 𝐿 − 𝐽 𝐾

Goal 𝑜 ∼ DPP 𝐿 ∗ , estimate 𝐿 ∗ . iid • Given 𝑍 1 , 𝑍 2 , … , 𝑍 • Approach: Maximum Likelihood Estimator. • Question: Rate of convergence of the MLE ?

Identification Id ∗ , ∀𝐾 ⊆ [𝑂] • DPP 𝐿 = DPP 𝐿 ∗ ⇔ det 𝐿 𝐾 = det 𝐿 𝐾 ±1 0 ±1 ⇔ 𝐿 = 𝐸𝐿 ∗ 𝐸 for some D = . ⋱ 0 ±1 ↓ ↓ ← + + + + + − − + + + + + − + + − • E.g.: K ∗ = 𝐸K ∗ 𝐸 = ⇝ + + + + − + + − ← + + + + + − − + 𝐿, 𝐿 ∗ = min Measure of the error of an estimator ℓ || 𝐿 − 𝐸𝐿 ∗ 𝐸|| 𝐺 𝑳 : 𝐸

Maximum likelihood estimation • Log-likelihood: Ψ 𝐿 = 𝑞 𝐾 ln det K − I 𝐾 𝐾 ⊆ 𝑂 𝐿 ∈ argmax • MLE: Ψ(𝐿) ∗ ln det K − I Ψ 𝐿 ≜ 𝔽 = 𝑞 𝐾 Ψ 𝐿 𝐾 𝐾 ⊆ 𝑂 = Ψ 𝐿 ∗ − 𝐿𝑀 𝐸𝑄𝑄 𝐿 ∗ , 𝐸𝑄𝑄 𝐿

Likelihood geometry ry Fisher information: −𝛼 2 Ψ 𝐿 ∗ Ψ 𝐿 Ψ 𝐿 𝐿 𝐿 𝐿 ∗ 𝐿 ∗ 𝛼 2 Ψ K ∗ < 0 𝛼 2 Ψ K ∗ = 0 What is the order of the first non degenerate derivative of 𝛀 at 𝑳 = 𝑳 ∗ ?

Determinantal Graphs & Ir Irreducibility Definition ∗ ≠ 0 . 𝐻 = 𝑂 , 𝐹 : 𝑗, 𝑘 ∈ 𝐹 ⇔ 𝐿 𝑗,𝑘 • 𝐿 ∗ is irreducible iff 𝐻 is connected. • Otherwise, 𝐿 ∗ is block diagonal. • Rk: 𝐿 ∗ is block diagonal ⇒ 𝑍 = union of independent DPPs • Write 𝑗 ∼ 𝑘 when 𝑗 and 𝑘 are connected in 𝐻 .

Main Results: Ir Irreducible case Theorem 1 𝐿 ∗ irreducible ⇔ 𝛼 2 Ψ(𝐿 ∗ ) is definite negative Statistical consequences: 𝐿, 𝐿 ∗ = 𝑃 ℙ 𝑜 − 1  ℓ 2  CLT

Main Results: Block diagonal case (1 (1) Theorem 2 Ker 𝛼 2 Ψ 𝐿 ∗ = 𝐼 ∈ ℝ 𝑂×𝑂 : 𝐼 𝑗,𝑘 = 0, ∀𝑗 ∼ 𝑘 𝜶 𝟑 𝛀 𝑳 ∗ is negative definite along directions supported on the blocks of 𝑳 ∗ . Theorem 3 𝐼 ⊗3 = 0 ∖ {0} : 𝛼 3 Ψ 𝐿 ∗ For 𝐼 ∈ Ker 𝛼 2 Ψ 𝐿 ∗ 𝐼 ⊗4 < 0 𝛼 4 Ψ 𝐿 ∗

Main Results: Block diagonal case (2 (2) Statistical consequences: 𝐿, 𝐿 ∗ = 𝑃 ℙ 𝑜 − 1  ℓ 6 ∗ = 𝑃 ℙ 𝑜 − 1  ℓ for all blocks 𝑇 of 𝐿 ∗ . 𝐿 𝑇 , 𝐿 2 𝑇

Conclusions • Rates of convergence of the MLE: if 𝐿 ∗ is irreducible 𝑜 −1/2 𝑜 −1/6 otherwise • Rate only determined by connectedness of the determinantal graph • Hidden constants can be arbitrarily large in 𝑂 : e.g., if 𝐻 is a path graph • In another paper we show that the sample complexity of a method-of-moment * estimator is determined by the cycle sparsity of 𝐻 . * Learning Determinantal Point Processes from Moments and Cycles , J. Urschel, V.-E. Brunel, A. Moitra, P. Rigollet, ICML 2017