Learning Determinantal Processes wit ith Moments and Cycles J. - PowerPoint PPT Presentation

Learning Determinantal Processes wit ith Moments and Cycles J. Urschel, V.-E. Brunel, A. Moitra, P. Rigollet ICML 2017, Sydney

Determinantal Point Processes (D (DPPs) DPP: Random subset of [𝑶] • For all 𝐾 ⊆ 𝑂 , ℙ 𝐾 ⊆ 𝑍 = det 𝑳 𝐾 𝑳 ∈ ℝ 𝑂×𝑂 , symmetric, 0 ≼ 𝐿 ≼ 𝐽 𝑂 : parameter ( kernel ) of the DPP • • 𝐿 𝐾 = 𝐿 𝑗,𝑘 𝑗,𝑘∈𝐾 2 ≤ ℙ 1 ∈ 𝑍 ℙ 2 ∈ 𝑍 . • ℙ 1 ∈ 𝑍 = 𝐿 1,1 , ℙ 1,2 ∈ 𝑍 = 𝐿 1,1 𝐿 2,2 − 𝐿 1,2 E.g. 𝑀 = 𝐿 𝐽 𝑂 − 𝐿 −1 . • A.k.a. 𝑀 -ensembles if 0 ≺ 𝐿 ≺ 𝐽 𝑂 : ℙ 𝑍 = 𝐾 ∝ det 𝑀 𝐾 ,

Binary ry representation DPP ⟷ Random binary vector of size 𝑶 , represented as a subset of [𝑶] . ↔ {1,4,5,7,9,10,12,15,19} 1 0 0 1 1 0 1 0 1 1 0 1 0 0 1 0 0 0 1 0 0 0 1 1 0 1 0 1 1 0 0 1 0 0 1 0 0 0 1 0 ↔ {3,4,6,8,9, 12,15,19} 𝑌 1 , … , 𝑌 𝑂 ∈ {0,1} 𝑂 ↔ 𝑍 ⊆ [𝑂] 𝑌 𝑗 = 1 ⇔ 𝑗 ∈ 𝑍 Model for correlated Bernoulli r.v.’s (such as Ising , …) featuring repulsion.

Applications of DPP’s DPPs have become popular in various applications: • Quantum physics ( fermionic processes ) [Macchi ‘74] • Document and timeline summarization [Lin, Bilmes ‘12; Yao et al. ‘16] • Image search [Kulesza, Taskar ‘11; Affandi et al. ‘14] • Bioinformatics [Batmanghelich et al. ‘14] • Neuroscience [Snoek et al. ‘13] • Wireless or cellular networks modelization [Miyoshi, Shirai ‘14; Torrisi, Leonardi ‘14; Li et al. ‘15; Deng et al. ‘15] And they remain an elegant and important tool in probability theory [Borodin ‘11]

Learning DPPs iid • Given 𝑍 1 , 𝑍 2 , … , 𝑍 𝑜 ∼ DPP 𝐿 , estimate 𝐿 . • Approach: Method of moments • Problem: Is 𝐿 identified ?

Identification: 𝓔 -similarity Id ′ = det 𝐿 • DPP 𝐿′ = DPP 𝐿 ⇔ det 𝐿 𝐾 , ∀𝐾 ⊆ [𝑂] 𝐾 ±1 0 [Oeding ‘11] ±1 ⇔ 𝐿′ = 𝐸𝐿𝐸 for some D = . ⋱ 0 ±1 ↓ ↓ ← + + + + + − − + + + + + − + + − • E.g.: K = ⇝ 𝐸𝐿𝐸 = + + + + − + + − ← + + + + + − − + • 𝐿 and 𝐸𝐿𝐸 are called 𝓔 -similar .

Method of f moments 𝑜 𝐿 𝑗,𝑗 = 1 • Diagonal entries: 𝑳 𝒋,𝒋 = ℙ 𝑗 ∈ 𝑍 𝑜 𝟐 𝑗∈𝑍 𝑙 𝑙=1 • Magnitude of the off-diagonal entries: + 𝑜 𝑘,𝑘 − 1 2 = 𝐿 𝑗,𝑗 𝐿 2 = 𝐿 𝑗,𝑗 𝐿 𝑗,𝑘 𝑘,𝑘 − ℙ 𝑗, 𝑘 ∈ 𝑍 𝐿 𝑗,𝑘 𝐿 𝑜 𝟐 𝑗,𝑘∈𝑍 𝑙 𝑙=1 • Signs (up to 𝓔 -similarity) ? 𝑜 𝐾 = 1 Use estimates of higher moments: det 𝐿 𝑜 𝟐 𝐾∈𝑍 𝑙 𝑙=1

Determinantal Graphs Definition 𝐻 = 𝑂 , 𝐹 : 𝑗, 𝑘 ∈ 𝐹 ⇔ 𝐿 𝑗,𝑘 ≠ 0 . ∗ ∗ 0 2 𝐿 = ∗ ∗ ∗ 1 0 ∗ ∗ 3 Examples: ∗ ∗ ∗ 0 2 ∗ ∗ ∗ 0 𝐿 = ∗ ∗ ∗ ∗ 1 3 0 0 ∗ ∗ 4

Cycle sparsity • Cycle basis : family of induced cycles that span the cycle space 𝐷 2 𝐷 1 + 𝐷 2 𝐷 1 • Cycle sparsity : length ℓ of the largest cycle needed to span the cycle space • Horton’s algorithm : Find a cycle basis with cycle lengths ≤ ℓ in 𝑃 𝐹 2 𝑂 ln 𝑂 −1 steps [Horton ’87; Amaldi et al. ‘10]

Cycle sparsity Theorem: 𝐿 is completely determined, up to 𝒠 -similarity, by its principal minors of order ≤ ℓ . for each cycle of length ≤ ℓ . Key: Signs of 𝐿 𝑗,𝑘 {𝑗,𝑘}∈𝐷

Learning the signs • Assumption: 𝐿 ∈ 𝒧 𝛽 , i.e., either 𝐿 𝑗,𝑘 = 0 or 𝐿 𝑗,𝑘 ≥ 𝛽 > 0 • All 𝐿 𝑗,𝑗 ’s and 𝐿 𝑗,𝑘 ’s are estimated within 𝒐 −𝟐/𝟑 -rate • 𝐻 is recovered exactly w.h.p. • Horton’s algorithm outputs a minimum basis ℬ • For all induced cycle 𝐷 ∈ ℬ + 2 −1 |𝐷| 2 det 𝐿 𝐷 = 𝐺 𝐷 𝐿 𝑗,𝑗 , 𝐿 𝑗,𝑘 𝐿 𝑗,𝑘 {𝑗,𝑘}∈𝐷 • Recover the sign of w.h.p. 𝐿 𝑗,𝑘 {𝑗,𝑘}∈𝐷

Main result Theorem: Let 𝐿 ∈ 𝒧 𝛽 with cycle sparsity ℓ and let 𝜁 > 0 . Then, the following holds with probability at least 1 − 𝑜 −𝐵 : 𝐿 in 𝑃 𝐹 3 + 𝑜𝑂 2 steps for which There is an algorithm that outputs 2ℓ 𝛽 2 𝜁 2 + ℓ 2 1 𝑜 ≳ ln 𝑂 ⇒ min 𝐿 − 𝐸𝐿𝐸 ∞ ≤ 𝜁 𝛽 𝐸 Near-optimal rate in a minimax sense.

Conclusions • Estimation of 𝐿 by a method of moments in polynomial time • Rates of estimation characterized by the topology of the determinantal graph through its cycle sparsity ℓ . • These rates are provably optimal (up to logarithmic factors) • Adaptation to ℓ .