Learning Determinantal Processes wit ith Moments and Cycles J. - - PowerPoint PPT Presentation

Learning Determinantal Processes wit ith Moments and Cycles

• J. Urschel, V.-E. Brunel, A. Moitra, P. Rigollet

ICML 2017, Sydney

Determinantal Point Processes (D (DPPs)

DPP: Random subset of [𝑶]

• For all 𝐾 ⊆ 𝑂 ,

ℙ 𝐾 ⊆ 𝑍 = det 𝑳𝐾

• 𝑳 ∈ ℝ𝑂×𝑂, symmetric, 0 ≼ 𝐿 ≼ 𝐽𝑂: parameter (kernel) of the DPP
• 𝐿

𝐾 = 𝐿𝑗,𝑘 𝑗,𝑘∈𝐾

• E.g.

ℙ 1 ∈ 𝑍 = 𝐿1,1 , ℙ 1,2 ∈ 𝑍 = 𝐿1,1𝐿2,2 − 𝐿1,2

2 ≤ ℙ 1 ∈ 𝑍 ℙ 2 ∈ 𝑍 .

• A.k.a. 𝑀-ensembles if 0 ≺ 𝐿 ≺ 𝐽𝑂: ℙ 𝑍 = 𝐾 ∝ det 𝑀𝐾 ,

𝑀 = 𝐿 𝐽𝑂 − 𝐿 −1.

Binary ry representation

1 0 0 1 1 0 1 0 1 1 0 1 0 0 1 0 0 0 1 0 0 0 1 1 0 1 0 1 1 0 0 1 0 0 1 0 0 0 1 0 𝑌1, … , 𝑌𝑂 ∈ {0,1}𝑂 ↔ {1,4,5,7,9,10,12,15,19} ↔ {3,4,6,8,9, 12,15,19} ↔ 𝑍 ⊆ [𝑂] DPP ⟷ Random binary vector of size 𝑶, represented as a subset of [𝑶]. 𝑌𝑗 = 1 ⇔ 𝑗 ∈ 𝑍 Model for correlated Bernoulli r.v.’s (such as Ising, …) featuring repulsion.

Applications of DPP’s

• Quantum physics (fermionic processes) [Macchi ‘74]
• Document and timeline summarization [Lin, Bilmes ‘12; Yao et al. ‘16]
• Image search [Kulesza, Taskar ‘11; Affandi et al. ‘14]
• Bioinformatics [Batmanghelich et al. ‘14]
• Neuroscience [Snoek et al. ‘13]
• Wireless or cellular networks modelization [Miyoshi, Shirai ‘14; Torrisi, Leonardi

‘14; Li et al. ‘15; Deng et al. ‘15]

DPPs have become popular in various applications: And they remain an elegant and important tool in probability theory [Borodin ‘11]

Learning DPPs

• Given 𝑍

1, 𝑍 2, … , 𝑍 𝑜 ∼ DPP 𝐿 , estimate 𝐿.

• Approach: Method of moments
• Problem: Is 𝐿 identified ?

iid

Id Identification: 𝓔-similarity

• DPP 𝐿′ = DPP 𝐿 ⇔ det 𝐿

𝐾 ′ = det 𝐿 𝐾 , ∀𝐾 ⊆ [𝑂]

⇔ 𝐿′ = 𝐸𝐿𝐸 for some D = ±1 ±1 ⋱ ±1 .

• E.g.: K =

+ + + + + + + + + + + + + + + + ⇝ 𝐸𝐿𝐸 = + − − + − + + − − + + − + − − +

• 𝐿 and 𝐸𝐿𝐸 are called 𝓔-similar.

← ← ↓ ↓

[Oeding ‘11]

Method of f moments

• Diagonal entries: 𝑳𝒋,𝒋 = ℙ 𝑗 ∈ 𝑍
• Magnitude of the off-diagonal entries:
• Signs (up to 𝓔-similarity) ?

Use estimates of higher moments: 𝐿𝑗,𝑗 = 1 𝑜

𝑙=1 𝑜

𝟐𝑗∈𝑍𝑙 𝐿𝑗,𝑘

2 = 𝐿𝑗,𝑗𝐿 𝑘,𝑘 − ℙ 𝑗, 𝑘 ∈ 𝑍

𝐿𝑗,𝑘

2 =

𝐿𝑗,𝑗 𝐿

𝑘,𝑘 − 1

𝑜

𝑙=1 𝑜

𝟐𝑗,𝑘∈𝑍𝑙

+

det 𝐿

𝐾 = 1

𝑜

𝑙=1 𝑜

𝟐𝐾∈𝑍𝑙

Determinantal Graphs

Definition 𝐻 = 𝑂 , 𝐹 : 𝑗, 𝑘 ∈ 𝐹 ⇔ 𝐿𝑗,𝑘 ≠ 0. Examples: 𝐿 = ∗ ∗ ∗ ∗ ∗ ∗ ∗ 𝐿 = ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗

1 2 2 1 3 4 3

Cycle sparsity

• Cycle basis: family of induced cycles that span the cycle space
• Cycle sparsity: length ℓ of the largest cycle needed to span the cycle

space

• Horton’s algorithm: Find a cycle basis with cycle lengths ≤ ℓ in

𝑃 𝐹 2𝑂 ln 𝑂 −1 steps [Horton ’87; Amaldi et al. ‘10]

𝐷1 𝐷2 𝐷1 + 𝐷2

Cycle sparsity

{𝑗,𝑘}∈𝐷

𝐿𝑗,𝑘 Theorem: 𝐿 is completely determined, up to 𝒠-similarity, by its principal minors of order ≤ ℓ. Key: Signs of for each cycle of length ≤ ℓ.

Learning the signs

• Assumption: 𝐿 ∈ 𝒧𝛽, i.e., either 𝐿𝑗,𝑘 = 0 or 𝐿𝑗,𝑘 ≥ 𝛽 > 0
• All 𝐿𝑗,𝑗’s and 𝐿𝑗,𝑘 ’s are estimated within 𝒐−𝟐/𝟑-rate
• 𝐻 is recovered exactly w.h.p.
• Horton’s algorithm outputs a minimum basis ℬ
• For all induced cycle 𝐷 ∈ ℬ

det 𝐿𝐷 = 𝐺𝐷 𝐿𝑗,𝑗, 𝐿𝑗,𝑘

2

+ 2 −1 |𝐷|

{𝑗,𝑘}∈𝐷

𝐿𝑗,𝑘

• Recover the sign of

w.h.p.

{𝑗,𝑘}∈𝐷

𝐿𝑗,𝑘

Main result

Theorem: Let 𝐿 ∈ 𝒧𝛽 with cycle sparsity ℓ and let 𝜁 > 0. Then, the following holds with probability at least 1 − 𝑜−𝐵: There is an algorithm that outputs 𝐿 in 𝑃 𝐹 3 + 𝑜𝑂2 steps for which 𝑜 ≳ 1 𝛽2𝜁2 + ℓ 2 𝛽

2ℓ

ln 𝑂 ⇒ min

𝐸

𝐿 − 𝐸𝐿𝐸 ∞ ≤ 𝜁 Near-optimal rate in a minimax sense.

Conclusions

• Estimation of 𝐿 by a method of moments in polynomial time
• Rates of estimation characterized by the topology of the

determinantal graph through its cycle sparsity ℓ.

• These rates are provably optimal (up to logarithmic factors)
• Adaptation to ℓ.