Optimal transport for Gaussian mixture models Yongxin Chen, Tryphon - - PowerPoint PPT Presentation

▶
optimal transport for gaussian mixture models
SMART_READER_LITE
LIVE PREVIEW

Optimal transport for Gaussian mixture models Yongxin Chen, Tryphon - - PowerPoint PPT Presentation

Feb 01, 2024 7 likes •358 views

Optimal transport for Gaussian mixture models Yongxin Chen, Tryphon T. Georgiou and Allen Tannenbaum Presented by: Zach Lucas Intro and Motivation A mixture model is a probabilistic model describing properties of populations with

slide-1
SLIDE 1

Optimal transport for Gaussian mixture models

Yongxin Chen, Tryphon T. Georgiou and Allen Tannenbaum Presented by: Zach Lucas

slide-2
SLIDE 2

Intro and Motivation

A mixture model is a probabilistic model describing properties of populations with subpopulations. To study OMT on certain submanifolds of probability densities. To retain the nice properties of OMT, herein, an explicit OMT framework on Gaussian mixture models is used. Data is sparsely distributed among subgroups. The difference between data within a subgroup is way less significant than that between subgroups.

slide-3
SLIDE 3

Gaussian Mixture Model (GMM) Learning

Unsupervised clustering based on naive Bayes

slide-4
SLIDE 4

GMM: Expectation - Maximization (EM)

slide-5
SLIDE 5

GMM: Expectation

slide-6
SLIDE 6

GMM: Maximization

slide-7
SLIDE 7

GMM: 2D example

https://www.youtube.com/watch?v=B36fzChfyGU

slide-8
SLIDE 8

OMT Background

slide-9
SLIDE 9

OMT Background: Kantorovich

Coupling The unique optimal transport T is the gradient of a convex function

slide-10
SLIDE 10

OMT Background: Kantorovich

The optimal coupling based on the transport map T in (2), where Id is the identity map. The square root of the minimum of the cost defines a Riemannian metric on , known as the Wasserstein metric . On this Riemannian-type manifold, the geodesic curve is given by Displacement Interpolation

slide-11
SLIDE 11

Gaussian marginal distributions

Denote the mean and covariance of Let X, Y be two Gaussian random vectors associated with respectively. Our new cost from (1) becomes

slide-12
SLIDE 12

Gaussian marginal distributions

The constraint is semidefinite constraint, so the (6) is a semidefinite programming (SDP). It turns out that the minimum is achieved by the unique minimizer in closed-form: With minimum value

slide-13
SLIDE 13

Gaussian marginal distributions

Displacement Interpolation as a Gaussian: Wasserstein Distance can be extended to singular Gaussian distributions

slide-14
SLIDE 14

OMT for GMM

Space of distributions: We view it as a discrete distribution on the Wasserstein space of Gaussian distributions:

slide-15
SLIDE 15

OMT for GMM

The discrete OMT problem:

slide-16
SLIDE 16
slide-17
SLIDE 17
slide-18
SLIDE 18

Geodesic

slide-19
SLIDE 19
slide-20
SLIDE 20

Notes

This is due to the fact that the restriction to the submanifold induces suboptimality in the transport plan. d is a very good approximation of W2 if the variances of the Gaussian components are small compared with the differences between the means. Only (9) must be solved to compute a new distance, which is extremely efficient with small distributions

slide-21
SLIDE 21

Barycenter of GMM

slide-22
SLIDE 22

Barycenter of GMM

Solve with fixed point iteration: Remark: unrealistic to solve (14) for more than 3 dimensions for both general and gaussian distributions

slide-23
SLIDE 23

Barycenter of GMM

Modified problem: Let as a discrete measure on

slide-24
SLIDE 24

Barycenter of GMM

The optimal v is gaussian. Denote the set of all such minimerzers For some probability vector The number of element N is bounded above by

slide-25
SLIDE 25

Barycenter of GMM

Barycenter with

slide-26
SLIDE 26

Numerical Examples

slide-27
SLIDE 27
slide-28
SLIDE 28

Geodesic

slide-29
SLIDE 29
slide-30
SLIDE 30
slide-31
SLIDE 31
slide-32
SLIDE 32
slide-33
SLIDE 33
slide-34
SLIDE 34

Barycenter

slide-35
SLIDE 35