Discriminative Regularization for Latent Variable Models with - - PowerPoint PPT Presentation

Discriminative Regularization for Latent Variable Models with Applications to Electrocardiography

Andrew C. Miller

with Ziad Obermeyer, John P . Cunningham, and Sendhil Mullainathan

Poster #53

Black-Box Predictors

(e.g. revascularization, future afib diagnosis, high troponin)

V1 II

Pr(y = 1 | x)

m(x)

discriminative/ predictive model

Ac Accuratel urately as assess (s (som

• me)

) disease ri risk sk. Wh What is the algorithm “s “seeing”? ”?

0.0 0.2 0.4 0.6 time (seconds) −1.0 −0.5 0.0 0.5 1.0 mV Pr(y = 1 | ˜ x) = 0.07 = 0.24 = 0.40 = 0.55 = 0.73

Gradient-based Explanations

rxm(x)

Follow naive gradient (e.g. saliency maps)

https://ai.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html

Wa Want to explore re realistic pre redictive feature res. Need Need t to mo model th the stru tructu ture in x Gr Gradient ent i is a s a l local o

• bjec

ect; i ; igno nores es gl globa bal structure in .

x ∈ X

θ

x

z

N

Generative Latent Variable Models

Observed data (e.g. Image, EKG) Per-observation (low-dimensional) latent variable Global variables

Structure vs. Noise e.g. probabilistic PCA ”structure” z ”noise”

Deep Generative Models:

Unsupervised Models Drawbacks

ST elevation How to make z sensitive to subtle features? maximum likelihood preserves features with high variability – like non-linear PCA

x

˜ x

gθ(·)

hλ(·)

µ, ln σ

z

Discriminatively Regularized VAE (DR-VAE)

Ldiscrim(θ, λ) = DKL(m(x) || m(˜ x))

Ldr-vae(θ, λ) = Lelbo(θ, λ) + β · Ldiscrim(θ, λ)

DR-VAE Objective

x

˜ x

gθ(·)

hλ(·)

µ, ln σ

z

m(x)

m(˜ x)

typical VAE obj.

• discrim. regularizer

EKGs: Discriminative-Generative Tradeoff

0.10 0.15 0.20 0.25 0.30 0.35

• discrim. rmse

0.06 0.07 0.08 0.09 0.10 0.11 0.12

• gen. rmse

VAE DR-VAE β = 1 DR-VAE β = 5 DR-VAE β = 10 DR-VAE β = 100 K=100 K=50 K=30 K=20 K=10

Age

E[(x − ¯ x)2]

E[(m(x) − m(¯ x))2]

(gen error) vs (discriminative error)

0.02 0.03 0.04 0.05 0.06 0.07

• discrim. rmse

0.07 0.08 0.09 0.10 0.11 0.12 0.13

• gen. rmse

MACE Little trade-off for flexible DGMs

Model-Morphs

0.0 0.2 0.4 0.6 time (seconds) −0.5 0.0 0.5 mV Pr(y = 1 | ˜ x) = 0.09 = 0.27 = 0.44 = 0.62 = 0.79 0.0 0.2 0.4 0.6 time (seconds) −1.0 −0.5 0.0 0.5 1.0 mV Pr(y = 1 | ˜ x) = 0.08 = 0.27 = 0.44 = 0.63 = 0.81

Model free DR-VAE

z(t+1) ← z(t) + δ · ∂m ∂x ∂x ∂z (z(t)) ˜ x(t+1) ← gθ(z(t+1))

Generate “morphing trajectory”, follow model gradient in latent space

More details at the poster!

• Technical details, application, and quantitative comparisons

Poster #53 (Wednesday night)