[PPT] - Compositional Methods for Learning and Inference in Deep PowerPoint Presentation

SLIDE 1

Compositional Methods for Learning and   Inference in Deep Probabilistic Programs

Jan-Willem van de Meent

Hao Wu Eli Sennesh Heiko Zimmermann Sam Stites

SLIDE 2

Deep Learning Success Stories

Ingredients for success

1. Abundance of (labeled) data and compute
2. A well-defined general notion of utility

14M images (ImageNet) Annotations available Very large corpora of text  (can self-supervise) 4.9M games (Self-play) Clear definition of success

Computer Vision Natural Language Reinforcement Learning

SLIDE 3

Do we still need models?

The Bitter Lesson Rich Sutton, March 13, 2019 The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin. Do we still need models or   just more data and compute? Max Welling, April 20, 2019 When you need to generalize to   new domains, i.e. extrapolate away   from the data, you will need a   generative model https://staff.fnwi.uva.nl/m.welling/ wp-content/uploads/Model-versus- Data-AI-1.pdf http://www.incompleteideas.net/ IncIdeas/BitterLesson.html

SLIDE 4

Do we still need models?

SLIDE 5

When are models useful?

Recommendation Large collection of small-data problems

We need inductive biases that

1. Improve generalization
2. Safe-guard against overconfident predictions

Science & Engineering High quality models and/or limited data Autonomous Vehicles Generalization to long tail events

SLIDE 6

Deep Probabilistic Models

Deep Learning

High-capacity models
Scalable to large datasets
Easy to try new models

Probabilistic Programming

Programs as inductive biases
Structured, interpretable
Also easy to try new models

SGD + AutoDiff   (very general) Monte Carlo Methods (more model specific) Stochastic Variational Inference (learn proposals using neural networks)

SLIDE 7

Structured Variational Autoencoders

[Kingma, Mohamed, Jimenez-Rezende, Welling, NIPS 2014]

Infer y from pixels x,  and z from y and x Inference Model (Encoder)

z x y

qφ(y, z | x) q(x)

<latexit sha1_base64="kwov9h8dYG1x2QhFDTYq6WBf2l8=">AF/3icfZRLb9QwEIBd6EJZXi0cuazYSxGrKmlLWyEOFaiCY6n6kupV5Tizu1YdJ7Wdviwf+An8Cm6IG0LiAD+Df0OcjcQmjvBKm5Hnm5dn7CjTOkg+DN36/Z8587dhXvd+w8ePnq8uPTkUKW5pHBAU57K4go4EzAgWaw3EmgSQRh6Po7J3TH12AVCwV+/o6g2FCxoKNGCW62DpdfHl+irMJW8YX5toOesXnxvZwmInXtkXPfy6d748lU8X+8FKUK6eL4SV0N9G07V7ujT/E8cpzRMQmnKi1EkYZHpoiNSMcrBdnCvICD0jYzgpRESUENTVmV7Ne1+ODSjVGgQtGZmSKISoifepoNVfZdOisAg62GrzaFxXmJQbCzqVlFiu10cw6g4TIzE0c8B2v23r+1JhsrA3C1U3bQCTEFRFuBYPi1wTGEkBUyNb6INzY8pkslxmHf1DgMJeNBAGXNE0SImKDL4Dak+J8MAiVS3CFGBwlph9az14ihY2pb6LZ5VX1hcS7DsfBO7nsFcoeXwNKGbNl83HnbehmE9AU1astftNMlb2Nxjcx+SHiSbGUJrTMgU46nw6hnN0OWcjPygfIapeuxc8uLaxsTzmE3a8eLaeuxeozN71o3LEHkOCFn3GagSQ6le7SXTI94SxhWplKb30rJv5vVeibwXZsfSjdfxSZHeuRNOLlYNbPzp9QKuM656pswcayjk0b1wJmDbA6YEcW713YfN184XB1JVxbefVxvb/9pnr5FtAz9BwtoxBtom30Ae2iA0TRZ/Qd/UK/O586XzpfO9+m6K25yuYpq3Oj7/b9jLg</latexit>

Assume independence  between digit y and style z Generative Model (Decoder)

z x y

pθ(x | y, z) p(y)p(z)

<latexit sha1_base64="+UXAQ6ZeCoF1fnHJT3XwyVfaI=">AGCnicfZTNbtQwEIDd0oWy/LVw5LJiL620qpJStRXiUIEqOJaqf1KzWjnO7K5VJzG20z/Lb8Aj8BTcEJwQN3gB3gY7G4lNHOFIyWjmz974pgzKlUQ/FlYvLPUuXtv+X73wcNHj5+srD49kXkhCByTnOXiLMYSGM3gWFHF4IwLwGnM4DS+eOvsp5cgJM2zI3XDYZjiSUbHlGBlVaOVbT6K1BQUXosu9bXpRSlNela8MQP3uTXrvehVj6+VqvWZYJWjlX6wEZSr5wthJfT30GwdjFaXvkdJToUMkUYlvI8DLgaiwUJQxMNyokcEwu8ATOrZjhFORQlw2aXs16FA71OM8UZKTmpnEqU6ymntLBsq4lU5sYRD1tpRxqFyUBSdZ3StOTbcbJTC2m1WpOYFWD04bs3RgeD7ZeDcHPHNBABSUWEu8HAPk1gIgCyCtndGoTbuz7DC8EZ/IMCh7lqBGRwRfI0xVmio0sg5tzuTwSZLAS4RnQUp7ofGmM8eIZan9LejeaN10brqFZgOSBN7GYOc42Wc9KEbti3XrYxzZsNp0t1at2GhctbOGxhQ8JDxLNCqE1J3BJWZ5/Yzn6HJOxn5SNsdUZ+xCMvsHJ9iLyKftOJ9Sjz1snMyhceMyT2AxSbE95yjnILDKhfvprqiaMpSJXVlN74Xzf7vZe3NZPumPpTuHcd63gkiVk5mPW98yeUiKTOuS5bsImoY7ODawF5A6w2JH2vgubt5svnGxuhFb+sNXfe13dfMvoOXqB1lCIdtAeo8O0DEi6DP6gX6h351PnS+dr51vM3RxofJ5hmqr8/Mv+kc3QA=</latexit><latexit sha1_base64="+UXAQ6ZeCoF1fnHJT3XwyVfaI=">AGCnicfZTNbtQwEIDd0oWy/LVw5LJiL620qpJStRXiUIEqOJaqf1KzWjnO7K5VJzG20z/Lb8Aj8BTcEJwQN3gB3gY7G4lNHOFIyWjmz974pgzKlUQ/FlYvLPUuXtv+X73wcNHj5+srD49kXkhCByTnOXiLMYSGM3gWFHF4IwLwGnM4DS+eOvsp5cgJM2zI3XDYZjiSUbHlGBlVaOVbT6K1BQUXosu9bXpRSlNela8MQP3uTXrvehVj6+VqvWZYJWjlX6wEZSr5wthJfT30GwdjFaXvkdJToUMkUYlvI8DLgaiwUJQxMNyokcEwu8ATOrZjhFORQlw2aXs16FA71OM8UZKTmpnEqU6ymntLBsq4lU5sYRD1tpRxqFyUBSdZ3StOTbcbJTC2m1WpOYFWD04bs3RgeD7ZeDcHPHNBABSUWEu8HAPk1gIgCyCtndGoTbuz7DC8EZ/IMCh7lqBGRwRfI0xVmio0sg5tzuTwSZLAS4RnQUp7ofGmM8eIZan9LejeaN10brqFZgOSBN7GYOc42Wc9KEbti3XrYxzZsNp0t1at2GhctbOGxhQ8JDxLNCqE1J3BJWZ5/Yzn6HJOxn5SNsdUZ+xCMvsHJ9iLyKftOJ9Sjz1snMyhceMyT2AxSbE95yjnILDKhfvprqiaMpSJXVlN74Xzf7vZe3NZPumPpTuHcd63gkiVk5mPW98yeUiKTOuS5bsImoY7ODawF5A6w2JH2vgubt5svnGxuhFb+sNXfe13dfMvoOXqB1lCIdtAeo8O0DEi6DP6gX6h351PnS+dr51vM3RxofJ5hmqr8/Mv+kc3QA=</latexit><latexit sha1_base64="+UXAQ6ZeCoF1fnHJT3XwyVfaI=">AGCnicfZTNbtQwEIDd0oWy/LVw5LJiL620qpJStRXiUIEqOJaqf1KzWjnO7K5VJzG20z/Lb8Aj8BTcEJwQN3gB3gY7G4lNHOFIyWjmz974pgzKlUQ/FlYvLPUuXtv+X73wcNHj5+srD49kXkhCByTnOXiLMYSGM3gWFHF4IwLwGnM4DS+eOvsp5cgJM2zI3XDYZjiSUbHlGBlVaOVbT6K1BQUXosu9bXpRSlNela8MQP3uTXrvehVj6+VqvWZYJWjlX6wEZSr5wthJfT30GwdjFaXvkdJToUMkUYlvI8DLgaiwUJQxMNyokcEwu8ATOrZjhFORQlw2aXs16FA71OM8UZKTmpnEqU6ymntLBsq4lU5sYRD1tpRxqFyUBSdZ3StOTbcbJTC2m1WpOYFWD04bs3RgeD7ZeDcHPHNBABSUWEu8HAPk1gIgCyCtndGoTbuz7DC8EZ/IMCh7lqBGRwRfI0xVmio0sg5tzuTwSZLAS4RnQUp7ofGmM8eIZan9LejeaN10brqFZgOSBN7GYOc42Wc9KEbti3XrYxzZsNp0t1at2GhctbOGxhQ8JDxLNCqE1J3BJWZ5/Yzn6HJOxn5SNsdUZ+xCMvsHJ9iLyKftOJ9Sjz1snMyhceMyT2AxSbE95yjnILDKhfvprqiaMpSJXVlN74Xzf7vZe3NZPumPpTuHcd63gkiVk5mPW98yeUiKTOuS5bsImoY7ODawF5A6w2JH2vgubt5svnGxuhFb+sNXfe13dfMvoOXqB1lCIdtAeo8O0DEi6DP6gX6h351PnS+dr51vM3RxofJ5hmqr8/Mv+kc3QA=</latexit><latexit sha1_base64="AnF+U5gmvKZR/Rp3+LdcpQrED24=">AGCnicfZTNbtQwEIDd0oWy/LVw5LJiL620qpJStRXiUIEqOJaqf1KzWjnO7K5VJzG20z/Lb8Aj8BTcEJwQN3gB3gY7G4lNHOFIyWjmz974pgzKlUQ/FlYvLPUuXtv+X73wcNHj5+srD49kXkhCByTnOXiLMYSGM3gWFHF4IwLwGnM4DS+eOvsp5cgJM2zI3XDYZjiSUbHlGBlVaOVbT6K1BQUXosu9bXpRSlNela8MQP3uTXrvehVj6+VqvWZYJWjlX6wEZSr5wthJfRtQ5Gq0vfoyQnRQqZIgxLeR4GXA01FoSBqYbFRI4Jhd4AudWzHAKcqjLBk2vZj0Kh3qcZwoyUnPTOJUpVlNP6WBZ15KpTQyinrZSDrWLkoCk6zuFaem240SGNvNLivTScwKMPrw3Rujg8H2y0G4uWMaiICkIsLdYGCfJjARAFmF7G4Nwu1dn+GF4Az+QYHDXDUCMrgieZriLNHRJRBzbvcngkwWAlwjOopT3Q+NMR48Q61Pae9G8Zro3VUK7AckCZ2M4e5Rs5aUK3bFuPexjGzabzpbqVTuNixa28NjCh4QHiWaF0JoTuKQsz7x+xnN0OSdjPymbY6ozdiGZ/YMT7EXk03acT6nHjZO5tC4cZknsJik2J5zlHMQWOXC/XRXVE0ZTamSurIb34tm/ey9mayfVMfSveOY71vPJLErBzM+t75E0pEUudcly3YRNSx2cG1gLwBVhvsSHvfhc3bzRdONjdCK3/Y6u+9rm6+ZfQcvUBrKEQ7aA+9RwfoGBH0Gf1Av9DvzqfOl87XzrcZurhQ+TxDtdX5+ReoeDcA</latexit>

Digit y Style z Goal: learn “disentangled” representation for y and z

SLIDE 8

Deep Probabilistic Programs

https://github.com/bleilab/edward

Edward

https://github.com/uber/pyro

Pyro Probabilistic Torch

https://github.com/probtorch/probtorch

Inference Model (Encoder) Generative Model (Decoder)

class Encoder(torch.nn.Module): def __init__(self, x_sz, h_sz, y_sz, z_sz): # intializes layers: h, y_log_weights, ... ... def forward(self, x, y_values=None): q = probtorch.Trace() h = self.h(x) y = q.concrete( self.y_log_weights(h), 0.66, value=y_values, name='y') hy = torch.cat([h, y], -1) z = q.normal(self.z_mean(hy), self.z_std(hy), name='z') return q class Decoder(torch.nn.Module): def __init__(self, x_sz, h_sz, y_sz, z_sz): # intializes layers: h, x_mean, ... ... def forward(self, x, q): p = probtorch.Trace() y = p.concrete(self.y_log_weights, 0.66, value=q['y'], name='y') z = p.normal(0.0, 1.0, value=q['z'], name='z') h = self.h(torch.cat([y, z], -1)) x = p.loss(self.bce, self.x_mean(h), x, name='x') return p

SLIDE 9

Learned Representations (Unsupervised)

Slant Width Height Style 1 Style 2 Thickness Style 3

Style Variables Generalization

[Esmaeli, Wu, Jain, Bozkurt, Siddharth, Paige, Brooks, Dy, van de Meent, AISTATS 2019]

Inductive Bias: Style features are uncorrelated with digit label, as well as with other features.

SLIDE 10

Model Composition

[Siddharth*, Paige*, van de Meent*, Desmaison, Wood, Goodman, Kohli, Torr, NIPS 2017]

based on

rk

Recurrent Recognition Loop

model architecture, the encoder uses a recurrent

based

frame

Reconstructions Decomposition

recurrent network to repeat-

Idea: Embed model for individual MNIST digits in a recurrent model for multiple object detection

SLIDE 11

ψi,u: A ⨉ K

xi,u,s: V zi,u,s: A

ρi: A ⨉ K

x i: V xu: V hi: H hu: H

*

ωi,u,s: A ρu: A ⨉ K

hi,u,s: H

ψi,u,s: A ⨉ K

xi,u,s: V Item Encoder Sentence Encoder Sentence Decoder

*

Broadcast Product

User Encoder

ρi,u: A ⨉ K

c

Concrete Dist

c c

⨉

Element-wise Product

⨉

Learn aspect-based representations of   users, items, and reviews (fully unsupervised)

[Esmaeli, Huang, Wallace, van de Meent, AISTATS 2019]

Example: Modeling Aspects in Reviews

SLIDE 12

Example: Modeling Aspects in Reviews

[Esmaeli, Huang, Wallace, van de Meent, AISTATS 2019]

Amber brown in color with very little head but a nice ring. Nicely carbonated. Smells like a camp fire, malts have a good sweet character with an abundance of smoke. Taste is quite good with smokiness being pungent but not overwelming. A sweet tasting bock with smokiness coming through around mid drink with a smooth mellow

finish. A good warming smoky beer.

Aspects: Look, Mouthfeel, Aroma, Taste, Overall Data: Beer reviews

SLIDE 13

Example: Modeling Aspects in Reviews

Appearance Aroma-Taste Palate Semantic golden black roasted citrus mouthfeel mouthfeel lagers try yellow tans coffee grapefruit bodied watery heineken hype white glass vanilla pine smooth rjt macro recommend

range

pour chocolate hops carbonation bodied import

verall

hazy head bourbon lemon medium refreshing euro founders color pitch

ak

floral drinkability carbonation lager favorite gold lacing malts clove drinkable crisp bmc stouts copper color sweet malt alcohol dry worse stout straw brown aroma aroma finish finish bad ipa amber ginger malt grass mouth thirst skunky cheers Food Service Ambiance Payment [Esmaeli, Huang, Wallace, van de Meent, AISTATS 2019]

Model learns topics that group into aspects  (fully unsupervised)

SLIDE 14

Where are we today?

Object recognition (~3 objects)

[Eslami et al, NeurIPS 2016]

Object tracking (~3 objects, ~10 frames)

[Kosiorek et al, NeurIPS 2018]

Product Reviews (~10 sentences, ~10 aspects)

[Esmaeli et al, AISTATS 2019]

Structured VAEs work well for O(10) variables

SLIDE 15

Still Hard: Clustering

Data Inference

We need methods that scale to > O(100) variables

N=60 N=80

Data Inference

SLIDE 16

Paths towards Scaling up Inference

1. Generative Model: Exponential family + Neural likelihood
2. Inference: Variational EM + Gradient descent

[Johnson, Duvenaud, Wiltschko, Adams, Datta, NIPS 2016] [Wiltschko, Neuron 2015]

Example: Deep State Space Models for Activity Recognition in Mice

SLIDE 17

Amortized Gibbs Samplers

Intuition: Same idea as Gibbs sampling or Variational EM, but learn proposals for conditional updates

[Wu, Zimmermann, Sennesh, Le, van de Meent, arXiv 2019]

SLIDE 18

Minimizing the Inclusive KL

Variational Autoencoders Wake-Sleep / EP Methods

argmin

φ

KL

qφ(z | x)
pθ(z | x)
<latexit sha1_base64="Fxzkf5QWYlUQGPsZeFZ9T13z3vM=">AJI3icfZbditGFMeV9COu+5FNc9kbUVOwizHSbrJZEhaW3YUsqTbkE0CHmNGo2Nr8Ggkj0a7tgf1bQp9lVLoRSm0vei7dEayU0vjUD2aOZ3/uec0fgcBymjmfS8f27d/uDjz6+0/qk/eln39xd+/el6+zJBcELknCEvE2wBkwyuFSUsngbSoAxwGDN8HszKy/uQKR0YS/ksURjGecjqhBEs9Nd47RVhMY8rHaOL+5KLn5wqR+UrN9XMXrVwU09BFi15RKBeRdLFS6RjN6yvjvY438MrLtQf+etA5carYnzvzq8oTEgeA5eE4Swb+l4qRwoLSQmDo3yDFJMZngKQz3kOIZspMpkC7e2+sofqUnCJXBSM1M4zmIsI2vSwFl9lkTaMYi62/XkSBmVEDI65XWrIC7abRTCRG98GZn6DovZRS5SnYEi8XJWKG9weNjXG9D3yvgoLAtTln+jq9QbVPxD3fgLyH8X90rxf0b4bMl5nVtc/s30M8EAK+F7r1X/QeB+RS2o9nK1+RaNwmDMtOXz0413D86Pv7jxqoTDZlYR/pFX6XhOYVjGWyNGDvn94ZDPp+h1sIM9gOpv3CQFgWUisjbicE2SOMY8VOUvAC8KhTbr5sBdUxkxGlOZqfV6YVtR/n4rvV46vgJSuWUgEdZPSN9tJGBbUE8VQ32eEfAsF2AOnkJBwsJsGesv1fELE0PDqDLRtuW6cVbhbhYlQkYaytwuS65BEF0mevUcTNLb3q7UomjmubSYpcWsLGZlMZHFIJB4R0p5HcybQvOGkIxKnTo0aUBpRG1PrAExXTdDbHOy6RHnN251nqbvtrquct5QMZ8EM3VubdXz82K43zgIEhZS6QV9hCaya162ix6bIhwy0K9fyNr4sas6+wUSdBrJXlP+6Y5AgkA9teJ4cUPELyzybAcpTpLOBEgwfB1A9NLyqOrh416tT7nDdqCN+VnJ25r6tbRTfw+Y5YNpV8l/zcjqYq4zthW7s4ptItnqKpLNVzfxbRKjQLTEsf+Rmebhp9sd06IG5rE3Um1360I8CWGYRTiF48q+Hwp83aecg3C1u2Nd0HXDdUuJnquQ7pWMuTqiJ83wdQ+MqxKTikTqMraJaF1udP3m63eHrzeH/gHg4c/PuicbP4GtJyvnK+druM7j5wT53vnwrl0iPOL87vzl/N36+fWb60/Wn9W6O1ba5v7Tu1q/fsfR3VPxQ=</latexit>

argmin

φ

KL

pθ(z | x)
qφ(z | x)
<latexit sha1_base64="ml9C1iyw8czhdz/HSuCj/PIguQ=">AJI3icfZbditGFMeV9COu+5FNc9kbUVOwizHSbrJZEhaW3YUsqTbkE0CHmNGo2Nr8Ggkj0a7tgf1bQp9lVLoRSm0vei7dEayU0vjUD2eOZ3/uec0egcBymjmfS8f27d/uDjz6+0/qk/eln39xd+/el6+zJBcELknCEvE2wBkwyuFSUsngbSoAxwGDN8HszKy/uQKR0YS/ksURjGecjqhBEs9Nd47RVhMY8rHaOL+5KLn58pFJF2sVDpG8y5auSimoYsWvaJQiMxXaq7J+vx4r+MNvPJy7YG/HnROnOq6GN+78ysKE5LHwCVhOMuGvpfKkcJCUsKgaKM8gxSTGZ7CUA85jiEbqTLZwq2tvJHapJwCZzUzBSOsxjLyJo0cFafJZF2DKLudj05UkYlhIxOed0qiIt2G4Uw0RtfRqa+w2J2kYtUZ6BIvJwVyhscHvb1BvS98j4KGyLU5a/4ytU21T8wx34Swj/V/dKcf9G+GyJeV3b3P4N9DMBwGuhe+9V/0FgPoXtaLbyNbnWTcKgzPTls1MN9w8P+v7+o4ZqKEx2JeEfaZW+1wSmVYwlcvSg7x8e2Uy6fgYbyDOYjuYbN0lBYJmIrI04XJMkjEPVfkG4IU+35t1c+CuqYwYjanM1Hq9sK0of7+VXi8dXwGp3DKQCOtfSN9tJGBbUE8VQ32eEfAsF2AOnkJBwsJsGesv1fELE0PDqDLRtuW6cVbhbhYlQkYaytwuS65BEF0mevUcTNLb3q7UomjmubSYpcWsLGZlMZHFIJB4R0p5HcybQvOGkIxKnTo0aUBpRG1PrAExXTdDbHOy6RHnN251nqbvtrquct5QMZ8EM3VubdXz82K43zgIEhZS6QV9hCayax62ix6bIhwy0I9fyNr4sas6+wUSdBrJXlP+6Y5AgkA9teJ4cUPELyzybAcpTpLOBEgwfB1A9NjyqOrh416tT7nDdqCN+VnJ25r6tbRTfw+Y5YNpV8l/zcjqYq4zthW7s4ptItnqKpLNVzfxbRKjQLTEsX3KzPNw0+OuaVED87M3Um1360I8CWGYRTiF48q+Hwp83aecg3C1u2Nd0HXDdUuJnquQ7pWMuTqiJ83wdQ+MqxKTikTqMraJaF1udP3m63eHrzeH/gHg4c/PuicbP4GtJyvnK+druM7j5wT53vnwrl0iPOL87vzl/N36+fWb60/Wn9W6O1ba5v7Tu1q/fsfRJNPxQ=</latexit>

p(z | x) q(z | x) (exclusive) q(z | x) (inclusive)

SLIDE 19

Minimizing the Inclusive KL

Variational Autoencoders Wake-Sleep / EP Methods

− ∇

|
|
= −∇φ Eqφ(z|x)

 log qφ(z | x) pθ(z | x)

<latexit sha1_base64="NRP6H2sY6s2vzw7Cc2SvRs6Qxt4=">AJsXicfZbfb9s2EMfldltr7Ve6Pu5FmLHCGTxDilM3yBAgaBZ0QIMuK5q2g2l4FHW2BVOUTFGJbUL7/a21/03IyW7tUS7BGR5Oe+vKOoO/sJDVPhuv817t3/7PMvHjxs2l9+9fU3x48+u5tGmecwA2Jaczf+zgFGjK4EaGg8D7hgCOfwjt/dqHn390CT8OYvRHLBIYRnrBwHBIs1NDo4N8nPyOGfYpHaOz87aCXVxKR+UrO1XMbrRwUhYGDFod5Lh1EksVKJiM0r87YCA2OEzG0n6D5PMPBmf1R0anNroc7RO1EYWxGKhbPLHRmGOyl3R0k/t94OFkqnwYHbTcrls0x+x4607r3Crb9ejRg39QEJMsAiYIxWk68FwVjMRchISCUs5SDCZ4QkMVJfhCNKhLY+dyqzb7yhHMdMACMVM4mjNMJiagxqOK2OkqlaGHh12fXgUGqVANJwqpWfpTbNgpgrI5B4Zn8FfPZdcYTFYEk0XKWS7fb73fUBnTc4ur1ctPiOc0+8CWqbEr+6Q78NQf1d1C3NsLXywxq2ry9tDv+ArOK6+0n13zlmE9j2ZiteHWvVJPCLSF+/eK7gTr/X8Y6e1VQDrqMrCO9EqXTcOjApfSyQk+O1z8xmWT9DjaQqzHlzY9OnADHIuapjRjckTiKMAskwnwS4Uu0WZeH7i7UExpGIUilev53LQK2aet1Hyx8C2QclkKAmH1hNSlPiDYFlRD+UCdZwQszTjogyeRH9MgXUbqJlteXnx1VaPSRNkW83qxEnfSaczFVEGp06bxHXCiktZhNQYd9PZqt3KR1+NcGszSYFYGszKYqcEgEHhHSFkVzOpC85qQmBY6VWhcg5JpaK5EaxBVWTzAJifqK+Js71ZnSfJhq6sqVzUV/UswlVfGVr28ygdHtYMgYCGkmijSd1u/bAed6mwcUFCvn4tK/9SRraO8zNCHdfnLHY74vrw0/Hi1x+NXBnmxgxRCXsSMcBCg+aqBLirF0VXdWr5an/MabcCb9LMTN7X2a2ka/h8hy+bTL5Lfm56U6bxnbCpXSTxjSdbNUWEs1XF/CdEQq5KYlB85Hp6sPnrcdbWJaqrHw+HsqzV64ZYHMAgneIEzkr7TsDxXSdkDLijljtTCV0VXKeQOFQlXtVKSh3l0S9191UNjMoUk/BYqDS28WidblTx9+ql3uy8Pep6ve7TP45b5u/AQ+t760frLblWc+sc+s369q6sUij30ANaIybveafzb+afonea6xtHluV1pz9D8AzfCM=</latexit>

− ∇φ KL

pθ(z | x)
qφ(z | x)

<latexit sha1_base64="KbsmZQP3CHWbFcU9P9knjc2hNM=">AKE3icjZbdbts2FMfl7qO19FmvdyNMGOFPXiGlKRp0CJA0CxogQZdVjRtAdMwKIq2BFMfpqjENqG9xYC9y4BdDAXaXewB9jY7lOTUEu1iBGTR5O/8zyFnSMnZn4iLOvfxo1Pv3s85u3mq0vzq69t3dr5lUQpJ/SCRCzibxycUOaH9EL4gtE3Mac4cBh97UxP1PzrS8oTPwpfikVMhwGehP7YJ1jA0GinYd/7EYXYXiExuavJnp2Jk1E4vlSxiM06CliQLfNdG8m2USkdlSzoCsjrcQGuzHYti6h2azFLtHrQ+aLXQ6ktsEW4jRsRjALZq0Jhjsh01VdseAfcnHkSgRfK/3F9vQBnJRicrH6M7batv5c3UO3bZaR8bRTsf7dz8G7kRSQMaCsJwkgxsC2KUmAufMArCaUJjTKZ4QgfQDXFAk6HMH25mVmZf2kM5jkJBQ1IxkzhIAiw8bVDBSXWUeOCY8qrbcnAolYpLE38SVq2cIGu1kEvHcNDyORPmE/PUx7DCiQJFtNMWv2Dgx5sQM/Kr729TLd4zNJrvkDBpuDvb8BfUPeDupWL21vhkwUOq9rqsrfQTzilYSV06PqP3McTuh6NGvrVWutmrhOvtIXTx4D3DvY69m7D2qLlerywn7EFR6Vh2YFDHmyOF+z41Jm4fAYryFIYRPO9GcWUYxHxBE4vSJREODQlQjzSYDn8Dav5tWBu/KFx/zAF4ks5zPdyg8/bgXzueNLSgq3jAqE4R+C94fui4IQ9kAzjOiYZJyqg6eRE7E3GQRwE27Sx/satGhQnY5vPKWYGbiRdx4QGUmB0WXVFOIC12q2tQi173dinWX2dC41ZaMxSY5Ya42kMogJvWFJaBdO60KwmJLxcpwqNa1Ds+bonVoMY1AkX65yoe8Tp1q1O4/h6q6sqZzUV9Uswk2faVj07ywa7tYMg6FxImMhTdEc9bBM9VLnYZRQePxeV/kNTtnezIkF36/KnGwJxHmqxfF8S8TPNfJkAymEPIlCwqmgiq8aqAqUH13o1vJVec5rtAav0s9GXNcus1tB1/DZhlhWmXyT/EyPpkjG2FdO0/iq0jWaorwp8uK+Q+I+BxKopu/5Gp6sPq4OeqoEtVXf7tDWXwNlA2FkUsHiYdjelTY91yOr3p+GFJugrsjSOhQcM1cogsfEVArGTMhokf18KEGBkWKiXkI2tIirTDR/u17q9c6r3b6917/y37ePUZcMv41vjO6Bi28cA4Np4a58aFQRq/N/5svGu8b/7W/KP5V/Ntgd5olDZ3jUpr/vMfkxOhdw=</latexit>

− ∇φ KL

qφ(z | x)
pθ(z | x)

<latexit sha1_base64="NRP6H2sY6s2vzw7Cc2SvRs6Qxt4=">AJsXicfZbfb9s2EMfldltr7Ve6Pu5FmLHCGTxDilM3yBAgaBZ0QIMuK5q2g2l4FHW2BVOUTFGJbUL7/a21/03IyW7tUS7BGR5Oe+vKOoO/sJDVPhuv817t3/7PMvHjxs2l9+9fU3x48+u5tGmecwA2Jaczf+zgFGjK4EaGg8D7hgCOfwjt/dqHn390CT8OYvRHLBIYRnrBwHBIs1NDo4N8nPyOGfYpHaOz87aCXVxKR+UrO1XMbrRwUhYGDFod5Lh1EksVKJiM0r87YCA2OEzG0n6D5PMPBmf1R0anNroc7RO1EYWxGKhbPLHRmGOyl3R0k/t94OFkqnwYHbTcrls0x+x4607r3Crb9ejRg39QEJMsAiYIxWk68FwVjMRchISCUs5SDCZ4QkMVJfhCNKhLY+dyqzb7yhHMdMACMVM4mjNMJiagxqOK2OkqlaGHh12fXgUGqVANJwqpWfpTbNgpgrI5B4Zn8FfPZdcYTFYEk0XKWS7fb73fUBnTc4ur1ctPiOc0+8CWqbEr+6Q78NQf1d1C3NsLXywxq2ry9tDv+ArOK6+0n13zlmE9j2ZiteHWvVJPCLSF+/eK7gTr/X8Y6e1VQDrqMrCO9EqXTcOjApfSyQk+O1z8xmWT9DjaQqzHlzY9OnADHIuapjRjckTiKMAskwnwS4Uu0WZeH7i7UExpGIUilev53LQK2aet1Hyx8C2QclkKAmH1hNSlPiDYFlRD+UCdZwQszTjogyeRH9MgXUbqJlteXnx1VaPSRNkW83qxEnfSaczFVEGp06bxHXCiktZhNQYd9PZqt3KR1+NcGszSYFYGszKYqcEgEHhHSFkVzOpC85qQmBY6VWhcg5JpaK5EaxBVWTzAJifqK+Js71ZnSfJhq6sqVzUV/UswlVfGVr28ygdHtYMgYCGkmijSd1u/bAed6mwcUFCvn4tK/9SRraO8zNCHdfnLHY74vrw0/Hi1x+NXBnmxgxRCXsSMcBCg+aqBLirF0VXdWr5an/MabcCb9LMTN7X2a2ka/h8hy+bTL5Lfm56U6bxnbCpXSTxjSdbNUWEs1XF/CdEQq5KYlB85Hp6sPnrcdbWJaqrHw+HsqzV64ZYHMAgneIEzkr7TsDxXSdkDLijljtTCV0VXKeQOFQlXtVKSh3l0S9191UNjMoUk/BYqDS28WidblTx9+ql3uy8Pep6ve7TP45b5u/AQ+t760frLblWc+sc+s369q6sUij30ANaIybveafzb+afonea6xtHluV1pz9D8AzfCM=</latexit>

− ∇

|
|
= −∇φEpθ (z|x)

 log pθ(z | x) qφ(z | x)

⇥

⇤

<latexit sha1_base64="KbsmZQP3CHWbFcU9P9knjc2hNM=">AKE3icjZbdbts2FMfl7qO19FmvdyNMGOFPXiGlKRp0CJA0CxogQZdVjRtAdMwKIq2BFMfpqjENqG9xYC9y4BdDAXaXewB9jY7lOTUEu1iBGTR5O/8zyFnSMnZn4iLOvfxo1Pv3s85u3mq0vzq69t3dr5lUQpJ/SCRCzibxycUOaH9EL4gtE3Mac4cBh97UxP1PzrS8oTPwpfikVMhwGehP7YJ1jA0GinYd/7EYXYXiExuavJnp2Jk1E4vlSxiM06CliQLfNdG8m2USkdlSzoCsjrcQGuzHYti6h2azFLtHrQ+aLXQ6ktsEW4jRsRjALZq0Jhjsh01VdseAfcnHkSgRfK/3F9vQBnJRicrH6M7batv5c3UO3bZaR8bRTsf7dz8G7kRSQMaCsJwkgxsC2KUmAufMArCaUJjTKZ4QgfQDXFAk6HMH25mVmZf2kM5jkJBQ1IxkzhIAiw8bVDBSXWUeOCY8qrbcnAolYpLE38SVq2cIGu1kEvHcNDyORPmE/PUx7DCiQJFtNMWv2Dgx5sQM/Kr729TLd4zNJrvkDBpuDvb8BfUPeDupWL21vhkwUOq9rqsrfQTzilYSV06PqP3McTuh6NGvrVWutmrhOvtIXTx4D3DvY69m7D2qLlerywn7EFR6Vh2YFDHmyOF+z41Jm4fAYryFIYRPO9GcWUYxHxBE4vSJREODQlQjzSYDn8Dav5tWBu/KFx/zAF4ks5zPdyg8/bgXzueNLSgq3jAqE4R+C94fui4IQ9kAzjOiYZJyqg6eRE7E3GQRwE27Sx/satGhQnY5vPKWYGbiRdx4QGUmB0WXVFOIC12q2tQi173dinWX2dC41ZaMxSY5Ya42kMogJvWFJaBdO60KwmJLxcpwqNa1Ds+bonVoMY1AkX65yoe8Tp1q1O4/h6q6sqZzUV9Uswk2faVj07ywa7tYMg6FxImMhTdEc9bBM9VLnYZRQePxeV/kNTtnezIkF36/KnGwJxHmqxfF8S8TPNfJkAymEPIlCwqmgiq8aqAqUH13o1vJVec5rtAav0s9GXNcus1tB1/DZhlhWmXyT/EyPpkjG2FdO0/iq0jWaorwp8uK+Q+I+BxKopu/5Gp6sPq4OeqoEtVXf7tDWXwNlA2FkUsHiYdjelTY91yOr3p+GFJugrsjSOhQcM1cogsfEVArGTMhokf18KEGBkWKiXkI2tIirTDR/u17q9c6r3b6917/y37ePUZcMv41vjO6Bi28cA4Np4a58aFQRq/N/5svGu8b/7W/KP5V/Ntgd5olDZ3jUpr/vMfkxOhdw=</latexit>

 |

= Epθ (z|x)

⇥ ∇φ logqφ(z | x) ⇤

<latexit sha1_base64="KbsmZQP3CHWbFcU9P9knjc2hNM=">AKE3icjZbdbts2FMfl7qO19FmvdyNMGOFPXiGlKRp0CJA0CxogQZdVjRtAdMwKIq2BFMfpqjENqG9xYC9y4BdDAXaXewB9jY7lOTUEu1iBGTR5O/8zyFnSMnZn4iLOvfxo1Pv3s85u3mq0vzq69t3dr5lUQpJ/SCRCzibxycUOaH9EL4gtE3Mac4cBh97UxP1PzrS8oTPwpfikVMhwGehP7YJ1jA0GinYd/7EYXYXiExuavJnp2Jk1E4vlSxiM06CliQLfNdG8m2USkdlSzoCsjrcQGuzHYti6h2azFLtHrQ+aLXQ6ktsEW4jRsRjALZq0Jhjsh01VdseAfcnHkSgRfK/3F9vQBnJRicrH6M7batv5c3UO3bZaR8bRTsf7dz8G7kRSQMaCsJwkgxsC2KUmAufMArCaUJjTKZ4QgfQDXFAk6HMH25mVmZf2kM5jkJBQ1IxkzhIAiw8bVDBSXWUeOCY8qrbcnAolYpLE38SVq2cIGu1kEvHcNDyORPmE/PUx7DCiQJFtNMWv2Dgx5sQM/Kr729TLd4zNJrvkDBpuDvb8BfUPeDupWL21vhkwUOq9rqsrfQTzilYSV06PqP3McTuh6NGvrVWutmrhOvtIXTx4D3DvY69m7D2qLlerywn7EFR6Vh2YFDHmyOF+z41Jm4fAYryFIYRPO9GcWUYxHxBE4vSJREODQlQjzSYDn8Dav5tWBu/KFx/zAF4ks5zPdyg8/bgXzueNLSgq3jAqE4R+C94fui4IQ9kAzjOiYZJyqg6eRE7E3GQRwE27Sx/satGhQnY5vPKWYGbiRdx4QGUmB0WXVFOIC12q2tQi173dinWX2dC41ZaMxSY5Ya42kMogJvWFJaBdO60KwmJLxcpwqNa1Ds+bonVoMY1AkX65yoe8Tp1q1O4/h6q6sqZzUV9Uswk2faVj07ywa7tYMg6FxImMhTdEc9bBM9VLnYZRQePxeV/kNTtnezIkF36/KnGwJxHmqxfF8S8TPNfJkAymEPIlCwqmgiq8aqAqUH13o1vJVec5rtAav0s9GXNcus1tB1/DZhlhWmXyT/EyPpkjG2FdO0/iq0jWaorwp8uK+Q+I+BxKopu/5Gp6sPq4OeqoEtVXf7tDWXwNlA2FkUsHiYdjelTY91yOr3p+GFJugrsjSOhQcM1cogsfEVArGTMhokf18KEGBkWKiXkI2tIirTDR/u17q9c6r3b6917/y37ePUZcMv41vjO6Bi28cA4Np4a58aFQRq/N/5svGu8b/7W/KP5V/Ntgd5olDZ3jUpr/vMfkxOhdw=</latexit>

Difficult: Need to approximate  expectation that depends on φ  Solution: reparameterization

r REINFORCE-style estimators

Easier: Expectation depends

n θ (generative parameters)

Solution: sample from pθ(z | x)

SLIDE 20

Reweighted Wake-sleep Style Methods

−∇φKL

pθ(z | x)
qφ(z | x)
= Epθ (z|x)

î ∇φ log qφ(z | x) ó

<latexit sha1_base64="R0BThqbzvpZdfwJkTHJoD98coXc=">AJeHicfZtb9s2EMfl7qGx9pRuL/dGmDHMHlxDcro06BAgSBZ0QIMuK5q2gGkYlERbhClKpqjENqHPsc+0j7CvMmDAjpLdWqJTAbJp8nf/u6OoO/spo5l03X9aDz759LPHx607S+/Orbw4fsmS3IRkJsgYl45+OMrJjaSkXepIDj2GXnrzy/0+tbIjKa8NdylZJxjGecTmAJUxNDv96jDj2GZ6gqYNeXCkHBelyrdIJWnTR2kExDR207BWFQsFirRbA1ecd+9Sx0eVE3Wdo3M6G9kf3NiIJTMHPbP3K5b8eHLYcQdueTnmwNsMOmdWdV1PHj38G4VJkMeEy4DhLBt5birHCgtJA0ZANs9IioM5npERDmOSTZW5Q5CErur72xmiZcEh7UzBSOsxjLyJjUcFafDSJwTETd7WZyrLRKSDI643UrPy5sG4VkCk+zjEz9hsX8OhcpZKCeDUvlDs4Pu7DBvTd8j46KkyLc5a/5ysUbCr+lz34KxJ+UHdLce9e+GKFeV1b3949HNBCK+F7n5U/Q+B+YzsRrOTr861bhL6Zavnp8D3D8+6nvDpw3VUOjsSsI7AZW+2wRmVYwlcvKk7x2fmEy6eQZbyNUYRPOjk6REYJmIDI4uQuSOMY8VAiLWYyX8Nps1/WBu6MyYjSmMlOb9cK0ovzjVrBeOr4lQeWEYkw/EJw20iQXUGYKkZwnhHhWS6IPngK+QkLs1UMX6rjFTqGhlFlArblunZW4U4WJUJGAGVOlyV3RARQe3r1HTSu95u1bJo5rkymJXBrA1mbTCRwSAi8Z6U8jqYN4UWDSEZlTp1aNqA0oianlgDYlCMQ2xysukR5/dudZ6m7e6rnLVUNGfAWbqytiqF1fFaNg4CJIspYIFOEJT2dUPG+qyrsQhI/D4hayNnzmqMyQoLNI9pryl3sC8X1acTx8p6IXxrkxR5SnWR8EAQSTRfN9AdqDy6MGzUq805b9AGvC0/e3FTe1PdKrqBL/bEsq3k+QXZjRVGd8Lm9plEd9GstNTJ2va+Y/o4AKaIlh+ZLr5dH2H8RpV7eogf7ZGyvb2bkQT0IyiKcktPKvh8KfNenBPhgLtTKOjQcJ1SoucoBL2SMQci+rUZPvTAuCoxqUgklLFtRJtyA83fa7Z6c/BmOPCOBsM/n3TOzjd/Aw6s760frK7lWU+tM+t369q6sQLrv1an1W89Pvi37bR/avcq9EFrY/OdVbvaw/8B/JNn9Q=</latexit>

∇θ log pθ(x) = Epθ (z|x) î ∇θ log pθ(x, z) ó

<latexit sha1_base64="nVHsr3Mh0Sm/s7K2+ae9+9efS+4=">AJUHicfZbfb9s2EMeV/lgd7UfT7XEvwowB9mAEktOlQYsAQbKgAxZ0WdG0BUzDoCTaIkxRCkUltgn9gXsdsIf9JwP2sB0lObNEpwJkU7zPfe9I0Xf2U0Yz6bp/7Tx4+OjxZ086u/bnX3z51dO9Z1+/z5JcBOQqSFgiPvo4I4xyciWpZORjKgiOfUY+PMzbf9wQ0RGE/5OLlMyjvGM0ykNsISpyV6COPYZnqBrB7Fk5qAgXagUHnto0S/sY8dG5xOlZ1f19MpBMQ0dbS5sdEpnI/tOw640XtqbDouBg1b9ih1P9ruvltejnw6kH3xKquy8mzJ7+jMAnymHAZMJxlI89N5VhIWnACMjmGUlxMczMoIhxzHJxqrcmcJpWN95YzVNuCQ8aLgpHGcxlpExqeGsORtEJiIZth6cqy0SkgyOuNLz8ubBuFZApvqcxM/YTF/DIXKaxABfFyXih3/BwABswcMv74KAwPU5ZfsdXKPhU/I9b8Lck/F/dLcW9e+GzJeZNbX179CvBSG8kbr7SfVfBeYzspnNxnr1WpsuoV+u9O3rU4AHhwcDb/ipRoKvbqS8I5AZeC2gVmVY4kcPR94h0cmk9bvYA25GoNsvneSlAgsE5HB8Sa3QRLHmIcKYTGL8aJQaG3XB+6WyojRmMpM1fbC9KL8015gLwPfkKAKy4hEGJ4Q3DYSZFMQpoRnGdEeJYLog+eQn7CwmwZw5fqeoXOoeVUuYBvadfBKtzJokTICKDM6bHklogAakq/uQa96M1oN2pRtNe5NJilwawMZmUwkcEgIvGWJeVNMG8LXbeEZFTqNKFpC0ojakZiLYhBkQ2xycl2RJzfu9V5mt5tdVPloqWiPwPM1IWxVb9cFKNh6yBIspAKDHCEprKnXzYUZV2Q0bg9QvZGL90VHdYIEFnkey35c+3JOL76tzI4809Gb8xyLMtpJTqLOGBIJovumgm0l5dGHYqlf1OW/RBrwuP1txU7ubhXdwq+35LKu5Nvkr81sqjK+FTa1yK+zmSjp0g6XzXcf0ABFdASw/JHrs2j9T+D45uUfv6sT9WtrNxIZ6EZJRFOCXHlf8gFPh2QDknwoFwx1DQoeE6pUTfUQh6JWMOZPSqnT70wLgqMalIJSxdUZ1uYHm7VbvTl4P9z3DvaHvz3vnpzWfwM61rfWd1bP8qwX1on1s3VpXVmB9af1z86jncedPzp/d/7d3anQB/W39Y3VuHbt/wAD+luS</latexit>

Approximate with any importance sampler 

(lots of probabilistic programming methods available,  can use learned qφ(z | x) as proposals)

SLIDE 21

Reweighted Wake-sleep Style Methods

Importance Sampling Use proposals qφ(z | x)  to sample from pθ(z | x) Variational Inference Learn proposals qφ(z | x) using samples from pθ(z | x) Better gradient estimates Better proposals Inference methods from  probabilistic programming Automatic differentiation  and neural networks

[Bornschein and Bengio, ICLR 2015] [Le, Kosiorek, Siddarth, Teh, Wood, UAI 2019]

SLIDE 22

Example: Unsupervised Tracking

Inferred Locations Reconstructions Fully unsupervised (learns sub-model for MNIST) Scales to O(100) frames and ~5 digits (possibly more)

SLIDE 23

Thinking Compositionally

Algorithm 1 Amortized Population Gibbs Sampling

1: for n in 1, . . . , N do 2:

Gφ = 0

3:

xn ∼ pDATA(x)

4:

for l in 1, . . . , L do

5:

zn,1,l ∼ qφ(z | xn)

6:

wn,1,l ← pθ(xn, zn,1,l) / qφ(zn,1,l)

7:

for k in 2, . . . , K do

8:

˜ z, ˜ w = zn,k1, wn,k1

9:

for b in 1, . . . , B do

10:

˜ z, ˜ w = RESAMPLE(˜ z, ˜ w)

11:

for l in 1, . . . , L do

12:

˜ z0 l

b ∼ qφ(· | xn, ˜

zl

b)

13:

˜ wl =

pθ(xn,˜ z0 l

b ,˜

zl

b) qφ(˜

zl

b|xn,˜

zl

b)

pθ(xn,˜ z l

b ,˜

zl

b) qφ(˜

z0 l

b |xn,˜

zl

b) ˜

wl

14:

˜ zl

b = ˜

z0 l

b

15:

Gφ = Gφ + PL

l=1 ˜ wl P

l0 ˜

wl0 d dφ log qφ(˜

zl

b | xn, ˜

zl

b)

16:

zn,k, wn,k = ˜ z, ˜ w

17: return Gφ, z, w

. Output: Gradients,

APG combines inference from

probabilistic programming  with SGD-based methods

Known building blocks (SMC

and RWS), but not trivial   to combine correctly

Can we define compositional

methods for importance sampling  and gradient estimation?

What (inference) DSL could define this sampler?

[Wu, Zimmermann, Sennesh, Le, van de Meent, arXiv 2019]

SLIDE 24

Static vs Dynamic Models

further

Recurrent Recognition Loop

model architecture, the encoder uses a recurrent

def detect_objects(image, canvas):

bjects = []

while not similar(image, objects):

bjects.append(

next_object(image, canvas)) return objects, canvas def next_object(image, objects): area = detect(image, objects) features = recognize(image, area) return area, features

SLIDE 25

further

Recurrent Recognition Loop

model architecture, the encoder uses a recurrent

def detect_objects(image, canvas):

bjects = []

while not similar(image, objects):

bjects.append(

next_object(image, canvas)) return objects, canvas def next_object(image, objects): area = detect(image, objects) features = recognize(image, area) return area, features

Dynamic Computation Graphs: Number of variable nodes   is data-dependent and/or stochastic

Static vs Dynamic Models

SLIDE 26

further

Recurrent Recognition Loop

model architecture, the encoder uses a recurrent

Static Computation Graphs: Set of nodes and dependency graph determinable from static analysis.

def detect_objects(image):

bjects = []

for k in range(3):

bject =

next_object(image, canvas) if not similar(image, objects):

bjects.append(object)

return objects def next_object(image, objects): area = detect(image, objects) features = recognize(image, area) return area, features

Static vs Dynamic Models

SLIDE 27

Anglican Venture WebPPL Birch Gen Turing

Static vs Dynamic Models

Prob. Prog.

Stan* Infer.NET

(*dynamic graphs, static support)

Static Graphs Dynamic Graphs

(**in non-eager mode)

Deep Learning

**

SLIDE 28

Programs as Importance Samplers

Weight Output Trace Program Input Unnormalized Density Prior Conditioning

Factor

γf (ξ; x) = ωf (ξ, x) pf (ξ; x)

z = ζf (ξ, x)

Output Computation

ξ = [α17! x1, ... ,αN 7! xN]

Address Value

z,ξ, w

;

f (α, x)

Address

SLIDE 29

Programs as Importance Samplers

γf (ξ; x) = ωf (ξ, x) pf (ξ; x)

z = ζf (ξ, x)

w = ωf (ξ; x)

ξ ∼ pf (·; x)

(measure semantics) (likelihood weighting semantics)

[Ścibior, Kammar, Vákár, Staton, Yang, Cai, Ostermann, Heunen, Gharamani POPL 2018]

Weight Output Trace Program Input

z,ξ, w

;

f (α, x)

Address

SLIDE 30

Model Combinators

[Sennesh, Ścibior, Wu, van de Meent, (arXiv)]

f f f … [x₁ , x₂ , x# ] z₁ ,τ₁ ,w₁ z₂ ,τ₂ ,w₂ z# ,τ# ,w# Input Model Output

z ⇤ [z1, . . . , zN] τ ⇤ [α1 7! τ1, . . . , αN 7! τN] w ⇤ w1 · w2 · . . . · wN

<latexit sha1_base64="GWMAgtldKVzasq9doZr5CPKaHcg=">AGbXicfZRb9MwFMfTsZVRbhuIJxCK1gl4qKakm7YJCWkCTexpGohdpLqnOS0DXMSYzu9zPIH5ZUvwFfASaOtsScKXHO+Z3/cZzjE1ASc+F5vxsrD1bXmg/XH7UeP3n67PnG5osLnuUshPMwIxm7CjAHEqdwLmJB4IoywElA4DK4/lL4LyfAeJylP8ScQj/BozQexiEW2jTY+IPCEZPoRrnu09ur3ob+KrjIhJlgutnZTtVfRehFgoDIpFQFc8iTChY6xjXJRgykXmVowhc4e2uitOtXJnKqFiu6ey/1Q621bOve2sosNqAzDTba3o5XDte+NWkfeQsxtlgc5WjKAvzBFIREsx5z/eo6EvMRBwSUC2Uc6A4vMYj6OlpihPgfVn+DOW2lt0yhSmdCZgJZsTLMaqjuOEL6yGcZilgtetBGtZPk/q1iCpK/ZgRjNWLD76mXMRZLPCH8FQ10u5YBkFJAclv3/9rKTX2d/t+N0DZSAMorwD72OvkxgxADSCjnc6/j7hzZDc0YJ3EFegRWrYaC3I8ySBKe6QiYQqp7flwhSnjMotkOiICOR/lb9kG1fKWUFLUJ0bOnXostemGCiJOroCGBsNgFuY0I4EhX4LZCH82Amaqn8hZmXSZmVvM3GJuLObGYsYWg0Dge74xr4O5KfTLEBLjUseUIQZGdLOI7uGEKYetjEMDoePY2nvMRgku9jOjwLDIWHFaprEYkziJBZeVX9lRcfr/KO03kx0bCyruQSCPlUXqrlOWjJ6V1VkegrJ06hiLalhxEO6hRqxGVcfhHpAaYHUmClK3Kd9sSvbkorvj7+50v+21j7aqhrXuvHa2nA+O7xw4R86Jc+acO2HjpJE2po3Z2t/mq+ab5tsFutKoYl46tdF8/w/dF1E5</latexit>

f f f … [x₁ , x₂ , x# ] τ₁ ,w₁ τ₂ ,w₂ z₁ z#-₁ z₂ z# ,τ# ,w# z ⇤ zN τ ⇤ [α1 7! τ1, . . . , αN 7! τN] w ⇤ w1 · w2 · . . . · wN

<latexit sha1_base64="/TwdK5GMvR5tuWA1YtPCGz1n5t8=">AGV3icfZRtb9MwEMfTsZVRnlZ4yZtonRAvqinpq1CQpAE7yaBmIPUl1VjnNtw5zE2E7XzvI35Avs04CTZtDEFa7UOHe/+59zunPAaCSk5903Nh5tbjUfbz9pPX32/MXLnfarS5FmnMAFSWnKrwMsgEYJXMhIUrhmHAcULgKbj7l/qsZcBGlyXe5YDCM8SJxhHB0phGO78QmXCF7rTrv3gli+jM+0i1EIkoApJnXsGiPBQIUzZFI984xEzJ1S8aYui6iYSqFea6gZzZ6podLdWZyzdStfsj98F7oE6O1auv9tRVZbMBkGu10vH2vWK698ctN58RZrvNRe1OgMCVZDIkFAsx8D0mhwpzGREKuoUyAQyTGzyBgdkmOAYxVEXZtdtadasEbtlcwlxq2xjOdVHMdia0Zx2kiRdVKsZEVi7hqDeKq4gDmLOX54cMfmZBOs/9IYxNZxQHVmFAM9Dq2+ePWndo4Ou3zvWNYRDWBJ+3+uaXx2YcICkRPqHXf+obzMs4zCP8jLsfw0HEw5SBrHODEdMgOiB/5QIUhExiEvh0JBSkPzreahOr7W2gpahpjYwm9EV70w1Qr1DVBYwokr4LaQxRwaDpwT6P39YC5rqafqXmRdJVZWMzCYu4s5s5iphaDQOI135hVwawu9LMmJKeFTl2G1jBqroVwDSfrctjKOK4hbBpZtcd8EuO8nikDjmXK82m5jeSURnEkhSr92o6Kkv9HGX892WntQPl/EKhTbZHm1ilaxuyK7iyGoGidKsbDCpYPwhpqwitUOQ5rQFYDy5nISXN+fVLyd5c9vb9g/3e18POyW5YW07b5xd53jO8fOifPFOXcuHNLoNa4buBFs3W/9bjab20t0o1HGvHYq9n+A8xmSZQ=</latexit>

α₁ α₂ α# α₁ α₂ α# … , … , f f f … x τ₁ ,w₁ τ₂ ,w₂ z₁ z#-₁ z₂ z# ,τ# ,w# α₁ α₂ α# z ⇤ [z1, . . . , zN] τ ⇤ [α1 7! τ1, . . . , αN 7! τN] w ⇤ w1 · w2 · . . . · wN

<latexit sha1_base64="GWMAgtldKVzasq9doZr5CPKaHcg=">AGbXicfZRb9MwFMfTsZVRbhuIJxCK1gl4qKakm7YJCWkCTexpGohdpLqnOS0DXMSYzu9zPIH5ZUvwFfASaOtsScKXHO+Z3/cZzjE1ASc+F5vxsrD1bXmg/XH7UeP3n67PnG5osLnuUshPMwIxm7CjAHEqdwLmJB4IoywElA4DK4/lL4LyfAeJylP8ScQj/BozQexiEW2jTY+IPCEZPoRrnu09ur3ob+KrjIhJlgutnZTtVfRehFgoDIpFQFc8iTChY6xjXJRgykXmVowhc4e2uitOtXJnKqFiu6ey/1Q621bOve2sosNqAzDTba3o5XDte+NWkfeQsxtlgc5WjKAvzBFIREsx5z/eo6EvMRBwSUC2Uc6A4vMYj6OlpihPgfVn+DOW2lt0yhSmdCZgJZsTLMaqjuOEL6yGcZilgtetBGtZPk/q1iCpK/ZgRjNWLD76mXMRZLPCH8FQ10u5YBkFJAclv3/9rKTX2d/t+N0DZSAMorwD72OvkxgxADSCjnc6/j7hzZDc0YJ3EFegRWrYaC3I8ySBKe6QiYQqp7flwhSnjMotkOiICOR/lb9kG1fKWUFLUJ0bOnXostemGCiJOroCGBsNgFuY0I4EhX4LZCH82Amaqn8hZmXSZmVvM3GJuLObGYsYWg0Dge74xr4O5KfTLEBLjUseUIQZGdLOI7uGEKYetjEMDoePY2nvMRgku9jOjwLDIWHFaprEYkziJBZeVX9lRcfr/KO03kx0bCyruQSCPlUXqrlOWjJ6V1VkegrJ06hiLalhxEO6hRqxGVcfhHpAaYHUmClK3Kd9sSvbkorvj7+50v+21j7aqhrXuvHa2nA+O7xw4R86Jc+acO2HjpJE2po3Z2t/mq+ab5tsFutKoYl46tdF8/w/dF1E5</latexit>

f₁ x τ₁ ,w₁ z₂ ,τ₂ ,w₂ z₁ α₁ α₂

z ⇤ z2 τ ⇤ [α1 7!τ1, α2 7!τ2] w ⇤ w1 · w2

<latexit sha1_base64="/NJovywuqrdFiYt7Gs4sz72K+Fo=">AGNXicfZTNbtNAEMed0kAJXykcubhNhThEle1WaYWEiEAVHAuiH1I2itbrSWK6tpfdZp0tS/EC/AsHLghrwCa8cVsR2xkZL1zG/+s57MrM9oKTj/Ghs3Nls3r23db/14OGjx0/a20/PRZJyAmckoQm/9LEAGsZwJkNJ4ZJxwJFP4cK/epf5L2bARZjEn+WCwTDCkzgchwRLYxq1vyEy4QrdaNt+8douHkaethFqIeJThaTOPANEeKAQpmyKR65x76AIMyETtGMXnDF37VXMW495erhUZybXTF3r29y3z7k+CRK5avP0qN1x9p182fWNW2w6fWu5TkfbmwIFCUkjiCWhWIiB6zA5VJjLkFDQLZQKYJhc4QkMzDbGEYihyouq7daqW8VwzeYS5lLXzRGWU13GcSW1opxnMRSlK0UG1mxiMpWPyorDmDOEp4dPviSCukn8wfwNj87/mBVeDTFLT69P6tVk63d9B1vSNdQTgEBeEeO13zqQITDhAXyPFh1+0d1xmWckbhH+RkWHYaDqYcJIkiHJsemAHRA3eoEMQi5ZCVQyE/oYF5V/OjOq7Wuha0DGxud+IrnphqlWqGuCxhRIVgW1hyjgwHTYnkavqgFzXU4/U/M86SqzqDGLGnNTY25qzLTGIJB4zTumZTCtCn2tCMlprlOVoRWMmqEP1nCyKodrGcVhE3DWu0xn0Q4q2fCgGOZ8GxarkM5pWEUSqEKv65HhfH/o4y/muykcqDs2/fVia6R5k7JW8bs8u7MhyBvnTLGgxKWDcIasJLVDEOa0BWAYuZyEhzTbnVS6m+Of23YN97+Nhp79bXFhb1nNr13pudaR1bc+WKfWmUa7Uav8abRb35v/mz+av5eohuNIuaZVrNP38BiYw9Wg=</latexit>

f₂

compose(f₂, f₁) map(f) reduce(f) unfold(f)

Idea: Static graphs with dynamic complexity

SLIDE 31

Sampling Primitives

3 Building Blocks for Importance Samplers Propose Resample Move

SLIDE 32

Inference Combinators

[Sennesh, Ścibior, Wu, van de Meent, (arXiv)]

Sampler Output zg, τg, wg ; g(x) z f , τ f , w f ; f (x; τg)

<latexit sha1_base64="+0yu9azyWTCiLKOTDF2BdPth/k0=">AGEnicfZTdbtMwFMfTscIH9vYJSBV64Q2qZqSbdomuJlAE1wOxD6kuqc5CQNcz6wna6d5WtegKfhDnHLC/AQvANOGthiV7hSe3rO73+Oc+JjLycx47zq7VwZ7F97SfvBw0ePl1dWn5yxrKA+nPoZyeiFhxmQOIVTHnMCFzkFnHgEzr3LN2X8fAyUxVn6kU9zGCQ4SuMw9jFXruHKF+RHVKDrYSR7HeR7RKCxQFzW/3MqrpRpv0AwxqRjR5szfiK3bITsv+JQE4c34vBGHP4Tv9JKbdnDla6z7VSrYxpubXSPrNk6Ga4uMhRkfpFAyn2CGeu7Ts4HAlMe+wSkjQoGOfYvcQR9ZaY4ATYQVcdkx74dFilc5RMOEy5Nd4L5SDZxnLCZV3OGWcpZ0uwSsumSdPrJc2MfZjkGS03H3wqGPeySRkPIFQvtdqwCDxSgBQf3r6Wwunt7/bcnQOpIRSCmnAPnZ76EBEAdIaOdzrufuHJpMXNCdwAzklVu6GgmqHnyUJTgP14sCXfXcgEKSsoFC2QyAvI4F6VvUjuq6U0hDNJEpbxVXS29HyjEiBekoUEvDLogNRAHjGcbEr3UBRPZLD8Wk6robWZqMFODuTaYa4MZGQwCjuc8Y9ECz3RZy0RH1V59DREw4ia6GAOx/V02KgYakg+io3eYxoluOxnlgPFPKPltFzFfETiJOZM1HFpquL0/yoV14sdaxsqvz1PHEuDVHdEdWSUVZ3Oagiqo9PEaNDAykGYQ0W0QdXjMAfMNbCeiZJU15SrX0qmcbaz7e5u7zf6x6t1xfWkvXUWrc2Ldc6sI6sd9aJdWr51u/WutZ63n7a/tb+3v7xwxdaNWaNaux2j/ABTqMUk=</latexit>

z f , τ f , w f ; f (x; τg) zg, τg, wg ; g(x, z f ) z0

f , τ0 f , w0 f

; f (x; τg)

<latexit sha1_base64="jaxWT4nKZC1aAGBnMQBGo1y5qcg=">AGbnicjZRtb9MwEMdT2MoITxtIvEGIap3YkKop2aZtgjcTaALeDcQepLqnOSahjkP2E6XzvIH5TVfgI+AnWascSqEK7Xu9/9fbmc7WUkYtxfrbu3F1abt9buW8/ePjo8ZPVtadnLM2pD6d+SlJ64WEGJErglEecwEVGAcegXPv8oOn0+AsihNvFpBoMYh0k0inzMlWu4+gv5IRXoejiSvQ7yPSLQRCAuq/8ZFVfKtF8jmGDSsUdbM76Q72p0KN/YCNk3YqEhFt6Khbdi4V8xHb0pY15o0yxrc6uzf8tbLjadbadcnWahlsZ3SNrtk6Ga0sMBamfx5Bwn2DG+q6T8YHAlEc+AWmjnEG/UscQl+ZCY6BDUT5NmTHng+LBK6ygkPBZdMdYz6WdRzHbOY1nKM04azuJVjJsmlc93pxXbEPRZSXzwPWfcSwsdD2CkBqYsWAQeyUGKrx/fS+H09nd7s6BNBAKQUW4h05PfUwgpABJhRzu9dz9wyaT5TQjcAs5GtPVUFDt8NM4xkmgXhz4su8OBIKE5R0OwTyUhKoZ1U/outKRtJsxSVW8aV6HxUz4gUqKeSRgR83QWxgQjgPF0Q6K3ZkIh69tPRFuOs9MG8y0wVw3mOsGM24wCDhe8Ix5HcxNoR+GEB+XOqYMTCibotgAcdNOdzYcWQg2Thq9B7TMa6n2kGFPOU6tNyFfExieKIM1HFZTMrSv6dpeLmZsdGQfrb8SxbJDqjihHRlnldJaHoBydOkaDGqYPwgIqpDWqOg4LwMwAqzOhSXVNueal1DTOdrbd3e2dL3vdo/XqwlqxXljr1pblWgfWkfXJOrFOLb/1uZW2itZ0+Xf7eftl+9UMvdOqcp5ZtdXe+gPI5FDT</latexit>

z ⇤ z f τ ⇤ τ f w ⇤ pg(τg; τ f , x) γg(τg; x) · w f · wg

<latexit sha1_base64="jmJUIhFSTxsoeSfx5f2tKihQT7Y=">AGhHicfZRb9MwFMdTGU2waPvFTrhIZUTck2tgkEmkATPA7ELlJTVU5ykoY5cWY7XTrLj7zC5+Oj8IadRqOJK1ypPf2f3zk+vhz7OU4Yd5zfnTt3V+7Z91cfdB8+evzk6dr6szNGChrAaUAwoRc+YoCTDE5wjFc5BRQ6mM49y8/av/5FChLSPaNz3IYpSjOkigJEFfSeO2PF8RUeDey+/Jdr1v/GUdS2VdXBQqV5GPhTYXHpRIr5lZQXNfzlJTrVwrojfPE1EUiHwcby3SsXzb6/ZuRzPRoOnSdZTylaxV4cUoTdGShLdoT9UShIRrfKGk+Vq0Y1GMZXe81ne2nWr0TMOtjf6RNR8n4/UV5oUkKFLIeIARY0PXyflIMqTAIOavmCQo+ASxTBUZoZSYCNRnZGqYdEtMrjOSw4l6acIj6RTRylbK62xIhknDVjFRaNkubqp82Mw6hzAnVxYfC8Z9Ump/CJG6RlXBIvRxAVJ8/fRBCmewvztwdw5kC6EQ1oR76AzUpw3EFCrkcO9gbt/aDJ5QXM/yBHY7oaCmo7AqKOPQvVsUEgh+5IeJCxgoLeDuH5BIdqrepH9F0pRE0D1GxlV8lXfTCFGEpvIEKijAEehfEpocBhYyTem9aQeUsjn9VJTVpIvMzGBmBnNjMDcGMzEYDzhasaiCRbtRFetRHxS5WmnwS0MqzckXMLxdjpkzBi1kHySGHuPaJwivZ8kB4o4obpbrhM+wUmacCZqvzSjkuz/Ucrfnuy4VZD+9n1xLA1SPS7VlVFWdTurJqiuThOjYQPTjbCEimDqthCZi3wLonNKmeKbf9KJnG2c62u7u982Wvf7RP1ir1gtrw9qyXOvAOrI+WyfWqRV0xp0fnZ+dX7ZtD+xd+/UcvdOpY5bjWG/wt+AFGc</latexit>

zk, τk, wk ; f (x) k ⇤ 1, . . . , K ak ∼ Discrete(w1, . . . , wK)

<latexit sha1_base64="Djx48EhAo3vTNbdQWN8EHXrfvBA=">AGfHicfZRtb9MwEMdT2MoITxu85E21TmgToUq2auQkCZgAmlvBmIPUl1VTnJtQ504s52uneUvw0fhW/BlEHYaRJNUpFJr3/3uf9eLz35KIi5c91fj3v219eaDjYf2o8dPnj7b3Hp+yWnGArgIKHs2scSJTAhYgEgeuUAY59Alf+5IPxX02B8Ygm38Q8hX6MR0k0jAIstGmw+eOVjYIRk+huMFEtp4UCn0g0lUgobTD7lJn9rdnaCKaY2MPdRchM7dnoJsOhPWm9a3kOIiEV3DmzEbKNLM0j8SKSR3HLRgJmQn6MeMBAgNpdUvdMsoXActIztTfYbLsdN39a9YVXLNon1uI5H2yt/UQhDbIYEhEQzHnPc1PRl5iJKCgi8k4pDiY4BH09DLBMfC+zJupWvayWyZwm85M1apujrEYqzKOY76wVoxDmghethKsZfk8Llv9uKzYg1lKmSk+/J5x4dOZ8Ycw1O87L1iGPslAya+f3ivpOkcHjrd/rCoIg7AgvK7r6E8VGDGApEC6h4531K0zacZSAv8g12BViDKcjP5CRsaApmQGumcBjWOchPrdQqB6Xl8iSHjGwPRMIp+SUDdE/8i2p5SqBS1CdGzu16LXnM0lUSODhoSCEyr5A4igEMu6I5Cb6sBM1VOP5WzPOkyM68x8xpzV2Puasy4xiAQeMV/zMpgVhW6qQiJca5TlSEVjOgbIVzBiaocrmUcVpB0HNV6j9koxqafNAWGBWVmpG4jMSZRHAkuC7+qR0XJ/6O0v5rstFKQ+fZ9eapqpL7L8iOjV/npzCclPzpljIUlzEzLCmrESlQxMyvAtAIWg7OCpBWymB5D6lvPq95x9cXlfsc76Ox/OWyfbBf34b10tq2di3POrZOrM/WuXVhBY2NRqdx3Oiu/27uNF83yzQe40i5oVepHfwDEO1Y3</latexit>

zk ⇤ zak τk ⇤ τak wk ⇤ 1 K

K

’

k⇤1

wk

<latexit sha1_base64="GKpjhB6zmst/ciDC4emRxXu41G0=">AGbHicfZTdTtswFMdTBh3rvmDbHdqEKGJcVFUCNAkJLQJbRI3bBofUtNVjnPaZnXiYDulxfJL7JH2FnuJXe9ydho04lRzpeb0/H/n2D3xOUFKIi5c91dt4cHiUv3h8qPG4ydPnz1fWX1xwWnGMJxjSi7ChAHEiVwLiJB4CplgOKAwGUw+mD0yzEwHtHkq5im0I3RIn6EUZCu3orP3w8YNK/7Y1UY+uocfLx1Q/xIpLaiGf32doVCrATFeXxh346jksEN8ra50bdZe8zhKWn5KnWeRb35OjIU9Oy2Rvpem23XytVw2vMJrHzmyd9VYXf/ohxVkMicAEcd7x3FR0JWIiwgT0XhmHFOERGkBHmwmKgXdlXjy13rgvywRu0omAiVBVd4zEUJVxFPOZ13L2aSJ42UuQTsuncdkbxOWMHZiklJnDh98zLgI6MXoIf1+8wPLMCAZKPnl43sl3db+bsvbOVAWwiAsCO/QbemPDQwYQFIgh3stb/+wyqQZSwn8g1yD2RBlKBncQSaNAc2RGeiaYRrHKAn1qwWsOl5X+pDwjIGpmfQDSkJdEP2QTU+ZO2MFzUJ0bK7rpPdVGCOipN/SQX0C2JRKbvoEUMgF3VT+Oztgosrbj+Uk3/Q+M60w0wpzW2FuK8ywvg0Jz/mJXBzE50bSUSwzyPnYZYGNETIJzDCTsdquzYt5B0GFVqj9gRqaeNAWGBGWmpW4iMSRHAkuC1Vo6Lk/1Fatzc7sQ5kvoNAnqgKqadRfmW0ld/OvFPyq1PGWFjCTLfMoQasRBU9MwdMLbBonDktciewyp5nz7iqcbHT9nbO5/3mscbxfxbdtacDWfb8ZwD59j5Jw5w52/tTe1N7Wtpd+1/V1+qvZ+hCrYh56ZRWfesvMyhTtQ=</latexit>

Input x

propose(f, g)

x

move(f, g) resample(f)

x z ⇤ z0

f

τ ⇤ τ0

f

w ⇤ pg(τg; τ f , x) γg(τg; x, z f ) · w f · wg · w0

f

<latexit sha1_base64="JPHXj/t4U6dfW6zViytC96C05uU=">AGqXicfZRb9MwFMfTMcotw0eEVK1TrCJakq2aZtASBNogscNsYtoqspxTtIw5zLb6dpafuPT8Apfhm+DnYbRxBWu1Jz8z+9c7Nj2MhIxbtu/G0t3lu82763cbz14+Ojxk9W1p+cszSmGM5ySlF56iAGJEjESdwmVFAsUfgwrv6oP0XI6AsSpMvfJBP0ZhEgURlxJg7XGCxeHVLhT2Xr5rt0qX14NAqlerq9z5CvNI8IdCZdLJRbQraDBlusqKaNaulFIe5YpoAiLbBuzuGDUL5t9q3Y94VyG7VpTsZy1ZqsINURyjBQn/ot3SmqpcW23VF/ZTrkPn2ptNTDvmxVAa2mwJBqsde9suRts0nNLoHFmzcTJYW2aun+I8hoRjghjrOXbG+wJRHmECqk7OIEP4CoXQU2aCYmB9UXxIVW/eLRK4ycYcxlyacoz4UFZxFLOZWhODNOGsqhKk0rJXFW9uJqxB+Mspbp5/1vOuJeOtd+HQO21omHheyQHKT5/fC+F3d3f7To7B7KGUPBLwjm0u+pXB0IKkJTI4V7X2T80mSynGYF/kK0x3Q0FtRw4VRsj8dWHAyx7Tl+4kLCcgl4O4Xop8dVc1UN0HCmlETQLUbGFXyWd98IESncrgoKCGC9CmLDJYB8xtMN6b6pB4xltfxIjIui8zEYCYGMzWYqcEMDcYFjhbMa+CeT3RdS0RHxZ56mlIDSPqovEXcLyeDhkVgxqSDSNj7RENY6TXM82AIp5SfVpuIj4kURxJkq/NKOi5P9Ryl8vdlxrSP97njiWBqmun2LKvYncUhKLZOFaN+BdMHYQEV0gpVHocFYFYDyzOhSXVNOfVLyTOd7ad3e2d073O0Xp5Ya1Yz61a9NyrAPryPpknVhnFm58b/xo/Gz8ar5unjYvm19n6FKjHlmVUYT/wF3TF7U</latexit>

Idea: Change the evaluation (sampling) strategy,   whilst leaving the target density unaffected.

SLIDE 33

Dealing with Missing Variables

def g(x0): u = sample(normal(enc_u(x0)), name='u') x1 = sample(categorical(enc_1(u)), name='x1') x2 = sample(normal(enc_2(u)), name='x2')

Proposal (variables: u, x1, x2)

γf (x1, x2, v ; x0) = pf (x0 | v) pf (v | x1, x2) pf (x1, x2)

pg(x1, x2,u; x0) = pg(x1, x2 | u) pg(u; x0)

def f(x0): x1 = sample(categorical(prior_1), name='x1') x2 = sample(normal(prior_2), name=‘x2') v = sample(normal(dec_v(x1, x2)), name='v')

bserve(normal(dec_0(v)), x0, name='x0')

Target (variables: v, x1, x2)

SLIDE 34

Dealing with Missing Variables

def g(x0): u = sample(normal(enc_u(x0)), name='u') x1 = sample(categorical(enc_1(u)), name='x1') x2 = sample(normal(enc_2(u)), name='x2') def f(x0): x1 = sample(categorical(prior_1), name='x1') x2 = sample(normal(prior_2), name=‘x2') v = sample(normal(dec_v(x1, x2)), name='v')

bserve(normal(dec_0(v)), x0, name='x0')

˜ pg(x1, x2,u; x0) = pg(x1, x2 | u) pg(u; x0) pf (v | x1, x2)

˜ γf (x1, x2, v ; x0) = pf (x0 | v) pf (v | x1, x2) pf (x1, x2) pg(u; x0)

Extended Target (variables: v, x1, x2, u) Extended Proposal (variables: u, x1, x2, v)

SLIDE 35

Dealing with Missing Variables

def g(x0): u = sample(normal(enc_u(x0)), name='u') x1 = sample(categorical(enc_1(u)), name='x1') x2 = sample(normal(enc_2(u)), name='x2') def f(x0): x1 = sample(categorical(prior_1), name='x1') x2 = sample(normal(prior_2), name=‘x2') v = sample(normal(dec_v(x1, x2)), name='v')

bserve(normal(dec_0(v)), x0, name='x0')

˜ pg(x1, x2,u; x0) = pg(x1, x2 | u) pg(u; x0) pf (v | x1, x2)

Extended Target (variables: v, x1, x2, u) Extended Proposal (variables: u, x1, x2, v)

˜ γf (x1, x2, v ; x0) = pf (x0 | v) pf (v | x1, x2) pf (x1, x2) pg(u; x0)

SLIDE 36

Dealing with Missing Variables

w f = ˜ γf (x1, x2, v ; x0) ˜ pg(x1, x2,u; x0) = pf (x0 | v) pf (x1, x2) pg(x1, x2 | u)

Proposal Conditioned Target Combinator

z1, 1,w1 ; (x0) z2, 2,w2 ; f (x0 ; 1) z2, 2, w1 · w2 · p(1 ; 2,x0) (1 ; x0) ; propose(f ,)(x0)

Conditioned reverse (ratio for variables common to both models)

SLIDE 37

Model Combinators

[Sennesh, Ścibior, Wu, van de Meent, (arXiv)]

Function Composition Reduce / Fold over Sequence define compositional model structure

z1, τ1, w1 ; g(x0 ; τ0(α1)) z2, τ2, w2 ; f(z1 ; τ0(α2)) z2, [α1 7!τ1, α2 7!τ2], w1·w2 ; compose(f,g)(x0 ; τ0)

<latexit sha1_base64="JSuV1lT5xw4uP+V4WLRnESWekxw=">AJlXicjZbtGEIZptU0i9ZC4vehFbuhKAWSAMEg5cYwmBowcmt7FDeokgCgIy+VIYrQ8dHdpi16wz9MH6k3fprMk1YqHGqUhcT37zT+zwx2uvIQFQtr2X3u9z7/4s7de/3Bl19/c39B/vfvhdxyilc0pjF/KNHBLAgksZSAYfEw4k9Bh8NYv9fyHK+AiKNfZbALCTLKFgElEg0zf3/nQ9WAaRSngcLyQHyAdmcT1yV1kSK5ewZEXmkbk1l7ez+r+uhI0UVBHf5/nYDQWnisZhEgvILTM63IqWmjdzx0IXkur79dzZalcMXBHWMBWCo+VoPNrMbdO1npXu9rhKzkcHebNIJMqyEQHmfzfIAsMgl2BJnUgTRAvikDPdpmG6B25IEiFj9+CfFW7d25OTmWX+fl26UT+WaNeZlsnVqjGvCwTK3DYLEGRkwuRv/MI5w+G9pFdXGZ74FSD4blRXhfz/Tt/uH5M0xAiSRkRYurYiZwpwmVAGe4JNxWQELomS5jiMCIhiJkq9l9uDgaPduZVBNfJRm+IvMeErnKB7t6ioSitDaMiziSom5lBHVFtatXlhXnMImiblO3/+UCunFmzoPXlGqhjQ2WxAtMSBm7cMCG6pYnvI9lkKu3r15kSvbOjm2nMnTvMlw8CvEObUt/GsRS3wUcWcPrack9MOKEl5wuBfytacXtwu9Yrw9UF0jBbY1pHJycWPlbLj7Hx3nb40WxipJ3Kuw2/p1e0VbeLtRLtycd8MuMRHVx/XH+g35TlmInd/tW9becREvYzWZnwTp5LBAH3F7YLCHBTnCvgOZTZ6awL0TKQW8v5Xox83Hv4E0NnVx3TcOpdEHfYh5Fd2d1T+bKtdBpwYDqXaVGLgPiY0OPcvfHpsMmr4e/Upsi6C6TtZisxdy0mJsWs2oxLkjSsca0DqZNod8aQnJV6DRlWANjeOL4HZxsypFWxEUDSVZBq/aEL0Oi6xknwImMuX7/XAdyxYIwkEJV83nbK4hu98L5ZrDXjYT0t+ep13mLpB4rtgyO6m1WbJ86yv0Wqjusg1zyFlk2TAebtNnqzdAB06wF3bgcZt3aoJq85gQgqQCn96BFTIjMGZK+WChAHLGlp4iGFBmX656h8pU1OEJQkYJlrgETgCxjPkoaXPvKdgx1o2mVg6FzgLHxbHOaJ1l78H5y5BwfPflMjx/Xp1y94yHxg/G2HCMp8a58bNxYVwatPesR3qfeuv+9/2z/qv+TyXa26t8vjNqV/t3HeT7E=</latexit>

z1, τ1, w1 ; g(x0 ; τ0(α1)) zn, τn, wn ; f(zn−1 ; τ0(αn)) for n = 2, . . . , N zN, [α1 7!τ1, . . . , αN 7!τN], QN

n=1 wn

; reduce(f,g)((x1, . . . , xN) ; τ0)

<latexit sha1_base64="CNlJ+0BzdoyoQqS3n/Nn85Iq/o=">AKLHicjZbPb+Q0FMfTKSw7w68tHLmkdFaSqFKWrZbAZWqXVbLaSgrurvSpIycxDMTxnGC7bQztdy/hxN/DOKCEFf+Bo48J5kliUNFqpm49ud93/Oznz1BRmIuXPf3rd72W2/fe+d+f/Due+9/8OGDnY9e8jRnIb4IU5Ky1wHimMQUX4hYEPw6YxglAcGvguVTPf7qCjMep/R7sc7wZYLmNJ7FIRLQNd3pLf0Az2MqM5amM8EwVgO7eB76i3WSh+RbIGm1N50l6/T5r+wCvBQ4miKmRn3AWSoajPMTKsen+RrOUvJl6DligXL+vp95GumLwFSKtrkJvOB+Ohqupa/vOl6W5O6qi8/aH+6rthFZOqHZC/6+TGTi5mUr6mac6HEmqTFfF5OXt7Sxlt8qm9ql9CK6jVHDHr9BYzrD7AjiGjtNx/ZkMwvb3/UTlHGR+rtvEtRkK9lWZ2k/Nu3HlwYLCx3BPE49cPYTExXZuz6eo4gQ4Na7E/Go5WxUqWIa2m4/1axos+ZhGtZ01fbDnHrjFY5sNr2rsnVnlcz7dufczqId5gqkICeJ84rmZuJSIiTgksFX9nOMhUs0xNoUpRgfimLslD2YPCwNi4pvs5WeqlUR3+CxEIN6noSJbzsbXOUip4s5cg0OXrpNkbJE3FCV5lKdPhRz/mXATpqsnjoEhVSxrOgJjOwSFEHeEZ1HkxPRkFJMdKvnj+REnXOT5yvMPHqs3AmlWId+I68GcQc1gYWjEnzve8UkHlOUsI/hfytWcnlyd+hqx5XkFhsl6CWEdHB87sKyOW3yOjpRp8aSYRcl7FXYX/0LPaCPvFuql2aMO+Oka0a4/nj/QT8vU1GL3b1T/VuG6BzXo6lNWAcPCWIYtleYJgmCSvCvcKgm3qWEuA5w3p7ST9ISQR7B15yz1O6alpGpQnYFuMgWh/V5aqk74DRjOBQ7yo59AlGERwDQ+V/0TZYqab7K7kqnNaZtcGsDebGYG4MZmEwPhaoY45E8zbQj+1hMSi0GnLkBZG4CKMOjRlkOGx1kLyRaxkXvE5gnS+UwzJBImT5/rmOxIHESCy6rcWVaxfRuKxhvO3vWCkh/B4F8pgwyDEixZaDVLNi+zRFhmorAOcs4MsiyYDjYz2epk6IDtQEXduBpqZuVYRVZRAuOBYSfhHFIRdrgk9IWYoicm6pQU3GiSU6MNV/3a6HRiHqIMO/YS4zDAQx3SctKX4RlOSa60LTK7p63C7hbvPaN5nZeHl4B0dPrucO/sq+qWu29Yn1qjSzPemydWd9Y59aFfZ+7f29vbXd6/S/63/R/PEu1tVTYfW42n/9c/Tvl0aQ=</latexit>

zn, τn, wn ; f(xn ; τ0(αn)) for n = 1, . . . , N (z1, . . . , zN), [α1 7!τ1, . . . , αN 7!τN], QN

n=1 wn

; map(f)((x1, . . . , xN) ; τ0)

<latexit sha1_base64="fzrh8L58mKqoBZn/D8i2iwjpoTs=">AJpHicfZb9s2FMcVb+vq7NZsj3tRZnezASGQkjUNtgUI2hUdhsHLuqUtYLoGJVG2ZuoyknYsE8zn2Vfa87IDiU50cWLAscUz+/8eXjIQ9pNaciFbf+z13nv/Q8efPiwu/Rx598+tmjg89f82TJPHLlJTRhb13MCQ1jciVCQcnblBEcuZS8cRfPtf3NijAeJvEfIkvJMKzOAxCDwvomh7s/YtcMgtjmbIkCQjRO2b+fMYzbM0kQjTdI6nsXluIkHWgnsS+z5TAxRx5skIp8oy4+HWq3DaTGMLcLzU39fgjMgKUzMYrHXb+r6w2YOt+HCozJpAPpS8uQkSdqNMPbgDgn4iuGWO7iJcgXcskigGFPohmOIjyVgRpsvhmq2vtQ3YxgFhx7J8Nc3BZupYhfhmOhpa5q1lXAbomOgQwVS5SNBhEbxTwbaRVToKv1HbzS54xAk3Z/K+NxR70Z5nm4teb5u37apHvSD/nDQhzTepWMNIVdy2tcTRST2Kys6fdSzj+z8MdsNp2z0LoziuZwePgb1L1lRGLhUcz52LFTMZGYidCjsEXQkpMUews8I2NoxjgifCLz7Qgruf+4YpcxuU7Xej3Vjv4Ii7nar+pJHPGit9EZJLHg9V6KQZdnUb3XjeqKY7JOE6bD9/9cuEm6zpP3DxVDWmovTCewYAQtU8CqK98etJ36ZIo+erlMyVt6/TEco6fqibDiF8izpltwV+LmMHCxCVz9q3lnJ7tgNIlSym5o2zN6clVqR8xW1yWoBdlCwjr6PTUgmW17PxzcqLaHs/yWRS8U2L38a/0jLbydq5euD3ZAT/PcFwX1x/nf+iXRSoqsdv3qv/KcDwj1WgqE9bBQ4IYge3lJVD+UAloRTw1diYS6oIvGdHbSyI3oT7sHfiSPUfpqmk4FS7gm9tBtGrVBQonjAVOASWe3lWyjyjBPhR7X6Hvmg5rVR9+Jdf5oFUmazFZi9m0mE2LmbcYRATeMcdlHVw2hf5qCIl5rtOUoQ2MwgXk7+BEUw63RgwaSDoPW7nHbBZhnc8kJQyLhOnz5zoUcxpGoeCytKu2Vxjf7wX25mAvGgHp/64rX6gW6bk03zLQqpdZvn3qKPNbqK6wHeSMtciYHawaZstT4YdsJe14Lxud6BJW7cswrIyKBecCAm/REKPi4yScyREgKOQZg0tuMkgoVQfrvo3y1ink3APp8QyF4SkHA5guEsaXvoCLMox0oWmVQ57ziGMDXeb07zJ2o3Xx0fOydGT3457Fz+Ut9xD40vjK2NgOMZT48L4ybg0rgyv83Mn7WSdTfr7i/d37tXBdrZK32+MGpP91/fGVdIg=</latexit>

Map over Sequence

SLIDE 38

Inference Combinators

Nested Importance Sampling Importance Resampling Application of a Transition Kernel define a compositional sampling strategy

z1, τ1, w1 ; g(x0) z2, τ2, w2 ; f(x0 ; τ1) z2, τ2, w1 · w2 ωg(τ1; τ2, x0) ; propose(f,g)(x0)

<latexit sha1_base64="Kk6ZedCe6ioT2YTap6rHM6H+Z7I=">ALQXicnVbdbtxEFHabUIqh0IAEF9y4ZCtJGtlb2gaUSpFLVW5I1SkrbRercb2sdfs+IeZ2WSd0UQ8Du/BU/AEiDvELTfMjJ3Gf41QJ9rsaM73fedn58yMX+CEMsf548bNre3br1/+wPzw4/ufPzJ3Z1PX9J8TQI4CXKck9c+oCTDE5YwjC8Lgig1Mfwyl89VfZXp0Bokmc/sbKAeYriLImSADG5tNjZ+tXzIU4yXpA8jxgBEDMKBSLa/tidOJDOTasa3rIscl4TECGoFDwQl1Y9zheubXkMrdX32cK1PDhF2IrHm4Wz10J6S1qgAPgU0q7EtJaYKolpLREpCcuzH9XyDTUPsrCO542Ul2QREJcrtYuWvjJW6v31MKEFRiVlJYaW8b786HSCMGdVGrWEfAigLewomuDzW4l6cQo0U8rnKxHr1JWNVJdF1XulZxSeCuUJXw6jrxqXgL8dJdjNIU9dnDnMpbHWI0bruwLoZ9WDK6K2xD/zrCu6XTK987ONKE/5HLtBmYOxSWGhd6i3JxYXkpJYFqpiKnMB5FI9saxaO98UgSR3t6e+rdetVui7u7zsTRw+pP3Hqye2RU43ixc+s3L8yDdQoZCzCidOY6BZvLBmBJgEGY3lp1cLBCMczkNEMp0DnXZ4WwTPN+w84zOCs2DZMDKyniC2F2dTjKXVamcxyjNG26sYSV1apu1VP20rzmBT5ESFH/68pszPN208+LpUHWl5MCZLB3KqEOI5OGn0+Ohj9cg+IvnTwR37IN9250+F0MgbCGuIeOLf96iFj+MFmNOfzadg8OB0DFmhQYrlCOwqnkmqjvEFkd18AgLVcyrMnBgS1/VtvRn/190Wc80VlUeLeGXYd/oTK6lHe0ekV7MAB+WqKsLa4+7lvQz6tSNGJ3rlX/gaAshmY0jYRV8LJABOT2CnJ5JMlO8E4hEDN3zmVf0DUBtb3kHZPjUO4d+cV3XSH7pkuqKJKr7VK0aVXdKLhnS1KEIVC7io8DCikLB8J75suYSPa7k/5RjtYsoepuxhznuY8x5m2cN4wNBAjus2cN0V+qUjxJZapyuDOzAsXwfhAI515VDPY9SBFMukV3tE4hSpeuYFyFdETtT5c5awJU7ShFe20WflWTXs6S96+xZJyD13/f5M9FDBj7W0bO2m2mt08bSsIeVHXYADImPWTVMAPYo+tT4YBcFD2wLpvB6B5X7duwrozMGUGJfPxCTQL5vHmMRShNcdrTk3SULitXhqh6UM1VOoAEqwLZWAIV+sdF5h6WuvKodU9VoSuXerntP+pZ3m9u9yfqTl9OJuz958ON09+jb+pa7bXxpfGWMDd4aBwZ3xvHxokRbP25fWf78+0vzN/Nv8y/zX8q6M0bNeczozXMf/8DaX+0MQ=</latexit>

zk, τ k, wk ; f(α, x) for k = 1, . . . , K a1, . . . , aK ∼ Discrete

w1, ..., wK

zak, τ ak, 1

K

P

k wk

; resample(f,K)(α,x) for k = 1, . . . , K

<latexit sha1_base64="2Zzk5E51nxkS2lUnH7+1kdn3Me0=">AJF3icfZbdjuM0FMfTBZYyfO3AJTdZWqSOFXJlJ0dIVYa7YcWqRcMK2Z2paYzctKT1sROgu3MNGOZ9+ANeAnEHeKWS65hXfATrLafJRJlca1f+fvc5xz7AYZwVy47l+DO2+9/c7d4fv7b3/wYcfXxv/5NznuYshLMwJSl7FSAOBCdwJrAg8CpjgGhA4GUQPzHjL6+AcZwm34sigyVF6wRHOERCd13uD879ANY4kRlL0gwALVn68vfFkqby5ix/YFys3z+iK2fbhCxI4mPiLZBjn29kDZDd4XsBUyStlPyo7tR7anrVep4I49b8qizcD6GJu+xTu7J9inIQIDyCURicm3I6XRqZp/7DK834qBWwkEbKZd1HKxqt183Y4YCqWn5Fxp8Zxexg3vfcpZKBlwRDMCk3E0duzxfHwGVdRmZ/bsQ7M3/AMhSAPgSr7ltBs45EPyaqxiJf3Ru7ULS+73/DqxujEq7Ty/27v2i9MKeQiJAgzhem4mlREzgkOi34uctDsxWsNCNxNEgS9lmQH6Hex90RiXCVxnW+Ow2tFPkdiovaeRJRXvZ3OKE0Eb/cSpHV5Qdu9AW0rLmCbpcy4v/oh5yJIt20egnKpOtI63XGy1hNqr1cQ6ZQuw5OrgOSg5Ivnj5V0naOZ4x0+VF2GwapGvGPX0Z8esdYvJqmZ4y8d7+h4B5TlTGfFG8o1nAmuST1FLD6twZAWsXZrenTk6NfquOU9m6m+xeMyior3auw2/oWJ6LW8W6pXZg92wE8KlLTFze39D/28WoqG7+6t6t8ylKyh6U0jYO8XiAGOr3ClFKkK8G/glAtvKXUdcFzBia9pB+kZKVzRz/kyFNK9YwqE21bjmvR5qipXiV9RxtFBEKTVXKs9wm04iIdK/+rsFWtae/kty0iZT9Jix9z0mJses+kxPgi0I8a8DeZdoR87QmJT6nRlSAcjes9f7eBEVw71Zow6SLbBvbVHbE2RWc80A4ZEysz+c43FhmCKBZf1uOpb4eR2Kz3enexZxyHzHQTymeqRYUDKlNGtdpmV6dNG2aqHmgrbQa5Zj6wKZgeb9dl6Z9gBh0UPLut2B5r2desirCuDcMFBSH345CLgsAjX4gIUyKjpY+7vSCErO5mr8JC7OcwEOUgWPHAFl5yvFlx8qcklU5UlNoRuX+yLuv59Znm9c9yfqN8OpN5s+O5wdPJ1fcoNrc+sz62J5VkPrRPrG+vUOrPCwa+Dvwf/DP4d/jz8bfj78I8KvTOobT61Wtfwz/8AC10/vQ=</latexit>

z1, τ1, w1 ; f(x0) z2, τ2, w2 ; g(z1) z3, τ3, w3 ; f(x0 ; τ2) z3, τ3, w1 · w2 · w3 ωg(τ2; τ3, z1) ; move(f,g)(x0)

<latexit sha1_base64="4ay26WcyuSzKeU4zqSy7yvL6dI=">AKVHicfZbdbts2FMfVZO0ab+ua9nI36uwg9iAEkr2mwboCQbuiu1tWLG2BKDAo6UjWQn2UpD9kgnmJvcSeaMBeZdjFSEperI+GgWOC/J0/znkIe3lOKbMtv+s7P72d17n9/f63x5VcPvn64/+gdzebEh3M/wxn54CEKOE7hnMUMw4ecAEo8DO+9q1dq/v0CI2z9DdW5HCZoCiNw9hHTA5N93f+cD2I4pTnJMtCRgBEz5TNnRV5xtdTxzJdhubqezl1TBcWCJvhcDW1R8KskeOKHCtyXJHRUEqM6pKTCpwocLItabrW80pko35QWi0PzRemGxLkczdCSYKmPBTD9aGbxIG5Gn0crnVvfTgSNeBmvkTXI7GsvInTEMhE+3O9cehaT2aDtgPMmbWx90gpjlGBWUFhvqM9lCHr+1kfKIG6MbdLIEITaNhGav5/P+E3GRr0w5uvKjpdnp28GnXDkrfzHyzLB8LlW+32g0+EZ3LVx6XOW2Ztg2u9X5y0UhZQonPk2wBw0E4sAbRYDQcyB0fjIQydyENtk7g9GHfPrJ1M9sdp+r0T42ynU37/3pBpk/TyBlPkaUXjh2zi45Iiz2sTzS7pxCjvwrFMGF7KYoAXrJdfnIg9Y72JrnKSzFYMVEx3jCWIz0dvW4yih5WhjMxSRujGEldWiT1US+pK17AKs+Icj/4fU6Zl63qPHg6VQ1peVfEaSQXlF4HEMr7QIfHAw/PQfC3b14KblvHE8sZPxNhkBQIc6Jbcm/FhHJjUkr5uR7yzk+6YDyOckx3FC24lRw29RPiFydVaCfFfSraPjY0tuq2Xrz0QWTcvipY6i5J0Ku41/qyLayNtavTR72gG/KlBaF1cf5xP0mzIVW7bt6r/QlAawbY3WwEr52WCMj5WeyGQluAvwxYVzyWVd0DkBdby462U4kGdHfvG+I2TdNI1KE2mr56Xo9qyqSnk3WtIoxOCrU8UHLgYUJYNhPtD02Al6sv+Eovus0ULaZoMesWs24xsxbjAkMdMc7r4Lwp9LEhxGZapymDGxiWD2bQwbGmHGqtGDaQfBa3co9IlCVzywHglhG1P2zjNkMx0nMK/mRdsqTm+3kvPNxV43HFL/PY+/Fi3S97A+MrJXLzN9fOoCVqoqrAOMiItsiyYDjZvs9XN0AH7RQvWduBZm3dqgirysCUWBc/nKf1WvnAZC1ES46KhJd8tmVCsLlf1G+tCpROoj3KwzCuAnMoLWL4lDSv13JXlmKhCUypP+s4TubZ825zmS9buvBsfOZOjp7+O+6c/Vq/cfeMb41tjaDjGM+PU+Nk4M84Nf+efXN3tPvd3l97/Z2e3dLdOdOZfPYqLXeg/8ABqiHwA=</latexit>

[Sennesh, Ścibior, Wu, van de Meent, (arXiv)]

SLIDE 39

Scaling up Amortized Inference

User-specified  importance sampler (inference combinators) User-specified  variational objective (next talk) Deep Generative Model  program pθ(x, z) Inference Model  program qφ(z | x) Better proposals Importance Sampling Use proposals qφ(z | x)  to sample from pθ(z | x) Variational Inference Learn proposals qφ(z | x) using samples from pθ(z | x) Better gradient estimates

SLIDE 40

Thank You!

Composing Modeling and Inference Operations   with Probabilistic Program Combinators 

E. Sennesh, A. Scibior, H. Wu, J.-W. van de Meent

ArXiv 2018 [https://arxiv.org/abs/1811.05965] Amortized Gibbs Samplers with Neural Sufficient Statistics 

H. Wu, H. Zimmermann, E. Sennesh, J.-W. Van de Meent

ArXiv 2019 [https://arxiv.org/abs/1911.01382] Structured Neural Topic Models for Reviews  

B. Esmaeli, H. Huang, B. Wallace, J.-W. van de Meent

AISTATS 2019 [http://proceedings.mlr.press/v89/esmaeili19b.html] Babak  Esmaeli Hao  Wu Alican  Bozkurt Eli  Sennesh Heiko  Zimmermann An Introduction to Probabilistic Programming  J-W van de Meent, B. Paige, H. Yang, F. Wood  ArXiv 2018 [https://arxiv.org/abs/1809.10756] Evaluating Combinatorial Generalization in VAEs 

A. Bozkurt, B. Esmaeili, D. H. Brooks,  
J. Dy, J.-W. van de Meent

ArXiv 2019 [https://arxiv.org/abs/1911.04594] Source Code (Apache 2.0)  https://github.com/probtorch/probtorch Sam Stites

Compositional Methods for Learning and Inference in Deep Probabilistic Programs

Deep Learning Success Stories

Ingredients for success

Do we still need models?

Do we still need models?

When are models useful?

We need inductive biases that

Deep Probabilistic Models

Deep Learning

Probabilistic Programming

SGD + AutoDiff (very general) Monte Carlo Methods (more model specific) Stochastic Variational Inference (learn proposals using neural networks)

Structured Variational Autoencoders

z x y

z x y

Deep Probabilistic Programs

Edward

Pyro Probabilistic Torch

Inference Model (Encoder) Generative Model (Decoder)

Learned Representations (Unsupervised)

Style Variables Generalization

Inductive Bias: Style features are uncorrelated with digit label, as well as with other features.

Model Composition

based on

based

Idea: Embed model for individual MNIST digits in a recurrent model for multiple object detection

Learn aspect-based representations of users, items, and reviews (fully unsupervised)

Example: Modeling Aspects in Reviews

Example: Modeling Aspects in Reviews

Aspects: Look, Mouthfeel, Aroma, Taste, Overall Data: Beer reviews

Example: Modeling Aspects in Reviews

Model learns topics that group into aspects (fully unsupervised)

Where are we today?

Structured VAEs work well for O(10) variables

Still Hard: Clustering

We need methods that scale to > O(100) variables

Paths towards Scaling up Inference

Amortized Gibbs Samplers

Intuition: Same idea as Gibbs sampling or Variational EM, but learn proposals for conditional updates

Minimizing the Inclusive KL

Variational Autoencoders Wake-Sleep / EP Methods

argmin

KL

argmin

KL

p(z | x) q(z | x) (exclusive) q(z | x) (inclusive)

Minimizing the Inclusive KL

Variational Autoencoders Wake-Sleep / EP Methods

− ∇

 log qφ(z | x) pθ(z | x)

− ∇φ KL

− ∇φ KL

− ∇

 log pθ(z | x) qφ(z | x)

⇤

 |

⇥ ∇φ logqφ(z | x) ⇤

Reweighted Wake-sleep Style Methods

−∇φKL

î ∇φ log qφ(z | x) ó

∇θ log pθ(x) = Epθ (z|x) î ∇θ log pθ(x, z) ó

Approximate with any importance sampler

Reweighted Wake-sleep Style Methods

Example: Unsupervised Tracking

Inferred Locations Reconstructions Fully unsupervised (learns sub-model for MNIST) Scales to O(100) frames and ~5 digits (possibly more)

Thinking Compositionally

probabilistic programming with SGD-based methods

and RWS), but not trivial to combine correctly

methods for importance sampling and gradient estimation?

What (inference) DSL could define this sampler?

Static vs Dynamic Models

Dynamic Computation Graphs: Number of variable nodes is data-dependent and/or stochastic

Static vs Dynamic Models

Static Computation Graphs: Set of nodes and dependency graph determinable from static analysis.

Static vs Dynamic Models

Anglican Venture WebPPL Birch Gen Turing

Static vs Dynamic Models

Stan* Infer.NET

Static Graphs Dynamic Graphs

Deep Learning

Programs as Importance Samplers

Compositional Methods for Learning and   Inference in Deep Probabilistic Programs

SGD + AutoDiff   (very general) Monte Carlo Methods (more model specific) Stochastic Variational Inference (learn proposals using neural networks)

Learn aspect-based representations of   users, items, and reviews (fully unsupervised)

Model learns topics that group into aspects  (fully unsupervised)

Approximate with any importance sampler 

probabilistic programming  with SGD-based methods

and RWS), but not trivial   to combine correctly

methods for importance sampling  and gradient estimation?

Dynamic Computation Graphs: Number of variable nodes   is data-dependent and/or stochastic

Idea: Change the evaluation (sampling) strategy,   whilst leaving the target density unaffected.