Faster Attend-Infer-Repeat with Tractable Probabilistic Models Karl - PowerPoint PPT Presentation

Faster Attend-Infer-Repeat with Tractable Probabilistic Models Karl Stelzner 1 , Robert Peharz 2 , Kristian Kersting 1,3 1 Machine Learning Group, TU Darmstadt 2 Machine Learning Group, University of Cambridge 3 Centre for Cognitive Science, TU Darmstadt ICML 2019 June 13, 2019

Deep Models with Tractable Components • Deep generative models are a powerful tool • Scaling is limited by effectiveness of approximate inference • Can we improve this by combining them with tractable models, such as Sum-Product Networks? primitive unspecified z 1 z 2 specified x

Deep Models with Tractable Components • Deep generative models are a powerful tool • Scaling is limited by effectiveness of approximate inference • Can we improve this by combining them with tractable models, such as Sum-Product Networks? primitive z 0 Neural Network primitive z 1 z 2 specified x

Deep Models with Tractable Components • Deep generative models are a powerful tool • Scaling is limited by effectiveness of approximate inference • Can we improve this by combining them with tractable models, such as Sum-Product Networks? primitive z 0 Neural Network primitive primitive SPN z 1 z 2 z 1 z 2 specified specified x x

Attend-Infer-Repeat i = 1, …, N max z iwhat Neural Network y i [Eslami, SM Ali, et al. "Attend, infer, repeat: Fast scene understanding with generative models." NIPS 2016]

Attend-Infer-Repeat i = 1, …, N max z iwhat Neural Network y i z iwhere N [Eslami, SM Ali, et al. "Attend, infer, repeat: Fast scene understanding with generative models." NIPS 2016]

Attend-Infer-Repeat i = 1, …, N max z iwhat Neural Network y i z iwhere N Specified Renderer x [Eslami, SM Ali, et al. "Attend, infer, repeat: Fast scene understanding with generative models." NIPS 2016]

Attend-Infer-Repeat i = 1, …, N max z iwhat Neural Network y i z iwhere N Specified Renderer x [Eslami, SM Ali, et al. "Attend, infer, repeat: Fast scene understanding with generative models." NIPS 2016]

Sum-Product Attend-Infer-Repeat Use SPN to model objects Each pixel in y is occluded (unobserved), or can be inferred deterministically from x and z where i = 1, …, N max SPN y 1 y i z iwhere N y 2 x x

Sum-Product Attend-Infer-Repeat Model background with another SPN 4 𝑞 𝑦 | 𝑂, 𝑨 '()*) = 𝑞 ,- (𝑦 ,-/0 ) ∏ 3/0 𝑞 5,6 (𝑦 3/0 ) i = 1, …, N max SPN y 1 y i z iwhere N y 2 SPN x y bg y bg x

Faster & More Robust Training

Background Model at Work

Thank you! Pacific Ballroom #89 github.com/stelzner/supair stelzner@cs.tu-darmstadt.de