advanced machine learning
play

Advanced Machine Learning CS 7140 - Spring 2019 Lecture 20: - PowerPoint PPT Presentation

Oct 13, 2023 •710 likes •1.11k views

Advanced Machine Learning CS 7140 - Spring 2019 Lecture 20: Generative Adversarial Networks Jan-Willem van de Meent Slide credits: Ian Goodfellow Variational Autoencoders Input Hidden Mean Encoding Hidden Reconstructed Images

  1. Advanced Machine Learning CS 7140 - Spring 2019 Lecture 20: Generative Adversarial Networks Jan-Willem van de Meent Slide credits: Ian Goodfellow

  2. Variational Autoencoders Input Hidden Mean 
 Encoding Hidden Reconstructed 
 Images Units Std Dev (random) Units Images 784 256 2-50 256 784 (28 x 28) (28 x 28)

  3. Variational Autoencoders Assume prior:

  4. <latexit sha1_base64="xCnLHJxvOenf3Kf6H0xnPDc3fOI=">AGrHicfZRb9MwFMfTMcoItw0eYnoSydVQLTNgkNTUMTPCA0pt2kulSO46TWHCdznN0sfzM+CHwb7DSCJg64D7V8fv9zjn1OTphTUgjf/9VbebD6sP9o7bH75Omz5y/WN16eFVnJET5FGc34RQgLTAnDp4Ii9yjmEaUnweXn409vNrzAuSsRNxl+NpChNGYoKg0Eez9Z8ghWKOIJVf1BCIORZw5IF8TjbdPc8FRZnOJNsL1Pev7r9I70HruWtmjGt6ZCEoTxUM3k1M/BQo/fKAymJFqpN5QKYzHRf1nigphDJPOZXERQwoaeZVMs5Z2pc0zlSnN06SuZjO1gf+2K+WZ2+CejNw6nU021j9AaIMlSlmAlFYFJPAz8VUQi4Iolg7LgucQ3QJEzRWwZTXExlVQLlNawnwVTGROYoYZMwrQwL2IdGrhonqK5Dox5M2x9OJXGS4QLkrCmKkyV64Ix7odqsxkFNISK3n86UBJf7T9bhS83VEthOoJoJdf6R/bSDhGLMa2d0aBdu7NpOXPKf4L+QbzGTDMcM3KEtTyCIJrjFSE/0+ALOi5NhcRIwlYNAKWXBC1RrKrsLlo23SkrQSLBqgTZ2t4SZi2rozoLu3zdW9hVF1Z3qp296KZh2cGWFlvaELcg3s4Qd8bEeUFoxqz7xEt01SexHZQuMXWNjUuqZ0wELY/5vBs3X2ubPW5V5liZdlkmIE9SqOsMshxzKDJuProbIuaUpEQUsrYrW0XY/1Xa3g52qJpN+Wd2WSQKadWYzbezOxTxqMmZW3ZgCW9i8J1gHkLrB/YkHreBe3pZm/O3o4Dfx82xrsH9STb8157bxhk7g7Dj7zmfnyDl1UO9DL+qlPdYf90/6k/50ga70as0rp7H68W8FM24q</latexit> Objective: Evidence Lower Bound (ELBO) Reconstruction 
 KL between approximate 
 log-likelihood posterior and prior Log marginal KL between approximate likelihood posterior and posterior

  5. <latexit sha1_base64="RPJ/9dykIGmaIKoms+rtG6vULy0=">AGOnicfZRLa9wEICVNtum7itpj72Y7iWBJdhpSAKlEFpCckxDXhAviyzPekXkRyQ5L6F/lp/QP9Brb6W30lt7qOx16NoylcFIM98pBkpzBkV0vO+zj14ON979HjhifP02fMXLxeXh2LrOAEjkjGMn4aYgGMpnAkqWRwmnPAScjgJDz/VOpPLoELmqWH8iaHYLjlI4pwdKIRovj4FJdazcQNHz0e5ytVxgosCR04QkxTRYx/oR2jur0nl6vFShA4U/sP7u5U5L53AzkBiY0PSKN729Fi31v1quHaE7+e9FE9kdL83dBlJEigVQShoU4871cDhXmkhIGJptCQI7JOY7hzExTnIAYqupAtNvQHvpDNc5SCSlpmCmciATLiSUsYdGUkokJDLwZthYOVeklAkHjtGkVJtpxgjGpjhVZioKWQFaHex+1MobLwb+GubuoVwiGrC3/IG5msDMQdIa2RrfeBvbNlMXvCcwT/IK7EyGw4pXJEsSbCpTnAJRJ+Z8zG1EgWHciMqCBPV97XWFjxFjU2ld4JZ5bVWKmgkWHVG7uZwcqNGujGgm67fN1a2EUXNu2+juxlN42LDraw2MKGuAXxdobQGRNyQVmWvsZz9BVn4ztoGyGqWtcumTmxkfY8phPuvF8Qi32oFWZA12yBeZxgU+cgy4FjmfHy0l1ROWE0oVKoWq9tK5r+38ro28F2dLMpy38Yqh1tkSRkVWM2z87uUMKjJlfusgOLeRObFq4DzFtgfcAlad47v/262ZPjtVXfW/U/r/e39+qXbwG9QW/RMvLRJtpGe2gfHSGCvqCf6Df607vrfet97/2Yog/mapvXqDF6v/4CPnVJ8w=</latexit> Implicit Generative Models z x • Can generate samples x ~ p G ( x ) • Cannot evaluate density p G ( x ) • Requirement: G ( z ; θ ) is differentiable

  6. Maximum Likelihood θ ∗ = arg max E x ∼ p data log p model ( x | θ ) θ

  7. Generative Adversarial Networks D tries to make D(G(z)) near 0, D (x) tries to be G tries to make near 1 D(G(z)) near 1 Di ff erentiable D function D x sampled from x sampled from data model Di ff erentiable function G Input noise z

  8. <latexit sha1_base64="9sNKaSQkROSgekjd1VD40+OaNo=">AGl3icfZRbT9swFIBTxjqWXYDtadpLtb4UqaAEPA2xMbgbYC4TU1VOclpauE4me1AwfI/29t+xf7NnDSMJo7mSu2xz3cu9jk9fkowF47zp7XwbPF5+8XS/vV6zdvl1dW313yJGMBXAQJSdi1jzgQTOFCYEHgOmWAYp/AlX/zJdf3QLjOKHn4j6FYwisc4QEIfjVZ+ezGajqQXREweKdXxYkzLQvlV72123vUO9v5VTrOI476cgTMBU8kCESPUK1ZqyPQJjMfBIEnVK61LlMRxNxPDJ08Ojp16x+Wdr58Yzudex3c76k6NZdiW/Zs9cPv4ORytdZ8MpVscU3FLo7luzdTJaXfzlhUmQxUBFQBDnA9dJxVAiJnBAQCeUcUhRcIMiGiRohj4UBavrd9kXnvuDuU4oQJoUDGTKOYxEhPjMId59TSY6MDAqmHLw6HMvYTAcUSrVn6sbNsLYawrX2QmQ59koOTZ0YGSTn9nq+9u7qoawiAsCXfP6etPHYgYAC2Rve2+u7NnMmnGUgJPkJNjeTYMKNwFSRwjGupSQ6AG+n08oDxjkF9Een4su65SyoBnqLYp9LY3r5wqXtyPsGit+rY/RyWX1RD9wb0OTrwcB+NmGemIBuejN70UyjrIHNDYzIWZArJ4hNMaElGOSUOM+4zm6JOxGZTMWNc5dEj5MQGR7TSTOeTrDBntUqc6bydpknEIv0JFLS1JgSCQs/9PdYTEhOMaCy1KvTCtM/2+l9fVgh6ralPm378tDZCBT4rGrL6d2aF6UlW5/JYNWMSq2KxwDWBaA8sHzk979z6dDOFy80Nd2tj83S7u39QTr4l6P1yepZrVr7VvH1ol1YQWtdb31nXrR/tD+3P7W/t4hi60Spv3VmW1T/8CRIZmeQ=</latexit> Objective: Minimax Game D − E x ∼ p DATA ( x ) [ log D ( x )] − E z ∼ p ( z ) [ log ( 1 − D ( G ( z )))] max min G • Generator G ( z ) generates fake images. • Discriminator D ( x ) classifies real images as 1 
 and fake images as 0. • Objective for G : Maximize probability that G fools D . • Objective for D : Minimize probability that D is fooled.

  9. <latexit sha1_base64="pXpRo69NxwuPNk2867tQLRln0Yw=">AAAGl3icfZRbT9swFIBTxjqW3cr2NO2lWl+KVFDCEPA2xMbgbYC4TXVVOclpauFcZjtQsPzP9rZfsX8zJw2jiaO5Unvs852LfU6Pl1LCheP8aS09WX7afrby3H7x8tXrN53Vtxc8yZgP535CE3blYQ6UxHAuiKBwlTLAkUfh0rv+kusvb4BxksRn4i6FUYTDmEyIj4U+Gnd+o4jEY4l8FsivSnVRhGf5NmTyUG/tdRsd6P2NnGkdJ1E3HSMBM8F9GWCBVb9QrSkbUZiIIaJJ2C2dlSrESDgVo0dP9w+e+sXmn62dG8/lftd2u+uPjubplPyaPXf58Dsad3rOhlOsrim4pdCzynU8Xl3+hYLEzyKIhU8x50PXScVIYiaIT0EnlHFIsX+NQxhqMcYR8JEsXlu/yaL2zB3JSRILiP2KmcQRj7CYGoc5zKun/lQHBlYNWx6OZO4lAE7CuGrlRcq2UQATXfkiMxl4NAMlTw/3lXQG258G7uaOqiEMgpJwd52B/tSBkAHEJbK7NXC3d00mzVhK4RFycizPhkEMt34SRTgOdKnBV0P9PghinjHILyKRF8meq5Qy4DmqbQq9jRaVMyV1Ty4mWPRWHbtbwPKLaujOgO6bfN0b2M8mDIkp6KY3sxfNNM4a2MxgMxNiBsTqGUJjTEg5oUls3GeyQBd9MjGD0gWmrHHukupxEmDDYzptxtMpMdjTWmVOVd4uiwRmoR49SqIkBYZFwvI/3S0RU0oiIrgs9cq0IvH/rbS+HuxAVZsy//Y8eaAM0vdo0ZjVtzM7VE+qKpffsgELWRWbF64BTGtg+cA5qeedW59upnCxueE6G+7JVm9vv5x8K9YH66PVt1xrx9qzjqxj69zyW2ut762r1o/2+/bn9rf20RxdapU276zKap/8BfDiZjU=</latexit> <latexit sha1_base64="h/1wASbY8y+dyFEiXmgpamckOK4=">AGLHicfZTLbtQwFEDd0oESXlNYshkxmyJGVKqthukqlCVZan6kprRyHuzFh1EmM7fVn+Jb4A8RVsEGKH2ME3EKdBTOKAI0VXvuf6Pu2IMyqV73+Zm7+10Ll9Z/Gud+/+g4ePukuPj2SWCwKHJGOZOImwBEZTOFRUMTjhAnASMTiOzl5b/fE5CEmz9EBdcRgmeJLSMSVYFVuj7smb5fBcX5rn3queF4FJpqPdKjgUkmiY6ywMRVhvJ5d/9T3XvT4aPcPOr2/RW/XD1XCqhj6q1N1pa+BjGckTSBVhWMrTwOdqLFQlDAwXphL4Jic4QmcFmKE5BDXZbA9Grag2Cox1mqICU1M40TmWA1dTYtLOu7ZFo4BlF3W20OtT0lBknad0qSoznhTGMi3aUkek4YjkYvb+7bQ/WH85CFY3TAMREFdEsOkPiq8JTARAWiGba4NgfdNleC4g7+QbzEbjYAULkiWJDiNdXgOxJwW9QkhlbkAm4gOo0T3A2OMA9+ghU2p98JZ5aXROqwFWLa+iV3NYDbRArpyoOu2s64d7H0bFqop2DF0olftNM5b2NxhcxcSDiSaEUKrT+CSsix18hnP0OWcjF2nbIapemyPZMUdj7FzIp+243xKHXa/0Zl9Y8dlsBikuCiz2HGQWCVCXvpLqiaMpQJXWlN64VTf9vVeibznZMfSjtP4r0jnFIErFyMOu1cyeUiLjO2SxbsImoYzeNawF5A6wKbMnivQuar5srHK2uBP5K8G6tv7VdvXyL6Cl6hpZRgDbQFnqL9tAhIugT+o5+ol+dD53Pna+dbzfo/Fxl8wTVufHb9nRJI=</latexit> Exercise • Suppose that we could evaluate p DATA ( x ) and p G ( x ) • Can you express D ( x ) in terms of p DATA ( x ) and p G ( x ) ? SOLUTION:

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

MACHINE LEARNING Kernel Canonical Correlation Analysis 1 ADVANCED MACHINE LEARNING ADVANCED

MACHINE LEARNING Kernel Canonical Correlation Analysis 1 ADVANCED MACHINE LEARNING ADVANCED

ADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING MACHINE LEARNING Kernel Canonical Correlation Analysis 1 ADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING Structure of todays and next weeks class 1) Briefly go through one

613 views • 28 slides
ADVANCED MACHINE LEARNING Non-linear regression techniques 1 1 ADVANCED MACHINE LEARNING

ADVANCED MACHINE LEARNING Non-linear regression techniques 1 1 ADVANCED MACHINE LEARNING

ADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING Non-linear regression techniques 1 1 ADVANCED MACHINE LEARNING Regression: Principle N Map N-dim. input x to a continuous output y . Learn a function of the type:

1.29k views • 61 slides
ADVANCED MACHINE LEARNING Kernel PCA 11 ADVANCED MACHINE LEARNING Overview Todays Lecture

ADVANCED MACHINE LEARNING Kernel PCA 11 ADVANCED MACHINE LEARNING Overview Todays Lecture

ADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING Kernel PCA 11 ADVANCED MACHINE LEARNING Overview Todays Lecture Brief Recap of Classical Principal Component Analysis (PCA) Derivation of kernel PCA Exercises to develop a

1.18k views • 28 slides
ADVANCED MACHINE LEARNING Mini-Project Overview Lecture : Prof. Aude Billard

ADVANCED MACHINE LEARNING Mini-Project Overview Lecture : Prof. Aude Billard

ADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING Mini-Project Overview Lecture : Prof. Aude Billard (aude.billard@epfl.ch) Teaching Assistants : Nadia Figueroa, Ilaria Lauzana, Brice Platerrier 11 ADVANCED MACHINE LEARNING Deadlines for

463 views • 24 slides
MACHINE LEARNING Spectral Clustering 1 ADVANCED MACHINE LEARNING Outline of Todays Lecture

MACHINE LEARNING Spectral Clustering 1 ADVANCED MACHINE LEARNING Outline of Todays Lecture

ADVANCED MACHINE LEARNING MACHINE LEARNING Spectral Clustering 1 ADVANCED MACHINE LEARNING Outline of Todays Lecture Introduce the principle of spectral clustering Show extension for other transformations of the space

616 views • 38 slides
Discrete and Continuous Reinforcement Learning (not part of exam material) 1 1 ADVANCED

Discrete and Continuous Reinforcement Learning (not part of exam material) 1 1 ADVANCED

ADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING Brief Overview of Discrete and Continuous Reinforcement Learning (not part of exam material) 1 1 ADVANCED MACHINE LEARNING Forms of Learning Supervised learning where the algorithm

1.27k views • 71 slides
Advanced Machine Learning Learning Kernels MEHRYAR MOHRI MOHRI@ COURANT INSTITUTE &amp;

Advanced Machine Learning Learning Kernels MEHRYAR MOHRI MOHRI@ COURANT INSTITUTE &amp;

Advanced Machine Learning Learning Kernels MEHRYAR MOHRI MOHRI@ COURANT INSTITUTE &amp; GOOGLE RESEARCH .. Outline Kernel methods. Learning kernels scenario. learning bounds. algorithms. Advanced Machine Learning -

776 views • 45 slides
ADVANCED MACHINE LEARNING Non-linear regression techniques (SVR and extensions, GPR, Gradient

ADVANCED MACHINE LEARNING Non-linear regression techniques (SVR and extensions, GPR, Gradient

ADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING Non-linear regression techniques (SVR and extensions, GPR, Gradient Boosting) 1 1 ADVANCED MACHINE LEARNING Regression: Principle N Map N-dim. input to a continuous

906 views • 89 slides
Computational Learning Theory - HT 2018 formerly Advanced Machine Learning (HT 2017) Introduction

Computational Learning Theory - HT 2018 formerly Advanced Machine Learning (HT 2017) Introduction

Computational Learning Theory - HT 2018 formerly Advanced Machine Learning (HT 2017) Introduction and Course Details Varun Kanade University of Oxford January 15, 2018 What is Machine Learning? 200-basic level categories Here: Six

466 views • 18 slides
ADVANCED MACHINE LEARNING Caveats and Techniques to Deal with Imbalanced Datasets (Adapted from

ADVANCED MACHINE LEARNING Caveats and Techniques to Deal with Imbalanced Datasets (Adapted from

ADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING Caveats and Techniques to Deal with Imbalanced Datasets (Adapted from H. He and E. A. Garcia, Learning from Imbalanced Data, IEEE Trans. Knowledge and Data Engineering, vol. 21, issue 9,

285 views • 24 slides
ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale Learning Dhruv Batra Virginia Tech What is this class about? Some of the most exciting developments in Machine Learning, AI, Statistics &amp;

925 views • 61 slides
CS 6784: Spring 2010 Advanced Topics in Machine Learning Review Guozhang Wang February 25, 2010

CS 6784: Spring 2010 Advanced Topics in Machine Learning Review Guozhang Wang February 25, 2010

CS 6784: Spring 2010 Advanced Topics in Machine Learning Review Guozhang Wang February 25, 2010 1 Machine Learning Tasks Machine learning tasks can be roughly categorized into three classes: su- pervised learning, unsupervised learning,

461 views • 7 slides
DD2434 - Advanced Machine Learning Gaussian Processes Carl Henrik Ek { chek } @csc.kth.se Royal

DD2434 - Advanced Machine Learning Gaussian Processes Carl Henrik Ek { chek } @csc.kth.se Royal

Introduction Recap Kernels Gaussian Processes References DD2434 - Advanced Machine Learning Gaussian Processes Carl Henrik Ek { chek } @csc.kth.se Royal Institute of Technology November 5th, 2015 Ek KTH DD2434 - Advanced Machine Learning

1.91k views • 170 slides
Advanced Introduction to Machine Learning, CMU-10715 Manifold Learning Barnabs Pczos

Advanced Introduction to Machine Learning, CMU-10715 Manifold Learning Barnabs Pczos

Advanced Introduction to Machine Learning, CMU-10715 Manifold Learning Barnabs Pczos Motivation Find meaningful low-dimensional structures hidden in high-dimensional observations. The human brain confronts the same problem in

1.06k views • 85 slides
ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale Learning Topics Bayes Nets (Finish) Parameter Learning Structure Learning Readings: KF 18.1, 18.3; Barber 9.5, 10.4 Dhruv Batra Virginia

351 views • 32 slides
Advanced Machine Learning CS 7140 - Spring 2019 Lecture 24: Bayesian Optimization Jan-Willem van

Advanced Machine Learning CS 7140 - Spring 2019 Lecture 24: Bayesian Optimization Jan-Willem van

Advanced Machine Learning CS 7140 - Spring 2019 Lecture 24: Bayesian Optimization Jan-Willem van de Meent Slide credits: Ryan Adams, Nando de Freitas Background: Multi-Armed Bandits Problem: Which machine has highest rate of payout?

805 views • 42 slides
Pr Protocol Design Challeng l Design Challenge Zebranet Monitor zebras: Vitals Loca2on

Pr Protocol Design Challeng l Design Challenge Zebranet Monitor zebras: Vitals Loca2on

Pr Protocol Design Challeng l Design Challenge Zebranet Monitor zebras: Vitals Loca2on Movement pa6erns Challenges: Large living area Ba6ery has to last long 2me Zebranet Z2Z DATA DATA DATA Z2R A5acker(s) Collar

382 views • 14 slides
Chad Voegele Selective Search for Object Recognition Outline 1. Individual contribution of

Chad Voegele Selective Search for Object Recognition Outline 1. Individual contribution of

Chad Voegele Selective Search for Object Recognition Outline 1. Individual contribution of region similarity measures 2. Importance of good base segmentation 3. Box overlap correspondence to recognition 3 Outline 1. Individual contribution

770 views • 41 slides
Conflation: its logic and some applications David Ripley University of Connecticut

Conflation: its logic and some applications David Ripley University of Connecticut

Conflation: its logic and some applications David Ripley University of Connecticut http://davewripley.rocks davewripley@gmail.com Conflation Conflation slenderness substance dualism y xR x y , and x yR x y and reading/revealing

401 views • 22 slides
Constraint Satisfaction Problems (CSPs) Russell and Norvig Chapter 6 CSP example: map coloring

Constraint Satisfaction Problems (CSPs) Russell and Norvig Chapter 6 CSP example: map coloring

Constraint Satisfaction Problems (CSPs) Russell and Norvig Chapter 6 CSP example: map coloring Given a map of Australia, color it using three colors such that no neighboring territories have the same color. 2 CSP example: map coloring 3

950 views • 63 slides
Slides Set 5: Probabilistic Networks Rina Dechter Darwiche chapter 3,4, Pearl: chapters 3

Slides Set 5: Probabilistic Networks Rina Dechter Darwiche chapter 3,4, Pearl: chapters 3

Algorithms for Reasoning with graphical models Slides Set 5: Probabilistic Networks Rina Dechter Darwiche chapter 3,4, Pearl: chapters 3 slides5 828X 2019 Outline Basics of probability theory DAGS, Markov(G), Bayesian networks

1.39k views • 120 slides
room Today at Tiny Tigers we had so much fun playing, learning and exploring together, the three

room Today at Tiny Tigers we had so much fun playing, learning and exploring together, the three

Todays adventures Giraffe, Tiger, Monkey and Zebra room Today at Tiny Tigers we had so much fun playing, learning and exploring together, the three rooms combined for the day, it was a great opportunity for the children to socialise and play

277 views • 7 slides
H517 Visualization Design, Analysis, &amp; Evaluation Week 12: Visualization Tasks &amp;

H517 Visualization Design, Analysis, &amp; Evaluation Week 12: Visualization Tasks &amp;

H517 Visualization Design, Analysis, &amp; Evaluation Week 12: Visualization Tasks &amp; Evaluation Khairi Reda | redak@iu.edu School of Informa5cs &amp; Compu5ng, IUPUI Raw user data Processed Data plots Visualization data pre- visual

690 views • 46 slides
Boosting Constraint Acquisition via Generalization Queries

Boosting Constraint Acquisition via Generalization Queries

1/18 Boosting Constraint Acquisition via Generalization Queries Chris'an Bessiere 1 Remi Cole1a 1 Abderrazak Daoudi 12 Nadjib Lazaar 1 Younes Mechqrane 12 El

576 views • 25 slides

More recommend