explicitly imposing constraints in deep networks via
play

EXPLICITLY IMPOSING CONSTRAINTS IN DEEP NETWORKS VIA CONDITIONAL - PowerPoint PPT Presentation

Nov 19, 2023 •264 likes •797 views

AAAI 19 EXPLICITLY IMPOSING CONSTRAINTS IN DEEP NETWORKS VIA CONDITIONAL GRADIENTS GIVES IMPROVED GENERALIZATION AND FASTER CONVERGENCE Sathya Ravi, Tuan Dinh, Vishnu Lokhande, Vikas Singh Department of Computer Sciences University of

  1. AAAI’ 19 EXPLICITLY IMPOSING CONSTRAINTS IN DEEP NETWORKS VIA CONDITIONAL GRADIENTS GIVES IMPROVED GENERALIZATION AND FASTER CONVERGENCE Sathya Ravi, Tuan Dinh, Vishnu Lokhande, Vikas Singh Department of Computer Sciences University of Wisconsin–Madison 11/14/2018

  3. DEEP LEARNING

  4. DEEP LEARNING min ! n L ( W ) Solve W ∈

  5. <latexit sha1_base64="+DqeC3o24NjVnLkKQwoKp/pcRUk=">ACBnicbVDLSsNAFJ3UV62vqEtBovQgpREBHUhFEVw4aKCMYUmhMl0g6dPJiZiCV058ZfceNCxa3f4M6/cdJmoa0Hhjmcy/3uMnjApGN9aW5+YXGpvFxZWV1b39A3t+5EnHJMLByzmLd9JAijEbEklYy0E05Q6DNi+4OL3LfvCRc0jm7lMCFuiHoRDShGUkmevntds+vwDohkn3fzy5HnvNAYVCzD9Rf9/Sq0TDGgLPELEgVFGh5+pfTjXEakhihoTomEYi3QxSTEjo4qTCpIgPEA90lE0QiERbja+YwT3ldKFQczViyQcq787MhQKMQx9VZmvK6a9XPzP6QyOHEzGiWpJBGeDApSBmUM81Bgl3KCJRsqgjCnaleI+4gjLFV0FRWCOX3yLEOG6cN8+ao2jwv0iDHbAHasAEx6AJrkALWACDR/AMXsGb9qS9aO/ax6S0pBU92+APtM8fbsyXVg=</latexit> <latexit sha1_base64="+DqeC3o24NjVnLkKQwoKp/pcRUk=">ACBnicbVDLSsNAFJ3UV62vqEtBovQgpREBHUhFEVw4aKCMYUmhMl0g6dPJiZiCV058ZfceNCxa3f4M6/cdJmoa0Hhjmcy/3uMnjApGN9aW5+YXGpvFxZWV1b39A3t+5EnHJMLByzmLd9JAijEbEklYy0E05Q6DNi+4OL3LfvCRc0jm7lMCFuiHoRDShGUkmevntds+vwDohkn3fzy5HnvNAYVCzD9Rf9/Sq0TDGgLPELEgVFGh5+pfTjXEakhihoTomEYi3QxSTEjo4qTCpIgPEA90lE0QiERbja+YwT3ldKFQczViyQcq787MhQKMQx9VZmvK6a9XPzP6QyOHEzGiWpJBGeDApSBmUM81Bgl3KCJRsqgjCnaleI+4gjLFV0FRWCOX3yLEOG6cN8+ao2jwv0iDHbAHasAEx6AJrkALWACDR/AMXsGb9qS9aO/ax6S0pBU92+APtM8fbsyXVg=</latexit> <latexit sha1_base64="+DqeC3o24NjVnLkKQwoKp/pcRUk=">ACBnicbVDLSsNAFJ3UV62vqEtBovQgpREBHUhFEVw4aKCMYUmhMl0g6dPJiZiCV058ZfceNCxa3f4M6/cdJmoa0Hhjmcy/3uMnjApGN9aW5+YXGpvFxZWV1b39A3t+5EnHJMLByzmLd9JAijEbEklYy0E05Q6DNi+4OL3LfvCRc0jm7lMCFuiHoRDShGUkmevntds+vwDohkn3fzy5HnvNAYVCzD9Rf9/Sq0TDGgLPELEgVFGh5+pfTjXEakhihoTomEYi3QxSTEjo4qTCpIgPEA90lE0QiERbja+YwT3ldKFQczViyQcq787MhQKMQx9VZmvK6a9XPzP6QyOHEzGiWpJBGeDApSBmUM81Bgl3KCJRsqgjCnaleI+4gjLFV0FRWCOX3yLEOG6cN8+ao2jwv0iDHbAHasAEx6AJrkALWACDR/AMXsGb9qS9aO/ax6S0pBU92+APtM8fbsyXVg=</latexit> <latexit sha1_base64="+DqeC3o24NjVnLkKQwoKp/pcRUk=">ACBnicbVDLSsNAFJ3UV62vqEtBovQgpREBHUhFEVw4aKCMYUmhMl0g6dPJiZiCV058ZfceNCxa3f4M6/cdJmoa0Hhjmcy/3uMnjApGN9aW5+YXGpvFxZWV1b39A3t+5EnHJMLByzmLd9JAijEbEklYy0E05Q6DNi+4OL3LfvCRc0jm7lMCFuiHoRDShGUkmevntds+vwDohkn3fzy5HnvNAYVCzD9Rf9/Sq0TDGgLPELEgVFGh5+pfTjXEakhihoTomEYi3QxSTEjo4qTCpIgPEA90lE0QiERbja+YwT3ldKFQczViyQcq787MhQKMQx9VZmvK6a9XPzP6QyOHEzGiWpJBGeDApSBmUM81Bgl3KCJRsqgjCnaleI+4gjLFV0FRWCOX3yLEOG6cN8+ao2jwv0iDHbAHasAEx6AJrkALWACDR/AMXsGb9qS9aO/ax6S0pBU92+APtM8fbsyXVg=</latexit> DEEP LEARNING min ! n L ( W ) Solve W ∈ L ( W ) = E ξ f ( W, ξ ) ξ = (x,y) ~ 𝒠

  7. Compute an estimate of gradient

  8. <latexit sha1_base64="aGnU6jnbxPGn2A/R7LrfwJsOIM=">ACFXicbVA9SwNBEN3z2/gVtbRZDIihjsR1EIQbSwsFIwRcuGY20x0yd7esTsnhCO/wsa/YmOhYivY+W/cxBR+PRh4vDfDzLw4U9KS7394I6Nj4xOTU9Olmdm5+YXy4tKlTXMjsCZSlZqrGCwqbFGkhReZQYhiRXW485x36/forEy1RfUzbCZwLWbSmAnBSVt+pRQZtBjx/wekR8i4dIEFGoIVbAQ5KqhcVpL6J1Z29E5Ypf9Qfgf0kwJBU2xFlUfg9bqcgT1CQUWNsI/IyaBRiSQmGvFOYWMxAduMaGoxoStM1i8FaPrzmlxdupcaWJD9TvEwUk1naT2HUmQDf2t9cX/MaObX3moXUWU6oxdeidq4pbyfEW9Jg4JU1xEQRrpbubgBA4JckiUXQvD75b+ktl3drwbnO5XDo2EaU2yFrbJ1FrBdshO2BmrMcHu2AN7Ys/evfovXivX60j3nBmf2A9/YJIWdtg=</latexit> <latexit sha1_base64="aGnU6jnbxPGn2A/R7LrfwJsOIM=">ACFXicbVA9SwNBEN3z2/gVtbRZDIihjsR1EIQbSwsFIwRcuGY20x0yd7esTsnhCO/wsa/YmOhYivY+W/cxBR+PRh4vDfDzLw4U9KS7394I6Nj4xOTU9Olmdm5+YXy4tKlTXMjsCZSlZqrGCwqbFGkhReZQYhiRXW485x36/forEy1RfUzbCZwLWbSmAnBSVt+pRQZtBjx/wekR8i4dIEFGoIVbAQ5KqhcVpL6J1Z29E5Ypf9Qfgf0kwJBU2xFlUfg9bqcgT1CQUWNsI/IyaBRiSQmGvFOYWMxAduMaGoxoStM1i8FaPrzmlxdupcaWJD9TvEwUk1naT2HUmQDf2t9cX/MaObX3moXUWU6oxdeidq4pbyfEW9Jg4JU1xEQRrpbubgBA4JckiUXQvD75b+ktl3drwbnO5XDo2EaU2yFrbJ1FrBdshO2BmrMcHu2AN7Ys/evfovXivX60j3nBmf2A9/YJIWdtg=</latexit> <latexit sha1_base64="aGnU6jnbxPGn2A/R7LrfwJsOIM=">ACFXicbVA9SwNBEN3z2/gVtbRZDIihjsR1EIQbSwsFIwRcuGY20x0yd7esTsnhCO/wsa/YmOhYivY+W/cxBR+PRh4vDfDzLw4U9KS7394I6Nj4xOTU9Olmdm5+YXy4tKlTXMjsCZSlZqrGCwqbFGkhReZQYhiRXW485x36/forEy1RfUzbCZwLWbSmAnBSVt+pRQZtBjx/wekR8i4dIEFGoIVbAQ5KqhcVpL6J1Z29E5Ypf9Qfgf0kwJBU2xFlUfg9bqcgT1CQUWNsI/IyaBRiSQmGvFOYWMxAduMaGoxoStM1i8FaPrzmlxdupcaWJD9TvEwUk1naT2HUmQDf2t9cX/MaObX3moXUWU6oxdeidq4pbyfEW9Jg4JU1xEQRrpbubgBA4JckiUXQvD75b+ktl3drwbnO5XDo2EaU2yFrbJ1FrBdshO2BmrMcHu2AN7Ys/evfovXivX60j3nBmf2A9/YJIWdtg=</latexit> <latexit sha1_base64="aGnU6jnbxPGn2A/R7LrfwJsOIM=">ACFXicbVA9SwNBEN3z2/gVtbRZDIihjsR1EIQbSwsFIwRcuGY20x0yd7esTsnhCO/wsa/YmOhYivY+W/cxBR+PRh4vDfDzLw4U9KS7394I6Nj4xOTU9Olmdm5+YXy4tKlTXMjsCZSlZqrGCwqbFGkhReZQYhiRXW485x36/forEy1RfUzbCZwLWbSmAnBSVt+pRQZtBjx/wekR8i4dIEFGoIVbAQ5KqhcVpL6J1Z29E5Ypf9Qfgf0kwJBU2xFlUfg9bqcgT1CQUWNsI/IyaBRiSQmGvFOYWMxAduMaGoxoStM1i8FaPrzmlxdupcaWJD9TvEwUk1naT2HUmQDf2t9cX/MaObX3moXUWU6oxdeidq4pbyfEW9Jg4JU1xEQRrpbubgBA4JckiUXQvD75b+ktl3drwbnO5XDo2EaU2yFrbJ1FrBdshO2BmrMcHu2AN7Ys/evfovXivX60j3nBmf2A9/YJIWdtg=</latexit> Compute an estimate of gradient W t +1 = W t � η t r ˜ L t ( W t )

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Imposing Hard Constraints on Deep Networks: Promises and Limitations min w R ( w ) s.t C ( w ) =

Imposing Hard Constraints on Deep Networks: Promises and Limitations min w R ( w ) s.t C ( w ) =

Imposing Hard Constraints on Deep Networks: Promises and Limitations min w R ( w ) s.t C ( w ) = 0 P. Marquez-Neila, M. Salzmann, and P. Fua EPFL Switzerland Motivation: 3D Pose Estimation Given a CNN trained to predict the 3D locations of

460 views • 13 slides
Enforcing constraints for interpolation and extrapolation in Generative Adversarial Networks

Enforcing constraints for interpolation and extrapolation in Generative Adversarial Networks

Enforcing constraints for interpolation and extrapolation in Generative Adversarial Networks Panos Stinis (joint work with T. Hagge, A.M. Tartakovsky and E. Yeung) Pacific Northwest National Laboratory space space space Supported by Deep

255 views • 14 slides
Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations

Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations

Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations deeplearning.cce2020@gmail.com Deep Networks Intuition Neural networks with multiple hidden layers - Deep networks [Hinton, 2006] Deep Networks Intuition

1.33k views • 28 slides
Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan Zelener Machine Learning Reading Group January 7 th 2016 The Graduate Center, CUNY Objectives Explain some of the trends of deep learning and

680 views • 44 slides
Deep Networks Andrea Passerini passerini@disi.unitn.it Machine Learning Deep Networks Need for

Deep Networks Andrea Passerini passerini@disi.unitn.it Machine Learning Deep Networks Need for

Deep Networks Andrea Passerini passerini@disi.unitn.it Machine Learning Deep Networks Need for Deep Networks Perceptron Can only model linear functions Kernel Machines Non-linearity provided by kernels Need to design appropriate kernels

563 views • 43 slides
Deep Networks Andrea Passerini passerini@disi.unitn.it Machine Learning Deep Networks Need for

Deep Networks Andrea Passerini passerini@disi.unitn.it Machine Learning Deep Networks Need for

Deep Networks Andrea Passerini passerini@disi.unitn.it Machine Learning Deep Networks Need for Deep Networks Perceptron Can only model linear functions Kernel Machines Non-linearity provided by kernels Need to design appropriate kernels

1.45k views • 34 slides
From Traditional Neural From Traditional NN . . . Networks to Deep Learning Need to Go Beyond .

From Traditional Neural From Traditional NN . . . Networks to Deep Learning Need to Go Beyond .

Why Traditional . . . How the Need for Fast . . . Faster Differentiation: . . . Beyond Traditional NN From Traditional Neural From Traditional NN . . . Networks to Deep Learning Need to Go Beyond . . . Constraints Are . . . and Beyond

548 views • 38 slides
Deep Learning With Constraints Yatin Nandwani Work done in collaboration with Abhishek Pathak

Deep Learning With Constraints Yatin Nandwani Work done in collaboration with Abhishek Pathak

Deep Learning With Constraints Yatin Nandwani Work done in collaboration with Abhishek Pathak Under the guidance of Prof. Mausam and Prof. Parag Singla Learning with Constraints: Motivation Modern day AI == Deep Learning (DL) [Learn from

810 views • 68 slides
E9 205 Machine Learning for Signal Processing Understanding Deep Networks 08-11-2019 Instructor

E9 205 Machine Learning for Signal Processing Understanding Deep Networks 08-11-2019 Instructor

E9 205 Machine Learning for Signal Processing Understanding Deep Networks 08-11-2019 Instructor - Sriram Ganapathy (sriramg@iisc.ac.in.in) Understanding Deep Networks Understanding Deep Networks Understanding Deep Networks SVHN dataset

366 views • 23 slides
Deep networks CS 446 The ERM perspective These lectures will follow an ERM perspective on deep

Deep networks CS 446 The ERM perspective These lectures will follow an ERM perspective on deep

Deep networks CS 446 The ERM perspective These lectures will follow an ERM perspective on deep networks: Pick a model/predictor class (network architecture). (We will spend most of our time on this!) Pick a loss/risk. (We will almost

608 views • 40 slides
Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and Courville [chapt. 6,7,8]; AIMA [sect. 21.1-21.3]; Sutton and Barto, Reinforcement Learning: an

527 views • 35 slides
Architecture Perceptron Highway Networks Highway Networks Allows training very deep

Architecture Perceptron Highway Networks Highway Networks Allows training very deep

Highway Networks for Visual Question Answering Aaditya Prakash PhD advisor: James Storer Brandeis University Architecture Perceptron Highway Networks Highway Networks Allows training very deep networks Srivastava et al trained

174 views • 16 slides
On the Expressive Power of Deep Neural Networks Maithra Raghu, Ben Poole, Jon Kleinberg, Surya

On the Expressive Power of Deep Neural Networks Maithra Raghu, Ben Poole, Jon Kleinberg, Surya

On the Expressive Power of Deep Neural Networks Maithra Raghu, Ben Poole, Jon Kleinberg, Surya Ganguli, Jascha Sohl Dickstein Tom Brady Deep Neural Networks Recent successes in using Deep neural networks for image classification,

1.32k views • 17 slides
Deep Neural Networks and Molecular Dynamics Roberto Car Princeton University MAX Conference

Deep Neural Networks and Molecular Dynamics Roberto Car Princeton University MAX Conference

Deep Neural Networks and Molecular Dynamics Roberto Car Princeton University MAX Conference ICTP, Trieste, Jan 29-31 2018 Deep Neural Networks When properly trained, deep NN are capable of learning the complex functional dependence of

442 views • 17 slides
Visualizing and Interpreting Deep Neural Networks Bolei Zhou Department of Information

Visualizing and Interpreting Deep Neural Networks Bolei Zhou Department of Information

Visualizing and Interpreting Deep Neural Networks Bolei Zhou Department of Information Engineering The Chinese University of Hong Kong Deep Neural Networks are Everywhere Playing Go Making Medical Decision Understanding Scenes Deep Neural

1.1k views • 46 slides
CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided

CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided

CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided Backpropagation, Deep Dream, Deep Art, Fooling Convolutional Neural Networks Mitesh M. Khapra Department of Computer Science and Engineering Indian Institute

1.07k views • 72 slides
Local Generation Network Credits Considering the proposed solution and potential alternatives

Local Generation Network Credits Considering the proposed solution and potential alternatives

Local Generation Network Credits Considering the proposed solution and potential alternatives Presentation slides and summary of discussion Updated 29 March 2016 Second Stakeholder Workshop Mercure Hotel, Sydney 15 March 2016 AEMC PAGE 1

444 views • 29 slides
Implementing constrained access Industry Forum 13 March 2018 AGENDA Agenda item Presenter

Implementing constrained access Industry Forum 13 March 2018 AGENDA Agenda item Presenter

Implementing constrained access Industry Forum 13 March 2018 AGENDA Agenda item Presenter Zaeen Khan, Executive Director Introduction and background Public Utilities Office Ashwin Raj, Project Lead Network connections and access reforms

521 views • 29 slides
Dispatch and Scheduling Proposed Decision Dispatch and Scheduling Proposed Decision Jonathan

Dispatch and Scheduling Proposed Decision Dispatch and Scheduling Proposed Decision Jonathan

Dispatch and Scheduling Proposed Decision Dispatch and Scheduling Proposed Decision Jonathan OSullivan Dundalk, October 2010 d lk b Key Themes y Holistic Market Approach Holistic Market Approach Overall objectives Energy

960 views • 9 slides
SESAR 2020 - PJ09 DCB A dvanced D emand &amp; C apacity B alancing 6 th 7 th March 2018

SESAR 2020 - PJ09 DCB A dvanced D emand &amp; C apacity B alancing 6 th 7 th March 2018

SESAR 2020 - PJ09 DCB A dvanced D emand &amp; C apacity B alancing 6 th 7 th March 2018 (Madrid) Hamid KADOUR PJ09 - Partner Organisations PJ09 ANSP Partner 24 Partner Organisations - EUROCONTROL Experimental Centre - Network Manager -

531 views • 20 slides
ENMAX GIS in Azure May 16,2019 Background Why the Azure cloud? In 2016 ENMAX IT

ENMAX GIS in Azure May 16,2019 Background Why the Azure cloud? In 2016 ENMAX IT

ENMAX GIS in Azure May 16,2019 Background Why the Azure cloud? In 2016 ENMAX IT Infrastructure launched a Data Centre Transformation project as ENMAX had out grown our on premise data center This move was also consistent with a

186 views • 15 slides
Enhanced data management techniques for real time logistics planning and scheduling Andrew Palmer

Enhanced data management techniques for real time logistics planning and scheduling Andrew Palmer

Enhanced data management techniques for real time logistics planning and scheduling Andrew Palmer 5 th International SRF Workshop, Cambridge 30 th November 2018 About LOGISTAR A consortium of 15 partners from across Europe, coordinated by

391 views • 11 slides
in the Acquisition Pipeline David W. Embley Christopher Almquist, Bill Barrett, Alan Cannaday,

in the Acquisition Pipeline David W. Embley Christopher Almquist, Bill Barrett, Alan Cannaday,

Research in the Acquisition Pipeline David W. Embley Christopher Almquist, Bill Barrett, Alan Cannaday, Robert Clawson, Jake Gehring, Doug Kennard, Tae Woo Kim, Steve Liddle, Peter Lindes, Deryle Lonsdale, Thomas Packer, Joseph Park, Pat

896 views • 46 slides
North &amp; South Country CAGs MAC Update 9 September 2015 DM: 13259889 Constrained connections

North &amp; South Country CAGs MAC Update 9 September 2015 DM: 13259889 Constrained connections

North &amp; South Country CAGs MAC Update 9 September 2015 DM: 13259889 Constrained connections for new generation Situation Preliminary Access Offers August 2015 Under the Access Code, Western Power Complication must make all reasonable

196 views • 8 slides

More recommend