imexnet a forward stable deep neural network
play

IMEXnet - A Forward Stable Deep Neural Network Eldad Haber, Keegan - PowerPoint PPT Presentation

Apr 05, 2024 •30 likes •90 views

IMEXnet - A Forward Stable Deep Neural Network Eldad Haber, Keegan Lensink, Eran Treister and Lars Ruthotto Jun 2019 Outline I Why Implicit I Implicit Explicit I Some results Why Implicit I For CNNs - depth is connected to field of view I

  1. IMEXnet - A Forward Stable Deep Neural Network Eldad Haber, Keegan Lensink, Eran Treister and Lars Ruthotto Jun 2019

  2. Outline I Why Implicit I Implicit Explicit I Some results

  3. Why Implicit I For CNN’s - depth is connected to field of view I Stability of the standard networks can be limited I Vanishing/Exploding gradients Goal: Develop a method that can deal with those problems

  4. Deep Networks and ODE’s ˙ Y = σ ( KY + b ) Y j +1 = Y j + h σ ( K j Y j + b j ) . ↔ I Deep Residual Networks equivalent to Forward Euler for ODE’s I Forward Euler have limitation on stability I Require many steps to converge

  5. Semi-Implicit methods Di ff erent stable integration technique that allows large steps Y j +1 = ( I + h K j ) − 1 ( Y j + h σ ( K j Y j + b j ) − K j Y j ) . ˙ Y = σ ( KY + b ) ↔ Implicit methods are used for I Computational Fluid Dynamics I Computational Electromagnetics I Nonlinear dynamics I Computer graphics

  6. Semi-Implicit methods Come to our poster and see how we apply these networks to many data sets

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Deep Learning Primer Nishith Khandwala Neural Networks Overview Neural Network Basics

Deep Learning Primer Nishith Khandwala Neural Networks Overview Neural Network Basics

Deep Learning Primer Nishith Khandwala Neural Networks Overview Neural Network Basics Activation Functions Stochastic Gradient Descent (SGD) Regularization (Dropout) Training Tips and Tricks Neural Network (NN)

828 views • 45 slides
Neural Networks Neural networks arise from attempts to model Neural Networks human/animal

Neural Networks Neural networks arise from attempts to model Neural Networks human/animal

Feed-forward Networks Network Training Error Backpropagation Deep Learning Feed-forward Networks Network Training Error Backpropagation Deep Learning Neural Networks Neural networks arise from attempts to model Neural Networks

380 views • 9 slides
Neural Network Optimization 1 CS 519: Deep Learning, Winter 2018 Fuxin Li With materials from

Neural Network Optimization 1 CS 519: Deep Learning, Winter 2018 Fuxin Li With materials from

Neural Network Optimization 1 CS 519: Deep Learning, Winter 2018 Fuxin Li With materials from Zsolt Kira Backpropagation learning of a network The algorithm 1. Compute a forward pass on the compute graph (DAG) from the input to all the

407 views • 16 slides
Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference

Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference

Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference Engine Song Han CVA group, Stanford University Apr 7, 2016 1 Intro about me and my advisor Fourth year PhD with Prof. Bill Dally at Stanford.

2.06k views • 70 slides
Deep Learning (jkim@bi.snu.ac.kr) 2015/05/7

Deep Learning (jkim@bi.snu.ac.kr) 2015/05/7

Deep Learning (jkim@bi.snu.ac.kr) 2015/05/7 1 History of Neural Network Research Neural network Deep belief net Back propagation Science

1.27k views • 108 slides
Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference

Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference

Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference Engine Song Han CVA group, Stanford University Jan 6, 2015 A few words about us Fourth year PhD with Prof. Bill Dally at Stanford.

966 views • 54 slides
Neural Networks Greg Mori - CMPT 419/726 Bishop PRML Ch. 5 Feed-forward Networks Network

Neural Networks Greg Mori - CMPT 419/726 Bishop PRML Ch. 5 Feed-forward Networks Network

Feed-forward Networks Network Training Error Backpropagation Deep Learning Neural Networks Greg Mori - CMPT 419/726 Bishop PRML Ch. 5 Feed-forward Networks Network Training Error Backpropagation Deep Learning Neural Networks Neural

712 views • 54 slides
Concise Introduction to Deep Neural Networks Outline: Classification problems Motivating

Concise Introduction to Deep Neural Networks Outline: Classification problems Motivating

Concise Introduction to Deep Neural Networks Outline: Classification problems Motivating Deep (large) Neural Network (DNN) classifiers Neurons and DNN architectures Numerical training of DNNs (supervised deep learning) Spiking

630 views • 33 slides
Neural Networks: What can a network represent Deep Learning, Fall 2020 1 Recap : Neural

Neural Networks: What can a network represent Deep Learning, Fall 2020 1 Recap : Neural

Neural Networks: What can a network represent Deep Learning, Fall 2020 1 Recap : Neural networks have taken over AI Tasks that are made possible by NNs, aka deep learning Tasks that were once assumed to be purely in the human domain

1.92k views • 150 slides
Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Learning in Neural Networks w 1,3 w 3,5 1 3 5 w w 1,4 3,6 w w 2,3 4,5 2 4 6 w w 2,4 4,6 Class #18: Back-Propagation; } A neural network can learn a classification function by Tuning Hyper-Parameters adjusting its weights to

259 views • 4 slides
Neural Networks: What can a network represent Deep Learning, Spring 2018 1 Recap : Neural

Neural Networks: What can a network represent Deep Learning, Spring 2018 1 Recap : Neural

Neural Networks: What can a network represent Deep Learning, Spring 2018 1 Recap : Neural networks have taken over AI Tasks that are made possible by NNs, aka deep learning 2 Recap : NNets and the brain In their basic form, NNets

2.14k views • 145 slides
IMPLEMENTATION OF A PARALLEL BATCH TRAINING ALGORITHM FOR DEEP NEURAL NETWORK YUPING LIN IFLYTEK

IMPLEMENTATION OF A PARALLEL BATCH TRAINING ALGORITHM FOR DEEP NEURAL NETWORK YUPING LIN IFLYTEK

IMPLEMENTATION OF A PARALLEL BATCH TRAINING ALGORITHM FOR DEEP NEURAL NETWORK YUPING LIN IFLYTEK LABORATORY FOR NEURAL COMPUTING FOR MACHINE LEARNING DEPARTMENT OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE YORK UNIVERSITY, TORONTO NOVEMBER

211 views • 20 slides
EN. 601.467/667 Introduc3on to Human Language Technology Deep Learning I Shinji Watanabe 1

EN. 601.467/667 Introduc3on to Human Language Technology Deep Learning I Shinji Watanabe 1

EN. 601.467/667 Introduc3on to Human Language Technology Deep Learning I Shinji Watanabe 1 Todays agenda Introduction of deep neural network Basics of neural network 2 Short bio Research interests Automatic speech

1.71k views • 62 slides
How to Evaluate Efficient Deep Neural Network Approaches Vivienne Sze ( @eems_mit)

How to Evaluate Efficient Deep Neural Network Approaches Vivienne Sze ( @eems_mit)

How to Evaluate Efficient Deep Neural Network Approaches Vivienne Sze ( @eems_mit) Massachusetts Institute of Technology In collaboration with Yu-Hsin Chen, Joel Emer, Yannan Wu, Tien-Ju Yang, Google Mobile Vision Team Slides available at

692 views • 48 slides
Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan Zelener Machine Learning Reading Group January 7 th 2016 The Graduate Center, CUNY Objectives Explain some of the trends of deep learning and

680 views • 44 slides
FLEET: Flexible Efficient Ensemble Training for Heterogenous Deep Neural Networks Hui Guan,

FLEET: Flexible Efficient Ensemble Training for Heterogenous Deep Neural Networks Hui Guan,

FLEET: Flexible Efficient Ensemble Training for Heterogenous Deep Neural Networks Hui Guan, Laxmikant Kishor Mokadam, Xipeng Shen, Seung-Hwan Lim, Robert Patton 1 Build an image classifier? Deep Neural Network (DNN) CPU GPU Training

957 views • 41 slides
J-P . Merlet HEPHAISTOS project INRIA 1 Assistance Robotics 2 Assistance Robotics is an

J-P . Merlet HEPHAISTOS project INRIA 1 Assistance Robotics 2 Assistance Robotics is an

Using robotics methods for mobility and medical monitoring of frail people J-P . Merlet HEPHAISTOS project INRIA 1 Assistance Robotics 2 Assistance Robotics is an INRIA team devoted to the assistance to HEPHAISTOS frail people (elderly,

916 views • 42 slides
High-Energy Particle Showers in Coincidence with Downward Lightning Leaders at the Telescope Array

High-Energy Particle Showers in Coincidence with Downward Lightning Leaders at the Telescope Array

High-Energy Particle Showers in Coincidence with Downward Lightning Leaders at the Telescope Array Cosmic Ray Observatory John Belz University of Utah 35 th ICRC Busan, 20170714 Unknowns In Lightning Production When we consider how much

743 views • 30 slides
Initiative Grant (TB-ELI) Grant Call 8 Work & Community Integration 12 October 2018 1

Initiative Grant (TB-ELI) Grant Call 8 Work & Community Integration 12 October 2018 1

Tote Board Enabling Lives Initiative Grant (TB-ELI) Grant Call 8 Work & Community Integration 12 October 2018 1 OBJECTIVES PROGRAMME Registration 09:30 10:00 1. OPPORTUNITY TO NETWORK 10:00 10:30 Lets Get to Know One

677 views • 41 slides
The Timing Model TIMMO Methodology Guest Lecture at Chalmers University Stefan Kuntz,

The Timing Model TIMMO Methodology Guest Lecture at Chalmers University Stefan Kuntz,

ITEA 2 06005: TIMMO Timing Model The Timing Model TIMMO Methodology Guest Lecture at Chalmers University Stefan Kuntz, Continental Automotive GmbH 10-02-2009 TIMMO Methodology Page 1 Welcome About Stefan Kuntz Studied Electrical

536 views • 14 slides
Safe model-based learning for robot control Felix Berkenkamp, Andreas Krause, Angela P. Schoellig

Safe model-based learning for robot control Felix Berkenkamp, Andreas Krause, Angela P. Schoellig

Safe model-based learning for robot control Felix Berkenkamp, Andreas Krause, Angela P. Schoellig @CDC Workshop on Learning for Control 16 th December 2018 The future of automation Felix Berkenkamp 2 The future of automation Large prior

734 views • 48 slides
This chart is a compartion between NTPv4 and IEEE1588v2 capabilities, and a summary of the

This chart is a compartion between NTPv4 and IEEE1588v2 capabilities, and a summary of the

This chart is a compartion between NTPv4 and IEEE1588v2 capabilities, and a summary of the applications requirements. It is an output of the TICTOC Paris Interim June 2008. 1588 wide-area 1588 constrained network NTPv4 Internet NTPv4

206 views • 3 slides
Why are nonlinear filters stable? Ramon van Handel Department of Operations Research &

Why are nonlinear filters stable? Ramon van Handel Department of Operations Research &

Why are nonlinear filters stable? Ramon van Handel Department of Operations Research & Financial Engineering 5th Oxford-Princeton Conference, March 27, 2009 Filtering models Markov additive process ( X t , Y t ) t 0 : ( X t , Y t ) t

680 views • 15 slides
Geant4: why move to 10.3.p01? Hans Wenzel LarSoft coordination meeting th 11 April 2017

Geant4: why move to 10.3.p01? Hans Wenzel LarSoft coordination meeting th 11 April 2017

Geant4: why move to 10.3.p01? Hans Wenzel LarSoft coordination meeting th 11 April 2017 Optical Photon Processes in GEANT4 Optical photon production in Geant4: Cerenkov Process. Scintillation Process (Birks suppression can be

378 views • 13 slides

More recommend