with deep neural networks
play

WITH DEEP NEURAL NETWORKS INTELLIGENT ROBOTICS SEMINAR PIA UK - PowerPoint PPT Presentation

Jan 02, 2023 •272 likes •481 views

OPTICAL FLOW ESTIMATION WITH DEEP NEURAL NETWORKS INTELLIGENT ROBOTICS SEMINAR PIA UK 25.11.2019 OUTLINE 1. Optical Flow Motivation 2. Neural Networks Basics 3. Optical Flow with Deep Neural Networks 1. PWC-Net Model 2. PWC-Net Results

  1. OPTICAL FLOW ESTIMATION WITH DEEP NEURAL NETWORKS INTELLIGENT ROBOTICS – SEMINAR PIA ČUK 25.11.2019

  2. OUTLINE 1. Optical Flow Motivation 2. Neural Networks Basics 3. Optical Flow with Deep Neural Networks 1. PWC-Net Model 2. PWC-Net Results 4. Discussion and Outlook 25.11.2019 2

  3. 1. OPTICAL FLOW • Motion estimation in video • “O ptical flow is the distribution of apparent velocities of movement of brightness patterns in an image.” ¹ • For subsequent frames, determine displacement vector for each pixel • https://www.youtube.com/watch?NR=1&v=-F38u9w6YII ¹ Horn, Berthold KP , And Brian G. Schunck. "Determining Optical Flow." Artificial Intelligence 17.1-3 (1981): 185-203. 25.11.2019 3 https://devblogs.nvidia.com/an-introduction-to-the-nvidia-optical-flow-sdk/, retrieved 18.11.2019

  4. 1. OPTICAL FLOW • Colour code for visualisation: Baghaie, Ahmadreza, Roshan D’Souza , and Zeyun Yu. "Dense descriptors for optical flow estimation: a comparative study." 25.11.2019 4 Journal of Imaging 3.1 (2017): 12. https://devblogs.nvidia.com/an-introduction-to-the-nvidia-optical-flow-sdk/, retrieved 18.11.2019

  5. 1. OPTICAL FLOW • Possible applications: visual odometry, autonomous driving, semantic segmentation… → Whenever motion conveys useful information Sun, Deqing, et al. "PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume." 25.11.2019 5 Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.

  6. 2. NEURAL NETWORKS BASICS • Inspired by neural networks in the human brain • Neuron as atomic unit • Deep neural networks: neurons organised in layers 25.11.2019 6

  7. 2.1. CONVOLUTIONAL NEURAL NETWORKS • Class of deep neural networks well-suited for computer vision • Use one filter kernel for whole image, “move” it along width, height axes → multiply at every position • Also called “feature extraction” 25.11.2019 7

  8. 3. OPTICAL FLOW WITH DEEP NEURAL NETWORKS • “Classical” approaches: complex optimization problems, computationally expensive → Not suitable for real-time applications • First DNN approaches: trade-off between accuracy and size of the model • No end-to-end training 25.11.2019 8

  9. 3.1. PWC-NET Sun, Deqing, et al. "PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2018. • Uses domain knowledge to reduce complexity • State-of-the-art accuracy with end-to-end training 25.11.2019 9

  10. 3.1. PWC-NET PWC: Pyramid, Warping, Cost volume 1. Feature extraction from input images with feature pyramid, i.e. convolutional layers • Reduction of spatial resolution 2. Optical flow estimation for every level of feature pyramid • Start with last convolutional layer, finish on input level • Warping and cost volume used in optical flow estimation 25.11.2019 10

  11. 3.1. PWC-NET Feature pyramid 1. Compute cost volume: find most similar pixel in features for other image Sun, Deqing, et al. "Models matter, so does training: An empirical study of cnns for optical flow estimation.“ 25.11.2019 11 arXiv preprint arXiv:1809.05571 (2018).

  12. 3.1. PWC-NET 2. Optical flow estimation: • Cost volume • Features of first image → Output OF for lowest level Sun, Deqing, et al. "Models matter, so does training: An empirical study of cnns for optical flow estimation.“ 25.11.2019 12 arXiv preprint arXiv:1809.05571 (2018).

  13. 3.1. PWC-NET 1. Upsample OF to match spatial dimensions 2. Warp the features of the 2nd image towards the 1st image Sun, Deqing, et al. "Models matter, so does training: An empirical study of cnns for optical flow estimation.“ 25.11.2019 13 arXiv preprint arXiv:1809.05571 (2018).

  14. WHY WARPING? • Second image becomes more similar to first image • Pixel displacement becomes smaller • For finding corresponding pixel in cost volume, only need to look at neighbourhood of pixel → Computationally much more effective 25.11.2019 14

  15. 3.1. PWC-NET Sun, Deqing, et al. "Models matter, so does training: An empirical study of cnns for optical flow estimation.“ 25.11.2019 15 arXiv preprint arXiv:1809.05571 (2018).

  16. 3.2. PWC-NET RESULTS • Inference fast enough for real-time application • PWC-Net-small for mobile applications Sun, Deqing, et al. "PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume." Proceedings of the IEEE 25.11.2019 16 Conference on Computer Vision and Pattern Recognition. 2018.

  17. 3.2. PWC-NET RESULTS • https://www.youtube.com/watch?v=rCoUcjSz9nQ Sun, Deqing, et al. "PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume." 25.11.2019 17 Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.

  18. 4. DISCUSSION AND OUTLOOK • First DNN model to outperform all classical approaches on all popular benchmarks • Code publicly available: https://github.com/NVlabs/PWC-Net • Follow-up paper: Sun, Deqing, et al. "Models Matter, So Does Training: An Empirical Study of CNNs for Optical Flow Estimation." arXiv preprint arXiv:1809.05571 (2018). • To be improved: occlusion detection, unsupervised training 25.11.2019 18

  19. THANK YOU FOR YOUR ATTENTION! 25.11.2019 19

  20. 1. OPTICAL FLOW • Error metric: Endpoint Error (EPE) • Euclidian distance between estimated and ground truth vector for one pixel: 𝑊 𝑓𝑡𝑢 − 𝑊 𝑕𝑢 • Compute average EPE for all pixels of an image pair: AEPE 25.11.2019 20

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan Zelener Machine Learning Reading Group January 7 th 2016 The Graduate Center, CUNY Objectives Explain some of the trends of deep learning and

680 views • 44 slides
Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking

Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking

Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In

987 views • 40 slides
Visualizing and Interpreting Deep Neural Networks Bolei Zhou Department of Information

Visualizing and Interpreting Deep Neural Networks Bolei Zhou Department of Information

Visualizing and Interpreting Deep Neural Networks Bolei Zhou Department of Information Engineering The Chinese University of Hong Kong Deep Neural Networks are Everywhere Playing Go Making Medical Decision Understanding Scenes Deep Neural

1.1k views • 46 slides
On the Expressive Power of Deep Neural Networks Maithra Raghu, Ben Poole, Jon Kleinberg, Surya

On the Expressive Power of Deep Neural Networks Maithra Raghu, Ben Poole, Jon Kleinberg, Surya

On the Expressive Power of Deep Neural Networks Maithra Raghu, Ben Poole, Jon Kleinberg, Surya Ganguli, Jascha Sohl Dickstein Tom Brady Deep Neural Networks Recent successes in using Deep neural networks for image classification,

1.32k views • 17 slides
Outline Evolution of neurocomputing Artificial neural networks Feed forward

Outline Evolution of neurocomputing Artificial neural networks Feed forward

Introduction to Neural Networks and Deep Learning Ling Guan References: S. Haykin, Neural Networks , 2 nd Edition, New Jersey: Prentice Hall, 1. 2004. C. M. Bishop, Neural Networks for Pattern Re cognition, New York: 2. Oxford University

451 views • 17 slides
Introduction to Artificial Intelligence Neural Networks - Deep Learning for NLP Janyl Jumadinova

Introduction to Artificial Intelligence Neural Networks - Deep Learning for NLP Janyl Jumadinova

Introduction to Artificial Intelligence Neural Networks - Deep Learning for NLP Janyl Jumadinova November 21, 2016 Neural Networks 2/20 Neural Networks 3/20 Neural Networks Neural computing requires a number of neurons , to be connected

813 views • 21 slides
Optimizing Deep Neural Networks Leena Chennuru Vankadara 26-10-2015 Table of Contents Neural

Optimizing Deep Neural Networks Leena Chennuru Vankadara 26-10-2015 Table of Contents Neural

Optimizing Deep Neural Networks Leena Chennuru Vankadara 26-10-2015 Table of Contents Neural Networks and loss surfaces Problems of Deep Architectures Optimization in Neural Networks Under fitting Proliferation of Saddle

1.67k views • 41 slides
Deep Neural Networks and Molecular Dynamics Roberto Car Princeton University MAX Conference

Deep Neural Networks and Molecular Dynamics Roberto Car Princeton University MAX Conference

Deep Neural Networks and Molecular Dynamics Roberto Car Princeton University MAX Conference ICTP, Trieste, Jan 29-31 2018 Deep Neural Networks When properly trained, deep NN are capable of learning the complex functional dependence of

442 views • 17 slides
CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided

CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided

CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided Backpropagation, Deep Dream, Deep Art, Fooling Convolutional Neural Networks Mitesh M. Khapra Department of Computer Science and Engineering Indian Institute

1.07k views • 72 slides
Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and Courville [chapt. 6,7,8]; AIMA [sect. 21.1-21.3]; Sutton and Barto, Reinforcement Learning: an

527 views • 35 slides
Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Learning in Neural Networks w 1,3 w 3,5 1 3 5 w w 1,4 3,6 w w 2,3 4,5 2 4 6 w w 2,4 4,6 Class #18: Back-Propagation; } A neural network can learn a classification function by Tuning Hyper-Parameters adjusting its weights to

259 views • 4 slides
Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream Umut Gcl and Marcel A. J. van Gerven Article overview by Ilya Kuzovkin Computational Neuroscience Seminar University of Tartu

852 views • 63 slides
Recurrent Neural Networks Deep neural networks have enabled major advances in machine learning

Recurrent Neural Networks Deep neural networks have enabled major advances in machine learning

Recurrent Neural Networks Deep neural networks have enabled major advances in machine learning and AI y t-1 y t y t+1 Computer vision h t-1 h t h t+1 Language translation Speech recognition h t-1 h t h t+1 Question answering x t-1 x t x t+1

454 views • 43 slides
Concise Introduction to Deep Neural Networks Outline: Classification problems Motivating

Concise Introduction to Deep Neural Networks Outline: Classification problems Motivating

Concise Introduction to Deep Neural Networks Outline: Classification problems Motivating Deep (large) Neural Network (DNN) classifiers Neurons and DNN architectures Numerical training of DNNs (supervised deep learning) Spiking

630 views • 33 slides
ICASSP 2017 Tutorial on Methods for Interpreting and Understanding Deep Neural Networks G.

ICASSP 2017 Tutorial on Methods for Interpreting and Understanding Deep Neural Networks G.

ICASSP 2017 Tutorial on Methods for Interpreting and Understanding Deep Neural Networks G. Montavon, W. Samek, K.-R. Mller Part 2: Making Deep Neural Networks Transparent 5 March 2017 Making Deep Neural Nets Transparent DNN transparency

448 views • 44 slides
Deep Neural Networks CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu Deep learning slides credit:

Deep Neural Networks CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu Deep learning slides credit:

Deep Neural Networks CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu Deep learning slides credit: Vlad Morariu Training (Deep) Neural Networks Computational graphs Improvements to gradient descent Stochastic gradient descent Momentum

814 views • 21 slides
3D Pattern Recognition Using Deep Neural Networks for Liquid Argon Time Projection Chambers

3D Pattern Recognition Using Deep Neural Networks for Liquid Argon Time Projection Chambers

3D Pattern Recognition Using Deep Neural Networks for Liquid Argon Time Projection Chambers (LArTPCs) Kazuhiro Terao SLAC National Accelerator Laboratory 1 Introduction This workshops charge : This meeting will focus on the options of

448 views • 32 slides
CreativeAI 3D (Geometric) Domain Niloy Mitra Iasonas Kokkinos Paul Guerrero Nils Thuerey

CreativeAI 3D (Geometric) Domain Niloy Mitra Iasonas Kokkinos Paul Guerrero Nils Thuerey

CreativeAI 3D (Geometric) Domain Niloy Mitra Iasonas Kokkinos Paul Guerrero Nils Thuerey Tobias Ritschel UCL UCL UCL TUM UCL Timetable Niloy Paul Nils Introduction 2:15 pm X X X 2:25 pm X Machine Learning Basics Theory

1.36k views • 107 slides
Intro to Image Understanding (CSC420) Projects Proposal Deadline : Nov 2 (Sunday), 11.59pm, 2014

Intro to Image Understanding (CSC420) Projects Proposal Deadline : Nov 2 (Sunday), 11.59pm, 2014

Intro to Image Understanding (CSC420) Projects Proposal Deadline : Nov 2 (Sunday), 11.59pm, 2014 Report Submission Deadline : December 7 (Sunday), 11.59pm, 2014 Max points for report: 30, max points for presentation: 20 Projects can be done

367 views • 9 slides
CSC420: Intro to Image Understanding Introduction Sanja Fidler September 11, 2014 Sanja Fidler

CSC420: Intro to Image Understanding Introduction Sanja Fidler September 11, 2014 Sanja Fidler

CSC420: Intro to Image Understanding Introduction Sanja Fidler September 11, 2014 Sanja Fidler Intro to Image Understanding 1 / 53 The Team Instructor : Sanja Fidler ( fidler@cs.toronto.edu ) Office : 283B in Pratt Office hours : Tuesday

955 views • 69 slides
Structure of Vision Problems Alan Yuille (UCLA). Machine Learning Theory of Machine Learning

Structure of Vision Problems Alan Yuille (UCLA). Machine Learning Theory of Machine Learning

Structure of Vision Problems Alan Yuille (UCLA). Machine Learning Theory of Machine Learning is beautiful and deep. But, how useful is it for vision? Vision rarely has an obvious vector space structure. Image Formation Images

424 views • 28 slides
Lecture: Motion Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab

Lecture: Motion Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab

Motion Lecture: Motion Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab 19-Nov-2019 1 St Stanfor ord University CS 131 Roadmap Motion Pixels Segments Images Videos Web Neural networks Convolutions Recognition

1.11k views • 84 slides
Junning Li April 17 th , 2012 Laboratory of Neuro Imaging University of California, Los Angeles

Junning Li April 17 th , 2012 Laboratory of Neuro Imaging University of California, Los Angeles

Junning Li April 17 th , 2012 Laboratory of Neuro Imaging University of California, Los Angeles Background Water molecules in biology tissue tend to diffuse faster along, relative to across, obstacle structures such as fibers or membranes.

429 views • 11 slides
Carlos Geijo 6th WMO Symposium on Data Assimilation. Maryland 7-11 October 2013 INTRODUCTION

Carlos Geijo 6th WMO Symposium on Data Assimilation. Maryland 7-11 October 2013 INTRODUCTION

Assimilation of Radar Data in a Convection Permitting NWP System using the Field Alignment Technique Carlos Geijo 6th WMO Symposium on Data Assimilation. Maryland 7-11 October 2013 INTRODUCTION DAbyFA , a method proposed by a group of MIT

755 views • 29 slides

More recommend