deep learning for vision
play

Deep Learning for Vision Presented by Kevin Matzen Wednesday, - PowerPoint PPT Presentation

Sep 15, 2022 •290 likes •717 views

Deep Learning for Vision Presented by Kevin Matzen Wednesday, April 9, 14 Quick Intro - DNN Feed-forward Sparse connectivity (layer to layer) Different layer types Recently popularized for vision [Krizhevsky, et. al. NIPS 2012]

  1. Deep Learning for Vision Presented by Kevin Matzen Wednesday, April 9, 14

  2. Quick Intro - DNN • Feed-forward • Sparse connectivity (layer to layer) • Different layer types • Recently popularized for vision [Krizhevsky, et. al. NIPS 2012] Wednesday, April 9, 14

  3. The Layers • Convolution • Loss functions • Fully connected • Image processing • Pooling • Neuron activation function • Normalization Wednesday, April 9, 14

  4. deeplearning.net/tutorial/lenet.html Wednesday, April 9, 14

  5. [Krizhevsky, NIPS 2012] Wednesday, April 9, 14

  6. Software • code.google.com/p/cuda-convnet/ [nvidia gpu] • github.com/UCB-ICSI-Vision-Group/decaf-release/ [deprecated; cpu-only] • caffe.berkeleyvision.org [cpu; nvidia gpu] • research.google.com/archive/ large_deep_networks_nips2012.html [proprietary; distributed system] Wednesday, April 9, 14

  7. DeepPose: Human Pose Estimation via Deep Neural Networks Alexander Toshev, Christian Szegedy - CVPR 2014 DeepFace: Closing the Gap to Human-Level Performance in Face Verification Yaniv Taigman, Ming Yang, Marc’Aurelio Ranzato, Lior Wolf - CVPR 2014 Wednesday, April 9, 14

  8. DeepPose: Human Pose Estimation via Deep Neural Networks Alexander Toshev, Christian Szegedy - CVPR 2014 DeepFace: Closing the Gap to Human-Level Performance in Face Verification Yaniv Taigman, Ming Yang, Marc’Aurelio Ranzato, Lior Wolf - CVPR 2014 Wednesday, April 9, 14

  9. Input: Uncropped photo Output: Joint locations Wednesday, April 9, 14

  10. Pipeline 1. Person detection 2. Joint position regression 3. Joint refinement Wednesday, April 9, 14

  11. Datasets Leeds Sports Pose (LSP) [Johnson, et. al. BMVC 2010] 14 joint locations 2000 main person - 150 px Frames Labeled in Cinema (FLIC) [Sapp, et. al. CVPR 2013] 5003 person detector every 10 frames of 30 movies 20k candidates Image Parse [Ramanan NIPS 2006] mturk 305 images 10 upperbody joints similar to leeds includes casual photos Buffy Stickmen 748 frames Wednesday, April 9, 14

  12. Person Detection • Input: Uncropped image • Output: Cropped image • LSP dataset - No person detector • FLIC dataset - Enlarged face detector Wednesday, April 9, 14

  13. Wednesday, April 9, 14

  14. Main difference Wednesday, April 9, 14

  15. Wednesday, April 9, 14

  16. Runtime • 0.1s per image - 12 cores (SotA - 1.5s, 4s) • Training stage 0 - 3 days • Training refinement - 7 days each Wednesday, April 9, 14

  17. Evaluation • Percentage of Correct Parts (PCP) • Correct if predicted limb is within 1/2 of correct limb length • Percentage of Detected Joints (PDJ) • Predicted and correct joints are within some factor of torso diameter Wednesday, April 9, 14

  18. Wednesday, April 9, 14

  19. Wednesday, April 9, 14

  20. Wednesday, April 9, 14

  21. Wednesday, April 9, 14

  22. DeepPose: Human Pose Estimation via Deep Neural Networks Alexander Toshev, Christian Szegedy - CVPR 2014 DeepFace: Closing the Gap to Human-Level Performance in Face Verification Yaniv Taigman, Ming Yang, Marc’Aurelio Ranzato, Lior Wolf - CVPR 2014 Wednesday, April 9, 14

  23. Pipeline • Detect faces • Correct out-of-plane rotation • Generate features via CNN • Classify Wednesday, April 9, 14

  24. Alignment Wednesday, April 9, 14

  25. Fiducial Detection • LBP histograms • Support Vector Regressor • Iteratively transform and predict • 6 fiducial points for 2D alignment • 67 fiducial points for 3D alignment Wednesday, April 9, 14

  26. 3D Alignment • Iterative affine camera PnP • 3D reference - Average mesh of USF Human-ID dataset • Considers fiducial covariance • Residuals applied to reference mesh • Affine warp texture Wednesday, April 9, 14

  27. CNN Architecture Wednesday, April 9, 14

  28. CNN Architecture Features Wednesday, April 9, 14

  29. CNN Architecture weight sharing no weight sharing Wednesday, April 9, 14

  30. Training softmax cross-entropy loss -log p k Wednesday, April 9, 14

  31. Sparsity • ReLU nonlinearly - rectified linear unit max(0, x) • 75% model parameters = 0 • Dropout - first fully connected layer Wednesday, April 9, 14

  32. Normalization • ReLU - unbounded • Normalize features to [0, 1] based on holdout Wednesday, April 9, 14

  33. Verification Metrics • Unsupervised - dot product • χ 2 similarity • Siamese network Wednesday, April 9, 14

  34. Χ 2 Similarity • Χ 2 (f 1 ,f 2 ) = Σ i w i (f 1 [i] - f 2 [i]) 2 /(f 1 [i] + f 2 [i]) • weights learned via svm Wednesday, April 9, 14

  35. Siamese Network FC 4096-to-1 - Wednesday, April 9, 14

  36. Datasets • Social Face Classification (SFC) • Presumably Facebook photos • 4.4 mil faces; 4,030 people • No overlap with other datasets Wednesday, April 9, 14

  37. Datasets • Labeled Faces in the Wild (LFW) • 13,323 faces; 5,749 celebs • 6,000 pairs • Restricted protocol - same/not same labels at training • Unrestricted protocol - identities during training • Unsupervised - no training on LFW Wednesday, April 9, 14

  38. Datasets • YouTube Faces (YTF) • 3,425 videos of 1,595 subjects • Subset of celebs from LFW Wednesday, April 9, 14

  39. SFC Training Perf Reduce data by Reduce data by Remove layers omitting people omitting examples from network Wednesday, April 9, 14

  40. LFW Perf Wednesday, April 9, 14

  41. Runtime • 0.18 s - feature extraction (1 core; 2.2 GHz) • 0.05 s - alignment • 0.33 s - total Wednesday, April 9, 14

  42. Questions? Wednesday, April 9, 14

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Deep Learning in Computer Vision Caner Hazrba Deep Learning in Action 24. June 15

Deep Learning in Computer Vision Caner Hazrba Deep Learning in Action 24. June 15

Deep Learning in Computer Vision Caner Hazrba Deep Learning in Action 24. June 15 Computer Vision Group 6 Postdocs, 16 PhD students Caner Hazrba | vision.in.tum.de Deep Learning in Computer Vision 2 Research in Computer Vision

960 views • 30 slides
FUNDAMENTALS OF DEEP LEARNING FOR COMPUTER VISION Twin Karmakharm DLI Certified Instructor

FUNDAMENTALS OF DEEP LEARNING FOR COMPUTER VISION Twin Karmakharm DLI Certified Instructor

FUNDAMENTALS OF DEEP LEARNING FOR COMPUTER VISION Twin Karmakharm DLI Certified Instructor This event is organised and run by... What we do: Project work and consultancy Training Deep Learning, HPC, GPU Deep Learning

651 views • 32 slides
Mocha.jl Deep Learning in Julia Chiyuan Zhang (@pluskid) CSAIL, MIT Deep Learning Learning

Mocha.jl Deep Learning in Julia Chiyuan Zhang (@pluskid) CSAIL, MIT Deep Learning Learning

Mocha.jl Deep Learning in Julia Chiyuan Zhang (@pluskid) CSAIL, MIT Deep Learning Learning with multi-layer (3~30) neural networks, on a huge training set. State-of-the-art on many AI tasks Computer Vision : Image Classification,

1.53k views • 23 slides
Deep Learning and its Applications in Vision and Robotics

Deep Learning and its Applications in Vision and Robotics

#dlvr Deep Learning and its Applications in Vision and Robotics http://juxi.net/workshop/deeplearning-applications-vision-robotics-2015/ Deep Learning and its Applications in Vision and Robotics #dlvr Programme 09:15-09:30 Welcome by Juxi,

388 views • 6 slides
APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston Practical DEEP

APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston Practical DEEP

APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston Practical DEEP LEARNING Examples Image Classification, Object Detection, Localization, Speech Recognition, Speech Translation, Action Recognition, Scene

885 views • 40 slides
EE-559 Deep learning 7. Networks for computer vision Fran cois Fleuret

EE-559 Deep learning 7. Networks for computer vision Fran cois Fleuret

EE-559 Deep learning 7. Networks for computer vision Fran cois Fleuret https://fleuret.org/dlc/ [version of: June 8, 2018] COLE POLYTECHNIQUE FDRALE DE LAUSANNE Tasks and data-sets Fran cois Fleuret EE-559 Deep learning

1.93k views • 170 slides
Deep Learning in Computer Vision Yikang Li MMLab, The Chinese University of Hong Kong Sep 22nd,

Deep Learning in Computer Vision Yikang Li MMLab, The Chinese University of Hong Kong Sep 22nd,

Deep Learning in Computer Vision Yikang Li MMLab, The Chinese University of Hong Kong Sep 22nd, 2017 @Microsoft Research Asia, China Outline 1. Introduction 2. Roadmap of Deep Learning 3. DL in CV: Object detection 4. DL in CV: Image

630 views • 45 slides
E9 205 Machine Learning for Signal Procesing Deep Learning for Audio and Vision 20-11-2019

E9 205 Machine Learning for Signal Procesing Deep Learning for Audio and Vision 20-11-2019

E9 205 Machine Learning for Signal Procesing Deep Learning for Audio and Vision 20-11-2019 Speech Recognition Noise Channel Automatic Speech Systems Courtesy Google Images Signal Modeling Short-term spectra integrated in mel frequency

555 views • 32 slides
Table of Contents Convolutional Neural Nets (CNNs) 1 Deep Q Learning 2 Lecture 6: CNNs and Deep

Table of Contents Convolutional Neural Nets (CNNs) 1 Deep Q Learning 2 Lecture 6: CNNs and Deep

Lecture 6: CNNs and Deep Q Learning 1 Emma Brunskill CS234 Reinforcement Learning. Winter 2019 1 With many slides for DQN from David Silver and Ruslan Salakhutdinov and some vision slides from Gianni Di Caro and images from Stanford CS231n,

1.17k views • 68 slides
Table of Contents Convolutional Neural Nets (CNNs) 1 Deep Q Learning 2 Lecture 6: CNNs and Deep

Table of Contents Convolutional Neural Nets (CNNs) 1 Deep Q Learning 2 Lecture 6: CNNs and Deep

Lecture 6: CNNs and Deep Q Learning 2 Emma Brunskill CS234 Reinforcement Learning. Winter 2018 2 With many slides for DQN from David Silver and Ruslan Salakhutdinov and some vision slides from Gianni Di Caro and images from Stanford CS231n,

850 views • 67 slides
Delving Deep into Computer Vision Caner Hazirbas Machine Learning Meetup #1 Delving Deep into

Delving Deep into Computer Vision Caner Hazirbas Machine Learning Meetup #1 Delving Deep into

Delving Deep into Computer Vision Caner Hazirbas Machine Learning Meetup #1 Delving Deep into Computer Vision FlowNet FuseNet PoseLSTM DDFF Caner Hazirbas | hazirbas@cs.tum.edu Delving Deep into Computer Vision 2 Delving Deep into

1.14k views • 45 slides
Deep multi-task learning with evolving weights Machine learning - computer vision published in

Deep multi-task learning with evolving weights Machine learning - computer vision published in

Deep multi-task learning with evolving weights Machine learning - computer vision published in European Symposium on Artificial Neural Networks (ESANN 2016) Soufiane Belharbi Romain Hrault Clment Chatelain Sbastien Adam

631 views • 44 slides
Impact of Deep Learning Speech Recogni4on Computer Vision

Impact of Deep Learning Speech Recogni4on Computer Vision

Impact of Deep Learning Speech Recogni4on Computer Vision Recommender Systems Language Understanding Drug Discovery and Medical Image

526 views • 21 slides
Deep learning 8.1. Computer vision tasks Fran cois Fleuret https://fleuret.org/dlc/ Dec 20,

Deep learning 8.1. Computer vision tasks Fran cois Fleuret https://fleuret.org/dlc/ Dec 20,

Deep learning 8.1. Computer vision tasks Fran cois Fleuret https://fleuret.org/dlc/ Dec 20, 2020 Computer vision tasks: classification, object detection, semantic or instance segmentation, Fran cois Fleuret Deep learning /

426 views • 20 slides
BBM413 Fundamentals of Image Processing Introduction to Deep Learning Erkut Erdem

BBM413 Fundamentals of Image Processing Introduction to Deep Learning Erkut Erdem

BBM413 Fundamentals of Image Processing Introduction to Deep Learning Erkut Erdem Hacettepe University Computer Vision Lab (HUCVL) What is deep learning? Y. LeCun, Y. Bengio, G. Hinton, "Deep Learning" , Nature,

1.12k views • 84 slides
Recent Trends in Computer Vision and Deep Learning Systems Yangqing Jia Lead Researcher and

Recent Trends in Computer Vision and Deep Learning Systems Yangqing Jia Lead Researcher and

Recent Trends in Computer Vision and Deep Learning Systems Yangqing Jia Lead Researcher and Manager of AI Platform, Facebook Computer Vision AlexNet So it begins. VGGNet Punch it. GoogLeNet We must go deeper. ResNet And we took the word

424 views • 40 slides
H Workshop on Photon Physics and Simulation at Hadron Colliders 2019 Ruggero Turra on

H Workshop on Photon Physics and Simulation at Hadron Colliders 2019 Ruggero Turra on

H Workshop on Photon Physics and Simulation at Hadron Colliders 2019 Ruggero Turra on behalf of the ATLAS and CMS Collaboration INFN Milano 7 May 2019 Table of contents 1 Higgs boson and its properties 2 ATLAS and CMS analyses 3

572 views • 41 slides
Characteristic number regression for fiducial facial feature extraction Presenter : Xin Fan

Characteristic number regression for fiducial facial feature extraction Presenter : Xin Fan

Characteristic number regression for fiducial facial feature extraction Presenter : Xin Fan Joint work with Prof. Zhongxuan Luo and Dr. Risheng Liu Published in IEEE TIP15 and ICME15 Outline Motivations Facial structure with

890 views • 38 slides
Effective Anti-thrombotic Therapy without Stenting: Intravascular OCT-based Management in Plaque

Effective Anti-thrombotic Therapy without Stenting: Intravascular OCT-based Management in Plaque

Effective Anti-thrombotic Therapy without Stenting: Intravascular OCT-based Management in Plaque Erosion (the EROSION study) Ik-Kyung Jang, MD, PhD In collaboration with Bo Yu, MD, PhD Harvard Medical School Harbin Medical University

421 views • 10 slides
Change-point methods for anomaly detection in fibrous media joint work with E. Spodarev, C. Re-

Change-point methods for anomaly detection in fibrous media joint work with E. Spodarev, C. Re-

Change-point methods for anomaly detection in fibrous media joint work with E. Spodarev, C. Re- denbach, D. Dresvyanskiy, T. Karaseva Vitalii Makogin | 10.10.2019 | Institute of Stochastics, Ulm University and S. Mitrofanov Seite 2 Problem

451 views • 19 slides
Effect of Fiducial Cut on All Backgrounds Plot on the left considers the effect of a fiducial

Effect of Fiducial Cut on All Backgrounds Plot on the left considers the effect of a fiducial

Effect of Fiducial Cut on All Backgrounds Plot on the left considers the effect of a fiducial cut (equal on all sides of the TPC) on the signal/sqrt(background) Plot on the right considers the same cut but only for pi0 events where the

249 views • 5 slides
FLASY2011 Short Baseline Neutrino Anomalies and Future Probes Sanjib Kumar Agarwalla

FLASY2011 Short Baseline Neutrino Anomalies and Future Probes Sanjib Kumar Agarwalla

FLASY2011 Short Baseline Neutrino Anomalies and Future Probes Sanjib Kumar Agarwalla (Sanjib.Agarwalla@ific.uv.es) IFIC/CSIC, University of Valencia, Spain Sanjib K. Agarwalla, FLASY2011, Valencia, Spain, 13/07/11 p.1/32 Definition of

532 views • 32 slides
Widowed stars probe BH kicks Bin C osmos Mathieu Renzo PhD in Amsterdam Collaborators:

Widowed stars probe BH kicks Bin C osmos Mathieu Renzo PhD in Amsterdam Collaborators:

Anton Pannekoek Institute Widowed stars probe BH kicks Bin C osmos Mathieu Renzo PhD in Amsterdam Collaborators: S. E. de Mink, E. Zapartas, Y. G otberg, E. Laplace, R. J. Farmer, S. Toonen, S. Justham, R. G. Izzard, D. J. Lennon,

297 views • 16 slides
BA BAO measure rement fro rom joint anal analysi ysis s of f gal galax axies es and and

BA BAO measure rement fro rom joint anal analysi ysis s of f gal galax axies es and and

BA BAO measure rement fro rom joint anal analysi ysis s of f gal galax axies es and and cosm smic voids ds (Cheng Zhao, Chuang, et al. 2018) & Ro Robust es esti timati ation n of covari riance matri rix of of galaxy

393 views • 24 slides

More recommend