In Introductio ion to Deep Learnin ing I2DL: Prof. Niessner, Prof. Leal-Taixé 1
The Team Lecturers Prof. Dr. Laura Prof. Dr. Matthias Leal-Taixé Niessner PhDs Patrick Andreas Dendorfer Rössler I2DL: Prof. Niessner, Prof. Leal-Taixé 2
What is is Computer r Vis ision? First defined in the 60s in artificial intelligence groups • “Mimic the human visual system” • Center block of robotic intelligence • I2DL: Prof. Niessner, Prof. Leal-Taixé 3
Hubel and Wie iesel • David Hubel and Torsten Wiesel were neurobiologists from Harvard Medical School • Experiment revealed several secrets of the human vision system • Won 2 Nobel prizes I2DL: Prof. Niessner, Prof. Leal-Taixé 4
Hubel and Wie iesel Experiment Recorded electrical activity from • individual neurons in the brains of cats. Slide projector to show specific • patterns to the cats noted specific patterns stimulated activity in specific parts of the brain. • Results: Visual cortex cells are sensitive to the orientation of edges but insensitive to their positio n I2DL: Prof. Niessner, Prof. Leal-Taixé 5
I2DL: Prof. Niessner, Prof. Leal-Taixé 6
Few Decades Later … Computer Vision I2DL: Prof. Niessner, Prof. Leal-Taixé 7
Engineering Computer Mathematics science Artificial Robotics Intelligence ML NLP Algorithms Speech Optimization Computer Optics Vision Image Neuroscience processing Physics Biology Psychology I2DL: Prof. Niessner, Prof. Leal-Taixé 8
Im Image Cla lassific ication Pre 2012 I2DL: Prof. Niessner, Prof. Leal-Taixé 9
Im Image Cla lassific ication Awesome A magic box Open the box Become magicians Post 2012 I2DL: Prof. Niessner, Prof. Leal-Taixé 10
Why Deep Learnin ing? I2DL: Prof. Niessner, Prof. Leal-Taixé 11
Deep Learnin ing His istory I2DL: Prof. Niessner, Prof. Leal-Taixé 12
The Empir ire Stri rikes Back 30 25 Deep Learning 20 Approaches 15 10 5 0 2010 2011 2012 2013 2014 Human 2015 2016 2017 AlexNet VGGNet ResNet Ensemble SENet ILSVRC top-5 error on ImageNet I2DL: Prof. Niessner, Prof. Leal-Taixé 13
What Has Changed? MNIST digit • 1998 recognition dataset LeCun 10 7 pixels used in • et al. training 2012 ImageNet image • Krizhevsky recognition dataset 10 14 pixels used in et al. • training I2DL: Prof. Niessner, Prof. Leal-Taixé 14
What Made this is Possible? Big Data Hardware Deep Models know Models are Models are where to learn from trainable complex I2DL: Prof. Niessner, Prof. Leal-Taixé 15
Deep Learnin ing Recognition ACM Turing Award 2019 (Nobel Prize of Computing) Yann LeCun, Geoffrey Hinton, and Yoshua Bengio I2DL: Prof. Niessner, Prof. Leal-Taixé 16
Deep Learnin ing and Computer Vis isio ion Credits: Dr. Pont-Tuset, ETH Zurich I2DL: Prof. Niessner, Prof. Leal-Taixé 17
Deep Learnin ing and Computer Vis isio ion Credits: Dr. Pont-Tuset, ETH Zurich I2DL: Prof. Niessner, Prof. Leal-Taixé 18
Deep Learnin ing Today Object Detection I2DL: Prof. Niessner, Prof. Leal-Taixé 19
Deep Learnin ing Today Self-driving cars I2DL: Prof. Niessner, Prof. Leal-Taixé 20
Deep Learnin ing Today Emoticon suggestion AlphaGo Machine translation I2DL: Prof. Niessner, Prof. Leal-Taixé 21
Deep Learnin ing Today Alpha Star I2DL: Prof. Niessner, Prof. Leal-Taixé 22
Deep Learnin ing Today Automated Text Generation [Karpathy et al.] I2DL: Prof. Niessner, Prof. Leal-Taixé 23
Deep Learnin ing Today Google Assistant (Google IO’19) I2DL: Prof. Niessner, Prof. Leal-Taixé 24
Deep Learnin ing Today Healthcare, cancer detection I2DL: Prof. Niessner, Prof. Leal-Taixé 25
Deep Learnin ing Market […] market research report Deep Learning Market […] “ the deep learning market is expected to be worth USD 1,722. 2.9 Mill llio ion by 20 2022 22. I2DL: Prof. Niessner, Prof. Leal-Taixé 26
Deep Learnin ing Job Pers rspective • Excellent Job Perspectives! – Automation requires ML/DL -> growth! – Top-notch companies will gladly hire you! • Many industries now: – IT-Companies – Cars, Logistic, Health Care, etc… – Manufacturing / Robotics, etc… I2DL: Prof. Niessner, Prof. Leal-Taixé 27
But: : Als lso Challenging! • High-level understanding is not enough – Need proper theory background – Need proper practical skillsets • Can be competitive! – Many good people – Downloading scripts / running code not enough – Deeper understanding often requires PhDs I2DL: Prof. Niessner, Prof. Leal-Taixé 28
Deep Learnin ing Cult lture I2DL: Prof. Niessner, Prof. Leal-Taixé 29
Deep Learnin ing Memes I2DL: Prof. Niessner, Prof. Leal-Taixé 30
Deep Learnin ing Memes I2DL: Prof. Niessner, Prof. Leal-Taixé 31
Deep Learnin ing Memes I2DL: Prof. Niessner, Prof. Leal-Taixé 32
Deep Learnin ing Memes I2DL: Prof. Niessner, Prof. Leal-Taixé 33
Deep Learnin ing Memes I2DL: Prof. Niessner, Prof. Leal-Taixé 34
Deep Learnin ing Memes I2DL: Prof. Niessner, Prof. Leal-Taixé 35
Deep Learnin ing at TUM I2DL: Prof. Niessner, Prof. Leal-Taixé 36
Many TUM Research Labs use DL Visual l Computin ing Lab (P (Pro rof. f. Niessner) r): • – Research in computer vision, graphics, and machine learning Dynamic ic Visio ion and Learn rnin ing Gro roup (Pro rof. f. Leal-Taixe) • – Research on Computer Vision; e.g., video editing/segmentation etc. • 3D Understanding Lab (Dr. Dai): – Research in 3D scenes and its semantics. • Computer Vision Group (Prof. Cremers) – Research in computer vision and pattern recognition Data Mining and Analytics Lab (Prof. Günnemann) • – Research methods for robust machine learning Computer Aided Medical Procedures (Prof. Navab) • – Research in machine learning for medical applications And probably many more • I2DL: Prof. Niessner, Prof. Leal-Taixé 37
Our r Research Labs • Visual Computing Lab (Prof. Niessner): https://niessnerlab.org/publications.html Twitter: https://twitter.com/MattNiessner – You outu tube: : htt https:/ ://www.y .youtube.com/channel/UCXN2nYjV jVT0 T0cR9G61RPEzK5 K5Q – – Facebook: https://www.facebook.com/matthias.niessner • Dynamic Vision and Learning Lab (Prof. Leal-Taixé): https://dvl.in.tum.de/publications.html – Twitter: https://twitter.com/lealtaixe – Youtube: https://www.youtube.com/channel/UCQVCsX1CcZQr0oUMZg6szIQ I2DL: Prof. Niessner, Prof. Leal-Taixé 38
Deep Learnin ing at TUM I2DL: Prof. Niessner, Prof. Leal-Taixé [Caelles et al., CVPR’ 17] One -Shot Video Object Segmentation 39
Deep Learnin ing at TUM I2DL: Prof. Niessner, Prof. Leal-Taixé [Dosovitskiy et al., ICCV’ 15] FlowNet 40
Deep Learnin ing at TUM CC3 CC2 CC1 Reshape Conv+BN+ReLU Pooling Upsample Concat Score DDFF [Hazirbas et al., IJCV’18] Deep Depth From Focus. I2DL: Prof. Niessner, Prof. Leal-Taixé 41
Deep Learnin ing at TUM • Video anonymization Control identity Source [Maximov et al., CVPR 2020] CIAGAN: Conditional identity I2DL: Prof. Niessner, Prof. Leal-Taixé anonymization generative adversarial networks. 42
Deep Learnin ing at TUM • Multiple object tracking with graph neural networks [Brasó and Leal-Taixé, CVPR 2020] Learning a Neural Solver for I2DL: Prof. Niessner, Prof. Leal-Taixé Multiple Object Tracking. 43
Deep Learnin ing at TUM I2DL: Prof. Niessner, Prof. Leal-Taixé [Yang et al., ECCV’ 18] Deep Virtual Stereo Odometry 44
Deep Learnin ing at TUM I2DL: Prof. Niessner, Prof. Leal-Taixé [Xie et al. Siggraph ’ 18] tempoGAN 45
Deep Learnin ing at TUM ScanNet Stats: -Kinect-style RGB-D sensors -1513 scans of 3D environments -2.5 Mio RGB-D frames -Dense 3D, crowd-source MTurk labels -Annotations projected to 2D frames I2DL: Prof. Niessner, Prof. Leal-Taixé [Dai et al., CVPR’17] ScanNet 47
Deep Learnin ing at TUM [Hou et al., CVPR’19] 3D Semantic Instance Segmentation I2DL: Prof. Niessner, Prof. Leal-Taixé 48
Deep Learnin ing at TUM I2DL: Prof. Niessner, Prof. Leal-Taixé [Thies et al., Siggraph’19 ]: Neural Textures 49
Deep Learnin ing at TUM I2DL: Prof. Niessner, Prof. Leal-Taixé [Thies et al., Siggraph’19 ]: Neural Textures 50
Deep Learnin ing at TUM I2DL: Prof. Niessner, Prof. Leal-Taixé [Thies et al., Siggraph’19 ]: Neural Textures 51
Deep Learnin ing at TUM I2DL: Prof. Niessner, Prof. Leal-Taixé [Thies et al., Siggraph’19 ]: Neural Textures 52
Deep Learnin ing at TUM [Roessler et al., ICCV’19 ]: Face Forensics++ I2DL: Prof. Niessner, Prof. Leal-Taixé 53
Context of Other Lectures at TUM I2DL: Prof. Niessner, Prof. Leal-Taixé 54
Machine Learning Optimization basics Introduction Introduction to to Deep NN Learning CNN Back- propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taixé 55
Recommend
More recommend
