In Introductio ion to Deep Learnin ing I2DL: Prof. Niessner, - - PowerPoint PPT Presentation

▶
in introductio ion to
SMART_READER_LITE
LIVE PREVIEW

In Introductio ion to Deep Learnin ing I2DL: Prof. Niessner, - - PowerPoint PPT Presentation

Oct 15, 2022 115 likes •810 views

In Introductio ion to Deep Learnin ing I2DL: Prof. Niessner, Prof. Leal-Taix 1 The Team Lecturers Prof. Dr. Laura Prof. Dr. Matthias Leal-Taix Niessner PhDs Patrick Andreas Dendorfer Rssler I2DL: Prof. Niessner, Prof.

slide-1
SLIDE 1

In Introductio ion to Deep Learnin ing

1

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-2
SLIDE 2

2

Lecturers PhDs

I2DL: Prof. Niessner, Prof. Leal-Taixé

  • Prof. Dr. Laura

Leal-Taixé

  • Prof. Dr. Matthias

Niessner Patrick Dendorfer Andreas Rössler

The Team

slide-3
SLIDE 3

What is is Computer r Vis ision?

  • First defined in the 60s in artificial intelligence groups
  • “Mimic the human visual system”
  • Center block of robotic intelligence

3

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-4
SLIDE 4

Hubel and Wie iesel

  • David Hubel and Torsten Wiesel

were neurobiologists from Harvard Medical School

  • Experiment revealed several

secrets of the human vision system

  • Won 2 Nobel prizes

4

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-5
SLIDE 5

Hubel and Wie iesel Experiment

  • Recorded electrical activity from

individual neurons in the brains of cats.

  • Slide projector to show specific

patterns to the cats noted specific patterns stimulated activity in specific parts of the brain.

  • Results: Visual cortex cells are

sensitive to the orientation of edges but insensitive to their position

5

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-6
SLIDE 6

6

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-7
SLIDE 7

Computer Vision

Few Decades Later…

7

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-8
SLIDE 8

Computer Vision Physics Psychology Biology Mathematics Engineering Computer science Artificial Intelligence ML Neuroscience Algorithms Optimization NLP Speech Robotics Optics Image processing

8

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-9
SLIDE 9

Pre 2012

Im Image Cla lassific ication

9

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-10
SLIDE 10

A

Awesome magic box

Become magicians

Post 2012

Open the box

Im Image Cla lassific ication

10

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-11
SLIDE 11

Why Deep Learnin ing?

11

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-12
SLIDE 12

Deep Learnin ing His istory

12

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-13
SLIDE 13

The Empir ire Stri rikes Back

13

5 10 15 20 25 30 2010 2011 2012 AlexNet 2013 2014 VGGNet Human 2015 ResNet 2016 Ensemble 2017 SENet ILSVRC top-5 error on ImageNet

Deep Learning Approaches

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-14
SLIDE 14
  • MNIST digit

recognition dataset

  • 107 pixels used in

training

  • ImageNet image

recognition dataset

  • 1014 pixels used in

training 1998 LeCun et al. 2012 Krizhevsky et al.

What Has Changed?

14

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-15
SLIDE 15

Big Data

Models know where to learn from

Hardware

Models are trainable

Deep

Models are complex

What Made this is Possible?

15

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-16
SLIDE 16

Deep Learnin ing Recognition

16

ACM Turing Award 2019 (Nobel Prize of Computing) Yann LeCun, Geoffrey Hinton, and Yoshua Bengio I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-17
SLIDE 17

Credits: Dr. Pont-Tuset, ETH Zurich

Deep Learnin ing and Computer Vis isio ion

17

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-18
SLIDE 18

Credits: Dr. Pont-Tuset, ETH Zurich

Deep Learnin ing and Computer Vis isio ion

18

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-19
SLIDE 19

Deep Learnin ing Today

19

Object Detection

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-20
SLIDE 20

Self-driving cars

Deep Learnin ing Today

20

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-21
SLIDE 21

AlphaGo Machine translation Emoticon suggestion

Deep Learnin ing Today

21

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-22
SLIDE 22

Deep Learnin ing Today

22

Alpha Star

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-23
SLIDE 23

Deep Learnin ing Today

23

Automated Text Generation [Karpathy et al.]

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-24
SLIDE 24

Deep Learnin ing Today

24

Google Assistant (Google IO’19)

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-25
SLIDE 25

Healthcare, cancer detection

Deep Learnin ing Today

25

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-26
SLIDE 26

Deep Learnin ing Market

[…] market research report Deep Learning Market […] “ the deep learning market is expected to be worth USD 1,722. 2.9 Mill llio ion by 20 2022 22.

26

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-27
SLIDE 27

Deep Learnin ing Job Pers rspective

  • Excellent Job Perspectives!

– Automation requires ML/DL -> growth! – Top-notch companies will gladly hire you!

  • Many industries now:

– IT-Companies – Cars, Logistic, Health Care, etc… – Manufacturing / Robotics, etc…

27

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-28
SLIDE 28

But: : Als lso Challenging!

  • High-level understanding is not enough

– Need proper theory background – Need proper practical skillsets

  • Can be competitive!

– Many good people – Downloading scripts / running code not enough  – Deeper understanding often requires PhDs

28

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-29
SLIDE 29

Deep Learnin ing Cult lture

29

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-30
SLIDE 30

Deep Learnin ing Memes

30

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-31
SLIDE 31

Deep Learnin ing Memes

31

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-32
SLIDE 32

Deep Learnin ing Memes

32

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-33
SLIDE 33

Deep Learnin ing Memes

33

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-34
SLIDE 34

Deep Learnin ing Memes

34

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-35
SLIDE 35

Deep Learnin ing Memes

35

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-36
SLIDE 36

Deep Learnin ing at TUM

36

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-37
SLIDE 37

Many TUM Research Labs use DL

  • Visual

l Computin ing Lab (P (Pro rof.

  • f. Niessner)

r):

– Research in computer vision, graphics, and machine learning

  • Dynamic

ic Visio ion and Learn rnin ing Gro roup (Pro rof.

  • f. Leal-Taixe)

– Research on Computer Vision; e.g., video editing/segmentation etc.

  • 3D Understanding Lab (Dr. Dai):

– Research in 3D scenes and its semantics.

  • Computer Vision Group (Prof. Cremers)

– Research in computer vision and pattern recognition

  • Data Mining and Analytics Lab (Prof. Günnemann)

– Research methods for robust machine learning

  • Computer Aided Medical Procedures (Prof. Navab)

– Research in machine learning for medical applications

  • And probably many more 

37

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-38
SLIDE 38

Our r Research Labs

  • Visual Computing Lab (Prof. Niessner):

https://niessnerlab.org/publications.html

– Twitter: https://twitter.com/MattNiessner – You

  • utu

tube: : htt https:/ ://www.y .youtube.com/channel/UCXN2nYjV jVT0 T0cR9G61RPEzK5 K5Q – Facebook: https://www.facebook.com/matthias.niessner

  • Dynamic Vision and Learning Lab (Prof. Leal-Taixé):

https://dvl.in.tum.de/publications.html

– Twitter: https://twitter.com/lealtaixe – Youtube: https://www.youtube.com/channel/UCQVCsX1CcZQr0oUMZg6szIQ

38

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-39
SLIDE 39

[Caelles et al., CVPR’ 17] One-Shot Video Object Segmentation

Deep Learnin ing at TUM

39

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-40
SLIDE 40

Deep Learnin ing at TUM

40

[Dosovitskiy et al., ICCV’ 15] FlowNet I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-41
SLIDE 41

Deep Learnin ing at TUM

41

CC3 CC2 CC1 Reshape Conv+BN+ReLU Pooling Upsample Concat Score

DDFF

[Hazirbas et al., IJCV’18] Deep Depth From Focus. I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-42
SLIDE 42

Deep Learnin ing at TUM

  • Video anonymization

42

[Maximov et al., CVPR 2020] CIAGAN: Conditional identity anonymization generative adversarial networks. I2DL: Prof. Niessner, Prof. Leal-Taixé

Source Control identity

slide-43
SLIDE 43

Deep Learnin ing at TUM

  • Multiple object tracking with graph neural networks

43

[Brasó and Leal-Taixé, CVPR 2020] Learning a Neural Solver for Multiple Object Tracking. I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-44
SLIDE 44

Deep Learnin ing at TUM

44

[Yang et al., ECCV’ 18] Deep Virtual Stereo Odometry I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-45
SLIDE 45

Deep Learnin ing at TUM

45

[Xie et al. Siggraph’ 18] tempoGAN I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-46
SLIDE 46

Deep Learnin ing at TUM

47

[Dai et al., CVPR’17] ScanNet

ScanNet Stats:

  • Kinect-style RGB-D

sensors

  • 1513 scans of 3D

environments

  • 2.5 Mio RGB-D frames
  • Dense 3D, crowd-source

MTurk labels

  • Annotations projected to

2D frames

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-47
SLIDE 47

Deep Learnin ing at TUM

48

[Hou et al., CVPR’19] 3D Semantic Instance Segmentation I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-48
SLIDE 48

Deep Learnin ing at TUM

49

[Thies et al., Siggraph’19 ]: Neural Textures I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-49
SLIDE 49

Deep Learnin ing at TUM

50

[Thies et al., Siggraph’19 ]: Neural Textures I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-50
SLIDE 50

Deep Learnin ing at TUM

51

[Thies et al., Siggraph’19 ]: Neural Textures I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-51
SLIDE 51

Deep Learnin ing at TUM

52

[Thies et al., Siggraph’19 ]: Neural Textures I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-52
SLIDE 52

Deep Learnin ing at TUM

53

[Roessler et al., ICCV’19 ]: Face Forensics++ I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-53
SLIDE 53

Context of Other Lectures at TUM

54

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-54
SLIDE 54

Introduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back- propagation RNN

55

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-55
SLIDE 55

Deep Learnin ing at TUM

56

Intro to Deep Learning

(Niessner, r, Leal-Taixe)

DL for Physics

(Th Thuerey)

ADL for Vision

(Niessner, r, Leal-Taixe)

DL for Medical Applicat.

(Menze)

DL in Robotics

(Bä Bäuml)

Machine Learning

(Günn nneman ann)

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-56
SLIDE 56

Why is is I2 I2DL so Im Import rtant?

  • Many of the other lectures / practical require it!

– Often only limited spots are available (e.g., in the prestigious Advanced Deep Learning for Computer Vision Class)

  • Always preparation for guided research / IDP

– Most topics require it – For career in AI/DL these are the best ways to get into

57

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-57
SLIDE 57

What’s New?

  • Last semester:

– Re-vamped content of slides / improved design – Re-organization of lecture structure

  • This semester:

– Re-vamping of the exercises (incl. sessions) – Earlier intro to pyTorch, more but smaller exercises

58

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-58
SLIDE 58

In Introductio ion to Deep Learnin ing

59

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-59
SLIDE 59

Moodle

  • Announcements via Moodle - IM

IMPORTANT!

– Sign up in TUM online for access: https://www.moodle.tum.de/ – We will share common information (e.g., regarding exam) – Ask content questions online so others benefit – Don’t post solutions – TAs will monitor Moodle (expect answers within hours)

  • We have a team of 11 TAs

60

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-60
SLIDE 60

Email il

  • Email list:
  • Do NOT email us personally!

– Cannot handle so many emails / hence will be ignored

  • Email list for organizational questions only!

– Content questions -> Moodle or Office Hours

61

i2 i2dl@vc.i .in.t .tum.de

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-61
SLIDE 61

Website

  • Links and slides will be shared on website

62

https://niessner.github.io/I2DL/

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-62
SLIDE 62

About the Lecture

  • Theory lectures
  • Every Monday 14:15-15:45

– Lectures are virtual and will be uploaded to youtube: https://www.youtube.com/channel/UCXN2nYjVT0cR9G61RPEzK5Q

  • Practical sessions

– Every Thursday 8:30-10:00 – There will be practical exercises

  • Guest lecture TBD

63

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-63
SLIDE 63

Gra radin ing System

  • Exam: TBA
  • Review: TUM Exam
  • Important: no retake exam
  • Practice: More information on Thursday!
  • Bonus 0.3 + questions in the final exam
  • Do not

t mis iss th the firs first pra ractic ical l sessio ion !!! !!!

64

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-64
SLIDE 64

Pre relim iminary Syll llabus

Lecture 1: Introduction to the lecture, Deep Learning, Machine Learning. Lecture 2: Machine Learning Basics, Linear regression, Maximum Likelihood Lecture 3: Introduction to Neural Networks, Computational Graphs Lecture 4: Optimization and Backpropagation Lecture 5: Scaling Optimization to large Data, Stochastic Gradient Descent Lecture 6: Training Neural Networks I Lecture 7: Training Neural Networks II Lecture 8: Training Neural Networks III Lecture 9: Introduction to CNNs Lecture 10: CNNs architectures; CNNs for object detection Lecture 11: Recurrent Neural Networks (RNNs) Lecture 12: Autoencoders, VAEs and GANs Lecture 13: Reinforcement Learning Lecture 14: Guest Lecture (TBD)

66

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-65
SLIDE 65

Other r Admin inistrative

  • External students welcome

– Will get Certificate / Schein at the end – Send email to list and we will add you to moodle

  • Lectures will be recorded and put on youtube

– https://www.youtube.com/channel/UCXN2nYjVT0cR9G 61RPEzK5Q

  • Again: check announcements on moodle

67

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-66
SLIDE 66

Off ffic ice Hours rs

  • We will have dedicated office hours regarding

– Theoretical help (e.g., specific lecture questions) – Help on exercises

  • Will be announced with first tutorial session and on

Moodle

68

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-67
SLIDE 67

Upcomin ing Lecture

  • Next Lecture: Lecture 2: Machine Learning basics
  • This Thursday: 1st Practical Session

– if you want bonus, do not miss it!

69

I2DL: Prof. Niessner, Prof. Leal-Taixé

slide-68
SLIDE 68

See you next week 

70

I2DL: Prof. Niessner, Prof. Leal-Taixé