Introduction to Computer Vision for Robotics AE640A Autonomous - - PowerPoint PPT Presentation

▶
introduction to computer vision for robotics
SMART_READER_LITE
LIVE PREVIEW

Introduction to Computer Vision for Robotics AE640A Autonomous - - PowerPoint PPT Presentation

Mar 12, 2023 432 likes •967 views

Introduction to Computer Vision for Robotics AE640A Autonomous Navigation 5 th March, 2019 Harsh Sinha Introduction to Computer Vision 1 Lecture Outline Introduction What is CV? Overview of the field A look at history

slide-1
SLIDE 1

Introduction to Computer Vision Harsh Sinha

Introduction to Computer Vision for Robotics

AE640A Autonomous Navigation

5th March, 2019

1

slide-2
SLIDE 2

Introduction to Computer Vision Harsh Sinha

Lecture Outline

  • Introduction

○ What is CV? ○ Overview of the field ○ A look at history ○ Hard Problem?

  • Human Vision System & the Machine

○ The human vision system ○ Fooling humans ○ The computer vision system

  • Images as matrices.

○ How cameras work to produce these matrices ○ Meaning of Intensity, Color etc ○ Shoutout to Image Processing

2

slide-3
SLIDE 3

Introduction to Computer Vision Harsh Sinha

Lecture Outline

  • Camera Model

○ Pinhole Camera Model ○ Intrinsic Camera Matrix ○ Camera Calibration

3

slide-4
SLIDE 4

Introduction to Computer Vision Harsh Sinha

Introduction

4

slide-5
SLIDE 5

Introduction to Computer Vision Harsh Sinha

What is Computer Vision?

5

Computer Vision System Image Information Image Processing Image

Universe

slide-6
SLIDE 6

Introduction to Computer Vision Harsh Sinha

What is Computer Vision?

6 Image Credits: CS131, Fall ‘18, Stanford

slide-7
SLIDE 7

Introduction to Computer Vision Harsh Sinha

What is Computer Vision?

  • Computer Vision is deals with extracting information regarding the 3D world

we live in using a single or a bunch of images.

  • Computer Vision like most other fields today, is at the junction of numerous

disciplines from Biology to Computer Science and has applications only limited by our imagination.

7

slide-8
SLIDE 8

Introduction to Computer Vision Harsh Sinha

Overview of the field

8 Image Credits: XKCD, 1425, 2014

slide-9
SLIDE 9

Introduction to Computer Vision Harsh Sinha

Overview of the field

9 Image Credits: XKCD, 1425, 2014 Image Credits: https://tinyurl.com/y53by9pr Image Credits: https://tinyurl.com/y53by9pr

slide-10
SLIDE 10

Introduction to Computer Vision Harsh Sinha

Overview of the field

10

Image

Universe

(Image Processing + ) Computer Vision System What kind of Information?

slide-11
SLIDE 11

Introduction to Computer Vision Harsh Sinha

Overview of the field

11

What kind of Information?

Image Credits: https://tinyurl.com/lxuex6o Image Credits: Karpathy, CVPR’15

slide-12
SLIDE 12

Introduction to Computer Vision Harsh Sinha

Overview of the field

Primary themes in Computer Vision are: 1. Object Detection

12

Recognition: Cat?

Image: https://tinyurl.com/yanp2o5e

slide-13
SLIDE 13

Introduction to Computer Vision Harsh Sinha

Overview of the field

Primary themes in Computer Vision are: 1. Object Detection

13

Recognition: Cat? Localization: Where is the cat?

Image: https://tinyurl.com/yanp2o5e

slide-14
SLIDE 14

Introduction to Computer Vision Harsh Sinha

Overview of the field

Primary themes in Computer Vision are: 1. Object Detection

14

Object Detection: Which Objects are here and where?

Image: https://tinyurl.com/y4ly96rd

slide-15
SLIDE 15

Introduction to Computer Vision Harsh Sinha

Overview of the field

Primary themes in Computer Vision are: 1. Object Detection 2. Segmentation

15

Segmentation: Which pixels belong to which object?

Credits: Own Work

slide-16
SLIDE 16

Introduction to Computer Vision Harsh Sinha

Overview of the field

Primary themes in Computer Vision are: 1. Object Detection 2. Segmentation 3. Image Modifications/Enhancements

16

Image Colorization: From Grayscale to Colored Images

Credits: Richard Zhang, CVPR 2016

slide-17
SLIDE 17

Introduction to Computer Vision Harsh Sinha

Overview of the field

Primary themes in Computer Vision are: 1. Object Detection 2. Segmentation 3. Image Modifications/Enhancements

17

Image Enhancement: Real Time Image Enhancement

Credits: Michael Gharbi, ACM Graphics 2017

slide-18
SLIDE 18

Introduction to Computer Vision Harsh Sinha

Overview of the field

Primary themes in Computer Vision are: 1. Object Detection 2. Segmentation 3. Image Modifications/Enhancements

18

Super Resolution: Upsampling Images while preserving quality

Credits: https://github.com/tensorlayer/srgan

slide-19
SLIDE 19

Introduction to Computer Vision Harsh Sinha

Overview of the field

Primary themes in Computer Vision are: 1. Object Detection 2. Segmentation 3. Image Modifications/Enhancements 4. Image to Text

19

Image Description: Automatic semantic description for images

Credits: Karpathy, CVPR 2015

slide-20
SLIDE 20

Introduction to Computer Vision Harsh Sinha

Overview of the field

Primary themes in Computer Vision are: 1. Object Detection 2. Segmentation 3. Image Modifications/Enhancements 4. Image to Text 5. Image Generation

20

Image Generation: A style based generator architecture for GANs

Credits: Tero Karras, arXiv 2018

slide-21
SLIDE 21

Introduction to Computer Vision Harsh Sinha

Overview of the field

Primary themes in Computer Vision are: 1. Object Detection 2. Segmentation 3. Image Modifications/Enhancements 4. Image to Text 5. Image Generation 6. Motion Estimation

21

Optical Flow: Lucas Kanade method for motion estimation

Credits: https://tinyurl.com/y5rloh3g

slide-22
SLIDE 22

Introduction to Computer Vision Harsh Sinha

Overview of the field

Primary themes in Computer Vision are: 1. Object Detection 2. Segmentation 3. Image Modifications/Enhancements 4. Image to Text 5. Image Generation 6. Motion Estimation 7. 3D reconstruction from Images

22

3D Reconstruction: REMODE, Real Time Reconstruction

Credits: Matia Pizzoli, ICRA 2014

slide-23
SLIDE 23

Introduction to Computer Vision Harsh Sinha

Overview of the field

Primary themes in Computer Vision are: 1. Object Detection 2. Segmentation 3. Image Modifications/Enhancements 4. Image to Text 5. Image Generation 6. Motion Estimation 7. 3D reconstruction from Images 8. Visual SLAM

23

3D Reconstruction: REMODE, Real Time Reconstruction

Credits: Matia Pizzoli, ICRA 2014

slide-24
SLIDE 24

Introduction to Computer Vision Harsh Sinha

Overview of the field

Primary themes in Computer Vision are: 1. Object Detection 2. Segmentation 3. Image Modifications/Enhancements 4. Image to Text 5. Image Generation 6. Motion Estimation 7. 3D reconstruction from Images 8. Visual SLAM 9. Biometrics and more ...

24

Biometrics : Fingerprint Detection, Apple Face ID

Credits:https://tinyurl.com/y2a7wybz, TheVerge Youtube

slide-25
SLIDE 25

Introduction to Computer Vision Harsh Sinha

A look at history

  • Robert Nathan started writing

computer programs for enhancing images from NASA’s spacecraft’s at Jet Propulsion Lab, NASA.

  • The Summer Vision Project: Project at

MIT to solve a significant part of visual system. Primary Objective was to divide the image into object, background and chaos regions, over the course of a summer.

25 Credits: EE604, nasa.gov Credits: https://tinyurl.com/y6bpo4nk

slide-26
SLIDE 26

Introduction to Computer Vision Harsh Sinha

A look at history

26 Credits: Prof. Tanaya Guha, EE698K

slide-27
SLIDE 27

Introduction to Computer Vision Harsh Sinha

A look at history

27 Credits: Prof. Tanaya Guha, EE698K

slide-28
SLIDE 28

Introduction to Computer Vision Harsh Sinha

Hard Problem?

  • Why are we still working on roughly the same problem as the “summer vision

project”?

  • Why is it that creating 3D models of chairs is easier than identifying them?

28

slide-29
SLIDE 29

Introduction to Computer Vision Harsh Sinha

Hard Problem?

  • Why are we still working on roughly the same problem as the “summer vision

project”?

  • Why is it that creating 3D models of chairs is easier than identifying them?

➔ There is a large between some ~1920x1080x3 numbers and the high-level abstract meaning we associate with them. ➔ Images are 2D representation of information from 3D world.

29

slide-30
SLIDE 30

Introduction to Computer Vision Harsh Sinha

Human Vision System & Computer Vision System

30

slide-31
SLIDE 31

Introduction to Computer Vision Harsh Sinha

The human vision system

31 Credits: https://tinyurl.com/y6bkhnqa

slide-32
SLIDE 32

Introduction to Computer Vision Harsh Sinha

The human vision system

32

slide-33
SLIDE 33

Introduction to Computer Vision Harsh Sinha

The human vision system

33 Credits: Ulas Bagci, UCF

slide-34
SLIDE 34

Introduction to Computer Vision Harsh Sinha

Fooling humans

34 Credits: Oleg Shuplyak, Pinterest Credits: https://tinyurl.com/y49rp7sd Credits: Wikipedia, Spinning Dancer

slide-35
SLIDE 35

Introduction to Computer Vision Harsh Sinha

The (human) (computer) vision system

35 Credits: CS131, Stanford

slide-36
SLIDE 36

Introduction to Computer Vision Harsh Sinha

Fooling Computers

36 Credits: Wikipedia, Barber Pole Illusion Credits: https://tinyurl.com/l5pwp6t

slide-37
SLIDE 37

Introduction to Computer Vision Harsh Sinha

Images as Matrices

37

slide-38
SLIDE 38

Introduction to Computer Vision Harsh Sinha

Camera Models

38

slide-39
SLIDE 39

Introduction to Computer Vision Harsh Sinha

Camera Models

39 Credits: https://tinyurl.com/y6qen2vb

slide-40
SLIDE 40

Introduction to Computer Vision Harsh Sinha

Camera Models

40 Credits: https://tinyurl.com/y6qen2vb

Not this one but models as in modelling a phenomena

slide-41
SLIDE 41

Introduction to Computer Vision Harsh Sinha

Camera Models

  • Like so many things in engineering, we create a simple “model” of a camera

to which is easy to understand and can approximate the actual functioning of a camera to a good degree.

  • There are different models:

■ Pinhole camera model ■ Lens model ■ ...

41

slide-42
SLIDE 42

Introduction to Computer Vision Harsh Sinha

Pinhole camera model

42

aperture

Credits: Wikipedia, Pinhole Camera Model

slide-43
SLIDE 43

Introduction to Computer Vision Harsh Sinha

Pinhole camera model

43

aperture

Credits: Wikipedia, Pinhole Camera Model

slide-44
SLIDE 44

Introduction to Computer Vision Harsh Sinha

Pinhole camera model

44

aperture

Credits: Wikipedia, Pinhole Camera Model

slide-45
SLIDE 45

Introduction to Computer Vision Harsh Sinha

Pinhole camera model

45

where x’i = yi and z = x3 where c is an offset in pixels Can we make this into a matrix multiplication of the form p’ = Mp?

slide-46
SLIDE 46

Introduction to Computer Vision Harsh Sinha

Intrinsic camera matrix

46 Credits: Edwin Olson, University of Michigan

slide-47
SLIDE 47

Introduction to Computer Vision Harsh Sinha

Intrinsic camera matrix

47 Credits: Edwin Olson, University of Michigan

slide-48
SLIDE 48

Introduction to Computer Vision Harsh Sinha

Intrinsic camera matrix

48 Credits: Edwin Olson, University of Michigan

slide-49
SLIDE 49

Introduction to Computer Vision Harsh Sinha

Intrinsic camera matrix

49 Credits: Edwin Olson, University of Michigan

slide-50
SLIDE 50

Introduction to Computer Vision Harsh Sinha

Intrinsic camera matrix

50 Credits: Edwin Olson, University of Michigan

slide-51
SLIDE 51

Introduction to Computer Vision Harsh Sinha

Camera calibration

51 Credits: Gaurav Pandey, Ford

Calibration Rig Image Pi Pci