COMP 150: Developmental Robotics Instructor: Jivko Sinapov - - PowerPoint PPT Presentation

▶

comp 150 developmental robotics

COMP 150: Developmental Robotics Instructor: Jivko Sinapov - - PowerPoint PPT Presentation

Nov 27, 2022 258 likes •880 views

COMP 150: Developmental Robotics Instructor: Jivko Sinapov www.cs.tufts.edu/~jsinapov This Week Theories of Vision Computer Vision Human Vision Project Breakouts Annoucements New reading assignment Thanks for the feedback!

slide-1

SLIDE 1

COMP 150: Developmental Robotics

Instructor: Jivko Sinapov www.cs.tufts.edu/~jsinapov

slide-2

SLIDE 2

This Week

Theories of Vision

– Computer Vision – Human Vision

Project Breakouts

slide-3

SLIDE 3

Annoucements

New reading assignment

slide-4

SLIDE 4

slide-5

SLIDE 5

Thanks for the feedback!

Overall you seemed happy
Many wished for more hands-on work with the

robots

“the 2 cool robot kids in the back”

slide-6

SLIDE 6

What is an image?

slide-7

SLIDE 7

A grayscale image

slide-8

SLIDE 8

An RGB image

slide-9

SLIDE 9

How did computer vision start?

In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to a computer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that!

slide-10

SLIDE 10

Computer vision vs human vision

What we see What a computer sees

slide-11

SLIDE 11

Intensity Levels

2
32
64
128
256 (8 bits)
512
…
4096 (12 bits)

slide-12

SLIDE 12

Intensity Levels

2
32
64
128
256 (8 bits)
512
…
4096 (12 bits)

slide-13

SLIDE 13

Image Plane v.s. Image Array

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 1]

slide-14

SLIDE 14

Point Operations

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 1]

slide-15

SLIDE 15

Local Operations

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 1]

slide-16

SLIDE 16

Global Operations

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 1]

slide-17

SLIDE 17

Thresholding an Image

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 1]

slide-18

SLIDE 18

Dark Image on a Light Background

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-19

SLIDE 19

Selecting a range

f intensity values

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-20

SLIDE 20

Generalized Thresholding

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-21

SLIDE 21

Thresholding Example (1)

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-22

SLIDE 22

Thresholding Example (2)

Original grayscale Image

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-23

SLIDE 23

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-24

SLIDE 24

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-25

SLIDE 25

Color

slide-26

SLIDE 26

The RGB Color Space

[http://www.arcsoft.com/images/topics/darkroom/what-is-color-space-RGB.jpg]

slide-27

SLIDE 27

The RGB Color Space

https://upload.wikimedia.org/wikipedia/commons/thumb/1/11/RGBCube_b.svg/2000px-RGBCube_b.svg.png

slide-28

SLIDE 28

3D Scatter Plot for a patch of skin

slide-29

SLIDE 29

The HSV Color Space

slide-30

SLIDE 30

Color Detection and Segmentation

slide-31

SLIDE 31

Discussion: how may we achieve this?

slide-32

SLIDE 32

Example Hand Tracking using Color

slide-33

SLIDE 33

Motion

slide-34

SLIDE 34

What is this?

slide-35

SLIDE 35

What is this?

slide-36

SLIDE 36

Bobick, Aaron F. "Movement, activity and action: the role of knowledge in the perception of motion." Philosophical Transactions of the Royal Society of London B: Biological Sciences 352.1358 (1997): 1257- 1265.

slide-37

SLIDE 37

Motion Energy Image (MEI)

[http://www.cse.ohio-state.edu/~jwdavis/CVL/Research/MHI/mhi.html]

slide-38

SLIDE 38

Average MEI for various viewing angles

slide-39

SLIDE 39

Motion History Image (MHI)

[http://www.cse.ohio-state.edu/~jwdavis/CVL/Research/MHI/mhi.html]

slide-40

SLIDE 40

Definitions

Image Sequence
Binary Images

indicating regions of motion

Binary Motion Energy Image

slide-41

SLIDE 41

Motion Energy

slide-42

SLIDE 42

Motion History

The result: more recently moving pixels appear brighter

slide-43

SLIDE 43

[http://www.cse.ohio-state.edu/~jwdavis/CVL/Research/MHI/mhi.html]

slide-44

SLIDE 44

Motion templates for finishing LEFT-ARM-RAISE and FAN-UP-ARMS.

[http://www.cse.ohio-state.edu/~jwdavis/CVL/Research/VirtualAerobics/aerobics.html]

slide-45

SLIDE 45

slide-46

SLIDE 46

Aerobics Dataset

slide-47

SLIDE 47

Video

slide-48

SLIDE 48

A. Bobick, S. Intille, J. Davis, F. Baird, C.

Pinhanez, L. Campbell, Y. Ivanov, A. Schutte, and A. Wilson (1999) ``The Kidsroom: A Perceptually-Based Interactive and Immersive Story Environment" Presence: Teleoperators and Virtual Environments, Vol. 8, No. 4, 1999, pp. 367-391.

slide-49

SLIDE 49

The Kid’s Room

[Bobick et al. 1996]

slide-50

SLIDE 50

slide-51

SLIDE 51

slide-52

SLIDE 52

The Blue Monster

[http://vismod.media.mit.edu/vismod/demos/kidsroom/kidsroom.html]

slide-53

SLIDE 53

slide-54

SLIDE 54

The Technology

[http://vismod.media.mit.edu/vismod/demos/kidsroom/kidsroom.html]

slide-55

SLIDE 55

Motion History Templates

Making a ‘Y’ Flapping Spinning

[http://vismod.media.mit.edu/vismod/demos/kidsroom/kidsroom.html]

slide-56

SLIDE 56

Detecting the Bed

[http://vismod.media.mit.edu/vismod/demos/kidsroom/kidsroom.html]

slide-57

SLIDE 57

Man Overboard Detector

[http://vismod.media.mit.edu/vismod/demos/kidsroom/kidsroom.html]

slide-58

SLIDE 58

slide-59

SLIDE 59

C++ Computer Vision Libraries

OpenCV:

– http://wiki.ros.org/vision_opencv – http://wiki.ros.org/cv_bridge/Tutorials – http://docs.opencv.org/2.4/doc/tutorials/tutorials.html

Point Cloud Library:

– http://pointclouds.org/

slide-60

SLIDE 60

Project Breakout

Identify your next steps…
Identify and find the tools that you need

– e.g., simulators, open-source libraries, datasets,

etc.

Break up the problem into parts so that you can

work on them in parallel

Design your pipeline – what goes in and what

goes out and what happens in between

slide-61

SLIDE 61