CS 378: Autonomous Intelligent Robotics Instructor: Jivko Sinapov - - PowerPoint PPT Presentation

▶
cs 378 autonomous intelligent robotics
SMART_READER_LITE
LIVE PREVIEW

CS 378: Autonomous Intelligent Robotics Instructor: Jivko Sinapov - - PowerPoint PPT Presentation

Nov 09, 2022 373 likes •1.09k views

CS 378: Autonomous Intelligent Robotics Instructor: Jivko Sinapov http://www.cs.utexas.edu/~jsinapov/teaching/cs378/ Computer Vision: 2D Images Announcements Homework 5 deadline extended to this Thursday Announcements Volunteers needed

slide-1
SLIDE 1

CS 378: Autonomous Intelligent Robotics

Instructor: Jivko Sinapov

http://www.cs.utexas.edu/~jsinapov/teaching/cs378/

slide-2
SLIDE 2

Computer Vision: 2D Images

slide-3
SLIDE 3

Announcements

  • Homework 5 deadline extended to this

Thursday

slide-4
SLIDE 4

Announcements

Volunteers needed for another study! As before, there will be extra credit

To sign up, email: – Rodolfo Rodriguez <rcorona@utexas.edu> – Jesse Thomason <thomason.jesse@gmail.com>

slide-5
SLIDE 5

Final Project Timeline

  • Project Proposal due: Mar. 29th Apr. 1st
  • Project Presentations / Demos: Last Week
  • f Class (May 3rd and 5th)
  • Final Report due: May 11th
slide-6
SLIDE 6

Project Proposal Guidelines

  • Work in groups of 2-3 (it's OK to work on

your own if you really want to)

  • Preferably, team up with people with

different skills than yours

  • Purpose of the proposal is to give you an
  • utline / roadmap
slide-7
SLIDE 7

Project Proposal Guidelines

  • Each proposal should be about 2-3 pages
  • Each proposal should include:

– What is the application / task / problem? – Any previous experience you may have in that area – What do you expect to achieve by the end of the semester? – How do you plan to evaluate whether it works or not? – A timeline / schedule of progress and milestones

slide-8
SLIDE 8

Project Proposal Guidelines

  • Organization: your proposal should have

sections and headings (don't just submit

  • ne long essay)
  • For example:

– Introduction / problem formulation – Proposed approach / software – Proposed evaluation – Summary of anticipated end result

slide-9
SLIDE 9

Project Ideas

Vending Machine Sonar Sensor

slide-10
SLIDE 10

Project Ideas

Write ROS code to allow the robot to use an LED light strip

slide-11
SLIDE 11

Project Ideas

Help the robot “see” something it currently cannot Help the robot “hear” something (e.g., the elevator sound) Help the robot “do” something (e.g., follow a person)

slide-12
SLIDE 12

Project Ideas

slide-13
SLIDE 13

Project Ideas

slide-14
SLIDE 14

Project Ideas

slide-15
SLIDE 15

Project Ideas

slide-16
SLIDE 16

Final Project Timeline

The most important thing is to start early, and discuss your ideas with the TA, mentors and myself. We'll point you to a starting point, describe functionality that already exists, and help refine your ideas.

slide-17
SLIDE 17

Final Project Timeline

  • Project Proposal due: Mar. 29th Apr. 1st
  • Project Presentations / Demos: Last Week
  • f Class (May 3rd and 5th)
  • Final Report due: May 11th
slide-18
SLIDE 18

Computer Vision: 2D Images

slide-19
SLIDE 19

Readings

  • Jain, Kasturi, and Schunck (1995).

Machine Vision, ``Chapter 1: Introduction,'' McGraw-Hill, pp. 1-24.

  • Jain, Kasturi, and Schunck (1995).

Machine Vision, ``Chapter 2: Binary Image Processing,'' McGraw-Hill, pp. 25-72.

slide-20
SLIDE 20

Readings (con't)

  • J. K. O'Regan and A. Noe, (2001).

``A sensorimotor account of vision and vis ual consciousness'' , Behavioral and Brain Sciences, 24(5), 939- 1011.

slide-21
SLIDE 21

What is an image?

slide-22
SLIDE 22

A grayscale image

slide-23
SLIDE 23

An RGB image

slide-24
SLIDE 24

How did computer vision start?

In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to a computer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that!

slide-25
SLIDE 25

Computer vision vs human vision

What we see What a computer sees

slide-26
SLIDE 26

Intensity Levels

  • 2
  • 32
  • 64
  • 128
  • 256 (8 bits)
  • 512
  • …
  • 4096 (12 bits)
slide-27
SLIDE 27

Intensity Levels

  • 2
  • 32
  • 64
  • 128
  • 256 (8 bits)
  • 512
  • …
  • 4096 (12 bits)
slide-28
SLIDE 28

Image Plane v.s. Image Array

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 1]

slide-29
SLIDE 29

Point Operations

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 1]

slide-30
SLIDE 30

Local Operations

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 1]

slide-31
SLIDE 31

Global Operations

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 1]

slide-32
SLIDE 32

Thresholding an Image

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 1]

slide-33
SLIDE 33

Dark Image on a Light Background

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-34
SLIDE 34

Selecting a range

  • f intensity values

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-35
SLIDE 35

Generalized Thresholding

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-36
SLIDE 36

Thresholding Example (1)

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-37
SLIDE 37

Thresholding Example (2)

Original grayscale Image

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-38
SLIDE 38

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-39
SLIDE 39

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-40
SLIDE 40

Area of a Binary Image

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-41
SLIDE 41

This figure now becomes important

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 1]

slide-42
SLIDE 42

Calculating the Position

  • f an Object

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-43
SLIDE 43

The center is given by

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-44
SLIDE 44

Horizontal and Vertical Projections

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-45
SLIDE 45

Horizontal and Vertical Projections

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-46
SLIDE 46

Projection Formulas

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-47
SLIDE 47

Diagonal Projection

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-48
SLIDE 48

The area and the position can be computed from the H and V projections

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-49
SLIDE 49

Neighbors and Connectivity

slide-50
SLIDE 50

4-Connected

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-51
SLIDE 51

8-connected

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-52
SLIDE 52

Examples of Paths

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-53
SLIDE 53

Boundary, Interior, and Background

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-54
SLIDE 54

An Image (a) and Its Connected Components (b)

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-55
SLIDE 55

Color Perception

slide-56
SLIDE 56

The RGB Color Space

[http://www.arcsoft.com/images/topics/darkroom/what-is-color-space-RGB.jpg]

slide-57
SLIDE 57

The RGB Color Space

https://upload.wikimedia.org/wikipedia/commons/thumb/1/11/RGBCube_b.svg/2000px-RGBCube_b.svg.png

slide-58
SLIDE 58

3D Scatter Plot for a patch of skin

slide-59
SLIDE 59

The HSV Color Space

slide-60
SLIDE 60

Color Detection and Segmentation

slide-61
SLIDE 61

Color Detection and Segmentation

slide-62
SLIDE 62

Discussion: how may we achieve this?

slide-63
SLIDE 63

Example Hand Tracking using Color

slide-64
SLIDE 64

Computer Vision in ROS

slide-65
SLIDE 65

Computer Vision in ROS

1) Subscribing to an image topic 2) Converting a ROS image to an OpenCV image 3) Copy an image 4) Convert an image to grayscale 5) Access and set individual pixel values

slide-66
SLIDE 66

Example Color Detection in ROS using OpenCV

slide-67
SLIDE 67

Resources

  • OpenCV in ROS:

– http://wiki.ros.org/vision_opencv – http://wiki.ros.org/cv_bridge/Tutorials – http://docs.opencv.org/2.4/doc/tutorials/tutorial s.html

slide-68
SLIDE 68

THE END

slide-69
SLIDE 69