Today Computer Vision Overview Project Brainstorm / Team-up - - PowerPoint PPT Presentation

▶
today
SMART_READER_LITE
LIVE PREVIEW

Today Computer Vision Overview Project Brainstorm / Team-up - - PowerPoint PPT Presentation

May 24, 2023 450 likes •729 views

Today Computer Vision Overview Project Brainstorm / Team-up Activity COMP 150: Probabilistic Robotics for Human-Robot Interaction Instructor: Jivko Sinapov www.cs.tufts.edu/~jsinapov Research Talk Announcement

slide-1
SLIDE 1

COMP 150: Probabilistic Robotics for Human-Robot Interaction

Instructor: Jivko Sinapov www.cs.tufts.edu/~jsinapov

Today

  • Computer Vision Overview
  • Project Brainstorm / Team-up Activity

Research Talk Announcement

  • https://www.radcliffe.harvard.edu/event/2019-

beyond-words-conference

slide-2
SLIDE 2

Another Talk

  • Title: “Integrating Plan Generation,

Execution, and Monitoring for Explainable, Normative, and Justified Agency”

  • Speaker: Pat Langley (founding editor of the

Journal of Machine Learning)

  • Time / Place: Feb 21, 3:00 pm, Halligan 102

Office Hours Today

  • Moved from 3 pm to 4 pm due to faculty

candidate talk

Preliminary Project Idea Presentations

  • Next Tuesday, in class
  • Sign up on shared document on Canvas

(document was shared via an announcement)

  • 2 to 3 slides per team
slide-3
SLIDE 3

Reading Assignment

  • Thrun, S., Fox, D., Burgard, W., & Dellaert, F.

(2001). Robust Monte Carlo localization for mobile robots. Artificial intelligence, 128(1-2), 99-141.

  • Fox, D. (2002). KLD-sampling: Adaptive

particle filters. In Advances in neural information processing systems (pp. 713-720).

Homework 2

slide-4
SLIDE 4

Simulation

  • 1. Randomly choose a starting location
  • 2. Observation – given the actual location,

simulate an image observation

  • 3. Given any x,y position, crop the reference

image, i.e., what would the observation be If the robot was at x,y?

Simulation

  • 1. Randomly choose a starting location
  • 2. Observation – given the actual location,

simulate an image observation

  • 3. Given any x,y position, crop the reference

image, i.e., what would the observation be If the robot was at x,y?

  • 4. Movement – according to a random angle

but fixed velocity

Simulation

  • 1. Randomly choose a starting location
  • 2. Observation – given the actual location,

simulate an image observation

  • 3. Given any x,y position, crop the reference

image, i.e., what would the observation be If the robot was at x,y?

  • 4. Movement – according to a random angle

but fixed velocity – after computing the noise-free end position, add some noise

Discussion

  • What are some implementation issues that you

foresee?

  • What are some design choices that you’ll have

to make?

  • What may be some variables that you need to

experiment with?

  • What are some computer vision operators and

functions you’ll need to use?

slide-5
SLIDE 5

What is Computer Vision? What is an image? A grayscale image An RGB Image

slide-6
SLIDE 6

Intensity Levels

  • 2
  • 32
  • 64
  • 128
  • 256 (8 bits)
  • 512
  • …
  • 4096 (12 bits)

How did computer vision start?

“In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to a computer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that!”

Human vs Computer Vision

What we see What a computer sees

Image Plane v.s. Image Array

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 1]

slide-7
SLIDE 7

Point Operations

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 1]

Local Operations

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 1]

Edge Detection

[https://www.mathworks.com/matlabcentral/fileexchange/51124-shannon-edge-detector-for-grayscale-images]

Early Example of Edge Detection by Robots

slide-8
SLIDE 8

Global Operations

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 1]

Thresholding an Image

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 1]

Dark Image on a Light Background

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

Selecting a range

  • f intensity values

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-9
SLIDE 9

Generalized Thresholding

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

Thresholding Example (1)

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

Thresholding Example (2)

Original grayscale Image

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2] [Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-10
SLIDE 10

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

Area of a Binary Image

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

This figure now becomes important

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 1]

Calculating the Position

  • f an Object

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-11
SLIDE 11

The center is given by

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

Horizontal and Vertical Projections

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

Projection Formulas

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

Diagonal Projection

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-12
SLIDE 12

The area and the position can be computed from the H and V projections

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

Neighbors and Connectivity 4-Connected

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

8-connected

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

slide-13
SLIDE 13

Examples of Paths

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

Boundary, Interior, and Background

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

An Image (a) and Its Connected Components (b)

[Jain, Kasturi, and Schunck (1995). Machine Vision, Ch. 2]

Color Perception

slide-14
SLIDE 14

The RGB Color Space

[http://www.arcsoft.com/images/topics/darkroom/what-is-color-space-RGB.jpg]

The RGB Color Space

https://upload.wikimedia.org/wikipedia/commons/thumb/1/11/RGBCube_b.svg/2000px-RGBCube_b.svg.png

3D Scatter Plot for a patch of skin

The HSV Color Space

slide-15
SLIDE 15

Color-based Tracking Color-based Tracking

How should we determine the min and max thresholds for each color channel?

Color Histograms

Color Histogram (4 x 4 x 4 = 64 bins)

Object Segmentation

Motion

slide-16
SLIDE 16

What is this? What is this?

What action is being performed? Motion Energy Image (MEI)

[http://www.cse.ohio-state.edu/~jwdavis/CVL/Research/MHI/mhi.html]

slide-17
SLIDE 17

Average MEI for various viewing angles Motion History Image (MHI)

[http://www.cse.ohio-state.edu/~jwdavis/CVL/Research/MHI/mhi.html]

Definitions

  • Image Sequence
  • Binary Images

indicating regions of motion

  • Binary Motion Energy Image

Motion Energy

slide-18
SLIDE 18

Motion History

The result: more recently moving pixels appear brighter

[http://www.cse.ohio-state.edu/~jwdavis/CVL/Research/MHI/mhi.html]

MHI pyramid

[http://www.cse.ohio-state.edu/~jwdavis/CVL/Research/MHI/mhi.html]

Motion templates for finishing LEFT-ARM-RAISE and FAN-UP-ARMS.

[http://www.cse.ohio-state.edu/~jwdavis/CVL/Research/VirtualAerobics/aerobics.html]

slide-19
SLIDE 19

Aerobics Dataset Video

  • A. Bobick, S. Intille, J. Davis, F. Baird, C.

Pinhanez, L. Campbell, Y. Ivanov, A. Schutte, and A. Wilson (1999) ``The Kidsroom: A Perceptually-Based Interactive and Immersive Story Environment" Presence: Teleoperators and Virtual Environments, Vol. 8, No. 4, 1999, pp. 367-391.

slide-20
SLIDE 20

The Kid’s Room

[Bobick et al. 1996]

The Blue Monster

[http://vismod.media.mit.edu/vismod/demos/kidsroom/kidsroom.html]

slide-21
SLIDE 21

The Technology

[http://vismod.media.mit.edu/vismod/demos/kidsroom/kidsroom.html]

Motion History Templates

Making a ‘Y’ Flapping Spinning

[http://vismod.media.mit.edu/vismod/demos/kidsroom/kidsroom.html]

Detecting the Bed

[http://vismod.media.mit.edu/vismod/demos/kidsroom/kidsroom.html]

slide-22
SLIDE 22

Man Overboard Detector

[http://vismod.media.mit.edu/vismod/demos/kidsroom/kidsroom.html]

OpenCV Book and Code

  • “Learning OpenCV”
  • Code from book is on github:

https://github.com/Itseez/opencv_extra/tree/m aster/learning_opencv_v2

slide-23
SLIDE 23

OpenCV Book and Code

  • “Learning OpenCV”
  • Code from book is on github:

https://github.com/Itseez/opencv_extra/tree/master /learning_opencv_v2

OpenCV Tutorials

  • Connected Components:

– http://nghiaho.com/?p=1102 – https://davidlavy.wordpress.com/opencv/connected

  • components-in-opencv/

OpenCV Tutorials

  • Circle Detection:

– http://docs.opencv.org/3.1.0/d4/d70/tutorial_hough_

circle.html#gsc.tab=0

OpenCV Tutorials

  • Face Detection:

– http://stackoverflow.com/questions/20757147/detect

  • faces-in-image

– https://github.com/Itseez/opencv_extra/blob/mast

er/learning_opencv_v2/ch13_ex13_4.cpp

slide-24
SLIDE 24

OpenCV Tutorials

  • Blog full of OpenCV examples:

– http://opencvexamples.blogspot.com/

Resources

  • OpenCV in ROS:

– http://wiki.ros.org/vision_opencv – http://wiki.ros.org/cv_bridge/Tutorials – http://docs.opencv.org/2.4/doc/tutorials/tutorials.html

Grabbing image data with ROS

  • Example ROS node that subscribes to an

image topic and does image processing: http://www.cs.tufts.edu/comp/50AIR/code/comp 50_computer_vision.zip

Next time...interest points and registration

slide-25
SLIDE 25

Later in the course...3D Vision Project Brainstorm and Team-up

  • Grab a piece of paper per team
  • Write down your topics of interests and specific

ideas for your project

  • Write any project related questions you may

have

THE END

slide-26
SLIDE 26

102 103