SLIDE 1 Spring 2017 CIS 493, EEC 492, EEC 592:
Autonomous Intelligent Robotics
Instructor: Shiqi Zhang
http://eecs.csuohio.edu/~szhang/teaching/17spring/
SLIDE 2 Proposal draft was due Feb 20 (1/2)
– Sujay Bajracharya – Alex Rhodes
- Simon Game with a Turtlebot
– Bri O’Neil – Justin Meinecke
- Autonomous Navigation and Visual Analysis
– Utkarsh Patel – Emre Hatay – Ghazal Zand
SLIDE 3 Proposal draft was due Feb 20 (2/2)
- Ball-following autonomous robot
– Elizabeth Barkett
- From Turtlesim to Turtlebot
– Lakiel Wade – Christian Loucka
- Turtlebot Security Patrol
– Emmanuel Manu
SLIDE 4 Written proposal (2-3 pages) will be due on March 9
- What is the problem?
- Why is it an interesting and/or important
problem?
- What techniques/algorithms will be used?
- What is the goal? Evaluations?
- Backup plans?
SLIDE 5
SLIDE 6 What’s an image ?
An image is simply a collectjon of pixels, each of which contains some data.
( A pixel is characterized by (x,y) )
Let us see some types of images...
Slides adapted from R. Maheshwari
SLIDE 7 Binary Image
Each Pixel has either 1 (White) or 0 (Black) Each pixel has 1 bit informatjon
( Binary images are seldom used )
1 1 1 1 1 1 1 1 1 1 1 1
SLIDE 8
Grayscale
Each Pixel has a value from 0 to 255. 0 : black and 255 : White Between 0 and 255 are shades of b&w. Each pixel has 1 byte informatjon It is stored as an array of bytes.
SLIDE 9
Grayscale Image
SLIDE 10
RGB Image
Each Pixel stores 3 values :- R : 0- 255 G: 0 -255 B : 0-255 Each pixel has 3 bytes of informatjon It is also stored as an array of bytes.
SLIDE 11
RGB image
SLIDE 12 Before moving to depth image, we must familiarize
- urselves with the basics of kinect.
What is a kinect camera ?
Kinect is a camera which gives R , G , B and depth informatjon of each pixel.
SLIDE 13 How does Kinect work?
Kinect has 3 components :-
- color camera ( takes RGB values)
- IR camera ( takes depth data )
- Microphone array ( for speech recognitjon )
SLIDE 14
Depth Image
SLIDE 15
But what’s the use of Depth ?? For this, Lets Discuss some Image Processing
SLIDE 16 Background/Foreground Subtractjon
- 1. On an Image ( Pixel Select Method )
- 2. On a Running Video ( Running Average Method )
SLIDE 17
Edge Detectjon (the gradient)
SLIDE 18
How can a Depth Image help in the above two ?? Back to kinect ….
SLIDE 19 How the Kinect Depth Sensor Works in 2 Minutes: htups://youtu.be/uq9SEJxZiUg
SLIDE 20 Player
A player is the (human) skeleton which is detected by kinect sdk. There can be multjple players. Each pixel stores the corresponding “player index”.
Player index = 1 Player index = 2 By default:- Player index = 0
SLIDE 21 Depth Image (Specifjc To Kinect sdk v1)
Each pixel stores :-
- Player index : 0 – 7 ( 3 bits )
- Depth( in mm) : 0 – 8192 ( 13 bits )
It is stored as an array of shorts.
( A short is a 16 bit data type)
P P P D D D D D D D D D D D D D Player Index Depth
SLIDE 22 Kinect has 3 streams
- ColorStream : contains RGB data as byte array
- DepthStream : contains depth data as short array
- SkeletonStream : a template
You need to enable these streams in the beginning as per your requirements.
SLIDE 23
What is a SkeletonStream ?
When skeletonstream is called , it recognizes skeletons and populates pixels of depthstream with player index. *If skeletonstream is not enabled, player index of all pixels of depthstream will remain 0.
SLIDE 24
Using skeletonstream, kinect sdk provides us with 20 joints.
Joints
