CS 7616 Pattern Recognition Introduction Aaron Bobick School of - - PowerPoint PPT Presentation

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

Aaron Bobick School of Interactive Computing

CS 7616 Pattern Recognition Introduction

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

Where are you?

• CS 7616 Pattern Recognition
• Web site: http://www.cc.gatech.edu/~afb/classes/CS7616-Spring2014/
• Will have posted calendar/syllabus with posted slides, problem sets with data, other

administrative stuff.

• T-square: The usual stuff. There is/will be a web page under resources that points to

the class web page listed above.

• Slides: PDFs will be posted by linking to the calendar. Hopefully draft by class time,

but I wouldn’t count on it.

• Piazza: You will all receive an invitation to Piazza for CS7616. If not, send us email.

This was invaluable for discussions for problem sets in CS4495. Not sure if here.

• Announcements will likely be done through both T-Square (and email) and Piazza
• Matlab access: if you don’t know how to get Matlab access, first ask a friend. Then

come see TAs or me. If you want to use Python/Numpy that’s OK.

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

Who are you?

• From the web site:

This is a graduate level for those interested in pattern recognition in general and for some elements as applied to computer vision. It is *not* going to be a comprehensive Machine Learning course.

• What do you know?
• A good foundation of probability and linear algebra. This class will have more

math in it than most Computer Science classes.

• Any Machine Learning background will help. Though the course won’t

technically presume ML as a background it will be much easier to grasp if you’ve seen things like graphical models or other inference structures.

• A good working knowledge of Matlab or Python with Numpy. We will likely be

doing things in Matlab in class. I am pretty sure that Octave will be OK though the lack of some plotting make may some figures harder to generate. Because this is my first time offering this class I can only speculate on this.

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

Who are we?

Professor:

Aaron Bobick afb@cc.gatech.edu Office: CCB 316 Office hours:

Tues 1-2pm (email is much better)

TA:

Abhijit Kundu abhijit.dgp@gmail.com Office: CCB 308 Office hours: TBD

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

What will you read?

Hastie, Tibshirani, and Friedman: The Elements of Statistical Learning

• Kevin Murphy’s book:

Machine Learning: a Probabilistic Perspective

(free pdf)

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

What you will do?

• NB: This is the first time I have taught this class. This is a

plan, maybe not even a plan but a goal, certainly not a commitment !

• Your grade is mostly project based set based.
• Communal projects (everyone does mostly the same): 60%
• Your unique final project 30%
• Class presentation (10% - but class size may revamp this)
• Class participation – max of 10% (ie it can only help your grade).
• There will not be a final exam.

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

When will you do them?

• You will have plenty of time for the projects. But they must be

submitted on time.

• Late submissions will only be accepted at full credit with prior
• approval. Otherwise 50% (yes half) reduction.
• TA/Prof *not* obligated to get back to you about permission

the weekend it is due! At your own risk.

• I will be very adamant about this. Not fair to others or the TA.

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

With whom will you do it?

• Honesty/Integrity policy (from web site):

Problem sets are to be done individually (or within your group of no more than two) but you may collaborate at the “white board level” helping each other with algorithms and general computation, BUT YOUR CODE MUST BE YOUR OWN.

• Do not hand in other people’s code unless you (1) say you

are, and (2) you want no credit for that section. We will be explicit about what previous or provided code you can use.

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

How will you do it?

• We will mostly support Matlab
• I know how to program Matlab.
• I am not going to use R.
• I am not going to (deeply) learn Python.
• All the algorithms and demos in Murphy’s book are available in Matlab.
• If you want to use some thing else
• Fine
• Your job to make it work.
• TA not obligated to help.

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

Any questions so far…

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

Who am I…

• Background: degrees in Math, CS but PhD in CogSci/AI;

interest in high level perception and cognition

• Faculty at the MIT Media Lab for a while – developed

lots of work in machine understanding of action from video

• Came here 14 years ago, and collected hats:
• Was GVU director
• Founding chair of the School of Interactive Computing
• But the most fun is being professor in Computational

Perception and Robotics!

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

Research I used to do…

Action recognition from video

• Lots of domains/levels of

complexity:

• Body motions
• Gesture recognition,
• Football plays
• Aware Home
• Surveillance

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

What I am doing now…

Outdoor

Robots that “see”… not (just) a question of geometry but understanding.

Indoor Human Robot Interaction

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

Two specific projects:

(Vision for) Human-robot collaboration in manufacturing (with BMW!) Affordance-based perception: Robot learning the “affordances” of objects and how to use in planning and acting.

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

Why am I teaching this class?

• Because I have to teach another class (besides Computer

Vision) …

• Because I used to teach PR from Duda and

Hart (and Stork) which was written somewhere between the Stone Age and the invention

• f the Prius…
• Because I keep running into PR problems and I really should

understand them better…

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

A few inspirations…

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

• Adaboost classification
• Weak classifiers: Haar-basis like functions (45,396 in total)

Boosting Simple Features [Viola and Jones CVPR 01]

18 Weak classifier Strong classifier 𝑔 𝑦 = 𝛽𝑢ℎ𝑢 𝑦

𝑈 𝑢=1

ℎ𝑢 𝑦 = 1 if 𝑔

𝑢(𝑦) > 0

−1

• therwise

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

Boosting Simple Features [Viola and Jones CVPR 01]

• Integral image
• A value at (x,y) is the sum of the

pixel values above and to the left

• f (x,y).
• The sum of original image values

within the rectangle can be computed: Sum = A-B-C+D 19 ICCV09 Tutorial Tae-

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

Body tracking in Microsoft Kinect for XBox 360

left hand right shoulder neck right foot

Input depth image Training labelled data Visual features Objective function Node training Labels are categorical Input data point Predictor model

Classification forest

Visual features Node parameters Weak learner Feature response

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick Input depth image (bg removed) Inferred body parts posterior

Body tracking in Microsoft Kinect for XBox 360

(2 videos here)

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

But what did I really want to do?

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

A “simple” detection problem

Is it a real person?

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

Anyone in the tunnel?

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

Questions we asked?

• Are we doing detection via decision (Viola Jones) or detection

then recognition? Or hybrid?

• What are the features we’re going to use?
• What’s our data source? Negatives are easy to get; how do we

get positives?

• Is there something about appearance that is invariant across

scenes/cameras? Or is every camera a new system? And then…

• What methods are we going to use to make the decision?
• I wasn’t sure. Had some ideas but not grounded well enough.
• This is why I am teaching this class…

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

What we’re going to do?

• Focusing on categorical labels - “Pattern Recognition” is really

decision theory applied to data

• Review the basics of probability, density estimation and

generative methods for decision making.

• Consider loss functions?
• Explore application of generative methods to some fixed data sets that

have “modest” numbers of features and variable numbers of training examples.

• Then investigate discriminative models and methods
• “Small” vs “large” numbers of features
• “Small” vs “large” numbers training examples; unbalanced
• Most things in the two texts that is focused on making a

decision

• Lots of forests, trees, boosting, bagging

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

What we’re not going to do?

• As little as possible on regression – the prediction of

continuous valued variables.

• Not so much on large pattern (many little variables) problems:
• Maybe no neural nets – or at least as little as possible. A little

about deep learning from the perspective of features.

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

Your first assignment!

• Due next Tuesday, Jan 14.
• Find an available data set that corresponds to “modest”

number of features and “small” number of classes

• Modest – plausible to try all or many possible subsets of features
• Small - maybe less than 5. 2 is ideal. 30 would be too many.
• Submit a one page description of the data, how we would get it

within a week. (Are you making it? That’s OK)

Introduction CS7616 Pattern Recognition – A. Bobick Introduction CS7616 Pattern Recognition – A. Bobick

Going forward

• For coming lectures:
• HTF: read ch 1&2
• Get yourself Matlab (and/or Python)
• Make sure you’re invited to Piazza