Tracking Articulated Objects Alexander (Sasha) Lambert CS7495 Fall - - PowerPoint PPT Presentation

Tracking Articulated Objects

Alexander (Sasha) Lambert

CS7495 – Fall 2014

2

Tracking From Depth

2

Kinect One sensor (Microsoft)

http://www.creativeapplications.net/ http://www.blogcdn.com/

• Infra-red point-clouds (structured light)
• Affordable sensors (Primesense)
• Large body of work on people-tracking

฀Complex tracking methods ฀ML-based approach (ex. Decision

forests)

3

Tracking – Robotics

3

• Move towards cheaper robot manipulators

฀Problem: poor encoders, flexible joints

• Would like a generalized approach
• Applications:

฀Human/Robot, Robot/Robot interaction

(ex. Task collaboration)

฀Self-calibration (Manipulators) ฀Interacting with every day objects

(drawers, doors, can-openers...)

• Realtime, robust to occlusion
• Useability and generality

4

Tracking

4

• DART (RSS '2014) -

Schmidt, Newcombe, Fox

฀Generalized framework ฀Realtime, GPU-optimized ฀Requires only kinematic model & part geometries

5

HEADER

5

6

HEADER

6

7

Kalman Filter

7

8

Extended Kalman Filter

8

• Non-linear functions (dynamics,

measurement)

฀Idea: local linearization ฀Preserves Gaussian shape

9

Extended Kalman Filter

9

Predictor Correcto r Correcto r Predictor

10

DART Tracker

10

Experiments → const. dyn. Model prior(t) = posterior(t-1)

11

DART Tracker

11

“Tracking Energy”

• Develop a measurement model
• θ – pose parameters (state)
• D – Depth map

12

Point Cloud Registration

12

http://taylorwang.files.wordpress.com/

• Correspondence between model and data
• ICP – Iterative Closest Point

฀Data association: many variants (CP, P2PL, KNN)

13

Point Cloud Registration

13

Non-Linear Optimization (Fitzgibbon '01)

• Gradient-based iteration
• Levenberg-Marquardt algorithm
• Signed Distance Function (SDF)

฀Can be pre-computed

• Schmidt et al. → use similar SDF mapping for articulated objects in 3D
• θ – pose parameters (state)
• x – 3d position (camera frame)
• u – pixel index

14

Objective Function

14

15

Objective Function

15

• Composite SDF for articulated

body

• k – frame index
• T – camera/frame transform

16

Optimization

16

1.Taylor-series expansion (Gauss-Newton approximation) 2.Jacobian computation

• 3. Iteration Update

17

Symmetric Formulation

17

Dealing with Occlusions

• Use 'Free Space' to constrain
• bjective function
• Augment probability with model-

point prediction

• SDF of observation constricting

model prediction

18

HEADER

18

Robotics/courses/CS7442-Computer_Vision/CS7495-presentation/slides/video/optim.mp4#Play Video

19

Results – Hand tracking

19

• Parallelized computation (GPU)
• Hand-pose dataset (Qian et al. '12)

–Models with 26 d.o.f –Frequent, rapid occlusions

• Qian et al. , Oikonomidis et. al

฀ICP + PSO

Average distance b/w prediction & ground truth (mm)

20

Results – Body tracking

20

• EVAL dataset (Ganaparthi et al. '12)

–Models with 48 d.o.f –% of joints within 10cm

• Ganaparthi et al.

–ICP + free-space

• Ye & Yang ('14)

–GMM –Shape estimation

21

Results - Servoing

21

• Grasping with robot manipulator
• Provide visual feedback

–Updated state to controller

• Improved accuracy

–3/10 vs 10/10

22

Results

22

box/GTRobotics/courses/CS7442-Computer_Vision/CS7495-presentation/slides/video/results.mp4

23

HEADER

23

THANK YOU!