poselets body part detectors trained using 3d human pose
play

Poselets: Body Part Detectors Trained Using 3D Human Pose - PowerPoint PPT Presentation

Jun 14, 2023 •196 likes •733 views

Poselets: Body Part Detectors Trained Using 3D Human Pose Annotations Lubomir Bourdev and Jitendra Malik Experiments Presented by Randall Smith Friday, November 2, 12 1 Outline Introduction Dataset Overview Annotations

  1. Poselets: Body Part Detectors Trained Using 3D Human Pose Annotations Lubomir Bourdev and Jitendra Malik Experiments Presented by Randall Smith Friday, November 2, 12 1

  2. Outline • Introduction • Dataset • Overview • Annotations • Distance Function • Segmentation • Experiments • Conclusion Friday, November 2, 12 2

  3. Dataset • Humans in 3D (H3D) • 2480 annotations • (1500 train / 500 test / 240 validate) • Java3D annotation tool Friday, November 2, 12 3

  4. Dataset Friday, November 2, 12 4

  5. Overview : Training Random Seed Select Nearby Train SVMs Prune Patches Patches Residual Error Friday, November 2, 12 5

  6. Overview : Detection Run All Mean Shift Compute Max Margin Poselets Cluster Activations Hough Friday, November 2, 12 6

  7. Annotations • Bounding box placed over annotated figure Friday, November 2, 12 7

  8. Annotations • Live Demo Friday, November 2, 12 8

  9. Annotations • Live Demo Friday, November 2, 12 9

  10. Annotations : Skeleton • Annotated skeleton Friday, November 2, 12 10

  11. Annotations : Keypoints • 20 manually annotated keypoints • 15 manually annotated segments Friday, November 2, 12 11

  12. Annotations : Query Mean Examples Distance 0 0 0 0.0001 0.0001 0.0002 0.0002 0.0003 0.0005 0.0005 • Query at green box Friday, November 2, 12 12

  13. Annotations : Query Mean Examples Distance 0 0.0017 0.0020 0.0020 0.0028 0.0031 0.0032 0.0035 0.0041 0.0043 Friday, November 2, 12 13

  14. Annotations : Query Mean Examples Distance 0 0.0019 0.0079 0.0093 0.0096 0.0104 0.0134 0.0139 0.0154 0.0169 Friday, November 2, 12 14

  15. Annotations : Query Mean Examples Distance 0 0.0067 0.0086 0.0167 0.0176 0.0178 0.0180 0.0183 0.0186 0.0198 Friday, November 2, 12 15

  16. Distance Function • Paper: computes a weighted sum of Euclidean distances with additive penalty. • Implementation: Procrustes distance plus penalty. • What is the Procrustes distance? Friday, November 2, 12 16

  17. Distance Function D proc ( x 1 , x 2 ) = min s,R,t k x 1 � ( sRx 2 + t ) k • Scale so that RMS is 1.0, translate to origin, and solve for rotation matrix R. • Non visible key points ignored Friday, November 2, 12 17

  18. Distance Function D ( x s , x r ) = D proc ( x s , s r ) + Penalty • Need to compute linear least squares / SVD to solve. • Is this very expensive? Friday, November 2, 12 18

  19. Distance Function • Live Demo: 2D Toy Example Friday, November 2, 12 19

  20. Segments : UpperClothes Friday, November 2, 12 20

  21. Segments : LowerClothes Friday, November 2, 12 21

  22. Segments : Faces Friday, November 2, 12 22

  23. Detection • A simple, occlusion free test Friday, November 2, 12 23

  24. Detection Task Time Features 0.69s Detect Poselets 0.56s Score 0.82s Cluster 0.61s Localize 0.11s Total 2.49s Friday, November 2, 12 24

  25. Detection : Example • Score: 14.10. How did the clusters vote? Friday, November 2, 12 25

  26. Detection : Votes • Inspect top hits • Inspect bottom hits Friday, November 2, 12 26

  27. Detection : Best Votes Friday, November 2, 12 27

  28. Detection : Best Votes Friday, November 2, 12 28

  29. Detection : Best Votes Friday, November 2, 12 29

  30. Detection : Best Votes Friday, November 2, 12 30

  31. Detection : Worst Votes Friday, November 2, 12 31

  32. Detection : Worst Votes Friday, November 2, 12 32

  33. Detection : Worst Votes Friday, November 2, 12 33

  34. Detection : Worst Votes Friday, November 2, 12 34

  35. Detection : Tests • Some samples from PASCAL VOC2007 • With varying degrees of occlusion • Comparison with Discriminatively Trained Deformable Part Models (DPM) • Some pictures taken from my iPhone 4S • Increasingly difficult in terms of occlusion Friday, November 2, 12 35

  36. Detection : Tests • PASCAL VOC2007 Friday, November 2, 12 36

  37. Detection : Tests • Some more difficult occlusion cases Friday, November 2, 12 37

  38. Detection : Comparison • Scores: 11.50 and 2.03. • DPM failed. Friday, November 2, 12 38

  39. Detection : Comparison • Score: 5.21. 31 poselet clusters contributed. • DPM HOG parts and bounding box shown. Friday, November 2, 12 39

  40. Detection : Comparison • Scores 12.48, 9.67, and 5.40. • DPM failed. Friday, November 2, 12 40

  41. Detection : Comparison • Score 12.01. • HOG parts and bounding box shown. Friday, November 2, 12 41

  42. Detection : Comparison • Score: 5.21. • HOG parts and bounding box shown. Friday, November 2, 12 42

  43. Detection : Comparison • Both fail. Friday, November 2, 12 43

  44. Detection : Comparison • When both succeed, DPM seems to get better bounding boxes. • The poselet algorithm always tries to get the best bounding box it can. • DPM has no way of degrading gracefully. Friday, November 2, 12 44

  45. Detection : Occlusion • Score: 22.4. 54 poselet clusters contributed. Friday, November 2, 12 45

  46. Detection : Occlusion • Score: 0.29. 3 poselet clusters contributed. Friday, November 2, 12 46

  47. Detection : Occlusion • Score: 0.38. 2 poselet clusters contributed. Friday, November 2, 12 47

  48. Detection : Occlusion • Score: 0.27. 1 poselet cluster contributed. Friday, November 2, 12 48

  49. Detection : Occlusion • Score: 0.21. 2 poselet clusters contributed. Friday, November 2, 12 49

  50. Detection : Occlusion • DPM fails on all of these. • Poselets do pretty poorly, but it still computes a bounding box. • Poselets have the chance of getting it right. Friday, November 2, 12 50

  51. Conclusions • Poselets are intuitive to find in an image. • If a body part is exposed, a poselet might match it. • Poselet ranking and scoring can be understood in an intuitive way. • Can handle some occlusion • Will always try to compute a bounding box. Friday, November 2, 12 51

  52. Conclusions • Sometimes poselet activations can be misleading. • Sometimes, some poselets should have higher scores than others. • This is sort of like getting the right answer for the wrong reasons. • The dataset is very labor intensive. Friday, November 2, 12 52

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Poselets: Body Part Detectors Trained Using 3D Human Pose Annotations LUBOMIR BOURDEV AND

Poselets: Body Part Detectors Trained Using 3D Human Pose Annotations LUBOMIR BOURDEV AND

Poselets: Body Part Detectors Trained Using 3D Human Pose Annotations LUBOMIR BOURDEV AND JITENDRA MALIK Outline H3D dataset Pipeline Analysis of Poselets fired Selective parts torso, legs and face Other cases Clutter, Rotation and

765 views • 37 slides
Human Pose Search using Deep Poselets Nataraj Jammalamadaka * Andrew Zisserman C. V. Jawahar *

Human Pose Search using Deep Poselets Nataraj Jammalamadaka * Andrew Zisserman C. V. Jawahar *

Human Pose Search using Deep Poselets Nataraj Jammalamadaka * Andrew Zisserman C. V. Jawahar * * CVIT, IIIT Hyderabad, India Visual Geometry Group, Department of Engineering, IIIT Hyderabad University of Oxford Human Pose: Gesture and

553 views • 33 slides
Learning Human Pose from Unaligned Data through Image Translation Tomas Jakab Ankush Gupta

Learning Human Pose from Unaligned Data through Image Translation Tomas Jakab Ankush Gupta

Learning Human Pose from Unaligned Data through Image Translation Tomas Jakab Ankush Gupta Andrea Vedaldi Hakan Bilen Presented by Triantafyllos Afouras Goal Learn human-body landmark detectors from unlabelled videos and unaligned

402 views • 27 slides
Articulated body dynamics Beyond human models How would you represent a pose? Quadraped animals

Articulated body dynamics Beyond human models How would you represent a pose? Quadraped animals

Articulated rigid bodies Articulated body dynamics Beyond human models How would you represent a pose? Quadraped animals Wavy hair Animal fur Plants Maximal vs. reduced coordinates How are things connected? Maximal coordinates Reduced

265 views • 10 slides
CEF IPA arm 11 102 the long part at each side of the human body, ending in a hand noun My

CEF IPA arm 11 102 the long part at each side of the human body, ending in a hand noun My

WORD UNIT NO PAGE DEFINITION PoS EXAMPLE CEF IPA arm 11 102 the long part at each side of the human body, ending in a hand noun My arms ache from carrying this bag. m one of the parts of the body of a human or animal that is used

172 views • 3 slides
Human Pose Estimation by Yannic Jnike - 04.11.2019 https://www.youtube.com/watch?v=mxKlUO_tjcg

Human Pose Estimation by Yannic Jnike - 04.11.2019 https://www.youtube.com/watch?v=mxKlUO_tjcg

Human Pose Estimation by Yannic Jnike - 04.11.2019 https://www.youtube.com/watch?v=mxKlUO_tjcg 1 Human Pose Estimation 1. What is Human Pose Estimation 2. OpenPose Pipeline 3. Bottom Up or Top Down Approach 2 What is Human Pose

3.09k views • 33 slides
Deep neural nets for human pose estimation in videos Tomas Pfister, James Charles, Andrew

Deep neural nets for human pose estimation in videos Tomas Pfister, James Charles, Andrew

Deep neural nets for human pose estimation in videos Tomas Pfister, James Charles, Andrew Zisserman Department of Engineering Science University of Oxford http://www.robots.ox.ac.uk/~vgg Aim: Estimate 2D upper body joint positions (wrist,

519 views • 28 slides
Human Pose Recovery And Gesture Recognition CS365 : Artificial Intelligence Khandesh

Human Pose Recovery And Gesture Recognition CS365 : Artificial Intelligence Khandesh

Human Pose Recovery And Gesture Recognition CS365 : Artificial Intelligence Khandesh Bhange(11196) And Piyush Kumar(11496) Problem Statement There are two parts of this project. 1 st Human Pose Recovery We have to label 14 limbs

389 views • 15 slides
7a A&P: Introduction to the Human Body - Body Compass 7a A&P: Introduction to the Human

7a A&P: Introduction to the Human Body - Body Compass 7a A&P: Introduction to the Human

7a A&P: Introduction to the Human Body - Body Compass 7a A&P: Introduction to the Human Body - Body Compass Class Outline 5 minutes Attendance, Breath of Arrival, and Reminders 10 minutes Lecture: AOIs of the

399 views • 38 slides
Human Pose Es,ma,on Greg Mori CMPT 888 Problem Human

Human Pose Es,ma,on Greg Mori CMPT 888 Problem Human

Human Pose Es,ma,on Greg Mori CMPT 888 Problem Human Figures in S,ll Images Detec,on of humans is possible for stereotypical poses Standing

503 views • 46 slides
Chirality Nets for Human Pose Regression Raymond A. Yeh*, Yuan-Ting Hu*, Alexander G. Schwing

Chirality Nets for Human Pose Regression Raymond A. Yeh*, Yuan-Ting Hu*, Alexander G. Schwing

Chirality Nets for Human Pose Regression Raymond A. Yeh*, Yuan-Ting Hu*, Alexander G. Schwing University of Illinois at Urbana-Champaign December 14, 2019 Yeh et al. 1 NeurIPS2019 Motivation Human Pose Regression Prior works: Concatenation

1.21k views • 16 slides
Domain Adapta,on for Upper Body Pose Tracking in Signed TV

Domain Adapta,on for Upper Body Pose Tracking in Signed TV

Domain Adapta,on for Upper Body Pose Tracking in Signed TV Broadcasts James Charles 1 , Tomas Pfister 2 , Derek Magee 1 , David Hogg 1 , Andrew Zisserman 2 1 School

646 views • 50 slides
5a A&P: Introduction to the Human Body - Cells 5a A&P: Introduction to the Human Body -

5a A&P: Introduction to the Human Body - Cells 5a A&P: Introduction to the Human Body -

5a A&P: Introduction to the Human Body - Cells 5a A&P: Introduction to the Human Body - Cells Class Outline 5 minutes Attendance, Breath of Arrival, and Reminders 10 minutes Lecture: AOIs of the erectors 5

830 views • 70 slides
5a A&P: Introduction to the Human Body - Cells 5a A&P: Introduction to the Human Body -

5a A&P: Introduction to the Human Body - Cells 5a A&P: Introduction to the Human Body -

5a A&P: Introduction to the Human Body - Cells 5a A&P: Introduction to the Human Body - Cells Class Outline 5 minutes Attendance, Breath of Arrival, and Reminders 10 minutes Lecture: AOIs of the erectors 5

987 views • 71 slides
5a A&P Introduction to the Human Body 1 - Cells 5a A&P Introduction to the Human Body 1 -

5a A&P Introduction to the Human Body 1 - Cells 5a A&P Introduction to the Human Body 1 -

5a A&P Introduction to the Human Body 1 - Cells 5a A&P Introduction to the Human Body 1 - Cells Class Outline 5 minutes Attendance, Breath of Arrival, and Reminders 10 minutes Lecture: AOIs of the erectors 5

1.34k views • 71 slides
6a A&P: Introduction to the Human Body - Tissues 6a A&P: Introduction to the Human Body -

6a A&P: Introduction to the Human Body - Tissues 6a A&P: Introduction to the Human Body -

6a A&P: Introduction to the Human Body - Tissues 6a A&P: Introduction to the Human Body - Tissues Class Outline 5 minutes Attendance, Breath of Arrival, and Reminders 10 minutes Lecture: AOIs of the gluteals 5

831 views • 60 slides
Data Structures in Processing The Object Class

Data Structures in Processing The Object Class

Data Structures in Processing The Object Class "Object" is the most general class Variables of type Object can hold any other type Object

309 views • 17 slides
Efficient Scoring in Lucene Stefan Pohl Nokia Berlin stefan.pohl@nokia.com Agenda

Efficient Scoring in Lucene Stefan Pohl Nokia Berlin stefan.pohl@nokia.com Agenda

Efficient Scoring in Lucene Stefan Pohl Nokia Berlin stefan.pohl@nokia.com Agenda Motivation Review: Query Processing Modes in Lucene Scoring Efficiency Optimization Experiments Motivation Speed ! Human Reaction Time: 200

1.02k views • 56 slides
Top-Down Performance Analysis Methodology for Workflows Ronny Tschter, Christian Herold, Bill

Top-Down Performance Analysis Methodology for Workflows Ronny Tschter, Christian Herold, Bill

Top-Down Performance Analysis Methodology for Workflows Ronny Tschter, Christian Herold, Bill Williams, Maximilian Knespel, Matthias Weber 1 Workflows Matter Common use cases today are not limited to single applications Pre- and

450 views • 23 slides
Typically represent objects by bounding boxes. People have tried Goal rotated bounding boxes

Typically represent objects by bounding boxes. People have tried Goal rotated bounding boxes

Lecture 6: Introduction to Detection Jonathan Krause Fei-Fei Li, Jonathan Krause Lecture 6 - 1 Typically represent objects by bounding boxes. People have tried Goal rotated bounding boxes before. Locate objects in images Fei-Fei Li,

515 views • 24 slides
Loops! Flow of Control: Loops (Savitch, Chapter 4) TOPICS while Loops do while

Loops! Flow of Control: Loops (Savitch, Chapter 4) TOPICS while Loops do while

Loops! Flow of Control: Loops (Savitch, Chapter 4) TOPICS while Loops do while Loops for Loops break Statement continue Statement CS 160, Fall Semester 2012 2 An Example While Loop

446 views • 11 slides
Functions A set of statements (lines of code) that can be run repeatedly Functions

Functions A set of statements (lines of code) that can be run repeatedly Functions

Functions A set of statements (lines of code) that can be run repeatedly Functions Goals: Learning Python by Lutz and Ascher Code reuse Procedural decomposition def printH(): Top-Down Design print "* *" print

172 views • 4 slides
Designing for the Cloud An Architects Perspective Christos Kozyrakis Computer Systems Lab

Designing for the Cloud An Architects Perspective Christos Kozyrakis Computer Systems Lab

Designing for the Cloud An Architects Perspective Christos Kozyrakis Computer Systems Lab Stanford University http: / / csl.stanford.edu/ ~ christos Good

423 views • 7 slides
CS/EE 3710 Computer Design Lab T Th 3:40pm-5:00pm Lectures in WEB 110, Labs in MEB 3133

CS/EE 3710 Computer Design Lab T Th 3:40pm-5:00pm Lectures in WEB 110, Labs in MEB 3133

CS/EE 3710 Computer Design Lab Fall 2010 CS/EE 3710 Computer Design Lab T Th 3:40pm-5:00pm Lectures in WEB 110, Labs in MEB 3133 (DSL) Instructor: Erik Brunvand MEB 3142 Office Hours: After class, when my door is open, or by

724 views • 16 slides

More recommend