Poselets: Body Part Detectors Trained Using 3D Human Pose - - PowerPoint PPT Presentation

▶
poselets body part detectors trained using 3d human pose
SMART_READER_LITE
LIVE PREVIEW

Poselets: Body Part Detectors Trained Using 3D Human Pose - - PowerPoint PPT Presentation

Jun 14, 2023 196 likes •737 views

Poselets: Body Part Detectors Trained Using 3D Human Pose Annotations Lubomir Bourdev and Jitendra Malik Experiments Presented by Randall Smith Friday, November 2, 12 1 Outline Introduction Dataset Overview Annotations

slide-1
SLIDE 1

Poselets: Body Part Detectors Trained Using 3D Human Pose Annotations

Lubomir Bourdev and Jitendra Malik

Experiments Presented by Randall Smith

1 Friday, November 2, 12

slide-2
SLIDE 2

Outline

  • Introduction
  • Dataset
  • Overview
  • Annotations
  • Distance Function
  • Segmentation
  • Experiments
  • Conclusion

2 Friday, November 2, 12

slide-3
SLIDE 3

Dataset

  • Humans in 3D (H3D)
  • 2480 annotations
  • (1500 train / 500 test / 240 validate)
  • Java3D annotation tool

3 Friday, November 2, 12

slide-4
SLIDE 4

Dataset

4 Friday, November 2, 12

slide-5
SLIDE 5

Overview : Training

Select Nearby Patches Prune Random Seed Patches Train SVMs

Residual Error

5 Friday, November 2, 12

slide-6
SLIDE 6

Overview : Detection

Compute Activations Max Margin Hough Mean Shift Cluster Run All Poselets

6 Friday, November 2, 12

slide-7
SLIDE 7

Annotations

  • Bounding box placed over annotated figure

7 Friday, November 2, 12

slide-8
SLIDE 8

Annotations

  • Live Demo

8 Friday, November 2, 12

slide-9
SLIDE 9

Annotations

  • Live Demo

9 Friday, November 2, 12

slide-10
SLIDE 10

Annotations : Skeleton

  • Annotated skeleton

10 Friday, November 2, 12

slide-11
SLIDE 11

Annotations : Keypoints

  • 20 manually annotated keypoints
  • 15 manually annotated segments

11 Friday, November 2, 12

slide-12
SLIDE 12

Annotations : Query

  • Query at green box

Mean Examples

0.0001 0.0001 0.0002 0.0002 0.0003 0.0005 0.0005

Distance

12 Friday, November 2, 12

slide-13
SLIDE 13

Annotations : Query

Mean Examples

0.0017 0.0020 0.0020 0.0028 0.0031 0.0032 0.0035 0.0041 0.0043

Distance

13 Friday, November 2, 12

slide-14
SLIDE 14

Annotations : Query

Mean Examples

0.0019 0.0079 0.0093 0.0096 0.0104 0.0134 0.0139 0.0154 0.0169

Distance

14 Friday, November 2, 12

slide-15
SLIDE 15

Annotations : Query

Mean Examples

0.0067 0.0086 0.0167 0.0176 0.0178 0.0180 0.0183 0.0186 0.0198

Distance

15 Friday, November 2, 12

slide-16
SLIDE 16

Distance Function

  • Paper: computes a weighted sum of

Euclidean distances with additive penalty.

  • Implementation: Procrustes distance plus

penalty.

  • What is the Procrustes distance?

16 Friday, November 2, 12

slide-17
SLIDE 17

Distance Function

  • Scale so that RMS is 1.0, translate to origin,

and solve for rotation matrix R.

  • Non visible key points ignored

Dproc(x1, x2) = min

s,R,t kx1 (sRx2 + t)k

17 Friday, November 2, 12

slide-18
SLIDE 18

Distance Function

  • Need to compute linear least squares /

SVD to solve.

  • Is this very expensive?

D(xs, xr) = Dproc(xs, sr) + Penalty

18 Friday, November 2, 12

slide-19
SLIDE 19

Distance Function

  • Live Demo: 2D Toy Example

19 Friday, November 2, 12

slide-20
SLIDE 20

Segments : UpperClothes

20 Friday, November 2, 12

slide-21
SLIDE 21

Segments : LowerClothes

21 Friday, November 2, 12

slide-22
SLIDE 22

Segments : Faces

22 Friday, November 2, 12

slide-23
SLIDE 23

Detection

  • A simple, occlusion free test

23 Friday, November 2, 12

slide-24
SLIDE 24

Detection

Task Time Features 0.69s Detect Poselets 0.56s Score 0.82s Cluster 0.61s Localize 0.11s Total 2.49s

24 Friday, November 2, 12

slide-25
SLIDE 25

Detection : Example

  • Score: 14.10. How did the clusters vote?

25 Friday, November 2, 12

slide-26
SLIDE 26

Detection : Votes

  • Inspect top hits
  • Inspect bottom hits

26 Friday, November 2, 12

slide-27
SLIDE 27

Detection : Best Votes

27 Friday, November 2, 12

slide-28
SLIDE 28

Detection : Best Votes

28 Friday, November 2, 12

slide-29
SLIDE 29

Detection : Best Votes

29 Friday, November 2, 12

slide-30
SLIDE 30

Detection : Best Votes

30 Friday, November 2, 12

slide-31
SLIDE 31

Detection : Worst Votes

31 Friday, November 2, 12

slide-32
SLIDE 32

Detection : Worst Votes

32 Friday, November 2, 12

slide-33
SLIDE 33

Detection : Worst Votes

33 Friday, November 2, 12

slide-34
SLIDE 34

Detection : Worst Votes

34 Friday, November 2, 12

slide-35
SLIDE 35

Detection : Tests

  • Some samples from PASCAL

VOC2007

  • With varying degrees of occlusion
  • Comparison with Discriminatively

Trained Deformable Part Models (DPM)

  • Some pictures taken from my iPhone 4S
  • Increasingly difficult in terms of occlusion

35 Friday, November 2, 12

slide-36
SLIDE 36

Detection : Tests

  • PASCAL

VOC2007

36 Friday, November 2, 12

slide-37
SLIDE 37

Detection : Tests

  • Some more difficult occlusion cases

37 Friday, November 2, 12

slide-38
SLIDE 38

Detection : Comparison

  • Scores: 11.50 and 2.03.
  • DPM failed.

38 Friday, November 2, 12

slide-39
SLIDE 39

Detection : Comparison

  • Score: 5.21. 31 poselet clusters contributed.
  • DPM HOG parts and bounding box shown.

39 Friday, November 2, 12

slide-40
SLIDE 40

Detection : Comparison

  • Scores 12.48, 9.67, and 5.40.
  • DPM failed.

40 Friday, November 2, 12

slide-41
SLIDE 41

Detection : Comparison

  • Score 12.01.
  • HOG parts and bounding box shown.

41 Friday, November 2, 12

slide-42
SLIDE 42

Detection : Comparison

  • Score: 5.21.
  • HOG parts and bounding box shown.

42 Friday, November 2, 12

slide-43
SLIDE 43

Detection : Comparison

  • Both fail.

43 Friday, November 2, 12

slide-44
SLIDE 44

Detection : Comparison

  • When both succeed, DPM seems to get

better bounding boxes.

  • The poselet algorithm always tries to get

the best bounding box it can.

  • DPM has no way of degrading gracefully.

44 Friday, November 2, 12

slide-45
SLIDE 45

Detection : Occlusion

  • Score: 22.4. 54 poselet clusters contributed.

45 Friday, November 2, 12

slide-46
SLIDE 46

Detection : Occlusion

  • Score: 0.29. 3 poselet clusters contributed.

46 Friday, November 2, 12

slide-47
SLIDE 47

Detection : Occlusion

  • Score: 0.38. 2 poselet clusters contributed.

47 Friday, November 2, 12

slide-48
SLIDE 48

Detection : Occlusion

  • Score: 0.27. 1 poselet cluster contributed.

48 Friday, November 2, 12

slide-49
SLIDE 49

Detection : Occlusion

  • Score: 0.21. 2 poselet clusters contributed.

49 Friday, November 2, 12

slide-50
SLIDE 50

Detection : Occlusion

  • DPM fails on all of these.
  • Poselets do pretty poorly, but it still

computes a bounding box.

  • Poselets have the chance of getting it right.

50 Friday, November 2, 12

slide-51
SLIDE 51

Conclusions

  • Poselets are intuitive to find in an image.
  • If a body part is exposed, a poselet might

match it.

  • Poselet ranking and scoring can be

understood in an intuitive way.

  • Can handle some occlusion
  • Will always try to compute a bounding box.

51 Friday, November 2, 12

slide-52
SLIDE 52

Conclusions

  • Sometimes poselet activations can be

misleading.

  • Sometimes, some poselets should have higher

scores than others.

  • This is sort of like getting the right answer for

the wrong reasons.

  • The dataset is very labor intensive.

52 Friday, November 2, 12