SLIDE 1
100Days Of Hands Video Dataset 131 Days boardgame diy drink food - - PowerPoint PPT Presentation
100Days Of Hands Video Dataset 131 Days boardgame diy drink food - - PowerPoint PPT Presentation
Dandan Shan 1 , Jiaqi Geng 1 *, Michelle Shu 2 *, David F. Fouhey 1 University of Michigan 1 , Johns Hopkins University 2 Sponsored by Procter & Gamble and Nokia Networks 100Days Of Hands Video Dataset 131 Days boardgame diy drink food
SLIDE 2
SLIDE 3
100K Frame-level Annotations
- Box around hand
- Side (left / right)
- Contact (no / self / other /
portable / furniture)
- Box around object in contact
- Association
SLIDE 4
Method
Hand Box Side State Offset Object Box Faster-RCNN Standard Detection Classification (if hand) Classification (if hand) Regression (if in contact) Standard Detection (another class)
[1] Ren et al. Faster R-CNN: Towards real-time object detection with region proposal networks. NIPS 2015. [2] Yang et al. A Faster Pytorch Implementation of Faster R-CNN (https://github.com/jwyang/faster-rcnn.pytorch).
simple greedy matching
- n hands and objects
SLIDE 5
Cross-Dataset Hand Detection
Faster RCNN with Resnet-101 backbone, AP (TP = IoU > 0.5) 100DOH VLOG VIVA Ego VGG >80 >70 >60 >40 <40 Train
- n
TV+Co VGG 100DOH VLOG VIVA TV+Co Ego 73.9 90.1 86.4 86.5 65.4 90.8 21.5 17.4 23.6 40.7 27.7 32.6 90.8 44.9 10.1 8.0 90.7 56.8 64.6 78.6 77.5 76.6 59.2 83.2 79.6 77.4 61.4 56.2 61.7 61.5 74.9 78.8 69.9 66.6 62.4 63.0
SLIDE 6
Qualitative Results
SLIDE 7
Method
Hand Box Side State Offset Object Box
L R
Hasson et al. Given location and side, can predict low-dimensional parameterization of hand
[1] Hasson et al. Learning joint reconstruction of hands and manipulated objects. CVPR 2019. [2] Romero et al. Embodied Hands: Modeling and Capturing Hands and Bodies Together. SIGGRAPH Asia 2017.
SLIDE 8
Enabling MANO at Scale
[1] Hasson et al. Learning joint reconstruction of hands and manipulated objects. CVPR 2019. [2] Romero et al. Embodied Hands: Modeling and Capturing Hands and Bodies Together. SIGGRAPH Asia 2017.
SLIDE 9
Hand Predictions
Input Prediction Input Prediction
SLIDE 10