deep convolutional poses for human
play

Deep Convolutional Poses for Human Interaction Recognition in - PowerPoint PPT Presentation

Oct 10, 2023 •447 likes •858 views

Deep Convolutional Poses for Human Interaction Recognition in Monocular Videos Marcel Sheeny de Moraes Supervisor: Neil Robertson HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Outline

  1. Deep Convolutional Poses for Human Interaction Recognition in Monocular Videos Marcel Sheeny de Moraes Supervisor: Neil Robertson

  2. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Outline • Introduction • Related Works • Methodology • Results • Conclusion and Future Works 2 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  3. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Outline • Introduction • Related Works • Methodology • Results • Conclusion and Future Works 3 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  4. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Introduction • Human Interaction Recognition • Surveillance • Human-Computer Interaction Automatic Video labeling • Kicking Hand shake High five 4 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  5. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Introduction • Goal of the project • To use Human Pose Estimation to recognize the Human Interaction in Monocular videos (RGB) . 5 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  6. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Outline • Introduction • Related Works • Methodology • Results • Conclusion and Future Works 6 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  7. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Related Works • Park and Arggawal, (Multimedia systems 2004) • Ellipse and Convex Hull features. • Hierarchical Bayesian Network. 9 types of interactions. • 78% of accuracy. • 7 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  8. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Related Works • Yun, et al. , (CVPR 2012) • Depth camera (kinect) to estimate the human pose. • 6 features from human pose. • Multiple Instance Learning. • 8 types of interactions. • 80% of accuracy using 3 frames. • 91% of accuracy whole sequence. 8 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  9. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Related Works • Hu, et al. , (ECCV 2014) • Yun, et al. (2012) dataset. • Positive actor features. • Hidden Markov Model to classify. • 76.2% of accuracy per frame and 83.3% for the whole sequence. • Zhu, et al., (AAAI 2016) • Yun, et al. (2012) dataset. • Deep LSTM network to recognize the interaction using the human pose estimation. • 90.41% of accuracy. 9 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  10. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Benchmark • Benchmark for the Two-Person Interaction dataset. Method Per frame Whole sequence 91.10% Yun, et al. (2012) 80.30% Hu, et al. (2014) 76.1% 83.33% 90.41% Zhu, et al. (2015) - 10 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  11. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Outline • Introduction • Related Works • Methodology • Results • Conclusion and Future Works 11 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  12. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Methodology 12 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  13. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Dataset • Two-person Interaction Detection Using Body-Pose Features (Yun, et al ., 2012). 8 interactions: approaching, departing, pushing, kicking, punching, • exchanging objects, hugging, and shaking hands. • 282 samples of interactions. Kicking Punching Hugging Shaking Hands 13 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  14. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Methodology 14 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  15. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Person Detection • “Faster R -CNN: Towards Real-Time Object Detection with Region Proposal Networks”, by Ren, et al (2015) • State-of-the-art method for object detection. • 0.1 s to detect the person in each image. • Very Deep Convolutional Neural Network (VGG-16). 15 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  16. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Person Detection Results 16 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  17. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Methodology 17 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  18. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Multi-Person Tracking • Kalman Filter with a linear motion model. • Hungarian Algorithm is used to assign detections and predictions. • Threshold methods are used to decide new/lost tracks. 18 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  19. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Methodology 19 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  20. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Human Pose Estimation • “Convolutional Pose Machines”, by Wei, et al (CVPR 2016) • 12 hierarchical Deep Convolutional Neural Networks (DCNN). • 12 different size inputs. • Current state-of-the-art for human pose estimation. PCKh @ 0.2 PC Benchmark using LSP dataset Method Head Shoulder Elbow Wrist Hip Knee Ankle Total AUC Pischulin, et al., ICCV’13 87.2 56.7 46.7 38.9 61.0 57.5 52.7 57.1 35.8 Chen and Yulle, NIPS’14 91.8 78.2 71.8 65.5 73.3 70.2 63.4 73.4 40.1 Carreira, et al. , CVPR’16 90.5 81.8 65.8 59.8 81.6 70.6 62.0 73.1 41.5 Fan et al. , CVPR’15 92.4 75.2 65.3 64.0 75.7 68.3 70.4 73.0 42.2 Tompson, et al., NIPS’14 90.6 79.2 67.9 63.4 69.5 71.0 64.2 72.3 47.3 Yang, et al. , CVPR’16 90.6 78.1 73.8 68.8 74.8 69.9 58.9 73.6 39.3 Pischulin, et al. , CVPR’16 97.0 91.0 83.8 78.1 91.0 86.7 82.0 87.1 63.5 Wei, et al. , CVPR 97.8 92.5 87.0 83.9 91.5 90.8 89.9 90.5 65.4 20 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  21. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Human Pose Estimation • “Convolutional Pose Machines”, by Wei, et al (CVPR 2016) 21 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  22. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Results 22 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  23. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Results https://www.youtube.com/watch?v=llLj50gE9GI 23 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  24. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Methodology 24 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  25. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Feature Extraction • 6 types of features were used • XY Joint Position (XY) • Distances from Related Joints (DRJ) • Distances from One Joint (DOJ) • Absolute Difference (AD) • Joint Angles (JA) • Velocity (VEL) 25 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  26. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Joint Position and Distance from Related Joints • Raw Joint Position (XY) 𝐺 𝑘 = 𝑄(𝑘) • Distance Related Joints (DRJ) 1 𝑘 − 𝑄 2 𝑘 | 𝐺 𝑘 = | 𝑄 26 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  27. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Distance from One Joint and Absolute Difference • Distance from One Joint (DOJ) 1 𝑘 1 − 𝑄 2 𝑘 2 | 𝐺 𝑘 1 , 𝑘 2 = | 𝑄 • Absolute difference (AD) 1 𝑘 − 𝑄 2 𝑘 | 𝐺 𝑘 = |𝑄 27 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  28. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Joint Angles and Velocity • Joint Angles (JA) 𝑄 𝑧 𝑘 1 −𝑄 𝑧 (𝑘 2 ) 𝐺 𝑘 1 , 𝑘 2 = 𝑢𝑏𝑜 −1 𝑄 𝑦 𝑘 1 −𝑄 𝑦 (𝑘 2 ) • Velocity (VEL) 𝐺(𝑘, 𝑢 1 , 𝑢 2 ) = 𝑄 𝑘, 𝑢 1 − 𝑄(𝑘, 𝑢 2 ) 28 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  29. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Methodology 29 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided

CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided

CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided Backpropagation, Deep Dream, Deep Art, Fooling Convolutional Neural Networks Mitesh M. Khapra Department of Computer Science and Engineering Indian Institute

1.07k views • 72 slides
of Objects and Human Poses Maryam Daneshi, Konstantin Bayandin May 28 th , 2013 1 Agenda

of Objects and Human Poses Maryam Daneshi, Konstantin Bayandin May 28 th , 2013 1 Agenda

Recognizing Human-Object Interactions in Still Images by Modeling the Mutual Context of Objects and Human Poses Maryam Daneshi, Konstantin Bayandin May 28 th , 2013 1 Agenda Introduction & Motivation Dataset description Model

935 views • 36 slides
Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image Denis Tom

Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image Denis Tom

Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image Denis Tom Chris Russell Lourdes Agapito We introduce a novel approach to solve the problem of 3D human pose estimation from a single RGB image Input

841 views • 18 slides
11/21/2018 ImageNet Classification with Deep Convolutional Neural Networks Prepared by Faizaan

11/21/2018 ImageNet Classification with Deep Convolutional Neural Networks Prepared by Faizaan

11/21/2018 ImageNet Classification with Deep Convolutional Neural Networks Prepared by Faizaan Naveed Won Mo (Andy) Jung Lassonde School of Engineering, York University Canada Objective Overview of Convolutional Neural Network

567 views • 17 slides
Convolutional Networks Lecture slides for Chapter 9 of Deep Learning Ian Goodfellow 2016-09-12

Convolutional Networks Lecture slides for Chapter 9 of Deep Learning Ian Goodfellow 2016-09-12

Convolutional Networks Lecture slides for Chapter 9 of Deep Learning Ian Goodfellow 2016-09-12 Convolutional Networks Scale up neural networks to process very large images / video sequences Sparse connections Parameter sharing

552 views • 27 slides
Food Image Recognition Using Very Deep Convolutional Networks Hamid Hassannejad 2 nd

Food Image Recognition Using Very Deep Convolutional Networks Hamid Hassannejad 2 nd

Food Image Recognition Using Very Deep Convolutional Networks Hamid Hassannejad 2 nd International Workshop on Multimedia Assisted Dietary Management Oct. 2016 Outlines Deep learning and food recognition Inception model Experiments

704 views • 16 slides
Multiscale Sparse Models in Deep Convolutional Networks Tomas Angls, Roberto Leonarduzi,

Multiscale Sparse Models in Deep Convolutional Networks Tomas Angls, Roberto Leonarduzi,

Multiscale Sparse Models in Deep Convolutional Networks Tomas Angls, Roberto Leonarduzi, Stphane Mallat, Louis Thiry, John Zarka, Sixin Zhang Collge de France cole Normale Suprieure Flatiron Institute www.di.ens.fr/data Deep

356 views • 33 slides
Deep Convolutional Networks are Useful in System Identification Antnio H. Ribeiro 1 , 2 , ,

Deep Convolutional Networks are Useful in System Identification Antnio H. Ribeiro 1 , 2 , ,

Deep Convolutional Networks are Useful in System Identification Antnio H. Ribeiro 1 , 2 , , Carl Andersson 1 , , Koen Tiels 1 , Niklas Wahlstrm 1 and Thomas B. Schn 1 1 Uppsala University, 2 UFMG, Equal contribution

332 views • 29 slides
Understanding (or not) Deep Convolutional Networks Stphane Mallat cole Normale Suprieure

Understanding (or not) Deep Convolutional Networks Stphane Mallat cole Normale Suprieure

Understanding (or not) Deep Convolutional Networks Stphane Mallat cole Normale Suprieure www.di.ens.fr/data Deep Neural Networks Approximations of high-dimensional functions from examples, for classification and regression.

387 views • 38 slides
High-Resolution Breast Cancer Screening with Multi-View Deep Convolutional Neural Networks

High-Resolution Breast Cancer Screening with Multi-View Deep Convolutional Neural Networks

High-Resolution Breast Cancer Screening with Multi-View Deep Convolutional Neural Networks Krzysztof J. Geras Joint work with Kyunghyun Cho , Linda Moy , Gene Kim , Stacey Wolfson and Artie Shen . GTC 2017 W HERE DEEP LEARNING IS USEFUL amount

772 views • 42 slides
Lecture 13: Introduction to Deep Learning Deep Convolutional Neural Networks Aykut Erdem

Lecture 13: Introduction to Deep Learning Deep Convolutional Neural Networks Aykut Erdem

Lecture 13: Introduction to Deep Learning Deep Convolutional Neural Networks Aykut Erdem November 2016 Hacettepe University Administrative Assignment 3 is out! It is due November 30, 2016 You will implement a 2-layer Neural

1.28k views • 114 slides
Fully Convolutional Network (FCN) Prof. Seungchul Lee Industrial AI Lab. Deep Learning for

Fully Convolutional Network (FCN) Prof. Seungchul Lee Industrial AI Lab. Deep Learning for

Fully Convolutional Network (FCN) Prof. Seungchul Lee Industrial AI Lab. Deep Learning for Computer Vision: Review Source: 6.S191 Intro. to Deep Learning at MIT 2 Segmentation Segmentation task is different from classification task because

522 views • 15 slides
I mproving*Object*Detection*with* Deep*Convolutional*Networks*via* Bayesian*Optimization*and*

I mproving*Object*Detection*with* Deep*Convolutional*Networks*via* Bayesian*Optimization*and*

I mproving*Object*Detection*with* Deep*Convolutional*Networks*via* Bayesian*Optimization*and* Structured*Prediction* Yuting Zhang * ,*KihyukSohn ,*Ruben*Villegas ,*Gang*Pan * ,* HonglakLee *

860 views • 84 slides
Deep Convolutional Neural Network for Computer Vision Products LI XU, R&D Director

Deep Convolutional Neural Network for Computer Vision Products LI XU, R&D Director

Deep Convolutional Neural Network for Computer Vision Products LI XU, R&D Director SenseTime Group Limited SenseTime Introduction SenseTime focuses on invention and development of computer vision and deep learning technologies. Our

1.03k views • 39 slides
Underwater sparse image classification using deep convolutional neural networks Mohamed Elawady

Underwater sparse image classification using deep convolutional neural networks Mohamed Elawady

Underwater sparse image classification using deep convolutional neural networks Mohamed Elawady Heriot-Watt University VIBOT Msc 2014 26 Nov 2015 Deep Learning Workshop, Lyon, 2015 1 About Me! 2014-Current: PhD in Imaging Processing

740 views • 47 slides
Convolutional Neural Networks (Part III) 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image

Convolutional Neural Networks (Part III) 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image

Convolutional Neural Networks (Part III) 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image Processing 2016 Prof. Luiz Velho Convolutional Neural Networks 1 Summary & References 08/11 ImageNet Classification with Deep Convolutional

444 views • 27 slides
A Fuller Understanding of Fully Convolutional Networks Evan Shelhamer* Jonathan Long* Trevor

A Fuller Understanding of Fully Convolutional Networks Evan Shelhamer* Jonathan Long* Trevor

A Fuller Understanding of Fully Convolutional Networks Evan Shelhamer* Jonathan Long* Trevor Darrell UC Berkeley in CVPR'15, PAMI'16 1 pixels in, pixels out colorization Zhang et al.2016 monocular depth + normals Eigen & Fergus 2015

414 views • 38 slides
Vehicle Velocity Prediction Using Artificial Neural Networks and Effect of Real-World Signals on

Vehicle Velocity Prediction Using Artificial Neural Networks and Effect of Real-World Signals on

21-23 20 Vehicle Velocity Prediction Using Artificial Neural Networks and Effect of Real-World Signals on Prediction Window Tushar D. Gaikwad , Zachary D. Asher, Aaron Rabinowitz, Farhang Motallebiaraghi, Thomas Bradley, Alvis Fong, Rick Meyer

1.19k views • 27 slides
Vehicle Localization based on Lane Marking Detection Yuncong Chen UCSD HRI intern 2014

Vehicle Localization based on Lane Marking Detection Yuncong Chen UCSD HRI intern 2014

Vehicle Localization based on Lane Marking Detection Yuncong Chen UCSD HRI intern 2014 Overview Input Goal Odometry lateral localization on highway (noisy GPS / IMU for now) give correct estimate on merge / split points

559 views • 36 slides
Monocular Depth Estimation Using Atrous Convolutions Group 5 - Faraz Saeedan Fabian Kessler,

Monocular Depth Estimation Using Atrous Convolutions Group 5 - Faraz Saeedan Fabian Kessler,

Monocular Depth Estimation Using Atrous Convolutions Group 5 - Faraz Saeedan Fabian Kessler, Dominik Straub, Steven Lang February 15, 2019 Technische Universitt Darmstadt Introduction Experiments Results Discussion Conclusion 1 Fabian

714 views • 45 slides
IN INVEST VESTOR R PRESENT PRESENTATIO TION AUG UGUST 20 UST 2019 19 SAF SAFE E HAR

IN INVEST VESTOR R PRESENT PRESENTATIO TION AUG UGUST 20 UST 2019 19 SAF SAFE E HAR

IN INVEST VESTOR R PRESENT PRESENTATIO TION AUG UGUST 20 UST 2019 19 SAF SAFE E HAR HARBOR BOR ST STATE TEMENT MENT This presentation does not constitute an offer or invitation for the sale or purchase of securities and has been

365 views • 32 slides
GANerated Hands for Real-Time 3D Hand Tracking from Monocular RGB Franziska Mueller, et, al. CVPR

GANerated Hands for Real-Time 3D Hand Tracking from Monocular RGB Franziska Mueller, et, al. CVPR

GANerated Hands for Real-Time 3D Hand Tracking from Monocular RGB Franziska Mueller, et, al. CVPR 2018 2018.11.20 20185209 Sangyoon Lee 1 Table of contents Motivation Challenges Background Contribution Solution Evaluation

426 views • 19 slides
T h e S t a t u s o f T e l e s c o p e A r r a y T h e S t a t u

T h e S t a t u s o f T e l e s c o p e A r r a y T h e S t a t u

S c i e n t i f i c A c t i v i t i e s o f I C R R S c i e n t i f i c A c t i v i t i e s o f I C R R - - - O O b s e r v a t i o n o f U l t r a - h i

699 views • 40 slides
Visu Visual I Inertial Su Subse sea 3 3D D Recon onstr tructi tion on For Subsea Model

Visu Visual I Inertial Su Subse sea 3 3D D Recon onstr tructi tion on For Subsea Model

Visu Visual I Inertial Su Subse sea 3 3D D Recon onstr tructi tion on For Subsea Model Generation and Real-Time Positioning w w w . Z u p t . c o m ZUPT Ts V VIEW O W ON THE A APPLIC ICAT ATIO IONS A AND T TECHNOLOGY

357 views • 33 slides

More recommend