SLIDE 1
Kitchen Scene Context based Gesture Recognition
ICPR2012 Contest
1
Overview of the contest
2
Kitchen Scene Context based Gesture Recognition ICPR2012 Contest 1 - - PowerPoint PPT Presentation
Kitchen Scene Context based Gesture Recognition ICPR2012 Contest 1 - - PowerPoint PPT Presentation
Kitchen Scene Context based Gesture Recognition ICPR2012 Contest 1 Overview of the contest 2 Organizers Atsushi Shimada Kyushu University, Japan Kazuaki Kondo Kyoto University, Japan Daisuke Deguchi Nagoya University,
SLIDE 2
SLIDE 3
Organizers
- Atsushi Shimada
– Kyushu University, Japan
- Kazuaki Kondo
– Kyoto University, Japan
- Daisuke Deguchi
– Nagoya University, Japan
- Géraldine Morin
– IRIT-ENSEEIHT , France
- Helman Stern
– Ben Gurion University of the Negev, Israel
3
SLIDE 4
Special Thanks
- Actors
– Tomo Asakura (Kyoto Univ.) – Keizaburo Takase (Kyoto Univ.) – Keisuke Yasuzawa (Kyoto Univ.) – Kenta Matsui (Kyoto Univ.) – Yusa Ko (Kyoto Univ.)
- Illustrations
– Toshiki Sonoda (Kyushu Univ., KUCC)
4
SLIDE 5
Contest Background
- Can gesture recognition be achieved by
motion analysis alone?
- Importance of
scene context for gesture recognition
– Strong relationship between action and situation surrounding an actor
5
SLIDE 6
Subject
- Gesture recognition in Kitchen
– Motion analysis – Recognition of ingredients, cooking tools, etc.
6
SLIDE 7
Cooking Menus
- ham and eggs
- omelet
- scrambled egg
- boiled egg
- Kinshi-tamago
7
SLIDE 8
Contest Task
- Label assignment to video frames
1 - breaking 2 - mixing 3 - baking 4 - turning 5 - cutting 6 - boiling 7 - seasoning 8 - pealing
8
SLIDE 9
Dataset
- RGBD video sequence
– Captured by Kinect @ 30fps
- Each cooking
– 5-10 minute long (2,000 to 12,000 frames)
9
SLIDE 10
Dataset
- Five menus
- Training set
– Five actors – Ground truth was given
- Test set
– Two actors – Ground truth was NOT given
10
SLIDE 11
Evaluation
- F-measure
– Harmonic mean of Precision & Recall
- For cooking menu
- 11
SLIDE 12
Applications
- 6 teams
– India – Singapore – Japan x 3 – Taiwan
- Invitation
– 4 teams
12
SLIDE 13
Evaluation Result
13
0.2 0.4 0.6 0.8 1
Motion feature only
SLIDE 14
Evaluation Result in Each Gesture
14
0.2 0.4 0.6 0.8 1 ID‐01 ID‐02 ID‐03 ID‐04 ID‐05 Reference
SLIDE 15
Awards ceremony
15
SLIDE 16
Awards
- Sponsored by SIG-CEA of HCG, IEICE,
Japan
– SIG-CEA
- A study group of “Cooking and Eating Activities”
- Researching “CEA” from viewpoint of
engineering, social and domestic science
- Supporting/improving dietary life with
information media technology
– Activity
- Domestic meeting: 4 times a year
- International Workshop: once a year
- CEA2009, 2010, 2011, 2012
16
SLIDE 17
First Prize
- Keisuke DOMAN, Chengying KUAI
– Chukyo University, Brother Industries, Ltd. – “Kitchen Scene Context based Gesture Recognition” Waterproof TV
17
SLIDE 18
Second Prize
- Tam V. Nguyen, Zheng Song, Manthan
Pancholi, Li Zhang, Shuicheng Yan
– National University of Singapore – “Context-aware Action Recognition in Kitchen Environment” Thermal mug Steam Roaster
18
SLIDE 19
Honorable Mention
- Shubham Bansal, Shubham Gupta,
Shubham Khandelwal, Dushyant Goyal
– The LNM Institute of Information Technology – “Kitchen Activity Recognition based on Scene Context” Microwave Egg Boiler
19
SLIDE 20
Honorable Mention
- Soichiro Hotta
– Mie University – GLAC feature with SVM for Kitchen Gesture Recognition Microwave Egg Boiler
20
SLIDE 21
Results of Winners (Total)
21
0.2 0.4 0.6 0.8 1
Doman et al. Nguyen et al. Bansal et al. Hotta et al.
SLIDE 22
Results of Winners (Each Gesture)
22
0.2 0.4 0.6 0.8 1 Doman et al. Nguyen et al. Bansal et al. Hotta et al.