Quantitative hierarchical representation and comparison of hand - PowerPoint PPT Presentation

Quantitative hierarchical representation and comparison of hand grasps from electromyography and kinematic data Francesca Stival 1,2 , Stefano Michieletto 1 , Enrico Pagello 1 , Henning Mü̈ller 2 , and Manfredo Atzori 2 1 Intelligent Autonomous Systems Lab (IAS-Lab) Department of Information Engineering (DEI), University of Padova, Italy 2 Information Systems Institute, University of Applied Sciences Western Switzerland (HES-SO Valais), Sierre, Switzerland Page 1/19

What is a taxonomy of hand grasps? Page 2/19

What is a taxonomy • Taxonomy is the practice and science of classification. • A taxonomy is a particular classification scheme (hierarchical or not). Page 3/19

Hand grasp taxonomies Several hand grasp taxonomies have been developed, based on qualitative parameters: • Grasp force • Opposition (palm, pad, side) • Thumb adduction / abduction • Virtual finger opposition Page 4/19

Usefulness of grasp taxonomies and current limits Usefulness: • Robotics: to compare the functionality of robotic hands with real human ones in different tasks • Prosthetics: to better define the requirements of prostheses and control methods • Physiology: to create a link between hand synergies and real life needs • Rehabilitation: to prioritize the hand functionalities to restore with higher priority Limits: • Lack of quantitative foundation • Subjectivity and interpretability • Quantitative evaluations and comparisons are not directly possible Page 5/19

Methods: what is the Ninapro database for hand movements? Page 6/19

Ninapro database • Publicly available (url: http://ninapro.hevs.ch) • 7 datasets • More than 120 subjects • 13 trans radial amputated subjects • ~50 different movements • 5 different acquisition setups Page 7/19

Ninapro data acquisition Kinematic data Cyberglove II sEMG Delsys Trigno Wireless System Page 8/19

Database: Movements Exercise 1 12 movements ! ! ! ! ! ! Hato, 2004 ! ! ! ! ! ! Sebelius, 2005 Exercise 2 Farrel, 2008 17 movements ! ! ! ! ! ! ! ! Crawford, 2005 Feix, 2008 DASH Score ! ! ! ! ! ! ! ! ! Exercise 3 23 movements ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! Exercise 4 9 force patterns Page 9/19

Recent results: Classification Accuracy vs Phantom Limb Sensation F ( x ) = 2 . 8 + 24 . 37 x 0 . 36 60 R 2 = 0 . 63 ± 0 . 06 Effect of clinical parameters on accuracy p = 0 . 005 ± 0 . 004 Classification Accuracy (%) 50 • Body mass index 40 • Remaining forearm percentage 30 • Phantom limb sensation intensity 20 • Years Since Amputation 10 0 1 2 3 4 5 Phantom Limb Sensation Intensity (a.u) Atzori et al., Journal of Rehabilitation Research and Development, 2016 Standardized comparison of different acquisition setups • Similar performance for pattern recognition tasks. Double Myo 8 Otto Bock ~ 300 € ~ 6000 € 12 Cometa 12 Delsys ~ 15’000 € Trigno ~ 18’000 € Pizzolato et al., PLOS One 2017 Page 10/19

Ongoing work Reproducing eye-hand coordination in hand prosthetics using gaze, computer vision and sEMG. Giordaniello et al., ICORR 2017, Cognolato et al., ICVS 2017 Page 11/19

Methods: subject specific quantitative taxonomies Page 12/19

Movement hierarchical representation and comparison: • Subjects: 40 intact subject (Ninapro DB2) • Windowing: 200 ms (10 ms overlap) • Signal feature extraction: Root Mean Square, Mean & Integrated Absolute Value, Waveform Length, Time Domain Statistics. • Movement hierarchical representation: one hierarchical tree for each modality-feature combination (5 sEMG and 5 kinematic trees / subj.) • Comparison of trees: edit distance (minimal-cost sequence of node edit operations required to obtain one tree from another) Page 13/19

Results: differences between taxonomies Page 14/19

Same subject, different signal features • IAV & MAV trees are almost identical (sEMG & kinematic) • RMS, IAV and MAV trees are very similar • Several characteristics are shared among different trees • TD trees are in general the most different Page 15/19

Inter-subject differences • Most subjects have a similar behavior (similar to Subj. 2). • Some subjects have different results (e.g. Subject 9) Page 16/19

Conclusions Page 17/19

Conclusions • Quantitative hierarchical representations of hand movements can be performed systematically with the proposed approach • The results can be compared and are usually similar considering different subjects and features • We are currently creating a quantitative taxonomy of hand movements that solves the limits of qualitative approaches. Page 18/19

Thanks a lot to: Francesca Stival Dr. Stefano Michieletto Prof. Henning Müller Prof. Enrico Pagello Page 19/19