Stereovision and augmented reality for closed-loop control of - - PowerPoint PPT Presentation

Stereovision and 
 augmented reality for 
 closed-loop control of 
 grasping in hand prosthesis

Markovic et al. (Germany) in Journal Neu. Eng. 2014
 
 Presented by Kory Mathewson at BLINC Journal Club
 July 24 2015

Inputs (Low dimension) Control Planning
 (high-level) Execution
 (low-level) Complex
 Tasks Closed-loop
 Control Salient points: user focus on the task,
 information bandwidth, user burden Learning Motor Info
 (feed-forward) Sensory Info 
 (feedback)

EEG, ECG, foot movements, tongue, EOG, 
 implantable neural electrodes and myoelectric sensors, 
 EMG Multichannel surface electromyography

Motor Info
 (feed-forward)

Sensory Info 
 (feedback) Can we enrich artificial controller 
 with extra information to allow 
 autonomous decision making?

Can we enrich artificial controller 
 with extra information to allow 
 autonomous decision making?
 Adding perception with sensor fusion. Stereovision Automatically reshape grip pattern. Operational responsibility (cognitive load) 
 shared between the system and the user.

direct mechanical (vibrotactile, haptic),
 electrocutaneous stimulation, vibration motors, 
 hybrid stimulation, invasive approaches, AR Motor Info
 (feed-forward)

Sensory Info 
 (feedback)

Augmented reality Artificial proprioception by projecting 
 the prosthetic into the field of view.

1) Compare fully vs semi-automatic control. 2) Evaluate user ability to share control. 3) Measure feasibility of utilizing AR feedback. Study Design 13 subjects 6 series 20 objects 1560 trials Auto-AR
 Semi-AR
 SEMI-AR-RE Semi-Vis-RE 4
 conditions

1) Compare fully vs semi-automatic control. 2) Evaluate user ability to share control. 3) Measure feasibility of utilizing AR feedback. Study Design 1) Semi-automatic control performed better. 2) User was able to share control 
 quickly and effectively. 3) User was able to use AR feedback to correct mistakes of automatic controller. Results

Discussion “It is likely that trained subjects will learn 
 to rely more on feed-forward control 
 and use feedback only when necessary” Agree? Only used four grip patterns, 
 how would we generalize? Does this reduce burden on the user? 
 How can we measure burden on the user? How else could AR / stereovision 
 be utilized as a feedback mechanism? Is this optimal integration of manual 
 and automatic control loops?