Human&Motor&Performance& - PowerPoint PPT Presentation

Human&Motor&Performance& in&Robot2Assisted&Surgery Ilana&Nisky 1 ,&Michael&Hsieh 2,3 ,&and&Allison&Okamura 1 1 Department&of&Mechanical&Engineering,&Stanford&University 2 Department&of&Urology,&Stanford&University 3 Lucile&Packard&Children’s&Hospital Presented&by&Allison&Okamura&for&the 2014&North&American&Summer&School&on&Surgical&RoboMcs 1

RoboMcs&for&Medical&IntervenMons Rehabilita)on Prosthe)cs Robot,assisted/surgery 2

Robot2Assisted&Minimally&Invasive&Surgery Design&does&not&fully&consider&the&sensorimotor&capabiliMes&of& • the&surgeon Training&methods&have&not&been&opMmized • Studying&the&sensorimotor&system&could&impact&both! 3

ComputaMonal&Motor&Control The&science&of&how&the&brain&controls&moMon&and& represents&the&external&world We&move&in&surprisingly& regular&ways… Morasso,&1981

A&Simple&Model&of&Motor&Control Bhanpuri&et&al.&Brain&2014 5

Effects&of&Arm&Dynamics Bhanpuri&et&al.&Brain&2014 6

AdaptaMon&to&PerturbaMons MarMn&et&al.,&1996 Shadmehr&and&Mussa2Ivaldi,&1994

OpMmality&and&Minimum&IntervenMon Trajectory&OpMmizaMon:& OpMmal&Feedback&Control Minimum&Jerk Minimum&intervenMon&principle Flash&and&&Hogan,&1985 Todorov&and&Jordan,&2002 8

Take&Home To&build&roboMc&systems&that&are& operated&by& humans ,&we&should: – Study&the& human/operator – Apply&findings&to&design,&control,& and&training Operators/interact &with&roboMc& devices& – This&allows&us&to&study&the& human/operator &in& unprecedented&ways 9

Surgery Open Minimally+Invasive Robot3Assisted IntuiMve&Surgical&

Sensorimotor&Performance&in&RAS Cognitive control strategies Surgeon action (e.g. movement) Tool action (e.g. tool moves) Sensory feedback Patient interaction Jarc&and&Nisky,&in&press 11

Sensorimotor&Performance&in&RAS Can&we&use&(and&extend)&what&we&know&about& human&motor&control& to&improve& design,&control,& and& training& in Robot2Assisted&Surgery? 12

Sensorimotor&Performance&in&RAS Compare&teleoperated&vs.&freehand&movements,& and&expert&vs.&novice&parMcipants – TeleoperaMon&vs.&freehand&&=>&&robot&design – Experts&vs.&novices&&=>&&skill&evaluaMon&and&training (1) Tool3:p+kinema:cs (2) Arm+posture+variability 13

Experimental&Setup 14

Experimental&Setup Pose+trackers+on+user+arm Grasp+fixture+–+ posi:on+and+force+sensing+ at+tool+:p s t t w e & designed&by&Taru&Roy 15

Experimental&Procedures Good Too slow long& reach& target short& reversal& target 16

TeleoperaMon 17

Freehand 18

KinemaMcs Variability 19

KinemaMcs Variability

Data&Analysis&2&Reach 50 50 100 posiMon pos [mm] [mm] 0 0 0 end&of&movement -50 -50 00 -0.2 0 0.2 0.4 0.6 0.8 0 0.5 1 -0.2 0 0.2 0.4 0.6 0.8 [mm/sec] 200 200 500 velocity vel [mm/sec] 0 0 0 -200 00 00 -0.2 0 0.2 0.4 0.6 0.8 0 0.5 1 -0.2 0 0.2 0.4 0.6 0.8 acceleraMon 1000 1000 2000 acc [mm/sec 2 ] [mm/sec 2 ] peak& deceleraMon 0 0 0 peak& acceleraMon -1000 00 00 -0.2 0 0.2 0.4 0.6 0.8 0 0.5 1 -0.2 0 0.2 0.4 0.6 0.8 speed [mm/sec] 200 200 400 peak&speed& fused&correcMve correcMve [&&mm/sec] &&&&&&&&&movement movement speed 100 100 200 0 0 0 -0.2 0 0.2 0.4 0.6 0.8 0 0.5 1 -0.2 0 0.2 0.4 0.6 0.8 speed der [mm/sec 2 ] [mm/sec 2 ]&&& 1000 1000 2000 speed&der. 500 500 1000 0 0 0 Nisky&et&al., -0.2 0 0.2 0.4 0.6 0.8 0 0.5 1 -0.2 0 0.2 0.4 0.6 0.8 Mme&[sec] Mme&[sec] Mme&[sec] time [sec] time [sec] time [sec] MMVR2013

Deviation from Straight Line Novice Expert Novice Expert 10mm Nisky&et&al.,&Surgical& Endoscopy&2014 First&trial Last&trial 22

Performance Endpoint&Error&*&Movement&Time novice tele novice free expert tele expert free Nisky&et&al.,&Surgical& Endoscopy&2014 23

Reach&Velocity&Skewness Increased&Peak&A&/&Peak&D& novice indicates&fused&correcMve& tele movements free expert 0 0.5 1 tele [&&mm/sec] 200 400 speed free 100 200 0 0 0 0.5 1 -0.2 0 0.2 0.4 0.6 0.8 [mm/sec 2 ]&&& speed&der. 2000 1000 1000 500 Expert 90 o % 0 0 -0.2 0 0.2 0.4 0.6 0.8 0 0.5 1 time [sec] time [sec] 180 o % Largest&in&teleoperated& 0 o % reaches&of&experts! !90 o %

Learning&effects novice tele novice free expert tele expert free Nisky&et&al., 2014 25

Learning&effects All&groups&learn&the&task&within&324& novice tele movement&blocks&in&the&first&session novice free TeleoperaMng&novices&also&learn&system& expert tele dynamics expert free Session&1 Session&2 Nisky&et&al., 2014 26

KinemaMcs Variability 27

Redundancy&and&Variability Human&arm&is&a& redundant & manipulator How&is&redundancy&resolved?& – Bernstein,&1967 Motor&system&constrains&only& task&relevant&variability – Uncontrolled&Manifold&Hypothesis& s Scholtz&ans&Schoner,&1999& w t – Minimum&intervenMon&principle& e Todorov&2002 28

Uncontrolled&Manifold&Hypothesis Task&space Joint&&space α w [degrees] xt [mm] zt [mm] 150 60 0 140 40 -20 Reach 20 -40 130 0 -60 120 0 0.5 1 0 0.5 1 0 0.5 1 normalized time normalized time normalized time α [degrees] Variability& 2&kinds&of&trial2to2trial& variability&in&joint&angles coordinaMon R V =log(V other /V task ) – Changes&task&performance:&&V task R V >0&stabilize – Doesn’t&change&task& performance:&V other R V =0&independent Nisky&et&al.,&ICRA&2013 29

Variability&in&Joint&Space&2&Uncontrolled&Manifold ( ) x [ t ] = F q [ t ] Forward&kinemaMcs ( ) x [ t ] − x [ t ] = J ( q [ t ]) q [ t ] − q [ t ] Linearize&FWD&kinemaMcs J ( q [ t ]) ⋅ e = 0 Calculate&null&space ( ) q UCM [ t ] = ee T q [ t ] − q [ t ] Project&variance&onto&null&and& ( ) − q UCM [ t ] q ORT [ t ] = q [ t ] − q [ t ] orthogonal&spaces& ⎛ ⎞ N ( ) ∑ 2 − 1 N − 1 Calculate&log&of& q UCM [ t ] d ucm ⎜ ⎟ variance&raMo R v [ t ] = log i = 1 ⎜ ⎟ N ∑ ( ) ⎜ ⎟ 2 − 1 N − 1 q ORT [ t ] d task ⎜ ⎟ ⎝ ⎠ Details&in&Nisky&et&al.,&ICRA&2013,& i = 1 Nisky&et&al.,&IEEE&TBME&2014&

Variability&PredicMons XY&movements&are&stabilized&&&&&&R V >0 Z&movements&are&not&&&&&&&&&&&&&&&&&&&R V =0 Larger&R V &of&experts + Skill+increases+R V +(Muller+and+Sternad,+2004) Smaller&R V &in&teleoperaMon

Trial2to2trial&Variability Experts Novices XY tele 6 6 XY free 4 4 ln V task Z tele Z free 2 2 0 0 0 0.5 1 0 0.5 1 0.5 1 0 0.5 normalized time normalized time tele -6 -6 free ln V joint -6.5 -6.5 0 0.5 1 0 0.5 1 normalized time normalized time XY tele 2 2 XY free R V [nu] Z tele 1 1 Z free 0 0 Nisky&et&al.,& IEEE&TBME&2014 0 0.5 1 0 0.5 1 normalized time normalized time 32

CoordinaMon&of&Arm&Posture&Variability peak speed movement end The&task&requires&only& 1.8 1.8 XY tele accurate&&XY&movements XY free &&&&&&&&&XY&movements&&&&R V >0 1.6 1.6 R V [nu] &&&&&&&&&Z&movements&&&&&&&R V =0 1.4 1.4 1.2 1.2 Experience Z tele 0.4 0.4 & Larger&R V &of&experts Z free 0.2 0.2 R V [nu] & 0 0 TeleoperaMon& -0.2 -0.2 & Experts&R V &increase Expert Novice Expert Novice & Novices&R V &decrease Nisky&et&al.,&IEEE& TBME&2014 33