Human&Motor&Performance& - - PowerPoint PPT Presentation

▶
human motor performance in robot2assisted surgery
SMART_READER_LITE
LIVE PREVIEW

Human&Motor&Performance& - - PowerPoint PPT Presentation

Dec 15, 2022 528 likes •1.01k views

Human&Motor&Performance& in&Robot2Assisted&Surgery Ilana&Nisky 1 ,&Michael&Hsieh 2,3 ,&and&Allison&Okamura 1 1 Department&of&Mechanical&Engineering,&Stanford&University 2

slide-1
SLIDE 1

Human&Motor&Performance& in&Robot2Assisted&Surgery

Ilana&Nisky1,&Michael&Hsieh2,3,&and&Allison&Okamura1

1Department&of&Mechanical&Engineering,&Stanford&University 2Department&of&Urology,&Stanford&University 3Lucile&Packard&Children’s&Hospital

1

Presented&by&Allison&Okamura&for&the 2014&North&American&Summer&School&on&Surgical&RoboMcs

slide-2
SLIDE 2

Rehabilita)on Robot,assisted/surgery Prosthe)cs

2

RoboMcs&for&Medical&IntervenMons

slide-3
SLIDE 3

Robot2Assisted&Minimally&Invasive&Surgery

  • Design&does&not&fully&consider&the&sensorimotor&capabiliMes&of&

the&surgeon

  • Training&methods&have&not&been&opMmized

Studying&the&sensorimotor&system&could&impact&both!

3

slide-4
SLIDE 4

ComputaMonal&Motor&Control

The&science&of&how&the&brain&controls&moMon&and& represents&the&external&world We&move&in&surprisingly& regular&ways…

Morasso,&1981

slide-5
SLIDE 5

A&Simple&Model&of&Motor&Control

5

Bhanpuri&et&al.&Brain&2014

slide-6
SLIDE 6

Effects&of&Arm&Dynamics

6

Bhanpuri&et&al.&Brain&2014

slide-7
SLIDE 7

AdaptaMon&to&PerturbaMons

MarMn&et&al.,&1996 Shadmehr&and&Mussa2Ivaldi,&1994

slide-8
SLIDE 8

OpMmality&and&Minimum&IntervenMon

8

Flash&and&&Hogan,&1985

Trajectory&OpMmizaMon:& Minimum&Jerk OpMmal&Feedback&Control Minimum&intervenMon&principle

Todorov&and&Jordan,&2002

slide-9
SLIDE 9

Take&Home

To&build&roboMc&systems&that&are&

  • perated&by&humans,&we&should:

– Study&the&human/operator – Apply&findings&to&design,&control,& and&training

Operators/interact&with&roboMc& devices&

– This&allows&us&to&study&the& human/operator&in& unprecedented&ways

9

slide-10
SLIDE 10

Open Minimally+Invasive Robot3Assisted

Surgery

IntuiMve&Surgical&

slide-11
SLIDE 11

Sensorimotor&Performance&in&RAS

11

Cognitive control strategies Surgeon action (e.g. movement) Tool action (e.g. tool moves) Patient interaction Sensory feedback

Jarc&and&Nisky,&in&press

slide-12
SLIDE 12

Sensorimotor&Performance&in&RAS

Can&we&use&(and&extend)&what&we&know&about&

human&motor&control&

to&improve&

design,&control,&and&training&

in Robot2Assisted&Surgery?

12

slide-13
SLIDE 13

Compare&teleoperated&vs.&freehand&movements,& and&expert&vs.&novice&parMcipants

– TeleoperaMon&vs.&freehand&&=>&&robot&design – Experts&vs.&novices&&=>&&skill&evaluaMon&and&training

13

Sensorimotor&Performance&in&RAS

(1) Tool3:p+kinema:cs (2) Arm+posture+variability

slide-14
SLIDE 14

Experimental&Setup

14

slide-15
SLIDE 15

Experimental&Setup

Grasp+fixture+–+ posi:on+and+force+sensing+ at+tool+:p

&designed&by&Taru&Roy s e w t

t

Pose+trackers+on+user+arm

15

slide-16
SLIDE 16

Experimental&Procedures

Too slow short& reversal& target long& reach& target Good

16

slide-17
SLIDE 17

TeleoperaMon

17

slide-18
SLIDE 18

Freehand

18

slide-19
SLIDE 19

19

KinemaMcs Variability

slide-20
SLIDE 20

KinemaMcs Variability

slide-21
SLIDE 21
  • 0.2

0.2 0.4 0.6 0.8 00 100

  • 0.2

0.2 0.4 0.6 0.8 00 500

  • 0.2

0.2 0.4 0.6 0.8 00 2000

  • 0.2

0.2 0.4 0.6 0.8 200 400

  • 0.2

0.2 0.4 0.6 0.8 1000 2000 time [sec] 0.5 1

  • 50

50 0.5 1 00 200 0.5 1 00 1000 0.5 1 100 200 0.5 1 500 1000 time [sec]

  • 0.2

0.2 0.4 0.6 0.8

  • 50

50 pos [mm]

  • 0.2

0.2 0.4 0.6 0.8

  • 200

200 vel [mm/sec]

  • 0.2

0.2 0.4 0.6 0.8

  • 1000

1000 acc [mm/sec2]

  • 0.2

0.2 0.4 0.6 0.8 100 200 speed [mm/sec]

  • 0.2

0.2 0.4 0.6 0.8 500 1000 time [sec] speed der [mm/sec2]

Data&Analysis&2&Reach

peak& acceleraMon peak&speed& peak& deceleraMon end&of&movement correcMve movement fused&correcMve &&&&&&&&&movement velocity [mm/sec] acceleraMon [mm/sec2] speed [&&mm/sec] speed&der. [mm/sec2]&&& Mme&[sec] Mme&[sec] Mme&[sec] posiMon [mm]

Nisky&et&al., MMVR2013

slide-22
SLIDE 22

Novice Expert

22

Deviation from Straight Line

Nisky&et&al.,&Surgical& Endoscopy&2014 First&trial Last&trial

10mm Novice Expert

slide-23
SLIDE 23

Performance

23

Endpoint&Error&*&Movement&Time

novice tele novice free expert tele expert free

Nisky&et&al.,&Surgical& Endoscopy&2014

slide-24
SLIDE 24

Reach&Velocity&Skewness

Increased&Peak&A&/&Peak&D& indicates&fused&correcMve& movements Largest&in&teleoperated& reaches&of&experts!

speed [&&mm/sec] speed&der. [mm/sec2]&&&

tele free tele free novice expert

0.5 1 0.5 1 100 200 0.5 1 500 1000 time [sec]

  • 0.2

0.2 0.4 0.6 0.8 200 400

  • 0.2

0.2 0.4 0.6 0.8 1000 2000 time [sec]

Expert !90o% 0o% 90o% 180o%

slide-25
SLIDE 25

Learning&effects

25

novice tele novice free expert tele expert free

Nisky&et&al., 2014

slide-26
SLIDE 26

Learning&effects

Session&1 Session&2

All&groups&learn&the&task&within&324& movement&blocks&in&the&first&session TeleoperaMng&novices&also&learn&system& dynamics

26

novice tele novice free expert tele expert free

Nisky&et&al., 2014

slide-27
SLIDE 27

27

KinemaMcs Variability

slide-28
SLIDE 28

Redundancy&and&Variability

Human&arm&is&a&redundant& manipulator How&is&redundancy&resolved?&

– Bernstein,&1967

Motor&system&constrains&only& task&relevant&variability

– Uncontrolled&Manifold&Hypothesis&

Scholtz&ans&Schoner,&1999&

– Minimum&intervenMon&principle&

Todorov&2002

s e w t

28

slide-29
SLIDE 29

0.5 1 20 40 60 normalized time xt [mm] Reach 0.5 1

  • 60
  • 40
  • 20

normalized time zt [mm] 0.5 1 120 130 140 150 normalized time α

w [degrees]

α [degrees]

Uncontrolled&Manifold&Hypothesis

2&kinds&of&trial2to2trial& variability&in&joint&angles

– Changes&task&performance:&&Vtask – Doesn’t&change&task& performance:&Vother

Variability& coordinaMon

RV=log(Vother/Vtask)

RV>0&stabilize RV=0&independent

29

Nisky&et&al.,&ICRA&2013

Task&space Joint&&space

slide-30
SLIDE 30

Forward&kinemaMcs Linearize&FWD&kinemaMcs Calculate&null&space Project&variance&onto&null&and&

  • rthogonal&spaces&

Calculate&log&of& variance&raMo

Details&in&Nisky&et&al.,&ICRA&2013,& Nisky&et&al.,&IEEE&TBME&2014&

J(q[t])⋅e = 0 qUCM[t] = eeT q[t]− q[t]

( )

qORT[t] = q[t]− q[t]

( )− qUCM[t]

x[t]− x[t] = J(q[t]) q[t]− q[t]

( )

x[t] = F q[t]

( )

Rv[t] = log qUCM[t]

( )

2 i=1 N

∑

ducm

−1 N −1

qORT[t]

( )

2 i=1 N

∑

dtask

−1 N −1

⎛ ⎝ ⎜ ⎜ ⎜ ⎜ ⎞ ⎠ ⎟ ⎟ ⎟ ⎟

Variability&in&Joint&Space&2&Uncontrolled&Manifold

slide-31
SLIDE 31

Variability&PredicMons

XY&movements&are&stabilized&&&&&&RV>0 Z&movements&are&not&&&&&&&&&&&&&&&&&&&RV=0 Larger&RV&of&experts

+ Skill+increases+RV+(Muller+and+Sternad,+2004)

Smaller&RV&in&teleoperaMon

slide-32
SLIDE 32

Trial2to2trial&Variability

32

Nisky&et&al.,& IEEE&TBME&2014

0.5 1 2 4 6 ln Vtask Experts 0.5 1 1 2 normalized time RV [nu] 0.5 1 2 4 6 Novices 0.5 1 1 2 normalized time 0.5 1

  • 6.5
  • 6

normalized time ln Vjoint 0.5 1

  • 6.5
  • 6

normalized time

XY tele XY free Z tele Z free

tele free

XY tele XY free Z tele Z free

0.5 1 normalized time 0.5 normalized time

slide-33
SLIDE 33

CoordinaMon&of&Arm&Posture&Variability

The&task&requires&only& accurate&&XY&movements &&&&&&&&&XY&movements&&&&RV>0 &&&&&&&&&Z&movements&&&&&&&RV=0 Experience & Larger&RV&of&experts

&

TeleoperaMon& & Experts&RV&increase & Novices&RV&decrease

33

Nisky&et&al.,&IEEE& TBME&2014

1.2 1.4 1.6 1.8 movement end XY tele XY free 1.2 1.4 1.6 1.8 RV [nu] peak speed Expert Novice

  • 0.2

0.2 0.4 Z tele Z free Expert Novice

  • 0.2

0.2 0.4 RV [nu]

slide-34
SLIDE 34

The&Effect&of&Movement&DirecMon

34

Nisky&et&al.,&IEEE&TBME&2014

  • 90

90 180 1 1.5 2 Direction [deg] Rv [nu] expert tele expert free novice tele novice free

  • 90

90 180

  • 7
  • 6
  • 5

Direction [deg] ln VTIM

  • 90

90 180

  • 9
  • 8
  • 7
  • 6

Direction [deg] ln VTRM

slide-35
SLIDE 35

1 2 1 2 1.6 1.8 2 2.2 2.4 2.6 ln Vtask [mm2] Expert Novice 1 2 1 2

  • 7
  • 6.5
  • 6
  • 5.5
  • 5

ln Vjoint [rad2] Expert Novice 1 2 1 2 0.8 1 1.2 1.4 1.6 1.8 2 RV[nu] Expert Novice

Changes&in&Variability&Between&Sessions

35

Nisky&et&al.,&IEEE&TBME&2014

Free -> Tele Tele -> Free

slide-36
SLIDE 36

Rv&and&Performance

36

Nisky&et&al.,&IEEE& TBME&2014

2 4 6

  • 1

1 2 3 Rv [nu] ln (Er*MT [mm*s]) expert tele expert free novice tele novice free

1.2 1.4 1.6 1.8 0.6 0.8 1

slide-37
SLIDE 37

RealisMc&Task:&Needle&driving

Clinically&relevant&movement Complexity

3D&movement& Tissue&interacMon OrientaMon&is&criMcal

CondiMons&and&parMcipants

Teleoperated&v.&open Experienced&surgeons&v.&novices

37

Nisky&et&al.,&in&preparaMon

slide-38
SLIDE 38

Experimental&Setup

38

Teleoperated&2&dVRK Open&–&magneMc&tracking& instrumented&needle&driver

slide-39
SLIDE 39

Needle&Driving&Task

39

1 2 3 4 5 6 7 8

slide-40
SLIDE 40

Learning&Curves

40

slide-41
SLIDE 41

Learning&Curves&Summary

Open&needle&driving&is&faster,&but&with&same& needle&path&length All&parMcipants&improve&movement&Mme Only&novices&improve&movement&length

41

slide-42
SLIDE 42

Conclusions

The dynamics of the master manipulator matter Experts have adapted and are better Experts exploit the redundancy of their arm more than novices

Especially in teleoperation

42

slide-43
SLIDE 43

Future&Work

Analysis&of&interacMon&forces&and& dynamic/modeling&of&user&in& teleoperaMon&and&freehand Analysis&of&redundancy& exploitaMon&in&needle&driving& experiment& What&is&the&role&of&hapMc& feedback?&

43

slide-44
SLIDE 44

Take&Home

To&build&roboMc&systems&that&are&

  • perated&by&humans,&we&should:

– Study&the&human/operator – Apply&findings&to&design,&control,& and&training

Operators/interact&with&roboMc& devices&

– This&allows&us&to&study&the& human/operator&in& unprecedented&ways

44

slide-45
SLIDE 45

Thank&you

Marie+Curie+Interna:onal+Outgoing+Fellowship Weizmann+Ins:tute+of+Science+Na:onal+Postdoctoral+ Award+for+Advancing+Women+in+Science+ Intui:ve+Surgical+Technology+Research+Grant

Marhew&Weber Sangram&PaMl Taru&Roy Ilana/Nisky Yuhang&Che Zhan&Fan&Quek

slide-46
SLIDE 46

QuesMons?

46