1 7/20/2014 3D sensing: Kinect TM Kinect IR projection CMU Summer - - PDF document

▶
1
SMART_READER_LITE
LIVE PREVIEW

1 7/20/2014 3D sensing: Kinect TM Kinect IR projection CMU Summer - - PDF document

Sep 30, 2022 151 likes •259 views

7/20/2014 Virtual Fixtures in Robotic III. Intelligent Augmentation Surgery Hawkeye King, Sina Nia Kosari, Fredrik Ryden, Howard Chizeck Goals Sensor-based real-time guidance Shared human-machine control Improved safety CMU

slide-1
SLIDE 1

7/20/2014

1

  • III. Intelligent Augmentation

CMU Summer School 7.2014 Blake Hannaford, University of Washington CMU Summer School 7.2014

Blake Hannaford, University of Washington

BioRobotics Lab

CMU Summer School 7.2014 Blake Hannaford, University of Washington

Virtual Fixtures in Robotic Surgery

  • Hawkeye King, Sina Nia Kosari, Fredrik Ryden,

Howard Chizeck

  • Goals
  • Sensor-based real-time guidance
  • Shared human-machine control
  • Improved safety

BioRobotics Lab

CMU Summer School 7.2014 Blake Hannaford, University of Washington

Task: Draw a straight line Virtual Fixture Example

BioRobotics Lab

CMU Summer School 7.2014 Blake Hannaford, University of Washington

Fixtures: Ruler

BioRobotics Lab

CMU Summer School 7.2014 Blake Hannaford, University of Washington

Virtual Fixture:

Remote

  • perator

BioRobotics Lab

CMU Summer School 7.2014 Blake Hannaford, University of Washington

Surgical guidance:

Remote

  • perator

Target Vital Structure

slide-2
SLIDE 2

7/20/2014

2

BioRobotics Lab

3D sensing: KinectTM

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

Kinect IR projection

wikipedia.org

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

CMU Summer School 7.2014 Blake Hannaford, University of Washington

Medical Virtual Fixture:

Remote

  • perator

Target Vital Structure

Sensing based virtual fixture

VF based On organ shape And Motion

BioRobotics Lab

CMU Summer School 7.2014 Blake Hannaford, University of Washington

Medical Virtual Fixture:

Remote

  • perator

“Hole” in VF Target Vital Structure

Sensing based virtual fixture

VF based On organ shape

BioRobotics Lab

CMU Summer School 7.2014 Blake Hannaford, University of Washington

Laser Resection Demo:

Real Time VF based On organ shape (butcher shop liver) Simulated Tumor VF Laser Tip

CMU Summer School 7.2014

Laser Tumor Resection

Google 13-Sept-2013 Blake Hannaford, University of Washington

Blake Hannaford, University of Washington

slide-3
SLIDE 3

7/20/2014

3

BioRobotics Lab

CMU Summer School 7.2014 Blake Hannaford, University of Washington

Haptics via Kinect

BioRobotics Lab

Haptic Rendering using Kinect

  • Haptic Rendering based

data from non-contact devices (Kinect)

  • Potentially useful in

many tele-operation tasks.

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

Comparison with a Traditional Haptic Algorithm

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

The Main Challenges

We need an algorithm for moving the proxy in the point cloud environment. The proxy should be constrained by the points as if the points represented surfaces. We must be able to interact with time varying point clouds.

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

New Definition of Proxy

Our proxy consists of three regions to detect states of free motion, contact and entrenchment.

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

New Definition of Proxy

Our proxy consists of three regions to detect states of free motion, contact and entrenchment.

CMU Summer School 7.2014 Blake Hannaford, University of Washington

slide-4
SLIDE 4

7/20/2014

4

BioRobotics Lab

New Definition of Proxy

Our proxy consists of three regions to detect states of free motion,

contact and

entrenchment.

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

New Definition of Proxy

Our proxy consists of three regions to detect states of free motion, contact and

entrenchment.

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

Proxy in Free Motion

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

Proxy in Free Motion

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

Proxy in Free Motion

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

Proxy in Free Motion

CMU Summer School 7.2014 Blake Hannaford, University of Washington

slide-5
SLIDE 5

7/20/2014

5

BioRobotics Lab

Proxy in Free Motion

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

Proxy in Contact

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

Proxy in Contact

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

Proxy in Contact

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

Proxy in Contact

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

Proxy in Contact

CMU Summer School 7.2014 Blake Hannaford, University of Washington

slide-6
SLIDE 6

7/20/2014

6

BioRobotics Lab

Proxy in Contact

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

Proxy in Contact

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

Proxy in Contact

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

Proxy in Contact

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

Proxy in Contact

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

Proxy in Contact

CMU Summer School 7.2014 Blake Hannaford, University of Washington

slide-7
SLIDE 7

7/20/2014

7

BioRobotics Lab

Proxy in Contact

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

Entrenched Proxy

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

Entrenched Proxy

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

Entrenched Proxy

CMU Summer School 7.2014 Blake Hannaford, University of Washington CMU Summer School 7.2014

Blake Hannaford, University of Washington

CMU Summer School 7.2014

Blake Hannaford, University of Washington

slide-8
SLIDE 8

7/20/2014

8

CMU Summer School 7.2014

Blake Hannaford, University of Washington

CMU Summer School 7.2014

Blake Hannaford, University of Washington

CMU Summer School 7.2014

Blake Hannaford, University of Washington

CMU Summer School 7.2014

Blake Hannaford, University of Washington

CMU Summer School 7.2014

Blake Hannaford, University of Washington

CMU Summer School 7.2014

Blake Hannaford, University of Washington

slide-9
SLIDE 9

7/20/2014

9

CMU Summer School 7.2014

Blake Hannaford, University of Washington

CMU Summer School 7.2014

Blake Hannaford, University of Washington

CMU Summer School 7.2014

Advanced Virtual Fixtures

Blake Hannaford, University of Washington

BioRobotics Lab

CMU Summer School 7.2014 Blake Hannaford, University of Washington

FLS

  • Basic curriculum from Society of American

Gastrointestinal and Endoscopic Surgeons (SAGES)

  • http://www.flsprogram.org
  • Block Transfer:
  • RPI Video of Block transfer Task

BioRobotics Lab

CMU Summer School 7.2014 Blake Hannaford, University of Washington

  • “Virtual Fixtures”

–Tactile Perceptual Overlays

  • “Mixed Autonomy”

–Automation

  • f sub-tasks

BioRobotics Lab

CMU Summer School 7.2014 Blake Hannaford, University of Washington

Putting Green VF

slide-10
SLIDE 10

7/20/2014

10

BioRobotics Lab

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

CMU Summer School 7.2014 Blake Hannaford, University of Washington

BioRobotics Lab

CMU Summer School 7.2014 Blake Hannaford, University of Washington

Ender's Game

CMU Summer School 7.2014 Blake Hannaford, University of Washington