Overview Overview Visual displays Visual displays Visual and - - PDF document

▶
overview overview
SMART_READER_LITE
LIVE PREVIEW

Overview Overview Visual displays Visual displays Visual and - - PDF document

Apr 28, 2023 30 likes •87 views

Overview Overview Visual displays Visual displays Visual and tactile displays Visual and tactile displays Tactile displays Tactile displays Integrated displays Integrated displays Alan Liu Alan Liu Applications

slide-1
SLIDE 1

1

Visual and tactile displays Visual and tactile displays

Alan Liu Alan Liu aliu@simcen.usuhs.mil aliu@simcen.usuhs.mil

Surgical Simulation Laboratory Surgical Simulation Laboratory National Capital Area Medical Simulation Center National Capital Area Medical Simulation Center Uniformed Services University Uniformed Services University http://simcen.usuhs.mil/mmvr2002 http://simcen.usuhs.mil/mmvr2002

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

Overview Overview

  • Visual displays

Visual displays

  • Tactile displays

Tactile displays

  • Integrated displays

Integrated displays

  • Applications in medical simulation

Applications in medical simulation

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

Color monitors Color monitors

  • CRT/LCD monitors

CRT/LCD monitors

  • Cheap, readily

Cheap, readily available, well available, well understood technology understood technology

  • Often sufficient in

Often sufficient in simulators simulators

– – with minimal visual with minimal visual requirements requirements – – Minimally invasive Minimally invasive procedures procedures

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

Stereoscopic Display Stereoscopic Display

  • Depth perception occurs when each eye sees a slightly

Depth perception occurs when each eye sees a slightly different image of the same object. different image of the same object.

  • The brain fuses the separate images into a 3

The brain fuses the separate images into a 3-

  • dimensional

dimensional representation representation

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

Stereoscopic eyewear Stereoscopic eyewear

  • Monitor alternately

Monitor alternately displays left/right image displays left/right image

  • Display synchronized with

Display synchronized with LCD shutter glasses LCD shutter glasses

  • CrystalEyes

CrystalEyes

– – Wireless, requires emitter Wireless, requires emitter for synchronization, cost: for synchronization, cost: about $700 for eyewear, about $700 for eyewear, $300 for emitter $300 for emitter

  • Vrex visualizer

Vrex visualizer

– – Wired, cost: about $30 Wired, cost: about $30 retail, does not require retail, does not require emitter, works with emitter, works with any any PC PC video display video display – – http://www.vrex.com http://www.vrex.com

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

Stereoscopic flat panel display Stereoscopic flat panel display

(Dimension Technologies) (Dimension Technologies)

www.dti3d.com

slide-2
SLIDE 2

2

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

Stereoscopic flat panel display Stereoscopic flat panel display

  • No eyewear required

No eyewear required

  • Perception of stereo

Perception of stereo effect is dependent on effect is dependent on user’s location user’s location

– – Easy to get left/right Easy to get left/right images crossed images crossed – – Harder to get used to Harder to get used to compared to compared to monitor/stereo eyewear monitor/stereo eyewear

www.dti3d.com

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

Head mounted display Head mounted display

  • CRT/LCD displays are worn on

CRT/LCD displays are worn on the head the head

  • Immersive display of a virtual

Immersive display of a virtual environment environment

– – Real Real-

  • time tracking of head

time tracking of head movements movements

  • Augmented display of real

Augmented display of real-

  • world environment

world environment

– –Need to align virtual and real Need to align virtual and real-

  • world environments

world environments

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

Retinal scanning display Retinal scanning display

[JOHNSTON95] [JOHNSTON95]

  • Research by U. of

Research by U. of Washington HITLab Washington HITLab

  • Developed by Microvision

Developed by Microvision

  • Coherent (laser) light

Coherent (laser) light scanned directly on retina scanned directly on retina

– – 800x600 (typical) 800x600 (typical) – – High contrast, daylight High contrast, daylight readable readable – – Monochrome/color displays Monochrome/color displays

  • Applications

Applications

– – Mobile medical wireless Mobile medical wireless display (TATRC) display (TATRC) – – Image guided surgery Image guided surgery

www.mvis.com www.mvis.com

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

CAVE: Cave Automatic Virtual Environment CAVE: Cave Automatic Virtual Environment [CRUZ [CRUZ-

  • NEIRA93]

NEIRA93]

  • Stereo projectors display

Stereo projectors display 3D images on each wall 3D images on each wall

– – Viewer(s) are immersed in Viewer(s) are immersed in 3D virtual environment 3D virtual environment – – Head movement can be Head movement can be tracked tracked

  • Developed at Univ. of

Developed at Univ. of Illinois at Chicago Illinois at Chicago

  • Commercially available

Commercially available

  • Applications

Applications

– – Medical readiness trainer Medical readiness trainer

www-vrl.umich.edu/mrt

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

Elumens VisionStation 800 Elumens VisionStation 800

  • Immersive 3D space

Immersive 3D space

  • Low cost: $10,000

Low cost: $10,000

  • Elumens Corporation:

Elumens Corporation: hemispherical 3D hemispherical 3D visualization systems visualization systems

  • Other products:

Other products: VisionDome VisionDome

  • Potential applications

Potential applications

– – Triage simulation Triage simulation

http://www.elumens.com/

Haptic displays Haptic displays

slide-3
SLIDE 3

3

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

Definitions Definitions

  • Tactile (touch) feedback: sensed by

Tactile (touch) feedback: sensed by receptors close to the skin (high bandwidth) receptors close to the skin (high bandwidth)

  • Force feedback: sensed by receptors placed

Force feedback: sensed by receptors placed deeper in the body (low bandwidth) deeper in the body (low bandwidth)

  • Haptic feedback: technically same as tactile

Haptic feedback: technically same as tactile feedback, but often extended to include both feedback, but often extended to include both tactile and force feedback as in haptics tactile and force feedback as in haptics

  • Source: Force and Touch Feedback For Virtual Reality, Burdea

Source: Force and Touch Feedback For Virtual Reality, Burdea

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

Why Use Force Feedback? Why Use Force Feedback?

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

Phantom Phantom

(Sensable Technologies) (Sensable Technologies)

  • Sensors report movement

Sensors report movement in 6 degrees of freedom in 6 degrees of freedom (dof) (dof)

  • 3 dof force feedback

3 dof force feedback

  • Force feedback loop must

Force feedback loop must be updated rapidly (1kHz be updated rapidly (1kHz refresh rate or better) refresh rate or better)

  • Widely used for in

Widely used for in surgical simulation surgical simulation research research

www.sensable.com A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

Virtual Laparoscopic Interface/ Virtual Laparoscopic Interface/ Laparoscopic Impulse Engine Laparoscopic Impulse Engine

(Immersion) (Immersion)

  • Interface device for

Interface device for simulated laparoscopic simulated laparoscopic procedures procedures

  • Haptic and non

Haptic and non-

  • haptic

haptic version version

www.immersion.com

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

Haptic Catheter/Needle Insertion Haptic Catheter/Needle Insertion Simulator Simulator

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

Endoscopy Simulator Endoscopy Simulator

  • Flexible scope

Flexible scope-

  • like

like interface interface

  • Computer generated

Computer generated imagery of anatomical imagery of anatomical structures structures

  • Different procedures

Different procedures can be simulated can be simulated

– – Bronchoscopy Bronchoscopy – – Endoscopy Endoscopy

slide-4
SLIDE 4

4

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

400 Pin Tactile Array 400 Pin Tactile Array

  • 400 individually

400 individually controlled probes controlled probes

– – 0.5 mm apart 0.5 mm apart

  • Developed for

Developed for research into the research into the spatial spatial-

  • temporal

temporal response properties in response properties in fingertips. fingertips.

  • Reference: Pawluk et al.

Reference: Pawluk et al.

http://www.med.jhu.edu/somlab/lab/research/methods/400pin/

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

Tactile displays Tactile displays

  • 5 x 5 pin pneumatic

5 x 5 pin pneumatic display display

– – 0.3N force per element 0.3N force per element – – 3 bits of force 3 bits of force resolution resolution

  • Tactile shape display

Tactile shape display

[WELLMAN97] [WELLMAN97]

– – Shape memory alloy Shape memory alloy based based – – 10 x 1 display 10 x 1 display – – Heat dissipation a Heat dissipation a problem problem

robotics.eecs.berkeley.edu/~ronf/tactile.html robotics.eecs.berkeley.edu/~ronf/tactile.html

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

VirTouch: Virtual Touch System VirTouch: Virtual Touch System

  • Designed to allow

Designed to allow blind and visually blind and visually impaired to navigate impaired to navigate computer screen computer screen

  • Provides tactile

Provides tactile feedback to fingertips feedback to fingertips

  • Allows user to

Allows user to recognize graphic recognize graphic shapes through touch shapes through touch

http://www.virtouch.co.il/vts.htm

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

CyberTouch™ Tactile Feedback CyberTouch™ Tactile Feedback

  • Tactile feedback

Tactile feedback

  • ption for
  • ption for

instrumented glove instrumented glove

  • Small vibrotactile

Small vibrotactile sensors on each finger sensors on each finger and the palm and the palm

  • RS

RS-

  • 232 interface

232 interface

http://www.immersion.com/products/3d/interaction/cybertouch.shtml

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

Virtual Workbench Environments Virtual Workbench Environments

[POSTON96] [POSTON96]

  • Combine 3D stereo display with haptic device.

Combine 3D stereo display with haptic device.

  • Virtual image is located within work volume of haptic

Virtual image is located within work volume of haptic device. device.

ReachIn Technologies www.reachin.se

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

http://www.krdl.org.sg/RND/biomed/virtual/ www.reachin.se

slide-5
SLIDE 5

5

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

Reachin Display Reachin Display

  • Components

Components

– – PHANTOM force PHANTOM force feedback feedback – – Stereo monitor Stereo monitor – – Semi Semi-

  • transparent

transparent mirror mirror

  • Swiss company:

Swiss company: Reachin Reachin Technologies Technologies

http://www.reachin.se/products/reachindisplay/

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

i.e.Med Reality Sculptor i.e.Med Reality Sculptor

  • Components

Components

– – Input device (such as Input device (such as PHANTOM or PHANTOM or tracking system) tracking system) – – Windows PC with Windows PC with OpenGL graphics OpenGL graphics card card – – Mirror and stand Mirror and stand – – TDI software TDI software development kit development kit

www.iemed.com/products/products/reality/index.htm

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

Resources Resources

  • Head mounted displays

Head mounted displays

– – http://www.vrnews.com/issuearchive/vrn1001/vrn1001tech.html http://www.vrnews.com/issuearchive/vrn1001/vrn1001tech.html – – http://www.cs.unc.edu/~us/web/headmounts.htm http://www.cs.unc.edu/~us/web/headmounts.htm

  • Tracking

Tracking

– – http://www.cs.unc.edu/~tracker/ref/biblio/index.html http://www.cs.unc.edu/~tracker/ref/biblio/index.html

  • Stereoscopic displays

Stereoscopic displays

– – http://www.stereographics.com/html/hp http://www.stereographics.com/html/hp-

  • paper.htm

paper.htm

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

Bibliography Bibliography

  • [

[JOHNSTON95]

JOHNSTON95] Johnston R.S., Willey S.R. "Development of a commercial retinal Johnston R.S., Willey S.R. "Development of a commercial retinal scanning display." Helmet scanning display." Helmet-

  • and Head

and Head-

  • Mounted Displays and

Mounted Displays and Symbology Symbology Design Design Requirements II, 18 Requirements II, 18-

  • 19 April 1995, Orlando, FL, USA, SPIE Proceedings Vol. 2465,

19 April 1995, Orlando, FL, USA, SPIE Proceedings Vol. 2465,

  • 1995. pp. 2
  • 1995. pp. 2-
  • 13, http://www.

13, http://www.hitl hitl. .washington washington.edu/publications/p .edu/publications/p-

  • 95

95-

  • 5/.

5/.

  • [POSTON96]

[POSTON96] Poston T., Poston T., Serra Serra L. "

  • L. "Dextrous

Dextrous virtual work" Communications of the virtual work" Communications of the ACM 39(5), May 1996. pp. 37 ACM 39(5), May 1996. pp. 37-

  • 45,

45, http://www.krdl.org.sg/RND/biomed/publications/ http://www.krdl.org.sg/RND/biomed/publications/dextroscope dextroscope/papers/CACM/ /papers/CACM/virtualWo virtualWo rkbench rkbench-

  • 1.html

1.html

  • [CRUZ

[CRUZ-

  • NEIRA93] Cruz

NEIRA93] Cruz-

  • Neira

Neira C., C., Sandin Sandin D.J., D.J., DeFanti DeFanti T.A. "Surround T.A. "Surround-

  • screen

screen projection projection-

  • based virtual reality: The design and implementation of the CAVE

based virtual reality: The design and implementation of the CAVE." ACM ." ACM SIGGRAPH ‘93 Proceedings, 1993. pp. 135 SIGGRAPH ‘93 Proceedings, 1993. pp. 135-

  • 142, www.

142, www.cs cs. .utah utah.edu/~ .edu/~thompson thompson/ /vissim vissim-

  • seminar/on

seminar/on-

  • line/CruzNeiraSig93.

line/CruzNeiraSig93.pdf pdf andwww andwww. .evl evl. .uic uic. .edu edu/EVL/RESEARCH/PAPERS/CRUZ/sig93.paper.html /EVL/RESEARCH/PAPERS/CRUZ/sig93.paper.html

  • Optical versus video see

Optical versus video see-

  • through head

through head-

  • mounted displays in medical visualization,

mounted displays in medical visualization, Rolland & Fuchs, Presence, 2000, Vol. 9, No. 3, 287 Rolland & Fuchs, Presence, 2000, Vol. 9, No. 3, 287-

  • 309.

309.

  • Virtual reality based surgical assistance and training system fo

Virtual reality based surgical assistance and training system for long duration space r long duration space missions, Montgomery et al., MMVR, Jan. 2001. missions, Montgomery et al., MMVR, Jan. 2001.

A Tutorial on Surgical Simulation: Past, Present, and Future – MMVR 2002

Bibliography Bibliography

  • SIGGRAPH 2001 Course 11

SIGGRAPH 2001 Course 11-

  • Tracking: Beyond 15 Minutes of

Tracking: Beyond 15 Minutes of Thought, Bishop, Welch, and Allen Thought, Bishop, Welch, and Allen (http://www. (http://www.cs cs. .unc unc.edu/~tracker/ref/s2001/tracker/index.html) .edu/~tracker/ref/s2001/tracker/index.html)

  • Image guidance of breast cancer surgery using 3D ultrasound

Image guidance of breast cancer surgery using 3D ultrasound images and augmented reality visualization, Sato et al., images and augmented reality visualization, Sato et al., IEEE IEEE Transactions on Medical Imaging Transactions on Medical Imaging, Oct. 1998, Vol. 17, No. 5, 681 , Oct. 1998, Vol. 17, No. 5, 681-

  • 693.

693.

  • Technologies for augmented

Technologies for augmented-

  • reality systems: realizing ultrasound

reality systems: realizing ultrasound guided needle biopsies, State et al., in guided needle biopsies, State et al., in Computer Graphics Computer Graphics proceedings, ACM SIGGRAPH, 1996, 439 proceedings, ACM SIGGRAPH, 1996, 439-

  • 446.

446.

  • Virtual reality technology

Virtual reality technology, Burdea and , Burdea and Coiffet Coiffet, Wiley, 1994 , Wiley, 1994

  • Wellman, P.S.,

Wellman, P.S., Peine Peine, W.J., and Howe, R.D., “Mechanical Design , W.J., and Howe, R.D., “Mechanical Design and Control of a High and Control of a High-

  • Bandwidth Shape Memory Alloy Tactile

Bandwidth Shape Memory Alloy Tactile Display,” proceedings of the International Symposium of Display,” proceedings of the International Symposium of Experimental Robotics, Barcelona, Spain, June 1997 Experimental Robotics, Barcelona, Spain, June 1997