Haptics for Surgical Simulation CPSC 599.86 / 601.86 Sonny Chan University of Calgary
Motivation • I taught you all this great, futuristic stuff this semester… - but where would I ever use it? • Few applications can support the high cost of haptic devices • Luckily for us (me), the medical community is well funded - surgery is a high-risk endeavour with very, very costly mistakes!
D E T P E NeuroTouch (NRC Canada) and LAP Mentor (Simbionix) C C A
neuroArm project, University of Calgary
Opportunities for Surgical Simulation What one is the simulator?
My Research Mission Virtual Surgical Environment Operating Room
Volumetric Isosurfaces
Volume Rendering • Implicit representations are now uncommon, but... • 3D medical imaging (CT, MRI, etc.) has resulted in an abundance of volume data • Can be rendered with (almost) the same algorithm for implicit surfaces!
Sampled Volume Data • Image values sampled on rectilinear grid • CT scans measure radiodensity in Hounsfield Units
Interpolation 0.3 0.6 (1,1) What is the value here? F ( x, y ) = (1 � x + b x c )(1 � y + b y c ) I ( b x c , b y c ) + ( x � b x c )(1 � y + b y c ) I ( d x e , b y c ) + (1 � x + b x c )( y � b y c ) I ( b x c , d y e ) + ( x � b x c )( y � b y c ) I ( d x e , d y e ) for x, y, z 2 R (0,0) 0.7 0.9
Isocontours & Isosurfaces Choose a threshold value, T, to determine the surface function T = -600 HU T = 300 HU S ( x, y, z ) = T − F ( x, y, z )
Isosurfaces in 3D
Rendering Algorithm • Our implicit surface rendering algorithm has two specific requirements: - Inside-outside function for S(p) S ( x, y, z ) = T − F ( x, y, z ) - The gradient of S(p) r S ( x, y, z ) = ???
Gradient Estimation • Estimate gradient using central difference: 0 1 S ( x + δ ,y ) − S ( x − δ ,y ) ∂ S ! ∂ x 2 δ r S ( x, y ) = ⇡ @ A ∂ S S ( x,y + δ ) − S ( x,y − δ ) ∂ y 2 δ • What’s the best choice for the δ value? ∂ S/ ∂ x ∂ S/ ∂ y
Isosurface Rendering • The full rendering algorithm: - Detect initial contact when S(p) < 0 - Find surface point with initial seed - Update surface point as the device moves by using tangent plane - Contact breaks when device is moved outside the constraint plane - Repeat from start… • Any issues?
Demo?
Deformable Tissue Simulation
Continuum Mechanics • Governs the stress/strain behaviour of materials • Includes Young’s modulus (elasticity) and Poisson ratio (incompressibility) • Think of it as an “advanced” version of mass-spring systems [from continuummechanics.org]
Finite Element Simulation • Partitions an object into small, discrete elements • Computes continuum mechanics on solid element mesh 1 − ( ) R K R x x − 0 e e e f -f x f plastic plastic R total e ( ) x − x 0 f elastic 1 − R x x e 0 original state plastic rest state deformed state 1 − ( ) K R x x − 1 − 0 e e ( ) R x x − 0 e [from M. Müller & M. Gross, Proc. Graphics Interface, 2004]
Data Representation CT isosurface tetrahedral mesh
Data Processing
Deformation Simulation • Co-rotational linear elastic FEM • Computes nodal displacements from external forces • Implementation from VEGA FEM library (Jernej Barbi č , USC)
Haptic Rendering • Extension of original isosurface rendering algorithm: - sampling deformed volume - force transfer - moving the proxy
Sampling the Deformed Volume p 1 • How do we find the image intensity value p 0 x deformed at a given location in deformed space, x ? tetrahedron p 2 r 1 x m � − 1 � � � x m r 0 r 1 r 2 r 3 p 0 p 1 p 2 p 3 x r 2 r 0 = 1 1 1 1 1 1 1 1 1 1 rest vertices barycentric coordinates
Force Transfer • Virtual coupling force is applied to mesh nodes with barycentric weighting F 0 F 1 x p F 2 x F c
Moving the Proxy • To avoid interpenetration, we must keep the proxy outside the isosurface, even when the deformable mesh elements themselves move! • If a tetrahedron’s vertices move, p i → q i • we displace the proxy’s with the tetrahedron: � � 1 x 0 � � � q 0 q 1 q 2 q 3 p 0 p 1 p 2 p 3 x p p = 1 1 1 1 1 1 1 1 1 1
forces displacements Haptic Deformation Visual Architecture Rendering Simulation Rendering displacements 1000 Hz ~10-100 Hz 60 Hz
Results
Results
Summary • Surgical simulation is a popular and promising application of high-fidelity computer haptics - can tolerate cost of hardware • Sampled volumetric images (medical image data) can be described as implicit surfaces - adapt the implicit surface rendering algorithm to compute forces
Recommend
More recommend
