Output Devices: Graphics, 3-D Sound, and Haptic Displays Output - PowerPoint PPT Presentation

Tilted surface Viewing Cone Reflector mirror Floor CRT projector (not shown) Tall structures will be clipped: stereo collapse effect The old Fakespace “ImmersaDesk” workbench

IR Controllers CRT Projector Mirrors Tilting mechanism Baron workbench (courtesy of BARCO Co.)

Baron Workbench-type display geometries V-desk

CRT Projector Screen Mirror CAVE 3-D large volume display (courtesy of Fakespace Co.) Invented at Un Illinois @ Chicago

Each wall is Image on floor driven by a created so that different the shadows are graphics behind the user workstation CAVE 3-D large volume display (courtesy of Fakespace Co.)

RAVE (“Re-configurable Virtual Environment”) � Modular construction ( 4 modules) that allows various viewing configuration, from flat wall, to angled theater, to CAVE; � Vertical wall image 2.3 m X 2.4 m; � Several CRT projectors (260 lumens, 1280x1024 resolution); � 500.000$

Link to VC 3.2 on book CD

Output Devices Wall-type displays � Accommodate more users � Using a single projector on a large wall means small image resolution; � Thus tiled displays place smaller images side-by-side so they need multiple projectors; � Images need to have overlap, to assure continuity; � However overlap from two projectors means intensity discontinuity (brighter images in the overlap areas) � Projectors need to modulate intensities to dim their light for overlap pixels.

Pano-Wall display Three projectors; Approx. 7 x 2 m 2

PanoWall display

Output Devices

Tiled composite image from four projectors

Tiled composite image from four projectors after adjustment

Dome-type displays � Multiple projectors arranged around a hemi-sphere (back projection) � V-dome (SEOS): 7 projectors � Image pre-distortion is necessary. � IMAX-3D � Polarized glasses can be used

Wall and Dome-type displays � Advantages: � Accommodate more users (tens to hundreds) � Give users more freedom of motion; � Disadvantages: � Large cost (up to millions of dollars); � Even with multiple projectors, resolution is much lower than for CRTs (because the area is large). Example PanoWall has 200,000 pixels/m 2 while a � monitor has 18,200,000 pixels/m 2 � To have equal numbers of pixels/unit must use more projectors (military)

Output Devices 3- -D Audio Displays D Audio Displays 3 Definition: Definition: Sound displays are computer interfaces that provide synthetic sound feedback to the user interacting with the virtual world. The sound can be monoaural (both ears hear the same sound) or binaural (each ear hears a different sound).

Output Devices 3- -D Audio Displays D Audio Displays 3 � 3-D audio should not be confused with stereo sound; � Human hearing model; � HRTF-based 3-D sound; � Convolvotron; � 3-D sound cards.

Stereo vs. 3-D sound Reflected sound should be also taken into account ….

Output Devices Human Hearing Model Human Hearing Model � Head attached polar coordinate system � azimuth, elevation, distance (range); � Different cues are utilized in order to infer azimuth, elevation & range cues;

Output Devices Head Related Transfer Function (HRTF)

Azimuth cues Maximum for θ =90, minimum when source in front or behind the head

Azimuth cues � The closest ear hears a sound with higher intensity (head shadow effect) � Interaural intensity difference (IID) � Detectable for high freq. Sounds (>1.5Khz) � For lower frequencies ITD dominates (near sources)

Elevation cues � Reflections in pinna help in elevation determination – Cones of confusion � Some frequencies are amplified other are suppressed.

3-D Sound Effect of pinna filtering of sound (elevation cues)

Range cues � Prior knowledge of a given source combined with the perceived intensity � Motion parallax: change of azimuth when user translates head � Large motion parallax indicates source is near

Range cues � Ratio of direct versus reflected sound � Direct sound energy drops with square of distance � Reflected sound energy does not change much with range � Small ratio of direct/reverberated sound: far source.

HRTF � Source position known, should model the sound reaching the inner ear: Head related transfer function � Depends on person, azimuth, elevation, frequency & range (only for near field sources). � Experimental evaluation.

Output Devices NASA again a pioneer in 3- - NASA again a pioneer in 3 D sound D sound � put microphones in dummy heads; � played localized sound and measured signal; � Determined the HRIR � FT->HRTF; � Worked on first circuitry;

� For a sound to be localized: convolve the appropriate HRTF with the sound (using FIR filters) – High computational load (increases with number of sources) � Not very good results when using HRTF from other persons. � Compromise: use “generic” HRTF.

…. The Convolvotron PC 3-D sound boards

The Huron workstation ….

Speaker based sound � Stereo sound � Multichannel 5.1 sound � Sound seems to stick in the room perimeter. � Sound coming from a location other than loudspeakers cannot be realized � Use of two loudspeakers to create surround sound (phantom speakers)

….

Cross- -talk effect talk effect Cross � Sound from one speaker reaches both ears: [ ] = [ ][ ] S left Y left H l,l H l,r S right Y right H r,l H r,r where H l,l is the HRTF between the left speaker and the left ear , H l,r is the HRTF between the right speaker and the left ear , Y left is the sound reaching the left ear Y right is the sound reaching the right ear

Cross- -talk effect cancellation talk effect cancellation Cross � Sound from both speakers is adjusted such that: -1 [ ] = [ ] [ ] S left Y left H l,l H l,r S right Y right H r,l H r,r where Y left and Y right are known (the output of the convolving process)

Output Devices Haptic Interfaces Haptics… … Haptics � Comes from Greek αφή meaning the sense of touch; � Groups touch feedback and force feedback

Output Devices Touch (tactile) Feedback Touch (tactile) Feedback � Relies on sensors in and close to the skin; � Conveys information on contact surface geometry, roughness, slippage, temperature; � Does not actively resist user contact motion; � Easier to implement than force feedback.

Output Devices Force Feedback Force Feedback � Relies on sensors on muscle tendons and bones/joints proprioception; � Conveys information on contact surface compliance, object weight, inertia; � Actively resist user contact motion; � More difficult to implement than touch feedback (no commercial products until mid 90s).

Haptic Interfaces Human touch sensing mechanism Human touch sensing mechanism � Most touch sensors are on the hand (much less density on other parts of the body); � Four primary types of sensors: 40 % are Meissner’s corpuscles – detect movement across the skin – velocity detectors 25% are Merkel’s disks – measure pressure and vibrations 13 % are Pacinian corpuscles – deeper in skin (dermis) – acceleration sensors Most sensitive to vibrations of about 250 Hz 19% are Rufini corpuscles – detect skin shear and temperature changes

Haptic Interfaces Skin touch sensors

Haptic Interfaces Sensorial adaptation Sensorial adaptation � Measure the decrease in electrical signals from the skin sensor over time, for a constant stimulus; � If the sensor produces a constant electrical discharge for a constant mechanical stimulus – called “Slow Adapting” (SA); � Detect constant or slowly changing forces (Merkel & Rufffini)

Haptic Interfaces Sensorial adaptation Sensorial adaptation � If the rate of electrical discharge drops rapidly over time for a constant stimulus – “Rapidly Adapting” (RA) � Detect vibrations, accelerations on the skin (high frequency forces): Meissner & Pacinian

Haptic Interfaces Spatial resolution Spatial resolution � Receptive field size of a sensor; � If the sensor has a large receptive field – it has low spatial resolution (Pacinian and Ruffini) SA-II, RA-II � If the receptive field is small – has high Spatial resolution (Meissner and Merkel) SA-I, RA-I � Temporal resolution

Haptic Interfaces Two-point limen test: 2.5 mm fingertip, 11 mm for palm, 67 mm for thigh

Haptic Interfaces

Haptic Interfaces Temperature sensing Temperature sensing • Thermoreceptors • Nociceptors (extreme temperatures, trigger pain) Proprioception Proprioception • Perception of one’s own body position • Sensors located at skeletal articulations. • Their discharge depends on the joint angle Kinesthesia Kinesthesia • Perception of one’s own body motion • Sensed through muscle contraction and stretching.

Sensory-motor control � Tactile, proprioceptive & kinesthetic sensing used by sensory-motor control to affect forces on an object or haptic interface � Precision grasp: dexterous object manipulation, fingers � Power grasp: higher force exertion: fingers & palm

Haptic Interfaces Human grasping configurations

Haptic Interfaces Maximum and sustained force exertion Maximum and sustained force exertion � Maximum force exerted during “power” grasp Averages 400 N (male) and 225 N (female); � Sustained force is much smaller than maximum, owing to fatigue and pain � No need for a force feedback interface to generate large forces.