IMGD 3xxx - HCI for Real, Virtual, and Teleoperated Environments: - - PowerPoint PPT Presentation

▶

imgd 3xxx hci for real virtual and teleoperated

IMGD 3xxx - HCI for Real, Virtual, and Teleoperated Environments: - - PowerPoint PPT Presentation

Feb 13, 2023 148 likes •527 views

IMGD 3xxx - HCI for Real, Virtual, and Teleoperated Environments: The Human Visual System and Visual Display Techniques by Robert W. Lindeman gogo@wpi.edu Introduction Vision is the most dominant sense Though other senses are better

slide-1

SLIDE 1

IMGD 3xxx - HCI for Real, Virtual, and Teleoperated Environments: The Human Visual System and Visual Display Techniques

by Robert W. Lindeman gogo@wpi.edu

slide-2

SLIDE 2

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 2

Introduction

Vision is the most dominant sense

 Though other senses are better at certain

things, like smell for memory recall

What types of visual elements are

common to interactive experiences?

How can we leverage the visual sense to

promote efficiency and effectiveness?

slide-3

SLIDE 3

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 3

Motivation

 We need to display the state of the world to the

user

 Display: a method of presenting information to any of

the senses

 We need to display the user to the user (maybe)  We need to feed each sense appropriately  We need to feed multiple senses in concert

 Display for one sense shouldn't get in the way of display

for another sense

 May need to quickly don/doff displays  For gaming, low-cost is important

slide-4

SLIDE 4

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 4

Some Things to Remember

Humans are animals, and hence, have

evolved over time.

Evolutionary forces have guided the

development of our senses.

Displays and cues that leverage this fact

have a better shot of being effective.

slide-5

SLIDE 5

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 5

General Types of Displays

The senses

 Visual  Auditory  Haptic  Olfactory  Gustatory

Display anchoring

World-fixed displays Body-worn displays Hand-held displays

slide-6

SLIDE 6

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 6

Visual Display Anchoring Points

World-fixed displays

 Fishtank VR  Projection VR

Body-worn displays

 Opaque HMDs  Transparent HMDs

Hand-held displays

 Palm VR  Boom-mounted screens

slide-7

SLIDE 7

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 7

Visual Display Types

 Monitors

 CRT, Plasma, LCD

 Surround-screens (e.g., CAVEs)  Tabletops  Hemispheric displays  Head-mounted displays (HMDs)  Arm-mounted displays  Virtual retinal displays  Autostereoscopic displays  3D displays

slide-8

SLIDE 8

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 8

Surround Screens

CAVEs

slide-9

SLIDE 9

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 9

Surround Screens (cont.)

CAVE

slide-10

SLIDE 10

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 10

Head-Mounted Displays (HMDs)

slide-11

SLIDE 11

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 11

Visual Cues

Depth is the main thing added by VR to

more-traditional displays

 How do we perceive depth?

Monoscopic cues Stereoscopic cues Motion-depth cues Physiological cues

slide-12

SLIDE 12

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 12

Monoscopic Cues

Overlap (Interposition) Shading & shadows Size Linear perspective Texture gradient Height in the image Atmospheric effects Brightness

slide-13

SLIDE 13

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 13

Stereoscopic Cues

This is based on the parallax of objects

appearing in two images.

Camera 1 / camera 2 effect Only good within about 5 meters of

viewer

slide-14

SLIDE 14

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 14

Motion Depth Cues

Changing relative position of head and

bjects

Can be user and/or object moving

 Train leaving a station  Use proprioception to disambiguate

http://www.youtube.com/watch?v=1AZAbSXmeoI

slide-15

SLIDE 15

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 15

Motion Depth Cues (cont.)

Head movement

slide-16

SLIDE 16

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 16

Physiological Cues

The eye changes during viewing Accommodation

 Muscular changes of the eye

Convergence

 Movements to bring images to same location

n both retinas

slide-17

SLIDE 17

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 17

Properties of Visual Displays

 Color  Spatial resolution  Contrast  Brightness  Number of channels  Focal distance  Opacity  Masking  Field of view  Field of Regard  Head position info  Graphics latency  Frame rate

slide-18

SLIDE 18

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 18

Number of Display Channels

Spatial multiplexing

 Different image in front of each eye

Temporal multiplexing (time interlacing)

 Use shutter glasses

Polarization multiplexing

 Use polarized glasses

Spectral multiplexing

 Red/blue left-eye/right-eye images

Binocular monoscopic Stereo takes twice the resources!

slide-19

SLIDE 19

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 19

Masking

How physical objects block virtual ones CAVE: Hands can break effect HMD: Not at all Fishtank: Display edges/bezel can break

effect

http://www.youtube.com/watch?v=Jd3-eiid-Uw

slide-20

SLIDE 20

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 20

Field of View vs. Field of Regard

Field of view (FOV)

 How much of the scene (in degrees) is visible

at any given time

Field of regard (FOR)

 Amount of space (in percent) of the virtual

world currently surrounding the user

Examples

 CAVE: 200° FOV facing forward, 75% FOR  HMD: 100° FOV, 100% FOR

slide-21

SLIDE 21

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 21

Hand-Held VR

PDAs are becoming more powerful

 Can track a tablet PC, and use as VR display

Call phones have cameras

 Can do AR

slide-22

SLIDE 22

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 22

Change Blindness

There is so much information for the

brain to process, we need to filter

Change blindness is when we miss things

that change from one instant to another

 http://www.youtube.com/watch?v=mAnKvo-fPs0

A public service announcement:

 http://www.youtube.com/watch?v=Ahg6qcgoay4&NR=1

Next example from:

 http://www.psych.ubc.ca/~rensink/flicker/  Show Movie

slide-23

SLIDE 23

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 23

Change Blindness

slide-24

SLIDE 24

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 24

Change Blindness (answer)

slide-25

SLIDE 25

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 25

Change Blindness (answer)

slide-26

SLIDE 26

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 26

Visuals in Games

Two main kinds

 Visuals for representing the world (player)  Visuals for representing the state of the

game (player)

Usually for the first type, more is better Usually for the second type, less is better

slide-27

SLIDE 27

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 27

Heads-Up Displays (HUDs)

What is a HUD?

 "A collection of persistent on-screen

elements whose purpose is to indicate player status."

(Greg Wilson, Gamasutra: http://www.gamasutra.com/features/20060203/wilson_pfv.htm)

Are HUDs good?

slide-28

SLIDE 28

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 28

Creating an Effective HUD

 How can we minimize HUD elements?  Decide what information the player needs, and

what he/she doesn't.

 Put as much of that information into the game

 E.g., speedometer in car, ammo count on weapon

 Off-load from visuals to something else

 Examples for what would work?

 Blink-in changes, then fade them out  Make things configurable

 View point, map type, transparency

 Camouflage the HUD using themes

slide-29

SLIDE 29

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 29

HUD-less

(Peter Jackson's King Kong)

slide-30

SLIDE 30

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 30

Integrated HUD Info

(Doom 3)

slide-31

SLIDE 31

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 31

Integrated HUD Info

(Project Gotham Racing 3)

slide-32

SLIDE 32

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 32

Semi-Opaic HUD

(Deus Ex: Invisible War)

slide-33

SLIDE 33

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 33

Themed HUD

(Metroid Prime)

slide-34

SLIDE 34

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 34

Need For Speed HUD Elements

slide-35

SLIDE 35

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 35

Need For Speed HUD Elements

slide-36

SLIDE 36

R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development 36

Good Readings

"Learn Faster to Play Better"

 http://www.gamasutra.com/view/feature/3392/learn_faster_to_p

lay_better_how_.php

"Off with their HUDs"

 http://www.gamasutra.com/features/20060203/wilson_01.shtml