15.1 I/O Technologies & VR Hao Li http://cs420.hao-li.com 1 In - - PowerPoint PPT Presentation

▶
15 1 i o technologies vr
SMART_READER_LITE
LIVE PREVIEW

15.1 I/O Technologies & VR Hao Li http://cs420.hao-li.com 1 In - - PowerPoint PPT Presentation

Nov 28, 2023 333 likes •1.07k views

Fall 2015 CSCI 420: Computer Graphics 15.1 I/O Technologies & VR Hao Li http://cs420.hao-li.com 1 In CS I/O is an abstraction http://stackoverflow.com/questions/236000/whats-a-turing-machine 2 In CG I/O is an object of study

slide-1
SLIDE 1

CSCI 420: Computer Graphics

Hao Li

http://cs420.hao-li.com

1

Fall 2015

15.1 I/O Technologies & VR

slide-2
SLIDE 2

In CS I/O is an abstraction

2

http://stackoverflow.com/questions/236000/whats-­‑a-­‑turing-­‑machine
slide-3
SLIDE 3

In CG I/O is an object of study

3

slide-4
SLIDE 4

ACM SIGGRAPH

4

  • Computer Graphics and Interactive Techniques
slide-5
SLIDE 5

Display Technologies

5

  • Liquid Crystal Display
slide-6
SLIDE 6

Liquid Crystals

6

  • Matter in a state that has

properties between liquid and solid crystals

  • Twisted nematics

Anisotropic pattern depending on electricity, heat, etc.

http://mrsec.wisc.edu/Edetc/courses/colorsymp/park/index.html
slide-7
SLIDE 7

Liquid Crystals

7

  • Off-state (left), On-state (right)
slide-8
SLIDE 8

LCD Light Path

8

http://electronics.howstuffworks.com/lcd2.htm
slide-9
SLIDE 9

LCD Light Path

9

http://electronics.howstuffworks.com/lcd2.htm

Light ¡ Source

slide-10
SLIDE 10

LCD Light Path

10

http://electronics.howstuffworks.com/lcd2.htm

Polarize

slide-11
SLIDE 11

LCD Light Path

11

http://electronics.howstuffworks.com/lcd2.htm

Twist ¡Light ¡ Polarization

slide-12
SLIDE 12

LCD Light Path

12

http://electronics.howstuffworks.com/lcd2.htm

Only ¡twisted ¡light ¡makes ¡it ¡through

slide-13
SLIDE 13

LCD Light Path

13

http://electronics.howstuffworks.com/lcd2.htm

Electrode ¡controls ¡crystals

slide-14
SLIDE 14

Cathode Ray Tube (CRT)

14

http://img.diytrade.com/cdimg/597243/7515356/0/1276139831/Sell_CRT_Monitor.jpg ¡ http://www.freepatentsonline.com/6741296-­‑0-­‑large.jpg
slide-15
SLIDE 15

Practical Display Issues

15

http://en.wikipedia.org/wiki/File:Shadow_mask_vs_aperture_grille.jpg
slide-16
SLIDE 16

Practical Display Issues

16

Have ¡to ¡convert ¡from ¡ RGB ¡to ¡display ¡pattern

http://en.wikipedia.org/wiki/File:Shadow_mask_closeup_cursor.jpg
slide-17
SLIDE 17

Subpixel Antialiasing

17

http://en.wikipedia.org/wiki/ClearType

Color ¡fringing

  • Clear Type (Microsoft, 1998), subpixel rendering
slide-18
SLIDE 18

Practical Display Issues

18

slide-19
SLIDE 19

Practical Display Issues

19

Nonlinear ¡relationship ¡between ¡brightness ¡and ¡ intensity

Perceptual Display-­‑related

slide-20
SLIDE 20

Practical Display Issues

20

Nonlinear ¡relationship ¡between ¡brightness ¡and ¡ intensity Nonlinear ¡relationship ¡between ¡intensity ¡and ¡ hardware ¡response

slide-21
SLIDE 21

Gamma Model: For Displays

21

slide-22
SLIDE 22

Gamma Model: For Displays

22

slide-23
SLIDE 23

Gamma Model: For Displays

23

slide-24
SLIDE 24

Why don’t we do this always?

24

slide-25
SLIDE 25

Detecting/Processing Motion

25

Visual ¡sensors ¡must ¡communicate!

http://darlingdarleen.blogspot.com/2008_01_01_archive.html

Discontinuous ¡motion ¡with ¡ same ¡average ¡velocity ¡as ¡implied ¡ continuous ¡motion.

slide-26
SLIDE 26

Other Displays

26

Digital ¡Light ¡Processing ¡(DLP) ¡

Spinning ¡color ¡wheel

LCD ¡Shutter ¡

Alternate ¡between ¡eyes

http://graphics.stanford.edu/courses/cs148-­‑11-­‑fall/lectures/displays.pdf
slide-27
SLIDE 27

Other Displays

27

No ¡black Max ¡black Four ¡primaries!

http://en.wikipedia.org/wiki/CMYK_color_ ¡ model
slide-28
SLIDE 28

Other Displays

28

Electronic ¡ink ¡

Different ¡appearance, ¡slow ¡update ¡rate

http://graphics.stanford.edu/courses/cs148-­‑11-­‑fall/lectures/displays.pdf
slide-29
SLIDE 29

Dealing with Input

29

Events


Notify ¡when ¡state ¡changes
 


Polling


Check ¡for ¡changed ¡state

http://graphics.stanford.edu/courses/cs148-­‑11-­‑fall/lectures/interaction.pdf
slide-30
SLIDE 30

Events

30

+ ¡


Efficient


  • ­‑


Need ¡to ¡track ¡state ¡ Need ¡to ¡decide ¡on ¡events ¡of ¡interest

http://graphics.stanford.edu/courses/cs148-­‑11-­‑fall/lectures/interaction.pdf
slide-31
SLIDE 31

Polling

31

+ ¡


Cleanly ¡deals ¡with ¡continuous ¡state ¡change
 


  • ­‑


Could ¡miss ¡a ¡state ¡change ¡ Considerable ¡overhead

http://graphics.stanford.edu/courses/cs148-­‑11-­‑fall/lectures/interaction.pdf
slide-32
SLIDE 32

Keyboards

32

http://www.headfuzz.co.uk/files/keyboard-­‑matrix2-­‑sch.png ¡ http://graphics.stanford.edu/courses/cs148-­‑11-­‑fall/lectures/input.pdf

Key ¡press ¡closes ¡circuit; ¡ character ¡map ¡used ¡to ¡ determine ¡which ¡key ¡(filter ¡ bounces)

slide-33
SLIDE 33

Optical Mice

33

http://www.bidouille.org/hack/mousecam

Digital ¡Image ¡Correlation

slide-34
SLIDE 34

Optical Mouse

34

http://www.blogcdn.com/www.engadget.com/media/2008/12/original-­‑mouse-­‑08dec03.jpg
slide-35
SLIDE 35

Multitouch

35

slide-36
SLIDE 36

Other Input Sources

36

http://0.tqn.com/d/compactiongames/1/0/J/A/1/gp2.jpg ¡ https://images-­‑na.ssl-­‑images-­‑amazon.com/images/G/01/videogames/detail-­‑page/B0045FGET2.01.lg.jpg

Game ¡controller, ¡joystick ¡

Communicate ¡with ¡station

Wii ¡remote ¡

Accelerometers, ¡IR ¡sensor

slide-37
SLIDE 37

Other Input Sources

37

Camera Kinect

slide-38
SLIDE 38

Other Input Sources

38

http://www.cyberware.com/products/scanners/lss.html ¡ http://home.12move.nl/~sh290334/dbase_force/cybergrasp.jpg ¡ http://upload.wikimedia.org/wikipedia/commons/1/13/Rosies_ct_scan.jpg ¡ http://www.nemusiccenter.com/product_images/u/377/SM58__69613_zoom.jpg ¡ http://onemillionlyrics.com/lyrics/scanner/rmu ¡ http://bssdigitalsound.files.wordpress.com/2008/02/midi-­‑mk249c.jpg
slide-39
SLIDE 39

Virtual Reality

39

slide-40
SLIDE 40

Technological Trends

  • Smaller, cheaper, more functions, more intimate, more

immersive

  • Intuitive to use
  • Interface over internals
  • Form over function
  • Human centered design

Technology becomes invisible

40

slide-41
SLIDE 41

Invisible Interfaces

Jun ¡Rekimoto, ¡Sony ¡CSL 41

slide-42
SLIDE 42

Graphical User Interfaces

42

  • Separation between real and digital worlds
  • WIMP (Windows Icons, Menus, Pointer) metaphor
slide-43
SLIDE 43

Ubiquitous Computing

43

  • Computing and sensing embedded in real world
  • Particle devices, RFID, motes, arduino, etc.
slide-44
SLIDE 44

Virtual Reality

44

  • Morton Heilig
  • Not in computers!
  • Surround sound

idea for the eyes…

  • Why use 18% of the

viewer’s FOV in 2D, when we can use 100% in 3D…

slide-45
SLIDE 45

Virtual Reality

45

  • Too expensive, Heilig’s

plans fell through

  • Sensorama! (early 60s)
  • Ivan Sutherland

continued (CRT’s, CGI), flight sims

slide-46
SLIDE 46

Virtual Reality

46

  • 1985…
slide-47
SLIDE 47

Virtual Reality

47

  • Head mounted display, gloves
  • Separation from the real world

Immersive VR

slide-48
SLIDE 48

Augmented Reality

48

  • Combines Real and Virtual Images
  • Both can be seen at the same time
  • Interactive in real-time
  • The virtual content can be interacted with
  • Registered in 3D
  • Virtual objects appear fixed in space

Defining Charactestics [Azuma 97]

Azuma, ¡R. ¡T. ¡(1997). ¡A ¡survey ¡of ¡augmented ¡reality. ¡Presence, ¡6(4), ¡355-­‑385

slide-49
SLIDE 49

Augmented Reality Examples

49 Magic ¡Leap

slide-50
SLIDE 50

Augmented Reality Examples

50 Google ¡Glass

slide-51
SLIDE 51

VR vs AR

51

Virtual Reality: Replaces Reality

  • Scene Generation: requires realistic images
  • Display Device: fully immersive, wide FOV
  • Tracking and Sensing: low accuracy used to be okay

Augmented Reality: Enhances Reality

  • Scene Generation: minimal rendering okay
  • Display Device: non-immersive, small FOV
  • Tracking and Sensing: high accuracy needed
slide-52
SLIDE 52

Milgram’s Reality-Virtuality Continuum

52

slide-53
SLIDE 53

Recent Advances

53

  • OLED-driven Low Persistency Displays
  • Less smearing and ghosting artefacts
  • Sliced time frame rendering
  • Darker games are an improvement
  • High-quality realtime 3D content
  • Low cost production
  • Wide-FOV (>110) Single Display
  • Cheap lenses
slide-54
SLIDE 54

Challenges: VR HMDs

54

Oculus Connect 2014, John Carmack:

  • Higher framerates without flicker problems
  • DK2 achieves 75Hz, optimal is 90-120 Hz
  • Resolution vs framerate vs bandwidth
  • Inaccurate positional tracking
  • Submillimeter tracking - SLAM

+IMU(Accelerometers/Magnetometers)

  • Relative velocity vs relative position?
  • No Jittering
slide-55
SLIDE 55

Virtual Reality Reloaded

Oculus VR 2012 / Crytek 2014

slide-56
SLIDE 56

Social Interactions in Cyberspace

slide-57
SLIDE 57

Immersive Virtual Worlds

slide-58
SLIDE 58

Very Low Cost

slide-59
SLIDE 59

Consuming VR

slide-60
SLIDE 60

Online Virtual Worlds

slide-61
SLIDE 61

Occlusions

slide-62
SLIDE 62

Facial Performance Sensing HMD

slide-63
SLIDE 63

Facial Performance Sensing HMD

slide-64
SLIDE 64

Facial Performance Sensing HMD

slide-65
SLIDE 65

Facial Performance Sensing HMD

slide-66
SLIDE 66

Ultra Thin Flexible Electronic Materials

slide-67
SLIDE 67

Live Demo

slide-68
SLIDE 68

Offline Training

slide-69
SLIDE 69

Online Operation

slide-70
SLIDE 70

Retargeting

slide-71
SLIDE 71

Social Interaction in CyberSpace

slide-72
SLIDE 72

http://cs420.hao-li.com

Thanks!

72