14.1 I/O Technologies & VR Hao Li http://cs420.hao-li.com 1 In - - PowerPoint PPT Presentation

▶
14 1 i o technologies vr
SMART_READER_LITE
LIVE PREVIEW

14.1 I/O Technologies & VR Hao Li http://cs420.hao-li.com 1 In - - PowerPoint PPT Presentation

Mar 23, 2024 148 likes •1.26k views

Fall 2018 CSCI 420: Computer Graphics 14.1 I/O Technologies & VR Hao Li http://cs420.hao-li.com 1 In CS I/O is an abstraction http://stackoverflow.com/questions/236000/whats-a-turing-machine 2 In CG I/O is an object of study 3 ACM

slide-1
SLIDE 1

CSCI 420: Computer Graphics

Hao Li

http://cs420.hao-li.com

1

Fall 2018

14.1 I/O Technologies & VR

slide-2
SLIDE 2

In CS I/O is an abstraction

2

http://stackoverflow.com/questions/236000/whats-a-turing-machine
slide-3
SLIDE 3

In CG I/O is an object of study

3

slide-4
SLIDE 4

ACM SIGGRAPH

4

  • Computer Graphics and Interactive Techniques
slide-5
SLIDE 5

Display Technologies

5

  • Liquid Crystal Display
slide-6
SLIDE 6

Liquid Crystals

6

  • Matter in a state that has

properties between liquid and solid crystals

  • Twisted nematics

Anisotropic pattern depending on electricity, heat, etc.

http://mrsec.wisc.edu/Edetc/courses/colorsymp/park/index.html
slide-7
SLIDE 7

Liquid Crystals

7

  • Off-state (left), On-state (right)
slide-8
SLIDE 8

LCD Light Path

8

http://electronics.howstuffworks.com/lcd2.htm
slide-9
SLIDE 9

LCD Light Path

9

http://electronics.howstuffworks.com/lcd2.htm

Light Source

slide-10
SLIDE 10

LCD Light Path

10

http://electronics.howstuffworks.com/lcd2.htm

Polariz e

slide-11
SLIDE 11

LCD Light Path

11

http://electronics.howstuffworks.com/lcd2.htm

Twist Light Polarizatio n

slide-12
SLIDE 12

LCD Light Path

12

http://electronics.howstuffworks.com/lcd2.htm

Only twisted light makes it through

slide-13
SLIDE 13

LCD Light Path

13

http://electronics.howstuffworks.com/lcd2.htm

Electrode controls crystals

slide-14
SLIDE 14

Cathode Ray Tube (CRT)

14

http://img.diytrade.com/cdimg/597243/7515356/0/1276139831/Sell_CRT_Monitor.jpg http://www.freepatentsonline.com/6741296-0-large.jpg
slide-15
SLIDE 15

Practical Display Issues

15

http://en.wikipedia.org/wiki/File:Shadow_mask_vs_aperture_grille.jpg
slide-16
SLIDE 16

Practical Display Issues

16

Have to convert from RGB to display pattern

http://en.wikipedia.org/wiki/File:Shadow_mask_closeup_cursor.jpg
slide-17
SLIDE 17

Subpixel Antialiasing

17

http://en.wikipedia.org/wiki/ClearType

Color fringing

  • Clear Type (Microsoft, 1998), subpixel rendering
slide-18
SLIDE 18

Practical Display Issues

18

slide-19
SLIDE 19

Practical Display Issues

19

Nonlinear relationship between brightness and intensity

Perceptual Display-related

slide-20
SLIDE 20

Practical Display Issues

20

Nonlinear relationship between brightness and intensity Nonlinear relationship between intensity and hardware response

slide-21
SLIDE 21

Gamma Model: For Displays

21

slide-22
SLIDE 22

Gamma Model: For Displays

22

slide-23
SLIDE 23

Gamma Model: For Displays

23

slide-24
SLIDE 24

Why don’t we do this always?

24

slide-25
SLIDE 25

Detecting/Processing Motion

25

Visual sensors must communicate!

http://darlingdarleen.blogspot.com/2008_01_01_archive.html

Discontinuous motion with same average velocity as implied continuous motion.

slide-26
SLIDE 26

Other Displays

26

Digital Light Processing (DLP)

Spinning color wheel

LCD Shutter

Alternate between eyes

http://graphics.stanford.edu/courses/cs148-11-fall/lectures/displays.pdf
slide-27
SLIDE 27

Other Displays

27

No black Max black Four primaries!

http://en.wikipedia.org/wiki/CMYK_color_ model
slide-28
SLIDE 28

Other Displays

28

Electronic ink

Different appearance, slow update rate

http://graphics.stanford.edu/courses/cs148-11-fall/lectures/displays.pdf
slide-29
SLIDE 29

Dealing with Input

29

Events


Notify when state changes
 


Polling


Check for changed state

http://graphics.stanford.edu/courses/cs148-11-fall/lectures/interaction.pdf
slide-30
SLIDE 30

Events

30

+ 


Efficient


Need to track state Need to decide on events of interest

http://graphics.stanford.edu/courses/cs148-11-fall/lectures/interaction.pdf
slide-31
SLIDE 31

Polling

31

+ 


Cleanly deals with continuous state change
 


Could miss a state change Considerable overhead

http://graphics.stanford.edu/courses/cs148-11-fall/lectures/interaction.pdf
slide-32
SLIDE 32

Keyboards

32

http://www.headfuzz.co.uk/files/keyboard-matrix2-sch.png http://graphics.stanford.edu/courses/cs148-11-fall/lectures/input.pdf

Key press closes circuit; character map used to determine which key (filter bounces)

slide-33
SLIDE 33

Optical Mice

33

http://www.bidouille.org/hack/mousecam

Digital Image Correlation

slide-34
SLIDE 34

Optical Mouse

34

http://www.blogcdn.com/www.engadget.com/media/2008/12/original-mouse-08dec03.jpg
slide-35
SLIDE 35

Multitouch

35

slide-36
SLIDE 36

Other Input Sources

36

http://0.tqn.com/d/compactiongames/1/0/J/A/1/gp2.jpg https://images-na.ssl-images-amazon.com/images/G/01/videogames/detail-page/B0045FGET2.01.lg.jpg

Game controller, joystick

Communicate with station

Wii remote

Accelerometers, IR sensor

slide-37
SLIDE 37

Other Input Sources

37

Camera Kinect

slide-38
SLIDE 38

Other Input Sources

38

http://www.cyberware.com/products/scanners/lss.html http://home.12move.nl/~sh290334/dbase_force/cybergrasp.jpg http://upload.wikimedia.org/wikipedia/commons/1/13/Rosies_ct_scan.jpg http://www.nemusiccenter.com/product_images/u/377/SM58__69613_zoom.jpg http://onemillionlyrics.com/lyrics/scanner/rmu http://bssdigitalsound.files.wordpress.com/2008/02/midi-mk249c.jpg
slide-39
SLIDE 39

Virtual Reality

39

slide-40
SLIDE 40

Technological Trends

  • Smaller, cheaper, more functions, more intimate, more

immersive

  • Intuitive to use
  • Interface over internals
  • Form over function
  • Human centered design

Technology becomes invisible

40

slide-41
SLIDE 41

Invisible Interfaces

Jun Rekimoto, Sony CSL 41

slide-42
SLIDE 42

Graphical User Interfaces

42

  • Separation between real and digital worlds
  • WIMP (Windows Icons, Menus, Pointer) metaphor
slide-43
SLIDE 43

Ubiquitous Computing

43

  • Computing and sensing embedded in real world
  • Particle devices, RFID, motes, arduino, etc.
slide-44
SLIDE 44

Virtual Reality

44

  • Morton Heilig
  • Not in computers!
  • Surround sound

idea for the eyes…

  • Why use 18% of the

viewer’s FOV in 2D, when we can use 100% in 3D…

slide-45
SLIDE 45

Virtual Reality

45

  • Too expensive, Heilig’s

plans fell through

  • Sensorama! (early 60s)
  • Ivan Sutherland

continued (CRT’s, CGI), flight sims

slide-46
SLIDE 46

Virtual Reality

46

  • 1985…
slide-47
SLIDE 47

Virtual Reality

47

  • Head mounted display, gloves
  • Separation from the real world

Immersive VR

slide-48
SLIDE 48

Augmented Reality

48

  • Combines Real and Virtual Images
  • Both can be seen at the same time
  • Interactive in real-time
  • The virtual content can be interacted with
  • Registered in 3D
  • Virtual objects appear fixed in space

Defining Charactestics [Azuma 97]

Azuma, R. T. (1997). A survey of augmented reality. Presence, 6(4),

slide-49
SLIDE 49

Augmented Reality Examples

49 Magic Leap

slide-50
SLIDE 50

Augmented Reality Examples

50 Google Glass

slide-51
SLIDE 51

VR vs AR

51

Virtual Reality: Replaces Reality

  • Scene Generation: requires realistic images
  • Display Device: fully immersive, wide FOV
  • Tracking and Sensing: low accuracy used to be okay

Augmented Reality: Enhances Reality

  • Scene Generation: minimal rendering okay
  • Display Device: non-immersive, small FOV
  • Tracking and Sensing: high accuracy needed
slide-52
SLIDE 52

Milgram’s Reality-Virtuality Continuum

52

slide-53
SLIDE 53

Recent Advances

53

  • OLED-driven Low Persistency Displays
  • Less smearing and ghosting artefacts
  • Sliced time frame rendering
  • Darker games are an improvement
  • High-quality realtime 3D content
  • Low cost production
  • Wide-FOV (>110) Single Display
  • Cheap lenses
slide-54
SLIDE 54

Challenges: VR HMDs

54

Oculus Connect 2014, John Carmack:

  • Higher framerates without flicker problems
  • DK2 achieves 75Hz, optimal is 90-120 Hz
  • Resolution vs framerate vs bandwidth
  • Inaccurate positional tracking
  • Submillimeter tracking -

SLAM+IMU(Accelerometers/Magnetometers)

  • Relative velocity vs relative position?
  • No Jittering
slide-55
SLIDE 55

Immersive Experience

slide-56
SLIDE 56

Virtual Reality Reloaded

Oculus VR 2012 / Crytek 2014

slide-57
SLIDE 57
slide-58
SLIDE 58

Social Interactions in Cyberspace

slide-59
SLIDE 59

Oculus Oculus Connect 3 (2016)

slide-60
SLIDE 60

Occlusions

slide-61
SLIDE 61

Facial Performance Sensing HMD

slide-62
SLIDE 62

Facial Performance Sensing HMD

slide-63
SLIDE 63

Facial Performance Sensing HMD

slide-64
SLIDE 64

Facial Performance Sensing HMD

slide-65
SLIDE 65

Ultra Thin Flexible Electronic Materials

slide-66
SLIDE 66

Live Demo

slide-67
SLIDE 67

Microsoft 2015

Olszewski et al. (2016)

slide-68
SLIDE 68
slide-69
SLIDE 69

High Dimensionality & Non-Linearity

slide-70
SLIDE 70

Training via Audio Alignment

slide-71
SLIDE 71

Mouth Animation

slide-72
SLIDE 72
slide-73
SLIDE 73
slide-74
SLIDE 74
slide-75
SLIDE 75
slide-76
SLIDE 76

Digital Humans

slide-77
SLIDE 77

VFX Production

Weta Digital (2014)

slide-78
SLIDE 78

Game Production

Epic Games / Cubic Motion / 3Lateral (2018)

slide-79
SLIDE 79

Pinscreen (2017)

slide-80
SLIDE 80

Real-Time Lighting Estimation

slide-81
SLIDE 81

NextGen Photoreal Avatars

slide-82
SLIDE 82

Deep Learning-Based Face Synthesis

slide-83
SLIDE 83

Deep Learning-Based Face Synthesis

slide-84
SLIDE 84

Deep Learning-Based Face Synthesis

slide-85
SLIDE 85

Deep Learning-Based Face Synthesis

slide-86
SLIDE 86
slide-87
SLIDE 87
slide-88
SLIDE 88

Hair Modeling

slide-89
SLIDE 89
slide-90
SLIDE 90
slide-91
SLIDE 91

91

Data-Driven Hair Modeling

slide-92
SLIDE 92

Saito et al. (2018)

Deep Learning for Hair Modeling

slide-93
SLIDE 93

Saito et al. (2018)

Deep Learning for Hair Modeling

slide-94
SLIDE 94

Deep Learning for Hair Modeling

Saito et al. (2018)

Deep Learning for Hair Modeling

slide-95
SLIDE 95

Saito et al. (2018)

Deep Learning for Hair Modeling

slide-96
SLIDE 96

Saito et al. (2018)

Deep Learning for Hair Modeling

slide-97
SLIDE 97

Saito et al. (2018)

Deep Learning for Hair Modeling

slide-98
SLIDE 98

Saito et al. (2018)

Deep Learning for Hair Modeling

slide-99
SLIDE 99

Saito et al. (2018)

Deep Learning for Hair Modeling

slide-100
SLIDE 100

Saito et al. (2018)

Deep Learning for Hair Modeling

slide-101
SLIDE 101

Hair Rendering

slide-102
SLIDE 102

Wei et al. (2018)

slide-103
SLIDE 103

Wei et al. (2018)

slide-104
SLIDE 104

Wei et al. (2018)

slide-105
SLIDE 105

What’s next?

slide-106
SLIDE 106

AI-Driven Graphics

slide-107
SLIDE 107

Created by Anyone

slide-108
SLIDE 108

VFX-Level Augmented Reality

slide-109
SLIDE 109

http://cs420.hao-li.com

Thanks!

109