Realistic Real-time Rendering Today and in the Future Ulf Assarsson - - PowerPoint PPT Presentation

▶
realistic real time rendering today and in the future
SMART_READER_LITE
LIVE PREVIEW

Realistic Real-time Rendering Today and in the Future Ulf Assarsson - - PowerPoint PPT Presentation

Dec 10, 2023 231 likes •1.13k views

Realistic Real-time Rendering Today and in the Future Ulf Assarsson Chalmers University of Technology Department of Computer Engineering The screen consists of pixels Department of Computer Engineering Grafikkort Department of Computer

slide-1
SLIDE 1

Realistic Real-time Rendering Today and in the Future

Ulf Assarsson

Chalmers University of Technology

slide-2
SLIDE 2

Department of Computer Engineering

The screen consists of pixels

slide-3
SLIDE 3

Department of Computer Engineering

Grafikkort

slide-4
SLIDE 4

Department of Computer Engineering

3D-Rendering

  • Objects are often made
  • f triangles
  • x,y,z- coordinate for each

vertex

Z X Y

slide-5
SLIDE 5

Department of Computer Engineering

4D Matrix Multiplication

ú ú ú ú û ù ê ê ê ê ë é ú ú ú ú û ù ê ê ê ê ë é

  • w

z y x t s t s t s

z z y y x x

1

slide-6
SLIDE 6

Department of Computer Engineering

Real-Time Rendering

slide-7
SLIDE 7

Department of Computer Engineering

+ =

l One application of texturing is to ”glue”

images onto geometrical object

Textures

slide-8
SLIDE 8

Department of Computer Engineering

Texturing: Glue images onto geometrical objects

  • Purpose: more realism, and this is a cheap way to do it

+ =

slide-9
SLIDE 9

Department of Computer Engineering

Light computation per triangle

light

slide-10
SLIDE 10

Department of Computer Engineering

More

Materials Reflections Shadows Fire Water Airlight

slide-11
SLIDE 11

Light Bounces

Typical test box (Cornell box), often compared to actual photograph:

slide-12
SLIDE 12

Light Bounces

0 bounces 1 bounce 2 bounces 4 bounces 8 bounces infinite bounces

slide-13
SLIDE 13

The Problem of Computer Graphics

  • Is not CG soon a solved problem?
  • Will not computers soon be fast enough?

Probably not…

~20 to ~1015 photons/s

The eye has a resolution of 130M receptors: 120M gray scale (rods / stavar) 7M color (cones / tappar)

slide-14
SLIDE 14
  • Eye sensitivity: ~20 photons/s to ~ 1015 photons/s
  • So, if we could trace only the photons that hit the eye, the

problem would be limited.

  • But, the only really guaranteed 100% correct way (still) is tracing

photons from light to eye.

photon

The problem of Computer Graphics

Sunny day outdoor scene: ~ 1021 photons/m2s

slide-15
SLIDE 15

Facts:

  • Eye sensitivity: ~20 to ~1015 photons/s
  • Sensitivity is logarithmic
  • i.e., difference between 100 or 200

photons is as noticable as for 1010 or 2*1010 photons

  • ~ 1021 photons/m2s
  • 1 photon costs ~10.000 cycles

10 billion years per square meter for 1 computer

slide-16
SLIDE 16

Solutions For games: Smart specialized algorithms. And cheat, cheat, cheat…. as long as it is not too noticable For movies: we typically trace ~100M – 10B photon bounces (and also cheat).

slide-17
SLIDE 17

Typically: 100M – 1B photon bounces

slide-18
SLIDE 18
slide-19
SLIDE 19
slide-20
SLIDE 20
slide-21
SLIDE 21
slide-22
SLIDE 22
slide-23
SLIDE 23
slide-24
SLIDE 24

Beäst + Unreal Engine

slide-25
SLIDE 25

Beast used in e.g.: WET (A2M) Mirror's Edge (EA Digital Illusions Creative Entertainment) Mortal Kombat (Midway) EVE Online (CCP Games) CrimeCraft (Vogster Entertainment LLC) Alpha Protocol (Obsidian Entertainment Inc) Dragon Age: Origins (BioWare) God of War III [8] (Sony Computer Entertainment) Gran Turismo (Polyphony Digital) and also the Unreal Engine

Illuminate Labs produkt “Beäst” för realistisk belystning i spel

slide-26
SLIDE 26

19 5 100 1 63 79 1 5 19 63 79 100 e.g. sorting:

Avalanche Studios Bosch Intel Fallout 4, NVIDIA

Our Research

Projects

Millions of lights (games)

Airlight (games)

Hair and Fur (games) Shadows (games) Scene compression Free Viewpoint Video GPGPU

Splinter Cell Sim City Infamous 2

slide-27
SLIDE 27

Beast + Unreal Engine

slide-28
SLIDE 28

Mix of graphics and photographing

Textures from photographs

slide-29
SLIDE 29
  • Istället för att modellera 3D-grafik och beräkna

realistiskt utseende:

– Fotografera/filma verkligheten och konvertera den till 3D-grafik.

  • Ofta vill man mixa modellerade och verkliga

konverterade objekt.

Mix of graphics and photographing

slide-30
SLIDE 30

http://www.cse.chalmers.se/~uffe/mindary/demo/v2.html

slide-31
SLIDE 31

Reflective surfaces (=view-dependent colors) are a problem

The color of a reflective surface is view dependent. So, what color should we use?

slide-32
SLIDE 32

1. View-dependent colors. (here also different exposure times for the two cameras). The more reflective surface, the larger the problem. “Solution”:

  • Supress reflections when

photographing.

  • Computer-generate the

reflections when visualizing the 3D scene. Very reflective surfaces (e.g. mirror) does not work.

slide-33
SLIDE 33

The most important 1st light bounces (i.e., sharp and glossy reflections)

slide-34
SLIDE 34

Unreal Engine 4

The most important 1st light bounces (i.e., sharp and glossy reflections) Combinig photo textures and computer-generated view- dependent reflections

slide-35
SLIDE 35

Increasing the amount of geometric details

slide-36
SLIDE 36

Triangles

slide-37
SLIDE 37

Voxels

Voxel Volume – element 1 bit Triangle 36 bytes

slide-38
SLIDE 38

Voxels

  • Desirable to be able to use very high resolutions
slide-39
SLIDE 39

Why use voxels? Autodesk fluid simulation

Voxels

slide-40
SLIDE 40

Voxels

RealFlow by nextLimit

slide-41
SLIDE 41

One possible data structure:

  • Grids – 3D array of

0:s and 1:s

Grid

Voxels

slide-42
SLIDE 42

One possible data structure:

Voxels

slide-43
SLIDE 43

Sparse Voxel Octree

0 1

1

Each node has eight children, representing an

  • ctant of the parent node’s volume.
slide-44
SLIDE 44

Sparse Voxel Octree

0 1

1

Each node has eight children, representing an

  • ctant of the parent node’s volume.
slide-45
SLIDE 45

Sparse Voxel Octree

  • SVO: Id Software, rage 6
  • 1.15 bits/ non-empty voxel
  • We: 0.08 bit/non-empty

voxel

slide-46
SLIDE 46

Voxels

  • Voxel = 1 bit.
  • We currently handle scene of res = 128.0003

– Naively: 262 TB – We => < 1GB

slide-47
SLIDE 47

Our Voxel DAGs

0 1

1

For identical subgraphs, only store one instance, and point to that instance.

slide-48
SLIDE 48

Visualizing Identical Subtrees

Epic Citadel

Resolution: 128K × 128K × 128K Number of nodes SVO: 5.5 billion DAG: 45 million (0.8%)

slide-49
SLIDE 49

Hairball

Resolution: 8K × 8K × 8K Number of nodes SVO: 781 million DAG: 44 million (5.6%) Identical colors are identical subvolumes of size 4 × 4 × 4

Visualizing Identical Subtrees

slide-50
SLIDE 50

Visualizing Identical Subtrees

Node occurrence 370 586 70 915 69 974 37 326 10 987 275 143

slide-51
SLIDE 51
slide-52
SLIDE 52
slide-53
SLIDE 53

Compress geometry and colors separately

with different specialized methods.

0 1

1

Geometry: Voxel colors: Three problems: 1. How can we compress the colors efficiently? 2. Connection between voxels and their colors 3. Fast color lookups

slide-54
SLIDE 54

From static scenes to dynamic (moving) scenes to Free Viewpoint Video

slide-55
SLIDE 55

frames

Voxel DAGs – dynamic scenes

3D geometry + time dimension

For every time step (=frame) in a dynamic scene, convert the whole voxelized 3D scene to a DAG: 70 frames 2048³ grid Length: 2.9 sec @24Hz 1.9 MiB (=2.3 GiB/hour) 5.2 Mbits/s

slide-56
SLIDE 56

Voxel DAGs – dynamic scenes

3D geometry + time dimension

70 frames 2048³ grid Length: 2.9 sec @24Hz 1.9 MiB (=2.3 GiB/hour) 5.2 Mbits/s

slide-57
SLIDE 57

Free Viewpoint Video

Want: 1. convert a real scene to 3D graphics, 24 times/second. 2. Render scene from any viewpoint.

slide-58
SLIDE 58

Med 2 eller fler kameror kan man beräkna djup – precis som våra ögon.

slide-59
SLIDE 59

Med 2 eller fler kameror kan man beräkna djup – precis som våra ögon.

slide-60
SLIDE 60

Med 2 eller fler kameror kan man beräkna djup – precis som våra ögon.

slide-61
SLIDE 61

Med 2 eller fler kameror kan man beräkna djup – precis som våra ögon. Varje kub ~1cm3. Önskar ~1mm3

slide-62
SLIDE 62

Free Viewpoint Video

480 frames 512³ grid 20 sec @24Hz 5.2 MiB 0.9 GiB/hour 2.1 Mbits/s

slide-63
SLIDE 63

500 photos, static scene. Precomputation time: hours We: 3 cameras, dynamic scene. Precomputation time: 5 minutes.

slide-64
SLIDE 64

Microsoft: ~100 cameras, triangles Precomputation time: ~100 hours.

slide-65
SLIDE 65

Our method:

  • Does not change or throw

away input data.

  • Higher geometrical detail.
  • Same data sizes.
  • Faster computation times
  • Much simpler and also robust
slide-66
SLIDE 66

But we still have no view-dependent colors. I.e., no reflections that change with the view position

slide-67
SLIDE 67

Deadpool (to demonstrate view-dependent reflections)

slide-68
SLIDE 68

Deadpool (to demonstrate desired quality of FVV with view-dependent reflections)

slide-69
SLIDE 69

Framtidens mediatekniker

  • Filma (4-10 kameror)
  • 3D-rekonstruera varje frame

– För varje tidssteg i filmen:

  • 3D rekonstruera scenen från

kamerornas foton.

  • Komprimera från TB till GB.

– streambar över internet

  • Spela upp filmen från valfri

synvinkel

– Dvs vi kan gå omkring i filmen medan den spelar.

slide-70
SLIDE 70

Framtidens mediatekniker

  • Science Fiction visar vägen

– Visar vad vi vill ha – Människan skaffar det hon vill ha (bland annat…)

slide-71
SLIDE 71

Star Trek

slide-72
SLIDE 72

Star Trek

60:ies 2000

slide-73
SLIDE 73

Star Trek - Tablets

80:ies 2010

slide-74
SLIDE 74

Star Trek - Holodeck

slide-75
SLIDE 75

Star Trek - Holodeck

slide-76
SLIDE 76

Star Trek

slide-77
SLIDE 77
slide-78
SLIDE 78

Oculus Rift HTC Vive

slide-79
SLIDE 79

Mid air display

slide-80
SLIDE 80

Mid air display

slide-81
SLIDE 81

Mid air displays 2015

slide-82
SLIDE 82

Mid air displays 2015

slide-83
SLIDE 83
slide-84
SLIDE 84
slide-85
SLIDE 85

Inget nytt under solen

  • Titta på Automan från 1983-1984.
slide-86
SLIDE 86

Inget nytt under solen

  • Titta på Automan från 1983-1984.
slide-87
SLIDE 87

Inget nytt under solen

  • Titta på Automan från 1983-1984.
slide-88
SLIDE 88
slide-89
SLIDE 89

The Future?

  • Much science fiction will become possible
  • We want to enter computer-generated virtual worlds

– Holodeck (maybe in a few decades) – Or plug into brain like in Matrix…

  • We want computer-generated objects to enter our real world

– 3D printers – Mid-air displays – Virtual matter (particles)