Image Based Lighting dr. Francesco Banterle - - PowerPoint PPT Presentation

▶
image based lighting
SMART_READER_LITE
LIVE PREVIEW

Image Based Lighting dr. Francesco Banterle - - PowerPoint PPT Presentation

Oct 30, 2023 454 likes •1.16k views

Image Based Lighting dr. Francesco Banterle francesco.banterle@isti.cnr.it Image Based Lighting: why? Image Based Lighting (IBL): To (re)light synthetic objects with real-world lighting Image Based Lighting: why? IBL is very

slide-1
SLIDE 1

Image Based Lighting

  • dr. Francesco Banterle

francesco.banterle@isti.cnr.it

slide-2
SLIDE 2

Image Based Lighting: why?

  • Image Based Lighting (IBL):
  • To (re)light synthetic objects with real-world

lighting

slide-3
SLIDE 3

Image Based Lighting: why?

  • IBL is very important:
  • advertisement: cars, forniture, etc.
  • visual effects: CGI, live motion, etc.
  • augmented reality
  • cultural heritage
slide-4
SLIDE 4

IBL: Capturing Lighting

  • The input of IBL is real-world lighting
  • HDR imaging is the key
  • A HDR photograph captures a limited portion of

light coming from the point of capture

slide-5
SLIDE 5

IBL: Capturing Lighting

slide-6
SLIDE 6

IBL: Capturing Lighting

  • Solution:
  • To capture HDR panoramic image 360x180
  • These images are typically called either

environment map or lightprobe

slide-7
SLIDE 7

IBL: Capturing Lighting

slide-8
SLIDE 8

IBL: Capturing Lighting

  • How capturing spherical (360x180) images?
  • Single shot panorama cameras:
  • SpheronVR: 50Mpix and 24 f-stops
  • iSTAR 360: 50Mpix and 27 f-stops
  • Roundshot: 160Mpix
  • These cameras may be expensive…
slide-9
SLIDE 9

IBL: Capturing Lighting

  • to capture a mirror sphere (e.g. xmas ball)
  • to capture a panoramic image from multiple directions and

exposure times. This requires post-processing; e.g. image stitching:

  • PTGui:
  • http://www.ptgui.com
  • Hugin (open source):
  • http://hugin.sourceforge.net/download/
slide-10
SLIDE 10

IBL: Capturing Lighting

slide-11
SLIDE 11

IBL: Capturing Lighting

slide-12
SLIDE 12

IBL: Capturing Lighting

0.289

slide-13
SLIDE 13

IBL: Capturing Lighting

0.289

slide-14
SLIDE 14

IBL: Capturing Lighting

0.289

slide-15
SLIDE 15

IBL: Capturing Lighting

0.289 0.537

slide-16
SLIDE 16

IBL: Capturing Lighting

x

θ

φ

x y z

slide-17
SLIDE 17

IBL: Longitude Latitude Mapping

D = 2 4 sin θ cos φ cos θ sin θ sin φ 3 5 θ = π ✓

1 2

  • 1 −

j height

  • − 1

2

◆ φ =

π2i width

θ

φ

slide-18
SLIDE 18

IBL: Longitude Latitude Mapping

  • Advantages:
  • easy mapping to understand/implement
  • Disadvantages:
  • not equal-area —>pixels cover different areas on the

sphere

  • squeezed at the poles —> to take this into account
slide-19
SLIDE 19

IBL: angular mapping

D =   cos φ sin θ sin φ sin θ − cos θ   φ = arctan(1 − 2y, 2x − 1) θ = π p (2x − 1)2 + (2y − 1)2

slide-20
SLIDE 20

IBL: angular mapping

  • Advantages:
  • avoiding undersampling at edges
  • Disadvantages:
  • not equal-area —>pixels cover different areas on the

sphere

  • a bit more complicated
slide-21
SLIDE 21

IBL: cube mapping

D = 1 p 1 + (2x − 1)2 + (2y − 1)2 2 4 2x − 1 2y − 1 1 3 5

x ∈ 1 3, 2 3

  • ∧ y ∈

1 2, 3 4

  • .

x y

slide-22
SLIDE 22

IBL: cube mapping

D = 1 p 1 + (2x − 1)2 + (2y − 1)2 2 4 2x − 1 2y − 1 1 3 5

x ∈ 1 3, 2 3

  • ∧ y ∈

1 2, 3 4

  • .

x y

slide-23
SLIDE 23

IBL: cube mapping

  • Advantages:
  • hardware support on the GPU
  • Disadvantages:
  • not equal-area (pixels are bigger at edges) —>pixels

cover different areas on the sphere

slide-24
SLIDE 24

… and now?

slide-25
SLIDE 25

Rendering

~ !i ~ n ~ !o x

slide-26
SLIDE 26

Rendering

Lo(x, ~ !o) = Le(x, ~ !o) + Z

Ω+ Li(x, ~

!i)fr(x, ~ !i, ~ !o)|~ n · ~ !i|d~ !i

slide-27
SLIDE 27

Rendering

slide-28
SLIDE 28

Rendering

Lo(x, ~ !o) = Le(x, ~ !o) + Z

Ω+ Li(~

!i)fr(x, ~ !i, ~ !o)|~ n · ~ !i|d~ !i

slide-29
SLIDE 29

Rendering

Lo(x, ~ !o) = Le(x, ~ !o) + Z

Ω+ Li(~

!i)fr(x, ~ !i, ~ !o)|~ n · ~ !i|d~ !i

slide-30
SLIDE 30

Rendering

  • How to solve this integral?
  • Creating light sources from the environment map
  • Sampling the environment map
slide-31
SLIDE 31

Light sources generation

  • Direction light sources are extracted from the

environment map.

  • Properties: direction, and HDR color
  • The integral is converted into:
  • Number of light sources is a parameter: more lights

more time. Few lights —> bias (integral not converged)

Lo(x, ~ !o) = Le(x, ~ !o) +

N

X

j=1

Lj

ifr(~

!j

i , ~

!o)|~ n · ~ !j

i |

slide-32
SLIDE 32

Light sources generation: uniform sampling

Subdivide the panorama in regular regions

slide-33
SLIDE 33

Light source generation: uniform sampling

Extracted light sources

slide-34
SLIDE 34

Light sources generation: median-cut sampling

Subdivide the panorama in regions of equal luminance

slide-35
SLIDE 35

Light sources generation: median-cut sampling

Subdivide the panorama in regions of equal luminance

slide-36
SLIDE 36

Light sources generation: median-cut sampling

Subdivide the panorama in regions of equal luminance

slide-37
SLIDE 37

Light sources generation: median-cut sampling

Subdivide the panorama in regions of equal luminance

slide-38
SLIDE 38

Light sources generation: median-cut sampling

Subdivide the panorama in regions of equal luminance

slide-39
SLIDE 39

Light sources generation: median-cut sampling

Subdivide the panorama in regions of equal luminance

slide-40
SLIDE 40

Light sources generation: median-cut sampling

Subdivide the panorama in regions of equal luminance

slide-41
SLIDE 41

Light sources generation: median-cut sampling

Subdivide the panorama in regions of equal luminance

slide-42
SLIDE 42

Light source generation: median-cut sampling

Extracted light sources

slide-43
SLIDE 43

Light source generation: median-cut sampling

Extracted light sources

slide-44
SLIDE 44

Light source generation: median-cut sampling

slide-45
SLIDE 45

Light source generation: median-cut sampling

slide-46
SLIDE 46

Sampling the Environment Map

  • Solving:
  • with monte-carlo methods; generating samples

according to a probability distribution:

  • Few samples —> noise

Lo(x, ~ !o) = Le(x, ~ !o) + 1 N

N

X

j=1

Li(~ !xj

i )fr(~

!xj

i , ~

!o)|~ n · ~ !xj

i |

p(xj) Lo(x, ~ !o) = Le(x, ~ !o) + Z

Ω+ Li(~

!i)fr(x, ~ !i, ~ !o)|~ n · ~ !i|d~ !i

slide-47
SLIDE 47

Sampling the Environment Map

slide-48
SLIDE 48

Sampling the Environment Map

slide-49
SLIDE 49

how to insert virtual

  • bjects?
slide-50
SLIDE 50

Differential Rendering

slide-51
SLIDE 51

Differential Rendering

slide-52
SLIDE 52

Differential Rendering

slide-53
SLIDE 53

Differential Rendering

slide-54
SLIDE 54

Differential Rendering

Images are courtesy of Karsch

slide-55
SLIDE 55

Differential Rendering

/

Synthetic Objects + Support Geometry Support Geometry Only

slide-56
SLIDE 56

Differential Rendering

Shadows to insert into the photograph (table)

slide-57
SLIDE 57

Differential Rendering

x

slide-58
SLIDE 58

Differential Rendering

slide-59
SLIDE 59

Differential Rendering

x

) (

+

slide-60
SLIDE 60

Differential Rendering

slide-61
SLIDE 61

there is more…

slide-62
SLIDE 62

nobody expects… the Spanish Inquisition

slide-63
SLIDE 63

Spatial IBL

  • A single environment can capture only distant light

sources

  • Nearby light sources are not modeled as local light

but as distant ones

  • To increase realism there is the need to model them

properly

slide-64
SLIDE 64

Spatial IBL

slide-65
SLIDE 65

Spatial IBL

slide-66
SLIDE 66

Spatial IBL

slide-67
SLIDE 67

Spatial IBL

slide-68
SLIDE 68

Spatial IBL

slide-69
SLIDE 69

Questions?