Models and Analyses for Image Synthesis Cyril Soler INRIA, LJK, - - PowerPoint PPT Presentation

▶
models and analyses for image synthesis
SMART_READER_LITE
LIVE PREVIEW

Models and Analyses for Image Synthesis Cyril Soler INRIA, LJK, - - PowerPoint PPT Presentation

Feb 29, 2024 264 likes •1.31k views

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion Models and Analyses for Image Synthesis Cyril Soler INRIA, LJK, Grenoble University C.Soler (INRIA) Models and Analyses for Image Synthesis

slide-1
SLIDE 1

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Models and Analyses for Image Synthesis

Cyril Soler

INRIA, LJK, Grenoble University

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 1 / 28

slide-2
SLIDE 2

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Outline

◮ Introduction ◮ 2 selected contributions

◮ Automatic instancing ◮ Fourier analysis of light transport

◮ ongoing / future work, and conclusion

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 2 / 28

slide-3
SLIDE 3

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Motivation 1/2

◮ movies

[Game of Thrones, HBO] C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 3 / 28

slide-4
SLIDE 4

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Motivation 1/2

◮ movies

[Game of Thrones, HBO] C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 3 / 28

slide-5
SLIDE 5

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Motivation 1/2

◮ movies

[Game of Thrones, HBO]

◮ create the models → geometry modeling, acquisition, processing

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 3 / 28

slide-6
SLIDE 6

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Motivation 1/2

◮ movies

[Game of Thrones, HBO]

◮ create the models → geometry modeling, acquisition, processing ◮ create materials → acquisition, filtering, reconstruction

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 3 / 28

slide-7
SLIDE 7

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Motivation 1/2

◮ movies

[Game of Thrones, HBO]

◮ create the models → geometry modeling, acquisition, processing ◮ create materials → acquisition, filtering, reconstruction ◮ compute lighting → solve an integral equation!

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 3 / 28

slide-8
SLIDE 8

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Motivation 1/2

◮ movies ◮ video games

[FarCry 3] C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 3 / 28

slide-9
SLIDE 9

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Motivation 1/2

◮ movies ◮ video games

[FarCry 3]

◮ tons of data → procedural modeling

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 3 / 28

slide-10
SLIDE 10

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Motivation 1/2

◮ movies ◮ video games

[FarCry 3]

◮ tons of data → procedural modeling ◮ everything moves → mechanics, fluid simulation

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 3 / 28

slide-11
SLIDE 11

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Motivation 1/2

◮ movies ◮ video games

[FarCry 3]

◮ tons of data → procedural modeling ◮ everything moves → mechanics, fluid simulation ◮ compute illumination in real time → needs some magic!

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 3 / 28

slide-12
SLIDE 12

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Motivation 1/2

◮ movies ◮ video games ◮ virtual prototyping

[J.Kˇ rivánek] C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 3 / 28

slide-13
SLIDE 13

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Motivation 1/2

◮ movies ◮ video games ◮ virtual prototyping

[J.Kˇ rivánek]

◮ needs quantitative accuracy → unbiased simulation

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 3 / 28

slide-14
SLIDE 14

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Motivation 2/2

Virtual prototyping Movie special effects Video games Scientific visualisation Virtual reality

Realism / Accuracy Speed

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 4 / 28

slide-15
SLIDE 15

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Motivation 2/2

Virtual prototyping Movie special effects Video games Scientific visualisation Virtual reality

Realism / Accuracy Speed

Ultimate goals:

◮ push speed and realism/accuracy, as far as possible

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 4 / 28

slide-16
SLIDE 16

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Motivation 2/2

Virtual prototyping Movie special effects Video games Scientific visualisation Virtual reality

Realism / Accuracy Speed

Ultimate goals:

◮ push speed and realism/accuracy, as far as possible ◮ provide scalability

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 4 / 28

slide-17
SLIDE 17

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

So, what do we do?

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 5 / 28

slide-18
SLIDE 18

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

So, what do we do?

  • 1. carefully simplify the physics

Real sapphire Real time simulation

[S.Guy, C.Soler. Siggraph 2004] C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 5 / 28

slide-19
SLIDE 19

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

So, what do we do?

  • 1. carefully simplify the physics
  • 2. choose the right math

Using Spherical Harmonics Using Zonal Harmonics 1300 ms 45ms

[Submitted to SIG Asia 2014. Col. with UDeM] C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 5 / 28

slide-20
SLIDE 20

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

So, what do we do?

  • 1. carefully simplify the physics
  • 2. choose the right math
  • 3. design efficient algorithms

[C.Soler, M.Bagher,... IEEE TVCG’2013] C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 5 / 28

slide-21
SLIDE 21

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

So, what do we do?

  • 1. carefully simplify the physics
  • 2. choose the right math
  • 3. design efficient algorithms
  • 4. play with the hardware

Direct illumination + Screen-space indirect illumination (8 ms)

[C.Soler, O.Hoel. FGO’2011] C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 5 / 28

slide-22
SLIDE 22

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Connexions to other scientific domains

◮ applied mathematics ◮ physics ◮ computational geometry ◮ signal processing ◮ perception

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 6 / 28

slide-23
SLIDE 23

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Contributions

Acquisition Modeling Data processing

Light transport

Simulation Display

Computational geometry Image manipulation Texture-based visibility SIG'98, TOG'00

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 7 / 28

slide-24
SLIDE 24

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Contributions

Acquisition Modeling Data processing

Light transport

Simulation Display

Computational geometry Plant growth simulation RFPT'03, TOG'03 Remote sensing RAMI'07 Image manipulation Geometry instantiation EGSR'00, SgpP'05, TOG'06 Texture-based visibility SIG'98, TOG'00

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 7 / 28

slide-25
SLIDE 25

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Contributions

Acquisition Modeling Data processing

Light transport

Simulation Display

Computational geometry Texture synthesis SIG'02 Plant growth simulation RFPT'03, TOG'03 Remote sensing RAMI'07 Texture analysis AFIG'08 Image manipulation Geometry instantiation EGSR'00, SgpP'05, TOG'06 Texture-based visibility SIG'98, TOG'00

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 7 / 28

slide-26
SLIDE 26

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Contributions

Acquisition Modeling Data processing

Light transport

Simulation Display

Computational geometry Real time rendering SIG'04, SigT'10, SigP'11, I3D'12, TVCG'13 Texture synthesis SIG'02 Plant growth simulation RFPT'03, TOG'03 Remote sensing RAMI'07 Texture analysis AFIG'08 Image manipulation Fourier analysis of light transport SIG'05,TOG'09,SigP'11,TOG'13,TOG'14 Geometry instantiation EGSR'00, SgpP'05, TOG'06 Texture-based visibility SIG'98, TOG'00

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 7 / 28

slide-27
SLIDE 27

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Contributions

Acquisition Modeling Data processing

Light transport

Simulation Display

Computational geometry Real time rendering SIG'04, SigT'10, SigP'11, I3D'12, TVCG'13 Texture synthesis SIG'02 Plant growth simulation RFPT'03, TOG'03 Remote sensing RAMI'07 Texture analysis AFIG'08 Image manipulation Fourier analysis of light transport SIG'05,TOG'09,SigP'11,TOG'13,TOG'14 Image decomposition SIG'09, CGF'13 Geometry instantiation EGSR'00, SgpP'05, TOG'06 Texture-based visibility SIG'98, TOG'00 Filtering SigP'11

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 7 / 28

slide-28
SLIDE 28

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Contributions

Acquisition Modeling Data processing

Light transport

Simulation Display

Computational geometry Real time rendering SIG'04, SigT'10, SigP'11, I3D'12, TVCG'13 Texture synthesis SIG'02 Plant growth simulation RFPT'03, TOG'03 Remote sensing RAMI'07 Texture analysis AFIG'08 BRDF modeling CGF'12 Image manipulation Fourier analysis of light transport SIG'05,TOG'09,SigP'11,TOG'13,TOG'14 Image decomposition SIG'09, CGF'13 Geometry instantiation EGSR'00, SgpP'05, TOG'06 Texture-based visibility SIG'98, TOG'00 Filtering SigP'11 BRDF acquisition SigP'13

◮ 11 papers in international journals (10 at TOG/Siggraph) ◮ 8 papers in international conferences (I3D,EG,EGSR)

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 7 / 28

slide-29
SLIDE 29

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Collaborations/Projects

2005 2006 2007 2008 2009 2011 2012 2013 2014 2004 1999 2000 2001 2002 2003 2010

MIT Cornell UDeM UCL Noveltis LIAMA CIRAD Daimler++ Eden Games++

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 8 / 28

slide-30
SLIDE 30

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Collaborations/Projects

2005 2006 2007 2008 2009 2011 2012 2013 2014 2004 1999 2000 2001 2002 2003 2010

MIT Cornell

PE.Landes

Texture analysis

M.Bagher

Fourier for RT rendering

L.Belcour

Spectral covariance

B.Zupancic A.Martinet

Instanciation BRDF C.Sensing

UDeM UCL Noveltis LIAMA CIRAD Daimler++ Eden Games++

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 8 / 28

slide-31
SLIDE 31

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Collaborations/Projects

2005 2006 2007 2008 2009 2011 2012 2013 2014 2004 1999 2000 2001 2002 2003 2010

ALTA Soleil Garden Genac II RealReflect

(IST European project) (ANR project) (FUI project) (FUI project) (ARC INRIA)

MIT Cornell

PE.Landes

Texture analysis

M.Bagher

Fourier for RT rendering

L.Belcour

Spectral covariance

B.Zupancic A.Martinet

Instanciation BRDF C.Sensing

UDeM UCL Noveltis LIAMA CIRAD Daimler++ Eden Games++

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 8 / 28

slide-32
SLIDE 32

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Collaborators

Research collaborators:

P .Dereffye MP .Cani F.Sillion F.Durand J.Thollot N.Holszchuch K.Subr S.Paris J.Kautz K.Bala D.Nowrouz- ezahrai

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 9 / 28

slide-33
SLIDE 33

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Collaborators

Research collaborators:

P .Dereffye MP .Cani F.Sillion F.Durand J.Thollot N.Holszchuch K.Subr S.Paris J.Kautz K.Bala D.Nowrouz- ezahrai

Past (and current) PhD students:

A.Martinet P-E.Landes M.Bagher L.Belcour B.Zupancic

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 9 / 28

slide-34
SLIDE 34

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Collaborators

Research collaborators:

P .Dereffye MP .Cani F.Sillion F.Durand J.Thollot N.Holszchuch K.Subr S.Paris J.Kautz K.Bala D.Nowrouz- ezahrai

Past (and current) PhD students:

A.Martinet P-E.Landes M.Bagher L.Belcour B.Zupancic

Assistants, engineers, co-authors:

D.Courtiol L.Boissieux O.Hoel I.Delore S.Guy K.Smith JC.Roche

...

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 9 / 28

slide-35
SLIDE 35

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Outline

◮ Introduction ◮ 2 selected contributions

◮ Automatic instancing ◮ Fourier analysis of light transport

◮ ongoing / future work, and conclusion

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 10 / 28

slide-36
SLIDE 36

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Problem statement

◮ origins of the problem:

◮ radiative exchanges in vegetation scenes [Soler, et al. TOG2003] [Soler, Sillion, Blaise, Dereffye. TOG 2003] C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 11 / 28

slide-37
SLIDE 37

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Problem statement

◮ origins of the problem:

◮ radiative exchanges in vegetation scenes [Soler, et al. TOG2003] [RAMI challenge website] C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 11 / 28

slide-38
SLIDE 38

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Problem statement

◮ origins of the problem:

◮ radiative exchanges in vegetation scenes [Soler, et al. TOG2003] ◮ compress geometry by instantiation [Martinet, Soler, et al. TOG2006] C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 11 / 28

slide-39
SLIDE 39

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Problem statement

◮ origins of the problem:

◮ radiative exchanges in vegetation scenes [Soler, et al. TOG2003] ◮ compress geometry by instantiation [Martinet, Soler, et al. TOG2006] ◮ in practice most scenes come without structure

Raw geometry

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 11 / 28

slide-40
SLIDE 40

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Instantiation algorithm

◮ Instantiation algorithm

◮ build tiles from polygon soup (See e.g. [Krayevoy’2006, Katz’2005] ) Raw geometry Tiles +tiles symmetry Hierarchical represention + symmetries C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 12 / 28

slide-41
SLIDE 41

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Instantiation algorithm

◮ Instantiation algorithm

◮ build tiles from polygon soup (See e.g. [Krayevoy’2006, Katz’2005] ) ◮ group hierarchically using: ⋆ a rotation-free similarity distance ⋆ and a symmetry detection algorithm Raw geometry Tiles +tiles symmetry Hierarchical represention + symmetries C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 12 / 28

slide-42
SLIDE 42

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Instantiation algorithm

◮ Instantiation algorithm

◮ build tiles from polygon soup (See e.g. [Krayevoy’2006, Katz’2005] ) ◮ group hierarchically using: ⋆ a rotation-free similarity distance ⋆ and a symmetry detection algorithm

◮ existing methods (e.g. [Kahzdan’2002]: brute force)

Raw geometry Tiles +tiles symmetry Hierarchical represention + symmetries C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 12 / 28

slide-43
SLIDE 43

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Instantiation algorithm

◮ Instantiation algorithm

◮ build tiles from polygon soup (See e.g. [Krayevoy’2006, Katz’2005] ) ◮ group hierarchically using: ⋆ a rotation-free similarity distance ⋆ and a symmetry detection algorithm

◮ existing methods (e.g. [Kahzdan’2002]: brute force) ◮ we used spherical harmonics!

Raw geometry Tiles +tiles symmetry Hierarchical represention + symmetries C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 12 / 28

slide-44
SLIDE 44

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Symmetry computation M2p(ω) =

  • s∈S

s × ω2pds (Generalized moment)

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 13 / 28

slide-45
SLIDE 45

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Symmetry computation M2p(ω) =

  • s∈S

s × ω2pds (Generalized moment) – For all p, all symmetries of S are symmetries of M2p

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 13 / 28

slide-46
SLIDE 46

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Symmetry computation M2p(ω) =

  • s∈S

s × ω2pds (Generalized moment) – For all p, all symmetries of S are symmetries of M2p – If ω is a symmetry axis of S, then ∇M2p(ω) = 0

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 13 / 28

slide-47
SLIDE 47

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Symmetry computation M2p(ω) =

  • s∈S

s × ω2pds (Generalized moment) – For all p, all symmetries of S are symmetries of M2p – If ω is a symmetry axis of S, then ∇M2p(ω) = 0 – M2p has a finite spherical harmonics decomposition !

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 13 / 28

slide-48
SLIDE 48

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Symmetry computation (2/2) Symmetry detection algorithm:

◮ decompose M2p into spherical harmonics

cm

l

=

  • Ω

M2p(ω)ym

l (ω)dω

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 14 / 28

slide-49
SLIDE 49

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Symmetry computation (2/2) Symmetry detection algorithm:

◮ decompose M2p into spherical harmonics

cm

l

= Sl

p

  • s∈S

s2pym

2l (s)ds

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 14 / 28

slide-50
SLIDE 50

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Symmetry computation (2/2) Symmetry detection algorithm:

◮ decompose M2p into spherical harmonics ◮ compute ∇M2p for all p analytically

M2p =

  • l,m

cm

l ym l

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 14 / 28

slide-51
SLIDE 51

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Symmetry computation (2/2) Symmetry detection algorithm:

◮ decompose M2p into spherical harmonics ◮ compute ∇M2p for all p analytically

∇M2p =

  • l,m

cm

l ∇ym l

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 14 / 28

slide-52
SLIDE 52

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Symmetry computation (2/2) Symmetry detection algorithm:

◮ decompose M2p into spherical harmonics ◮ compute ∇M2p for all p analytically ◮ look for the zeroes, this gives the axes of the symmetries

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 14 / 28

slide-53
SLIDE 53

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Symmetry computation (2/2) Symmetry detection algorithm:

◮ decompose M2p into spherical harmonics ◮ compute ∇M2p for all p analytically ◮ look for the zeroes, this gives the axes of the symmetries ◮ for each axis, align M2p with this axis using SH rotation.

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 14 / 28

slide-54
SLIDE 54

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Symmetry computation (2/2) Symmetry detection algorithm:

◮ decompose M2p into spherical harmonics ◮ compute ∇M2p for all p analytically ◮ look for the zeroes, this gives the axes of the symmetries ◮ for each axis, align M2p with this axis using SH rotation.

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 14 / 28

slide-55
SLIDE 55

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Symmetry computation (2/2) Symmetry detection algorithm:

◮ decompose M2p into spherical harmonics ◮ compute ∇M2p for all p analytically ◮ look for the zeroes, this gives the axes of the symmetries ◮ for each axis, align M2p with this axis using SH rotation. ◮ compute the symmetry angle/sign

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 14 / 28

slide-56
SLIDE 56

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Similarity distance

◮ compute the L2 square norm m(cm l )2 ◮ concatenate for all p

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 15 / 28

slide-57
SLIDE 57

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Similarity distance

◮ compute the L2 square norm m(cm l )2 ◮ concatenate for all p

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 15 / 28

slide-58
SLIDE 58

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Results

[A.Martinet, C.Soler,... ACM TOG 2006] C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 16 / 28

slide-59
SLIDE 59

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Outline

◮ Introduction ◮ 2 selected contributions

◮ Automatic instancing ◮ Fourier analysis of light transport

◮ ongoing / future work, and conclusion

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 17 / 28

slide-60
SLIDE 60

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Monte-Carlo Light Transport

◮ collect light bouncing in a scene toward a captor

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 18 / 28

slide-61
SLIDE 61

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Monte-Carlo Light Transport

◮ collect light bouncing in a scene toward a captor

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 18 / 28

slide-62
SLIDE 62

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Monte-Carlo Light Transport

◮ collect light bouncing in a scene toward a captor

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 18 / 28

slide-63
SLIDE 63

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Monte-Carlo Light Transport

◮ collect light bouncing in a scene toward a captor

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 18 / 28

slide-64
SLIDE 64

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Monte-Carlo Light Transport

◮ collect light bouncing in a scene toward a captor

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 18 / 28

slide-65
SLIDE 65

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Monte-Carlo Light Transport

◮ collect light bouncing in a scene toward a captor

L(p) =

  • x∈P

C(x, p)dx

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 18 / 28

slide-66
SLIDE 66

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Monte-Carlo Light Transport

◮ collect light bouncing in a scene toward a captor

L(p) =

  • x∈P

C(x, p)dx L(p) ≈ 1 N

N

  • i=1

C(x, p) P(x)

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 18 / 28

slide-67
SLIDE 67

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Monte-Carlo Light Transport

◮ collect light bouncing in a scene toward a captor ◮ many variants! (MLT, Photon Mapping, BDPT, ...)

L(p) =

  • x∈P

C(x, p)dx L(p) ≈ 1 N

N

  • i=1

C(x, p) P(x)

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 18 / 28

slide-68
SLIDE 68

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Monte-Carlo Light Transport

◮ collect light bouncing in a scene toward a captor ◮ many variants! (MLT, Photon Mapping, BDPT, ...)

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 19 / 28

slide-69
SLIDE 69

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Monte-Carlo Light Transport

◮ collect light bouncing in a scene toward a captor ◮ many variants! (MLT, Photon Mapping, BDPT, ...)

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 19 / 28

slide-70
SLIDE 70

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Monte-Carlo Light Transport

◮ collect light bouncing in a scene toward a captor ◮ many variants! (MLT, Photon Mapping, BDPT, ...) ◮ eventually the image can be very smooth.

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 19 / 28

slide-71
SLIDE 71

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Monte-Carlo Light Transport

◮ collect light bouncing in a scene toward a captor ◮ many variants! (MLT, Photon Mapping, BDPT, ...) ◮ eventually the image can be very smooth.

Can we predict in advance–and leverage–image smoothness?

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 19 / 28

slide-72
SLIDE 72

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Let’s use Fourier analysis!

◮ the local Fourier spectrum in the image tells us a lot!

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 20 / 28

slide-73
SLIDE 73

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Let’s use Fourier analysis!

◮ the local Fourier spectrum in the image tells us a lot!

◮ image-space sampling densities (2D spatial component, image) [Soler, Subr, et al. TOG’2009] C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 20 / 28

slide-74
SLIDE 74

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Let’s use Fourier analysis!

◮ the local Fourier spectrum in the image tells us a lot!

◮ image-space sampling densities (2D spatial component, image) ◮ light-space variance (2D angular component, scene) [Soler, Subr, et al. TOG’2009] C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 20 / 28

slide-75
SLIDE 75

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Let’s use Fourier analysis!

◮ the local Fourier spectrum in the image tells us a lot!

◮ image-space sampling densities (2D spatial component, image) ◮ light-space variance (2D angular component, scene) ◮ image-space reconstruction filters (4D/5D spectrum, scene) [Belcour, Soler, et al. TOG’2011] C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 20 / 28

slide-76
SLIDE 76

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Let’s use Fourier analysis!

◮ the local Fourier spectrum in the image tells us a lot!

◮ image-space sampling densities (2D spatial component, image) ◮ light-space variance (2D angular component, scene) ◮ image-space reconstruction filters (4D/5D spectrum, scene) ◮ density estimation kernels (2D spatial component, scene) [Belcour, Soler. Siggraph’2012 poster] C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 20 / 28

slide-77
SLIDE 77

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

So, can we predict the local image spectrum??

◮ spectra are additive!

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 21 / 28

slide-78
SLIDE 78

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

So, can we predict the local image spectrum??

◮ spectra are additive! ◮ add up spectra of local light fields along light paths

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 21 / 28

slide-79
SLIDE 79

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

So, can we predict the local image spectrum??

◮ spectra are additive! ◮ add up spectra of local light fields along light paths ◮ ...and analyze the effect of the various events Primal space

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 21 / 28

slide-80
SLIDE 80

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

So, can we predict the local image spectrum??

◮ spectra are additive! ◮ add up spectra of local light fields along light paths ◮ ...and analyze the effect of the various events Primal space

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 21 / 28

slide-81
SLIDE 81

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

So, can we predict the local image spectrum??

◮ spectra are additive! ◮ add up spectra of local light fields along light paths ◮ ...and analyze the effect of the various events Primal space Fourier space

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 21 / 28

slide-82
SLIDE 82

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

So, can we predict the local image spectrum??

◮ spectra are additive! ◮ add up spectra of local light fields along light paths ◮ ...and analyze the effect of the various events Primal space Fourier space

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 21 / 28

slide-83
SLIDE 83

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

So, can we predict the local image spectrum??

◮ spectra are additive! ◮ add up spectra of local light fields along light paths ◮ ...and analyze the effect of the various events Occlusion Convolution Primal space Fourier space

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 21 / 28

slide-84
SLIDE 84

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

So, can we predict the local image spectrum??

◮ spectra are additive! ◮ add up spectra of local light fields along light paths ◮ ...and analyze the effect of the various events Occlusion Convolution Primal space Fourier space

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 21 / 28

slide-85
SLIDE 85

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

So, can we predict the local image spectrum??

◮ spectra are additive! ◮ add up spectra of local light fields along light paths ◮ ...and analyze the effect of the various events Space travel Occlusion Shear Convolution Primal space Fourier space

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 21 / 28

slide-86
SLIDE 86

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

So, can we predict the local image spectrum??

◮ spectra are additive! ◮ add up spectra of local light fields along light paths ◮ ...and analyze the effect of the various events Space travel Reflectance Occlusion Shear Angular product Convolution Primal space Fourier space

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 21 / 28

slide-87
SLIDE 87

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

So, can we predict the local image spectrum??

◮ spectra are additive! ◮ add up spectra of local light fields along light paths ◮ ...and analyze the effect of the various events b Space travel Reflectance Occlusion Shear Angular product Convolution Primal space Fourier space

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 21 / 28

slide-88
SLIDE 88

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

So, can we predict the local image spectrum??

◮ spectra are additive! ◮ add up spectra of local light fields along light paths ◮ ...and analyze the effect of the various events ◮ we need a good representation for this b Space travel Reflectance Occlusion Shear Angular product Convolution Primal space Fourier space

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 21 / 28

slide-89
SLIDE 89

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

So, can we predict the local image spectrum??

◮ spectra are additive! ◮ add up spectra of local light fields along light paths ◮ ...and analyze the effect of the various events ◮ we need a good representation for this b Space travel Reflectance Occlusion Shear Angular product Convolution Primal space Fourier space Spectral covariance (4x4 matrix!)

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 21 / 28

slide-90
SLIDE 90

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

So, can we predict the local image spectrum??

◮ spectra are additive! ◮ add up spectra of local light fields along light paths ◮ ...and analyze the effect of the various events ◮ we need a good representation for this b Space travel Reflectance Occlusion Shear Angular product Convolution Primal space Fourier space Spectral covariance (4x4 matrix!) Left-right product Sum of matrices Product + p-inverse

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 21 / 28

slide-91
SLIDE 91

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Scattering and Absorption [TOG’2013]

◮ Fourier analysis:

◮ absorption increases bandwidth ◮ scattering reduces bandwidth

Σ′ = Σ + A Σ′ =

  • Σ−1 + S

−1

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 22 / 28

slide-92
SLIDE 92

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Scattering and Absorption [TOG’2013]

◮ Fourier analysis:

◮ absorption increases bandwidth ◮ scattering reduces bandwidth

◮ Derived prediction formulas for

◮ 3D frequency spectrum of the fluence ◮ reconstruction kernels for progressive photon beams [To be presented at Sig’2014] C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 22 / 28

slide-93
SLIDE 93

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Selected contributions

◮ Introduction ◮ 2 selected contributions ◮ Ongoing/Future work, and Conclusion

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 23 / 28

slide-94
SLIDE 94

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Ongoing work: spherical filtering

◮ idea: efficient filtering using isotropic function bases

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 24 / 28

slide-95
SLIDE 95

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Ongoing work: spherical filtering

◮ idea: efficient filtering using isotropic function bases ◮ lots of possible applications:

◮ rendering measured materials in real time [Brush alum. MERL database] C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 24 / 28

slide-96
SLIDE 96

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Ongoing work: spherical filtering

◮ idea: efficient filtering using isotropic function bases ◮ lots of possible applications:

◮ rendering measured materials in real time ◮ anti-aliasing sub-pixel curvature C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 24 / 28

slide-97
SLIDE 97

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Ongoing work: spherical filtering

◮ idea: efficient filtering using isotropic function bases ◮ lots of possible applications:

◮ rendering measured materials in real time ◮ anti-aliasing sub-pixel curvature ◮ efficient reconstruction for 4D BRDFs ◮ lightfield illumination [Purple satin. MERL database] C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 24 / 28

slide-98
SLIDE 98

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Ongoing work: Compressed Material Acquisition

◮ Idea: Use compressive sensing for BRDF acquisition

◮ BRDFs are very sparse signals ◮ according to E.Candès work, a few photographs should be

sufficient to recover a BRDF

◮ 1 PhD student working on it (Benoit Zupancic)

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 25 / 28

slide-99
SLIDE 99

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Future work: Dimensional analysis of light transport

◮ reflectance, visibility, scattering,...

− → are all linear operators in light field space

◮ eigen analysis will probably provide:

◮ optimal representation bases (made of harmonic functions!) ◮ new computation methods C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 26 / 28

slide-100
SLIDE 100

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Future work: Dimensional analysis of light transport

◮ reflectance, visibility, scattering,...

− → are all linear operators in light field space

◮ eigen analysis will probably provide:

◮ optimal representation bases (made of harmonic functions!) ◮ new computation methods

Λ0 Λ1 αΛ0 + TLe

Best Ambient Illum. Plausible rendering

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 26 / 28

slide-101
SLIDE 101

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Conclusion

◮ a very active and connected scientific domain ◮ most of the time: chose the right mathematical tool ◮ interesting perspectives ◮ all my contributions have needed several iterations to publish

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 27 / 28

slide-102
SLIDE 102

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Conclusion

◮ a very active and connected scientific domain ◮ most of the time: chose the right mathematical tool ◮ interesting perspectives ◮ all my contributions have needed several iterations to publish

while(!succeed=try());

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 27 / 28

slide-103
SLIDE 103

Introduction Automatic instancing (2006) Fourier Analysis of Light Transport (2005-2014) Conclusion

Questions? Thank you!

C.Soler (INRIA) Models and Analyses for Image Synthesis June 24, 2014 28 / 28