by Discarding Sample Taps Robert Toth Intel, Advanced Rendering - - PowerPoint PPT Presentation

β–Ά
by discarding sample taps
SMART_READER_LITE
LIVE PREVIEW

by Discarding Sample Taps Robert Toth Intel, Advanced Rendering - - PowerPoint PPT Presentation

Mar 17, 2024 282 likes β€’966 views

Avoiding Texture Seams by Discarding Sample Taps Robert Toth Intel, Advanced Rendering Technology 2014-03-15 1 Part I Seams: Atlases and Ptex Texture Seams Atlas Source: Microsoft DirectX SDK Visual & Parallel Computing Group 3

slide-1
SLIDE 1

1

Avoiding Texture Seams by Discarding Sample Taps

Robert Toth Intel, Advanced Rendering Technology 2014-03-15

slide-2
SLIDE 2

Part I Seams: Atlases and Ptex

slide-3
SLIDE 3

3

Visual & Parallel Computing Group

Texture Seams

Atlas

Source: Microsoft DirectX SDK

slide-4
SLIDE 4

4

Visual & Parallel Computing Group

Texture Seams

Atlas

Source: Microsoft DirectX SDK

Connected Connected Connected

slide-5
SLIDE 5

5

Visual & Parallel Computing Group

Texture Seams

Atlas

Source: Microsoft DirectX SDK

Texture filter

slide-6
SLIDE 6

6

Visual & Parallel Computing Group

Texture Seams

Ptex

slide-7
SLIDE 7

7

Visual & Parallel Computing Group

Texture Seams

Ptex Connected Connected

slide-8
SLIDE 8

8

Visual & Parallel Computing Group

Texture Seams

Ptex Texture filter

slide-9
SLIDE 9

9

Visual & Parallel Computing Group

Realtime Ptex implementations

Algo gorith ithm Wide de filt lter Mem emory

  • ry

Loo

  • okup

kups McDonald, Burley: SIGGRAPH 2011

Real-time Ptex (Per-Face Texture Mapping)

Yes Large 1 Kim, Hillesland, Hensley: SIGGRAPH Asia 2011

A Space-efficient and hardware-friendly Implementation of Ptex

No Small 1 McDonald: GDC 2013

Eliminating Texture Waste: Borderless Ptex

Yes* Small 5/10 *: over edges only, not corners

slide-10
SLIDE 10

Part II Analysis: What, and Why?

slide-11
SLIDE 11

11

Visual & Parallel Computing Group

Goal

Need to determine pixel colors

  • Scene is a continuous signal
  • Display has finite number of pixels
slide-12
SLIDE 12

12

Visual & Parallel Computing Group

Two interpretations

Interpretation 1:

  • This is a sampling and reconstruction problem!

Interpretation 2:

  • This is an error minimization problem!

Regardless: integrate scene modulated by filter function, f(x)

slide-13
SLIDE 13

13

Visual & Parallel Computing Group

Integrate

𝑄 = 𝑒𝑗 𝒗𝑗 π’š 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

slide-14
SLIDE 14

14

Visual & Parallel Computing Group

Integrate

𝑄 = 𝑒𝑗 𝒗𝑗 π’š 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

Pixel color value

slide-15
SLIDE 15

15

Visual & Parallel Computing Group

Integrate

𝑄 = 𝑒𝑗 𝒗𝑗 π’š 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

Integrate over each contributing surface

slide-16
SLIDE 16

16

Visual & Parallel Computing Group

Integrate

Pixel filter function 𝑄 = 𝑒𝑗 𝒗𝑗 π’š 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

slide-17
SLIDE 17

17

Visual & Parallel Computing Group

Integrate

Texture Texcoords 𝑄 = 𝑒𝑗 𝒗𝑗 π’š 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

slide-18
SLIDE 18

18

Visual & Parallel Computing Group

Integrate

𝑄 = 𝑒𝑗 𝒗𝑗 π’š 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

slide-19
SLIDE 19

19

Visual & Parallel Computing Group

Integrate

𝑄′ = 𝑑𝑗 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

MSAA: 𝑄 = 𝑒𝑗 𝒗𝑗 π’š 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

slide-20
SLIDE 20

20

Visual & Parallel Computing Group

Integrate

Resolve filter MSAA: Many† visibility samples

  • > Riemann integral

† a few

𝑄 = 𝑒𝑗 𝒗𝑗 π’š 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

𝑄′ = 𝑑𝑗 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

slide-21
SLIDE 21

21

Visual & Parallel Computing Group

Integrate

Pixel shader MSAA: Pass the problem on to the developer οƒ  Pass the problem on to the texture sampler 𝑄 = 𝑒𝑗 𝒗𝑗 π’š 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

𝑄′ = 𝑑𝑗 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

slide-22
SLIDE 22

22

Visual & Parallel Computing Group

Solution

𝑄′ = 𝑑𝑗 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

slide-23
SLIDE 23

23

Visual & Parallel Computing Group

Solution: Simple

𝑑𝑗

𝑇 = 𝑒𝑗 𝒗𝑗 π’š

𝑔 π’š π‘’π’š

Ω𝑄

𝑄′ = 𝑑𝑗 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

slide-24
SLIDE 24

24

Visual & Parallel Computing Group

Solution: Simple

𝑑𝑗

𝑇 = 𝑒𝑗 𝒗𝑗 π’š

𝑔 π’š π‘’π’š

Ω𝑄

𝑄′ = 𝑑𝑗 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

Extrapolation

slide-25
SLIDE 25

25

Visual & Parallel Computing Group

Solution: Simple

𝑑𝑗

𝑇 = 𝑒𝑗 𝒗𝑗 π’š

𝑔 π’š π‘’π’š

Ω𝑄

𝑄′ = 𝑑𝑗 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

𝑄′ = 𝑑𝑗

𝑇 𝑔 π’š π‘’π’š Ω𝑗 𝑗

β‰  𝑄

slide-26
SLIDE 26

26

Visual & Parallel Computing Group

Solution: Traverse

𝑑𝑗

π‘ˆ = π‘’π‘˜ π’—π‘˜ π’š

𝑔 π’š π‘’π’š

Ξ©π‘˜ π‘˜

= 𝑄

𝑄′ = 𝑑𝑗 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

slide-27
SLIDE 27

27

Visual & Parallel Computing Group

Solution: Traverse

𝑑𝑗

π‘ˆ = π‘’π‘˜ π’—π‘˜ π’š

𝑔 π’š π‘’π’š

Ξ©π‘˜ π‘˜

= 𝑄

𝑄′ = 𝑑𝑗 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

slide-28
SLIDE 28

28

Visual & Parallel Computing Group

Solution: Traverse

𝑄′ = 𝑑𝑗

π‘ˆ 𝑔 π’š π‘’π’š Ω𝑗 𝑗

= 𝑄 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

= 𝑄 𝑑𝑗

π‘ˆ = π‘’π‘˜ π’—π‘˜ π’š

𝑔 π’š π‘’π’š

Ξ©π‘˜ π‘˜

= 𝑄

𝑄′ = 𝑑𝑗 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

slide-29
SLIDE 29

29

Visual & Parallel Computing Group

Solution: Traverse

𝑑𝑗

π‘ˆ = π‘’π‘˜ π’—π‘˜ π’š

𝑔 π’š π‘’π’š

Ξ©π‘˜ π‘˜

= 𝑄 𝑄′ = 𝑑𝑗

π‘ˆ 𝑔 π’š π‘’π’š Ω𝑗 𝑗

= 𝑄 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

= 𝑄

𝑄′ = 𝑑𝑗 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

slide-30
SLIDE 30

30

Visual & Parallel Computing Group

Solution: Traverse

Connectivity 𝑑𝑗

π‘ˆ = π‘’π‘˜ π’—π‘˜ π’š

𝑔 π’š π‘’π’š

Ξ©π‘˜ π‘˜

= 𝑄 𝑄′ = 𝑑𝑗

π‘ˆ 𝑔 π’š π‘’π’š Ω𝑗 𝑗

= 𝑄 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

= 𝑄

𝑄′ = 𝑑𝑗 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

slide-31
SLIDE 31

31

Visual & Parallel Computing Group

Solution: Traverse

Neighboring textures 𝑑𝑗

π‘ˆ = π‘’π‘˜ π’—π‘˜ π’š

𝑔 π’š π‘’π’š

Ξ©π‘˜ π‘˜

= 𝑄 𝑄′ = 𝑑𝑗

π‘ˆ 𝑔 π’š π‘’π’š Ω𝑗 𝑗

= 𝑄 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

= 𝑄

𝑄′ = 𝑑𝑗 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

slide-32
SLIDE 32

32

Visual & Parallel Computing Group

Solution: Traverse

Curvature 𝑑𝑗

π‘ˆ = π‘’π‘˜ π’—π‘˜ π’š

𝑔 π’š π‘’π’š

Ξ©π‘˜ π‘˜

= 𝑄 𝑄′ = 𝑑𝑗

π‘ˆ 𝑔 π’š π‘’π’š Ω𝑗 𝑗

= 𝑄 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

= 𝑄

𝑄′ = 𝑑𝑗 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

slide-33
SLIDE 33

33

Visual & Parallel Computing Group

Solution: Traverse

𝑑𝑗

π‘ˆ β‰ˆ π‘’π‘˜ π’—π‘˜ π’š

𝑔 π’š π‘’π’š

Ξ©π‘˜ π‘˜

= 𝑄 𝑄′ = 𝑑𝑗

π‘ˆ 𝑔 π’š π‘’π’š Ω𝑗 𝑗

β‰ˆ 𝑄 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

= 𝑄

𝑄′ = 𝑑𝑗 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

slide-34
SLIDE 34

34

Visual & Parallel Computing Group

New solution: Discard

𝑑𝑗

𝐸 =

𝑒𝑗 𝒗𝑗 π’š 𝑔 π’š π‘’π’š

Ω𝑗

𝑔 π’š π‘’π’š

Ω𝑗

𝑄′ = 𝑑𝑗 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

slide-35
SLIDE 35

35

Visual & Parallel Computing Group

𝑑𝑗

𝐸 =

𝑒𝑗 𝒗𝑗 π’š 𝑔 π’š π‘’π’š

Ω𝑗

𝑔 π’š π‘’π’š

Ω𝑗

New solution: Discard

𝑄′ = 𝑑𝑗 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

slide-36
SLIDE 36

36

Visual & Parallel Computing Group

New solution: Discard

Normalize filter weight = 𝑑𝑗

𝐸 =

𝑒𝑗 𝒗𝑗 π’š 𝑔 π’š π‘’π’š

Ω𝑗

𝑔 π’š π‘’π’š

Ω𝑗

𝑄′ = 𝑑𝑗 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

slide-37
SLIDE 37

37

Visual & Parallel Computing Group

New solution: Discard

𝑑𝑗

𝐸 =

𝑒𝑗 𝒗𝑗 π’š 𝑔 π’š π‘’π’š

Ω𝑗

𝑔 π’š π‘’π’š

Ω𝑗

𝑄′ = 𝑑𝑗 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

𝑄′ = 𝑑𝑗

𝐸 𝑔 π’š π‘’π’š Ω𝑗 𝑗

= 𝑒𝑗 𝒗𝑗 π’š 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

= 𝑄

slide-38
SLIDE 38

38

Visual & Parallel Computing Group

New solution: Discard

𝑑𝑗

𝐸 =

𝑒𝑗 𝒗𝑗 π’š 𝑔 π’š π‘’π’š

Ω𝑗

𝑔 π’š π‘’π’š

Ω𝑗

𝑄′ = 𝑑𝑗 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

𝑄′ = 𝑑𝑗

𝐸 𝑔 π’š π‘’π’š Ω𝑗 𝑗

= 𝑒𝑗 𝒗𝑗 π’š 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

= 𝑄

slide-39
SLIDE 39

39

Visual & Parallel Computing Group

New solution: Discard

Resolve filter Texture filter 𝑑𝑗

𝐸 =

𝑒𝑗 𝒗𝑗 π’š 𝑔 π’š π‘’π’š

Ω𝑗

𝑔 π’š π‘’π’š

Ω𝑗

𝑄′ = 𝑑𝑗

𝐸 𝑔 π’š π‘’π’š Ω𝑗 𝑗

= 𝑒𝑗 𝒗𝑗 π’š 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

= 𝑄

𝑄′ = 𝑑𝑗 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

slide-40
SLIDE 40

40

Visual & Parallel Computing Group

New solution: Discard

Local information 𝑑𝑗

𝐸 =

𝑒𝑗 𝒗𝑗 π’š 𝑔 π’š π‘’π’š

Ω𝑗

𝑔 π’š π‘’π’š

Ω𝑗

𝑄′ = 𝑑𝑗 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

𝑄′ = 𝑑𝑗

𝐸 𝑔 π’š π‘’π’š Ω𝑗 𝑗

= 𝑒𝑗 𝒗𝑗 π’š 𝑔 π’š π‘’π’š

Ω𝑗 𝑗

= 𝑄

slide-41
SLIDE 41

41

Visual & Parallel Computing Group

New solution: Discard

Restrict all the edges?

slide-42
SLIDE 42

42

Visual & Parallel Computing Group

New solution: Discard

Restrict all the edges?

  • No, only texture boundaries

Interiors are less problematic Sampler would need edge information

slide-43
SLIDE 43

43

Visual & Parallel Computing Group

John McDonald [2013] * (Traverse)

C = lookup(texture, texcoord(p)) W = lookup(one, texcoord(p)) for each neighboring patch K: C += lookup(K.texture, K.texcoord(p)) W += lookup(K.one, K.texcoord(p)) return C/W

* Eliminating Texture Waste: Borderless Ptex https://developer.nvidia.com/gdc-2013/

slide-44
SLIDE 44

44

Visual & Parallel Computing Group

New solution (Discard)

C = lookup(texture, texcoord(p)) W = lookup(one, texcoord(p)) //for each neighboring patch K: // C += lookup(K.texture, K.texcoord(p)) // W += lookup(K.one, K.texcoord(p)) return C/W

slide-45
SLIDE 45

Part III Really?

slide-46
SLIDE 46

46

Visual & Parallel Computing Group

Assumptions

Traverse Surface is not curved Texture is not stretched Discard Resolve filter and texture filter are the same

slide-47
SLIDE 47

47

Visual & Parallel Computing Group

Assumptions

Traverse Surface is not curved Texture is not stretched Discard Resolve filter and texture filter are the same

They should be!

slide-48
SLIDE 48

48

Visual & Parallel Computing Group

Assumptions

Traverse Surface is not curved Texture is not stretched Discard Resolve filter and texture filter are the same

Quality factors

Traverse Discard Texture filter Resolve filter (silhouettes) Geometry/texture curvature Texture filter Resolve filter

slide-49
SLIDE 49

49

Visual & Parallel Computing Group

Achilles heel

DX/GL do not specify texture content all the way to the texture boundary

  • Bad for magnification

Desired appearance Appearance without texture data at border

slide-50
SLIDE 50

50

Visual & Parallel Computing Group

Achilles heel

Solved by Purnomo et al., 2004 * Still not in DirectX/OpenGL APIs

* B. Purnomo, J. Cohen and S. Kumar, "Seamless Texture Atlases” ACM SIGGRAPH/Eurographics SGP 2004

slide-51
SLIDE 51

51

Visual & Parallel Computing Group

Results

Traverse Discard

slide-52
SLIDE 52

52

Visual & Parallel Computing Group

Results

Traverse Discard

slide-53
SLIDE 53

53

Visual & Parallel Computing Group

Results

Traverse Discard

slide-54
SLIDE 54

54

Visual & Parallel Computing Group

Results

Traverse Discard

slide-55
SLIDE 55

55

Visual & Parallel Computing Group

slide-56
SLIDE 56

56

Visual & Parallel Computing Group

Larger error More pixels

slide-57
SLIDE 57

57

Visual & Parallel Computing Group

1 in 1000 pixels have an error β‰₯ 0.36

slide-58
SLIDE 58

58

Visual & Parallel Computing Group

1.0 0.36

1 in 1000 pixels have an error β‰₯ 0.36

slide-59
SLIDE 59

59

Visual & Parallel Computing Group

Traverse better

slide-60
SLIDE 60

60

Visual & Parallel Computing Group

Discard better

slide-61
SLIDE 61

61

Visual & Parallel Computing Group

Performance

1 2 3 4 5 1x 2x 4x 8x 0,0 0,2 0,4 0,6 0,8 1x 2x 4x 8x

Render time [ms] MSAA rate MSAA rate

NVIDIA GTX 680 195W GPU Intel Iris Pro 5200 47W CPU+GPU

McDonald 2013 Discard

slide-62
SLIDE 62

62

Visual & Parallel Computing Group

ISO performance

slide-63
SLIDE 63

63

Visual & Parallel Computing Group

Realtime Ptex implementations

Algo gorith ithm Wide de filt lter Mem emory

  • ry

Loo

  • okup

kups McDonald, Burley: SIGGRAPH 2011

Real-time Ptex (Per-Face Texture Mapping)

Yes Large 1 Kim, Hillesland, Hensley: SIGGRAPH Asia 2011

A Space-efficient and hardware-friendly Implementation of Ptex

No Small 1 McDonald: GDC 2013

Eliminating Texture Waste: Borderless Ptex

Yes* Small 5/10 *: over edges only, not corners

slide-64
SLIDE 64

64

Visual & Parallel Computing Group

Realtime Ptex implementations

Algo gorith ithm Wide de filt lter Mem emory

  • ry

Loo

  • okup

kups McDonald, Burley: SIGGRAPH 2011

Real-time Ptex (Per-Face Texture Mapping)

Yes Large 1 Kim, Hillesland, Hensley: SIGGRAPH Asia 2011

A Space-efficient and hardware-friendly Implementation of Ptex

No Small 1 McDonald: GDC 2013

Eliminating Texture Waste: Borderless Ptex

Yes* Small 5/10 *: over edges only, not corners

slide-65
SLIDE 65

65

Visual & Parallel Computing Group

Realtime Ptex implementations

Algo gorith ithm Wide de filt lter Mem emory

  • ry

Loo

  • okup

kups McDonald, Burley: SIGGRAPH 2011

Real-time Ptex (Per-Face Texture Mapping)

Yes Large 1 Kim, Hillesland, Hensley: SIGGRAPH Asia 2011

A Space-efficient and hardware-friendly Implementation of Ptex

No Small 1 McDonald: GDC 2013

Eliminating Texture Waste: Borderless Ptex

Yes* Small 5/10 Toth: JCGT 2013

Avoiding Texture Seams by Discarding Filter Taps

Yes Small 1/2 *: over edges only, not corners

slide-66
SLIDE 66

Thank you for listening

slide-67
SLIDE 67