WetBrush: GPU-based 3D Painting Simulation at the Bristle Level - - PowerPoint PPT Presentation

GTC 2016

WetBrush: GPU-based 3D Painting Simulation at the Bristle Level

Zhili Chen Byungmoon Kim Adobe Research

GTC 2016

2

Oil painting with brush

• Complex interactions
• Bristle-Bristle
• Bristle-Paint
• Paint-Paint
• Motivation:
• Exhaustive simulation?

GTC 2016

3

Oil Painting Simulation

Brush Sim Renderer User Input

Brush head movement Two-way Bristle-Paint Liquid Interaction

Paint Liquid

GTC 2016

4

Brush Model

• Previous works on brush model

2D Stamps 2D Surface wrapped around skeleton Individual bristles

Chu et al., 2002 Expresii DiVerdi et al., 2010 D-Brush in Photoshop Natural brushes in Photoshop

• No variations from

brush dynamics

• No details from individual

bristles with large deformation

• Brush shape not realistic without

inter-bristle interactions

GTC 2016

5

Brush Model

• Our model

Bristle vertices Bristle samples

50-100 bristles ~10 vertices per bristle

𝐪𝑗 𝐪𝑗+1 𝑚0 𝐪𝑗 𝐪𝑗+1 𝐪𝑗−1 𝜄

Bending Stretching

GTC 2016

6

Brush Model

• Inter-bristle interactions
• Collision
• Friction

GTC 2016

7

Paint Liquid Model

Height field Volumetric grid Particles

Chu, et al. 2010 Microsoft FreshPaint

Overhang Multi-layer color mixing

?

• Track volume and mass better
• Less blurring from grid-sampling
• Model paint carried by brush

GTC 2016

8

Hybrid Fluid Representation

Brush

Fluid particles

• Adaptive Hybrid Fluid Representation based on
• Distance to brush
• Velocity

GTC 2016

9

Hybrid Fluid Representation

Brush

Fluid particles

Particles

• Adaptive Hybrid Fluid Representation based on
• Distance to brush
• Velocity
• Close to the brush
• OR Fast Moving
• Cover smaller region

GTC 2016

10

Hybrid Fluid Representation

Brush

Fluid particles

Particles Density Grid

• Adaptive Hybrid Fluid Representation based on
• Distance to brush
• Velocity
• Close to the brush
• OR Fast Moving
• Cover smaller region
• Further away from the brush
• AND Slow moving
• Cover larger region

GTC 2016

11

Fluid Conversions

Grid & Particles Visualized Only Particles Visualized

• FLIP
• Particles sample from velocity grid
• Avoid expensive neighbor searching

Grid Fluid Particle Fluid

• Sample new

particles

• Convert and delete

stationary particles

GTC 2016

12

Hybrid Fluid

Grid Only Grid+Particle

• Less volume loss
• Better surface tracking
• Sharper surface detail
• Sharper color mixing

GTC 2016

13

Brush-Paint Interaction

Brush Sim Grid Fluid Particle Fluid

• Brush emit new pigment particles
• Brush carries and pushes existing fluid particles
• Brush pick up color from paint on canvas
• Thick paint affect brush motion and shape

GTC 2016

14

Implementation

• CUDA implementation
• Both brush and paint liquid in CUDA
• CUDA+OpenGL interop. for rendering
• Rendering
• Ray casting on paint density grid and pigment grid
• Screen space particle rendering

GTC 2016

15

Implementation

k_ForwardPosition Hash SolveSelfCollision k_ProjectConstraints k_UpdateFinalSim k_UpdateRendering k_UpdateSorted FindBrushBoundingB

• x

k_InterpolatePaintNo des k_UpdatePaintNodesRendering k_VoxelizeBrush k_FilterBrushVoxelization k_BrushDistanceMap k_Advection_Particles k_RasterizeParticleTrilinear k_NormalizeRasterization k_MixBrushVelocity Diffusion PostProcessVelocity k_AdvectPaintDensityGrid SpawnNewPaticles k_BrushPickupPigment k_BrushPaintNodesDistanceToPaintSurface k_PaintNodeEmitNewParticlesBasedOnDistanceToPaint k_PIC k_FindConvertParticlesToPaintDensity k_ExtendPigment SaveActiveDomain

Forward Brush Inter-Bristle Collision Bristle Stretching and Bending Update Brush Attributes for Rendering Update Brush Paint Nodes that holds pigment Voxelize Brush Create Brush Distance Field Find Brush Bounding Box Forward FLIP Particles Rasterize Particles

• nto Velocity

Grid Diffusion and Dissapation Mix with Brush Velocity Field Grid Fluid Simulation Brush Pickup Pigment Sample New Particles from Density Grid Brush Vertices emit new Paint Particles Resample FLIP Particle Velocities Convert inactive particles to density grid Save current window back to full- canvas texture

GTC 2016

16

Performance GTX Titan X

• Benchmark sequence: 46 fps, 92K Particles (~2M at maximum)

5% 30% 24% 24% 8% 9% Grid-based liquid simulation

(6.5ms)

Particle-based liquid simulation

(5.2ms)

Grid-particle transfer

(5.2ms)

Brush simulation

(1.1ms)

Rendering

(2.0ms)

Bristle-particle transfer

(1.6ms)

GTC 2016

17

Performance

• Faster stroke requires higher framerate
• Faster stroke => more stroke samples per unit time
• Maximum distance between stroke sample ~= 3-5 pixels

Stroke Sample distance 2px Stroke Sample distance 30px

GTC 2016

18

Optimization

GTC 2016

19

Optimization

• Adaptive simulation window (vs fixed simulation window)

Axis-aligned bounding box for brush Active simulation window thrust::minmax_element() For grid fluid simulation

46 fps-> 75 fps (63%)

GTC 2016

20

Optimization

• Particle insertion

Num_Particles Particle Array AtomicInc(Num_Particles)

GTC 2016

21

Optimization

New Particle Array

• Pre-allocate maximum #

particle space for each cell

• No more than 27 particles within one grid cell
• No more than one particle in one subcell

Marker Array

• 1 => a new particle is

added for the element Cell 𝑗 Cell 𝑗 + 1

• Optimized Particle insertion

Cell 𝑗 − 1

1 1 1 1 1 1

GTC 2016

22

Optimization

New Particle Array

• Pre-allocate maximum #

particle space for each cell

Marker Array

• 1 => a new particle is

added for the element Cell 𝑗 Cell 𝑗 + 1

• Optimized Particle insertion

Cell 𝑗 − 1

1 1 1 1 1 1

Compact New Particle Array thrust::remove_if() Shrink List Append Particle Array

75 fps-> 92 fps (22%)

GTC 2016

23

Kernel Pipeline

k_ForwardPosition Hash SolveSelfCollision k_ProjectConstraints k_UpdateFinalSim k_UpdateRendering k_UpdateSorted FindBrushBoundingB

• x

k_InterpolatePaintNo des k_UpdatePaintNodesRendering k_VoxelizeBrush k_FilterBrushVoxelization k_BrushDistanceMap k_Advection_Particles k_RasterizeParticleTrilinear k_NormalizeRasterization k_MixBrushVelocity Diffusion PostProcessVelocity k_AdvectPaintDensityGrid SpawnNewPaticles k_BrushPickupPigment k_BrushPaintNodesDistanceToPaintSurface k_PaintNodeEmitNewParticlesBasedOnDistanceToPaint k_PIC k_FindConvertParticlesToPaintDensity k_ExtendPigment SaveActiveDomain

GTC 2016

24

Kernel Pipeline

k_ForwardPosition Hash SolveSelfCollision k_ProjectConstraints k_UpdateFinalSim k_UpdateRendering k_UpdateSorted FindBrushBoundingB

• x

k_InterpolatePaintNo des k_UpdatePaintNodesRendering k_VoxelizeBrush k_FilterBrushVoxelization k_BrushDistanceMap k_Advection_Particles k_RasterizeParticleTrilinear k_NormalizeRasterization k_MixBrushVelocity Diffusion PostProcessVelocity k_AdvectPaintDensityGrid SpawnNewPaticles k_BrushPickupPigment k_BrushPaintNodesDistanceToPaintSurface k_PaintNodeEmitNewParticlesBasedOnDistanceToPaint k_PIC k_FindConvertParticlesToPaintDensity k_ExtendPigment SaveActiveDomain

Low occupancy, GPU almost idle

GTC 2016

25

Dependencies

k_ForwardPosition Hash SolveSelfCollision k_ProjectConstraints k_UpdateFinalSim k_UpdateRendering k_UpdateSorted FindBrushBoundingB

• x

k_InterpolatePaintNo des k_UpdatePaintNodesRendering k_VoxelizeBrush k_FilterBrushVoxelization k_Advection_Particles k_RasterizeParticleTrilinear k_NormalizeRasterization k_MixBrushVelocity Diffusion PostProcessVelocity k_PaintNodeEmitNewParticlesBasedOnDistanceToPaint k_PIC k_FindConvertParticlesToPaintDensity k_ExtendPigment SaveActiveDomain k_BrushDistanceMap k_AdvectPaintDensityGrid SpawnNewPaticles k_BrushPickupPigment k_BrushPaintNodesDistanceToPaintSurface

GTC 2016

26

Dependencies

k_ForwardPosition Hash SolveSelfCollision k_ProjectConstraints k_UpdateFinalSim k_UpdateRendering k_UpdateSorted FindBrushBoundingB

• x

k_InterpolatePaintNo des k_UpdatePaintNodesRendering k_VoxelizeBrush k_FilterBrushVoxelization k_BrushDistanceMap k_Advection_Particles k_RasterizeParticleTrilinear k_NormalizeRasterization k_MixBrushVelocity Diffusion PostProcessVelocity k_PaintNodeEmitNewParticlesBasedOnDistanceToPaint k_PIC k_FindConvertParticlesToPaintDensity k_ExtendPigment SaveActiveDomain

• Brush # Vertices << Grid Dimension, Particle #
• Low occupancy
• First half of fluid simulation do not depend on brush

k_BrushDistanceMap k_AdvectPaintDensityGrid SpawnNewPaticles k_BrushPickupPigment k_BrushPaintNodesDistanceToPaintSurface

GTC 2016

27

Dependencies

k_ForwardPosition Hash SolveSelfCollision k_ProjectConstraints k_UpdateFinalSim k_UpdateRendering k_UpdateSorted FindBrushBoundingB

• x

k_InterpolatePaintNo des k_UpdatePaintNodesRendering k_VoxelizeBrush k_FilterBrushVoxelization k_BrushDistanceMap k_Advection_Particles k_RasterizeParticleTrilinear k_NormalizeRasterization k_MixBrushVelocity Diffusion PostProcessVelocity k_AdvectPaintDensityGrid SpawnNewPaticles k_BrushPaintNodesDistanceToPaintSurface k_PaintNodeEmitNewParticlesBasedOnDistanceToPaint k_PIC k_FindConvertParticlesToPaintDensity k_ExtendPigment SaveActiveDomain

• Only for rendering
• Can take place anytime

in a frame

k_BrushPickupPigment

GTC 2016

28

Dependencies

k_ForwardPosition Hash SolveSelfCollision k_ProjectConstraints k_UpdateFinalSim k_UpdateRendering k_UpdateSorted FindBrushBoundingB

• x

k_InterpolatePaintNo des k_UpdatePaintNodesRendering k_VoxelizeBrush k_FilterBrushVoxelization k_BrushDistanceMap k_Advection_Particles k_RasterizeParticleTrilinear k_NormalizeRasterization k_MixBrushVelocity Diffusion PostProcessVelocity k_AdvectPaintDensityGrid SpawnNewPaticles k_BrushPickupPigment k_BrushPaintNodesDistanceToPaintSurface k_PaintNodeEmitNewParticlesBasedOnDistanceToPaint k_PIC k_FindConvertParticlesToPaintDensity k_ExtendPigment SaveActiveDomain

• More parallelization

GTC 2016

29

Optimization

k_ForwardPosi tion Hash SolveSelfCollision k_ProjectCo nstraints k_UpdateFinalSim k_UpdateRendering k_UpdateSorted k_InterpolatePaintNo des k_UpdatePaintNodesRendering k_VoxelizeBrush k_FilterBrushVoxelization k_BrushDistanceMap k_Advection_Particles k_RasterizeParticleTrilinear k_NormalizeRasterization k_MixBrushVelocity k_AdvectPaintDensityGrid SpawnNewPaticles k_BrushPickupPigment k_BrushPaintNodesDista nceToPaintSurface k_PaintNodeEmitNewParticle sBasedOnDistanceToPaint k_PIC k_FindConvertParticle sToPaintDensity k_ExtendPigment SaveActiveDomain

1 2 3 4

Sync 0->1,3,4 Sync 0->3 Sync 0->2 Sync 2->0

cudaEventRecord() cudaStreamWaitEvent()

FindBrushBoundingB

• x

PostProcessVelocity Diffusion

• Using CUDA Streams

92 fps-> 117 fps (27%)

GTC 2016

30

Optimization

• 46 fps -> 117 fps (154%)
• Shared memory, Loop unrolling, Adjusting block size
• 100~200fps on GTX Titan X
• 30~50 fps on Surface Book

GTC 2016

31

Results

GTC 2016

32

Results

Better 3D shape Finer surface details More pigment variations along strokes

Thick “Impasto” Style A lot of overhang

Thinner painting style

Thank you!