> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Computer Graphics Basics Probabilistic Morphable Models Summer School, June 2017 Sandro Schönborn University of Basel
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Computer Graphics • Artificial Image Computation • Focus: Photorealistic Rendering • Computer graphics is more: visualization, non-photorealistic rendering, animation, … 2
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Image Formation Study of light: • Light is emitted by source • Light travels through space • Light interacts with objects • Light is reflected • Light is refracted • Light is captured by sensor Computer Graphics: Simulation of light 3
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Computer Graphics Compromise • Inspired by physical cameras Compromise: • Light and matter interaction • Models to achieve results • Light propagation which are good enough • Optimal goal: • Finding good-looking and S imulation of physical reality simple approximations • Unrealistic! Infeasible • Simple models • The perfect model? • Surface rendering • Unknown parameters (volume, interacting media, …) • Computational capacity • Lambert and Phong reflectance 4
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Rendering Geometry: Correspondence Shading: Value • Light transport & refraction • Light-matter interaction • Scene setup • Color values • Correspondence • Needs correspondence Image point ↔ face point Camera 5
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Geometry sensor Object Coordinate transforms Camera model Mesh Model, View transform Projection 6
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Shading: Light-Matter Interaction Reflectance Models Transform incoming light into outgoing light 7
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Modern Graphics Pipeline • Common design • Specialized hardware • Efficient, parallel • Programmable: Shaders (blue parts) • OpenGL (ES, WebGL), Direct3D, Vulkan, Metal In scalismo-faces: Close to standard design fully controllable 8
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Geometry 9
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL 3D Scene • 3D scene Objects in a world Object • Camera takes the picture Image lives on image plane Camera Typically 4 steps to image: Model Transform View Transform Projection Viewport Transform Multiple coordinate systems! 10
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Model and View Transform 𝑧 𝑦 𝑨 Model World Eye Reference frame Camera frame Model View Transform Transform 11 𝑈 # 𝑈 $
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Pinhole Camera: Perspective Projection • Image formation on sensor Pinhole camera image plane (3D -> 2D) Single point aperture • Condition for sharp image: A sensor pixel captures light Camera captures Light leaves surface rays passing from a single point in scene in all directions through aperture sensor • Image plane coordinates by aperture perspective division: 𝑦 𝑨 ⁄ 𝑦′ 𝑧′ = 𝑔 ∗ 𝑧 𝑨 - Focal length f Object distance z A sensor point captures • Non-linear division operation light from a single point in scene only 12
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Perspective Effect • Perspective division distorts image non-linearly • Effect depends on relation of object size and distance 13
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Our Transformations • Model-View • Describes our face-to-image 𝑈 #$ 𝑦 = 𝑆 /,1,2 𝒚 + 𝒖 transform completely • 9 Parameters: • Projection 𝒬 𝑦 = 𝑔 𝑦 • (3) Translation 𝒖 𝑧 𝑨 • (3) Rotation 𝜒, 𝜔, 𝜘 • (1) Focal length 𝑔 • Viewport 𝑥 • (2) Image Offset 𝒖 ?? 2 (𝑦 + 1) • 2 Constants: 𝑈 $7 (𝑦) = + 𝒖 ?? ℎ 2 (1 − 𝑧) • (2) Image size / sampling 14
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Rasterization • Camera: 3D → 2D transformation for points • Raster Image in image plane • Establishes correspondence to 3D surface for each pixel ℎ • Basis: geometric primitives (4,2) (0,0) 𝑥 Pixel grid, cell-centered 15
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Primitives: Triangles • Triangle meshes for surface parametrization: • Triangle • Position within triangle 𝐵 • Parametrization within triangle • Barycentric coordinates 𝝁 𝑄 = 𝑣𝐵 + 𝑤𝐶 + 𝑥𝐷 𝑄 • Barycentric interpolation 𝝁 = 𝑣, 𝑤, 𝑥 𝑔 𝑄 = 𝑣𝑔 𝐵 + 𝑤𝑔 𝐶 + 𝑥𝑔 𝐷 𝑣 + 𝑤 + 𝑥 = 1 • In/out Test: 𝑣, 𝑤, 𝑥 ≥ 0 • All BCC valid (non-negative) 𝐷 𝐶 counter-clockwise winding! 16
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Rasterization: Correspondence 𝝁 𝝁 • Each image pixel is mapped to surface point • Point identification by parameterization with triangle and barycentric coordinate 17
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Rasterization of Triangle Primitives Vertex shader • For each triangle: • Find Vertex position (corners) • Determine bounding box • For all pixels in box: • Inside triangle? • Find BCC in plane: correspondence to 3D through BCC • Draw the pixel Efficient! No ray (0,0) Fragment shader intersections (not perfect though) 18
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Visibility Issues • Multiple triangles might cover the same pixel • 3D surface occludes background • Only the most frontal part is visible in the image • Needs special care during rendering: Hidden Surface Removal Triangles behind are drawn on top of those in front 19
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Hidden Surface Removal: Z-Buffer • Keep additional Z-Buffer: • Store depth information for each pixel • Draw a pixel only if it lies in front of previous drawing • Standard approach In front • Easy and versatile • Extensible to shadowing • Issues: Precision, single value per pixel 20
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Illumination 21
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Illumination • Color in image is result of light and surface interactions • Shading: simulate light interaction and transport • Illumination is global: Lights scatters through scene, interaction with many objects • Global transport • Local interaction 22
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Reflectance Models: BRDF Geometry 𝑴 Light direction • 𝑶 • 𝑶 Surface normal 𝑾 Viewing direction • 𝑾 𝑴 Spectrum Albedo (color) • Bidirectional Reflectance Distribution Function Eye and most cameras: 3 color sensor types • • RGB Model: spectral distribution is sampled 𝑔 𝜇 Z , 𝑴, 𝜇 [ , 𝑾, 𝒚 = d𝑀 [ 𝑾 for red , green and blue d𝐹 Z 𝑴 𝑑 = 𝑠, , 𝑐 , 𝑠, , 𝑐 ∈ [0,1] incoming light ( irradiance ) into outgoing light ( radiance ) 23
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Lambertian Reflectance • Diffuse Reflectance • Surface absorbs all radiation and reemits into every direction • Not directional • Constant BRDF • Brightness of surface depends on incident energy 𝐽 `abb = 𝑙 `abb ∗ 𝐽 d ∗ cos 𝑴, 𝑶 • Deep surface interaction: Albedo: 𝑙 `abb (colored) Observed Reflection Light intensity coefficient intensity 24
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Lambertian Reflectance: Examples 25
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Specular Highlights • Mirror-like reflectance • Highly directional • Reflection cone due to surface roughness • Mostly without deep surface interaction 𝑙 hijk not colored 𝐽 hijk = 𝑙 hijk ∗ 𝐽 d ∗ cos 𝑺, 𝑾 n • Parameter: 𝑜 Phong exponent Width of specular cone 26
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE PROBABILISTIC MORPHABLE MODELS | JUNE 2017 | BASEL Phong Specularity: Examples Specular reflection Specular & diffuse 27
Recommend
More recommend
