14 Mipmaps and normal map AA Steve Marschner CS5625 Spring 2019
Overview Basic sampling problem • Texture mapping defines a signal in image space • That signal needs to be filtered: convolved with a filter • Approximating this drives all the basic algorithms Antialiasing nonlinear shading • Basic sampling su ffi ces only if pixel and texture are linearly related • Normal mapping is the most important nonlinearity
Texture mapping from 0 to infinity When you go close…
Texture mapping from 0 to infinity When you go far…
Solution: pixel filtering Problem: Perspective produces very high image frequencies Solution • Would like to render textures with one (few) samples/pixel • Need to filter first!
Solution: pixel filtering point sampling area averaging
Pixel filtering in texture space Sampling is happening in image space • therefore the sampling filter is defined in image space • sample is a weighted average over a pixel-sized area • uniform, predictable, friendly problem! Signal is defined in texture space • mapping between image and texture is nonuniform • each sample is a weighted average over a di ff erent sized and shaped area • irregular, unpredictable, unfriendly! This is a change of variable • integrate over texture coordinates rather than image coordinates
Pixel footprints image space texture space
How does area map over distance? At optimal viewing distance: • One-to-one mapping between pixel area and texel area When closer • Each pixel is a small part of the texel • magnification When farther • Each pixel could include many texels • “minification” upsampling downsampling magnification minification
How to get a handle on pixel footprint We have a nonlinear mapping to deal with R 2 : u 7! x ( u ) R 2 ! I • image position as a function of texture coordinates: I • but that is too hard Instead use a local linear approximation • hinges on the derivative of u = (u,v) wrt. x = (x,y) u ( x + ∆ x ) ≈ u ( x ) + ∂ u ∂ x ∆ x " ∂ u ∂ u # ∂ u Matrix derivative, ∂ x ∂ y ∂ x = or Jacobian ∂ v ∂ v ∂ x ∂ y
Sizing up the situation with the Jacobian image space texture space � ∂ u � ∂ y , ∂ v � ∂ u � ∂ x , ∂ v ∂ y ∂ x ( 0 , 1 ) ( 1 , 0 ) y v x ψ ( x ) x u
How to tell minification from magnification Di ff erence is the size of the derivative • but what is “size”? � � ∂ u • area: determinant of Jacobian: � � � � ∂ x � � • max-stretch: 2-norm of Jacobian (requires a singular-value computation) • Frobenius norm of matrix (RMS of 4 entries, easy to compute) • max dimension of bounding box of quadrilateral footprint: max-abs of 4 entries (conservative) Take your pick; magnification is when size is more than about 1
Solutions for Minification For magnification, use a good image interpolation method • bilinear (usual) or bicubic filter (fancier, smoother) are good picks • nearest neighbor (box filter) will give you Minecraft-style blockies For minification, use a good sampling filter to average • box (simple, though not usually easier) • gaussian (good choice) Challenge is to approximate the integral e ffi ciently! • mipmaps • multi-sample anisotropic filtering (based on mipmap)
Mipmap image pyramid MIP Maps • Multum in Parvo: Much in little, many in small places • Proposed by Lance Williams Stores pre-filtered versions of texture Supports very fast lookup • but only of circular filters at certain scales [Akenine-Möller & Haines 2002]
Filtering by Averaging Each pixel in a level corresponds to 4 pixels in lower level • Average • Gaussian filtering
<latexit sha1_base64="EOhaTlnq84sPc1CEnsnq2xc0kI=">ACkHichVHLSsNAFJ3Ed31VXboZLIqrkrQRIygqbsSVglWhKWUyvamDk0mYmRLyPf4P+78Cz/Baepb0QsDh3PuPXM5N0w5U9pxnix7YnJqemZ2rjK/sLi0XF1ZvVJim0aMITeRMSBZwJaGmOdykEkgcrgO705G+vUApGKJuNTDFDox6QsWMUq0obrVhyCEPhN5GBMt2X1RCSJaB6kRGpGOM6KD3xf4C38hz4sgoFKCYXcS3URBN+9Bv94Db54VQIQvfe9utWaU/d3HFP4J3DrTlm1w2dU1nm3+hj0EprFIDTlRKm26S6k4/sKQfjnikwu96RPrQNFCQG1cnLQAu8aZgejhJpntC4ZD9P5CRWahiHptPsd6u+ayPyN62d6cjv5EykmQZBx9FGc6waPr4B6TQDUfGkCoZGZXTG+JSUmbG1bKELxGw/eaeAya7hvw30O4atRdr7534dWOTsdpoFm0jbQNnLRLjpCp+gctRC1Fq2mtW8d2Ku2bx/ax+NW23qdWUNfyj57AW6uzOc=</latexit> Using the MIP Map Find the MIP Map level where the pixel has a 1-to-1 mapping • How to do this? Find size of pixel footprint in texture space • Pick level where that size corresponds to a texel " ∂ u ∂ u # • How to measure “size?” ∂ x ∂ y Compute derivatives to find pixel footprint • How does (u,v) change as (x,y) changes? ∂ v ∂ v • 2x2 derivative matrix ∂ x ∂ y
<latexit sha1_base64="Ed/S/lmVK3C8svdBGmcHY4buFsE=">ACtHichVFNa9tAEF2pH0nVtHaYy9LTSGhYCTHSdxDSaAXH12o4DXdVfrkb1ktVJ2RyZG6Bfm1lv+RX5C13K/65IHC483b2aWN3GupMUwvPX8Bw8fPd7afhI83Xn2fLex9+LcZoURMBCZysxFzC0oqWGAEhVc5AZ4GisYxpcfVvXhAoyVmf6EyxzGKZ9pmUjB0UmTxo2i7ylT2YzZK4NlwBQkuM8Sw0XJcm5QckWL6he/rpiRszkefG7Tt3Szf/E/3jl/fbF5vtQTVpNMNW9yh0oP+SqBXWaJ7ekRr9SeMrm2aiSEGjUNzaURTmOC5Xs4WCKmCFhZyLSz6DkaOap2DHZR16Rd84ZUqTzLinkdbq7x0lT61dprFzphzn9u/aStxUGxWYdMel1HmBoMV6UVIoihldXZBOpQGBaukIF0a6v1Ix5y4idHcO6hA67Xa3c0jX5D6Qbo/Qzhvt6JO693HTvOst06DbJNX5DXZJxE5IWekR/pkQIQXeUPvi8f9Y5/5woe1fe+97wkf8DX3wB0wNhN</latexit> <latexit sha1_base64="Lkf9jmjtz0lYWX0ispQDjwxun1Q=">ACwniclVFNT9tAEF27tFD3K6VHLiuiSlSqIjsEkR4QSOXAkUoEkGIrWm/GyZb1rtkdIyInf7I3/kV/Qjd2y1e5MNJqnt68mVm9SQspLIbhje/WHn5anXtdfDm7bv3H1of10+tLg2HAdSm/OUWZBCwQAFSjgvDLA8lXCWXnxf1s+uwFih1QnOCkhyNlEiE5yho0atG0n3aCz1JaQ4TDO2XWNtoI6zePMF7FBTMomKTl4g5fL2IjJlOcf6VPi6+eI74/efacybfioMlfmpSMWu2w098JXdD/QdQJ62jv/yZ1HI9av+Kx5mUOCrlk1g6jsMCkWq7hEhZBXFoGL9gExg6qFgONqnqEyzoZ8eMaNewpzd7vqFhu7SxPnTJnOLWPa0vyqdqwxKyfVEIVJYLizaKslBQ1Xd6TjoUBjnLmAONGuL9SPmXOLXRXD2oTet1uv7dNG7Ad/QP9WxNOu52o1/n2o9c+OGrcIGtkg2ySLRKRXJAjsgxGRDu7XvgKU/7h/5P/9K3jdT3/vZ8Ig/Cn/8BcFLhpw=</latexit> Given derivatives: what is level? Need to reduce the matrix to a single number • aka. choosing a matrix norm; several choices available with di ff erent tradeo ff s • elementwise max partial derivative: ✓� � � � � � � � ◆� ∂ u ∂ v ∂ u ∂ v � � � � � � � � l = log max � , � , � , � � � � � � � � ∂ x ∂ x ∂ y ∂ y � � � � � • root-mean-square of partial derivatives: s✓ ∂ u ◆ 2 ◆ 2 ◆ 2 ◆ 2 ✓ ∂ v ✓ ∂ u ✓ ∂ v l = log + + + ∂ x ∂ x ∂ y ∂ y • either way, you get a non-integer level at which to look up
Using the MIP Map In level, find texel and • Return the texture value: point sampling (but still better)! • Bilinear interpolation • Trilinear interpolation Level k Level ( k + 1)
Memory Usage What happens to size of texture? • level 1 takes 1/4 the memory of level 0 • level 2 takes 1/16, etc. • in total, adds 1/3 to the storage requirements
MIPMAP Multi-resolution image pyramid • Pre-sampled computation of MIPMAP • 1/3 more memory Bilinear or Trilinear interpolation
Point sampling point sampled mini fj cation mipmap mini fj cation
Point sampling point sampled mini fj cation mipmap mini fj cation
Reference: gaussian sampling by point sampled 512x supersampling mini fj cation mipmap mini fj cation
Reference: gaussian sampling by point sampled 512x supersampling mini fj cation mipmap mini fj cation
Texture minification with a mipmap
Texture minification with a mipmap
Texture minification: supersampling vs. mipmap
Texture minification: supersampling vs. mipmap
How to do better? RIP maps and summed-area tables • can look up axis-aligned rectangular areas • diagonals still a big problem! Elliptical Weighted Average (EWA) filter • perform multiple lookups • accumulate using filtering weights • MIP map pyramid still helps!
Recommend
More recommend
