1 Sampling and Aliasing Image Processing pipeline Artifacts due to - PDF document

Outline Foundations of Computer Graphics § Basic ideas of sampling, reconstruction, aliasing (Fall 2012) § Signal processing and Fourier analysis CS 184, Lectures 19: Sampling and Reconstruction § Implementation of digital filters http://inst.eecs.berkeley.edu/~cs184 § Section 14.10 of FvDFH (you really should read) § Post-raytracing lectures more advanced topics § No programming assignment § But can be tested (at high level) in final Some slides courtesy Tom Funkhouser Acknowledgements: Thomas Funkhouser and Pat Hanrahan Sampling and Reconstruction Sampling and Reconstruction § An image is a 2D array of samples § Discrete samples from real-world continuous signal (Spatial) Aliasing (Spatial) Aliasing § Jaggies probably biggest aliasing problem 1

Sampling and Aliasing Image Processing pipeline § Artifacts due to undersampling or poor reconstruction § Formally, high frequencies masquerading as low § E.g. high frequency line as low freq jaggies Outline Motivation § Basic ideas of sampling, reconstruction, aliasing § Formal analysis of sampling and reconstruction § Signal processing and Fourier analysis § Important theory (signal-processing) for graphics § Implementation of digital filters § Also relevant in rendering, modeling, animation § Section 14.10 of FvDFH Ideas Sampling Theory Analysis in the frequency (not spatial) domain § Signal (function of time generally, here of space) § Sum of sine waves, with possibly different offsets (phase) § Each wave different frequency, amplitude § Continuous: defined at all points; discrete: on a grid § High frequency: rapid variation; Low Freq: slow variation § Images are converting continuous to discrete. Do this sampling as best as possible. § Signal processing theory tells us how best to do this § Based on concept of frequency domain Fourier analysis 2

Fourier Transform Fourier Transform § Tool for converting from spatial to frequency domain § Simple case, function sum of sines, cosines + ∞ ∑ F ( u ) e 2 π iux § Or vice versa f ( x ) = u = −∞ § One of most important mathematical ideas 2 π ∫ F ( u ) = f ( x ) e − 2 π iux dx § Computational algorithm: Fast Fourier Transform 0 § One of 10 great algorithms scientific computing § Continuous infinite case § Makes Fourier processing possible (images etc.) § Not discussed here, but look up if interested ∞ ∫ f ( x ) e − 2 π iux Forward Transform: F ( u ) = dx −∞ + ∞ ∫ Inverse Transform: f ( x ) = F ( u ) e 2 π iux du −∞ Fourier Transform Fourier Transform: Examples 1 § Simple case, function sum of sines, cosines Single sine curve + ∞ (+constant DC term) ∑ F ( u ) e 2 π iux f ( x ) = u = −∞ 2 π ∫ f ( x ) e − 2 π iux F ( u ) = dx 0 § Discrete case x = N − 1 ( ) − i sin 2 π ux / n ( ) ∑ F ( u ) = f ( x ) cos 2 π ux / N ⎡ ⎤ 0 ≤ u ≤ N − 1 , ⎣ ⎦ + ∞ ∑ F ( u ) e 2 π iux f ( x ) = x = 0 u = N − 1 f ( x ) = 1 ( ) + i sin 2 π ux / n ( ) ∑ u = −∞ ⎡ ⎤ F ( u ) cos 2 π ux / N 0 ≤ x ≤ N − 1 , ⎣ ⎦ 2 π N ∫ F ( u ) = f ( x ) e − 2 π iux dx u = 0 0 Fourier Transform Examples 2 Fourier Transform Properties ∞ ∞ ∫ ∫ Forward Transform: F ( u ) = f ( x ) e − 2 π iux Forward Transform: F ( u ) = f ( x ) e − 2 π iux dx dx −∞ −∞ + ∞ + ∞ ∫ ∫ Inverse Transform: f ( x ) = F ( u ) e 2 π iux du Inverse Transform: f ( x ) = F ( u ) e 2 π iux du −∞ −∞ § Common examples § Common properties § Linearity: F ( af ( x ) + bg ( x )) = aF ( f ( x )) + bF ( g ( x )) f ( x ) F ( u ) ∞ ∫ F ( f '( x )) = f '( x ) e − 2 π iux § Derivatives: [integrate by parts] dx δ ( x − x 0 ) − 2 π iux 0 e −∞ = 2 π iuF ( u ) § 2D Fourier Transform 1 δ ( u ) ∞ ∞ ∫ ∫ Forward Transform: F ( u , v ) = f ( x , y ) e − 2 π iux e − 2 π ivy dxdy −∞ π e − ax 2 ae − π 2 u 2 / a −∞ ∞ § Convolution (next) + ∞ ∫ ∫ Inverse Transform: f ( x , y ) = F ( u , v ) e 2 π iux e 2 π ivy dudv −∞ −∞ 3

Sampling Theorem, Bandlimiting Sampling Theorem, Bandlimiting § A signal can be reconstructed from its samples, § A signal can be reconstructed from its samples, if if the original signal has no frequencies above the original signal has no frequencies above half half the sampling frequency – Shannon the sampling frequency – Shannon § The minimum sampling rate for a bandlimited § The minimum sampling rate for a bandlimited function is called the Nyquist rate function is called the Nyquist rate § A signal is bandlimited if the highest frequency is bounded. This frequency is called the bandwidth § In general, when we transform, we want to filter to bandlimit before sampling, to avoid aliasing Antialiasing Ideal bandlimiting filter § Sample at higher rate § Formal derivation is homework exercise § Not always possible § Real world: lines have infinitely high frequencies, can ’ t sample at high enough resolution § Prefilter to bandlimit signal § Low-pass filtering (blurring) § Trade blurriness for aliasing Outline Convolution 1 § Basic ideas of sampling, reconstruction, aliasing § Signal processing and Fourier analysis § Convolution § Implementation of digital filters § Section 14.10 of FvDFH 4

Convolution 2 Convolution 3 Convolution 4 Convolution 5 Convolution in Frequency Domain Practical Image Processing ∞ ∫ Forward Transform: F ( u ) = f ( x ) e − 2 π iux § Discrete convolution (in spatial domain) with filters for dx various digital signal processing operations −∞ + ∞ ∫ F ( u ) e 2 π iux du Inverse Transform: f ( x ) = § Easy to analyze, understand effects in frequency domain −∞ § E.g. blurring or bandlimiting by convolving with low pass filter § Convolution (f is signal ; g is filter [or vice versa]) + ∞ + ∞ ∫ ∫ h ( y ) = f ( x ) g ( y − x ) dx = g ( x ) f ( y − x ) dx −∞ −∞ h = f * g or f ⊗ g § Fourier analysis (frequency domain multiplication) H ( u ) = F ( u ) G ( u ) 5

Outline Discrete Convolution § Previously: Convolution as mult in freq domain § Basic ideas of sampling, reconstruction, aliasing § But need to convert digital image to and from to use that § Useful in some cases, but not for small filters § Signal processing and Fourier analysis § Previously seen: Sinc as ideal low-pass filter § Implementation of digital filters § But has infinite spatial extent, exhibits spatial ringing § In general, use frequency ideas, but consider implementation issues as well § Section 14.10 of FvDFH § Instead, use simple discrete convolution filters e.g. § Pixel gets sum of nearby pixels weighted by filter/mask 2 0 -7 5 4 9 1 -6 -2 Outline Implementing Discrete Convolution § Fill in each pixel new image convolving with old § Implementation of digital filters § Not really possible to implement it in place § Discrete convolution in spatial domain a + width b + width ∑ ∑ I new ( a , b ) = f ( x − a , y − b ) I old ( x , y ) § Basic image-processing operations x = a − width y = b − width § Antialiased shift and resize § More efficient for smaller kernels/filters f § Normalization § If you don ’ t want overall brightness change, entries of filter must sum to 1. You may need to normalize by dividing § Integer arithmetic § Simpler and more efficient § In general, normalization outside, round to nearest int Basic Image Processing Blurring § Blur § Used for softening appearance § Sharpen § Convolve with gaussian filter § Same as mult. by gaussian in freq. domain, so § Edge Detection reduces high-frequency content § Greater the spatial width, smaller the Fourier width, more blurring occurs and vice versa All implemented using convolution with different filters § How to find blurring filter? 6

Blurring Blurring Blurring Blurring Blurring Blurring Filter § In general, for symmetry f(u,v) = f(u) f(v) § You might want to have some fun with asymmetric filters § We will use a Gaussian blur § Blur width sigma depends on kernel size n (3,5,7,11,13,19) Spatial Frequency exp − u 2 ⎡ ⎤ 1 f ( u ) = σ = floor ( n / 2) / 2 ⎢ ⎥ 2 σ 2 2 πσ ⎣ ⎦ 7

Discrete Filtering, Normalization Basic Image Processing § Gaussian is infinite § Blur § In practice, finite filter of size n (much less energy beyond 2 § Sharpen sigma or 3 sigma). § Must renormalize so entries add up to 1 § Edge Detection § Simple practical approach § Take smallest values as 1 to scale others, round to integers § Normalize. E.g. for n = 3, sigma = ½ All implemented using convolution with different filters 2 πσ 2 exp − u 2 + v 2 ⎡ ⎤ ( ) 1 ⎥ = 2 π exp − 2 u 2 + v 2 ⎡ ⎤ f ( u , v ) = ⎢ ⎣ ⎦ 2 σ 2 ⎣ ⎦ ⎛ ⎞ ⎛ ⎞ 0.012 0.09 0.012 1 7 1 ⎜ ⎟ ⎟ ≈ 1 ⎜ ⎟ ≈ 0.09 0.64 0.09 7 54 7 ⎜ ⎜ ⎟ 86 ⎝ 0.012 0.09 0.012 ⎠ ⎝ 1 7 1 ⎠ Sharpening Filter Blurring § Unlike blur, want to accentuate high frequencies § Take differences with nearby pixels (rather than avg) ⎛ ⎞ − 1 − 2 − 1 f ( x , y ) = 1 ⎜ ⎟ − 2 − 2 19 ⎜ ⎟ 7 ⎝ − 1 − 2 − 1 ⎠ Blurring Blurring 8