Image Processing (10) RNDr. Martin Madaras, PhD. - - PowerPoint PPT Presentation

▶
image processing 10
SMART_READER_LITE
LIVE PREVIEW

Image Processing (10) RNDr. Martin Madaras, PhD. - - PowerPoint PPT Presentation

Sep 24, 2023 225 likes •1.02k views

Principles of Computer Graphics and Image Processing Image Processing (10) RNDr. Martin Madaras, PhD. martin.madaras@stuba.sk Computer Graphics Image processing Representing and manipulation of 2D images Modeling Representing and

slide-1
SLIDE 1

Principles of Computer Graphics and Image Processing

Image Processing (10)

  • RNDr. Martin Madaras, PhD.

martin.madaras@stuba.sk

slide-2
SLIDE 2

Computer Graphics

2

 Image processing

 Representing and manipulation of 2D images

 Modeling

 Representing and manipulation of 2D and 3D objects

 Rendering

 Constructing images from virtual models

 Animation

 Simulating changes over time

slide-3
SLIDE 3

3

  • Ask questions, please!!!
  • Be communicative
  • www.slido.com #PPGSO10
  • More active you are, the better for you!

How the lectures should look like #1

slide-4
SLIDE 4

4

  • Image Filtering / Image Manipulation
  • Pixel operations
  • Filtering
  • Composition
  • Quantization
  • Warping and Morphing
  • Sampling, Reconstruction and Aliasing

Image Processing

slide-5
SLIDE 5

5

  • Change the color values of every pixel
  • P’ = F(P)

Pixel operations

slide-6
SLIDE 6

6

  • Add random value to each color channel
  • Clamp to <0,1> range

Adding noise

slide-7
SLIDE 7

7

 Simply scale pixel values and clamp to <0,1> range

Change brightness

slide-8
SLIDE 8

8

 Compute mean luminance L = 0.3r+0.59g+0.11b  Scale deviation from L and clamp to <0,1>

Change contrast

slide-9
SLIDE 9

9

 r’, g’, b’ = 0.3r+0.59g+0.11b

Grayscale

slide-10
SLIDE 10

10

 Discretized convolution of functions  Each pixel is a linear combination of pixels in its neighborhood

Linear filtering

slide-11
SLIDE 11

11

 Convolve with a filter that sums to 1

Blur

slide-12
SLIDE 12

12

 Convolve with a filter that finds differences

between pixels

Edge detection

slide-13
SLIDE 13

13

 Sum edges with original image

Sharpen

slide-14
SLIDE 14

14

 Each pixel is non-linear function of input pixels  A non-linear filter is one that cannot be done with convolution or

Fourier multiplication

 The median is NOT linear because

 med( f{i} + g{i} ) != med( f{i} ) + med( g{i} )

 med( {1,2,3} + {5,4,6}) = med( {6,6,9} ) = 6  med( {1,2,3} ) + med( {5,4,6} ) = 2 + 5

Non Linear filtering

slide-15
SLIDE 15

15

 Combine multiple images, produce new image  Per-pixel operation  blend = F(base, source)  Image dimensions may not be identical

Blending

slide-16
SLIDE 16

16

 Linear combination of pixels from both images  blend(x,y) = base(x,y)(1-t) + source(x,y)t  t is from <0,1>

t = 0 t = 0.5 t = 1 base blend source

Normal Blending

slide-17
SLIDE 17

17

 Linear combination of pixels from both images  blend(x,y) = base(x,y)(1-t) + source(x,y)t  t is from <0,1>

t = 0 t = 0.5 t = 1 base blend source

Normal Blending

slide-18
SLIDE 18

18

 Compute value v that is a linear combination of v0 and v1  Use argument t to specify how close v is to v1 using <0,1>

range v = v0*(1-t)+v1*t float lerp(float v0, float v1, float t) { return v0*(1-t)+v1*t; }

Linear Interpolation

slide-19
SLIDE 19

19

 Add pixels from both images  Clamp the result into the <0,1> range  blend(x,y) = base(x,y) + source(x,y)

base source base + source

Additive Blending

slide-20
SLIDE 20

20

 Subtract pixels from both images  Clamp the result into the <0,1> range  blend(x,y) = base(x,y) - source(x,y)

base source base - source

Subtract

slide-21
SLIDE 21

21

 Compute the difference between images  White source inverts base  blend(x,y) = abs(source(x,y)-base(x,y))

base source |source-base|

Difference

slide-22
SLIDE 22

22

 How to blend parts of the image?  We need a way to select sections of the image  Idea. How about using another image ... ?

Masking

slide-23
SLIDE 23

23

 White region is transparent  Black regions are opaque  Use a 1-bit image to select areas of interest base mask masked area

Mask

slide-24
SLIDE 24

24

 Blend two images with mask  blend(x,y) = base(x,y) if mask(x,y) is 1  blend(x,y) = source(x,y) if mask(x,y) is 0 base + source composite

Mask composition

slide-25
SLIDE 25

25

 Masks are 1-bit, no smooth edges  What about transparent objects such as glass?  We need to work with additional image?

Mask Problems

slide-26
SLIDE 26

26

 Idea. Store pixel transparency per pixel!  Alvy Ray Smith, late1970s  Pixel = (r, g, b, a), added alpha channel  Let a define the opacity of the pixel  a = 0, pixel is transparent  a = 1, pixel is opaque  Opacity depth is usually same as for colors

Alpha Blending

slide-27
SLIDE 27

27

 Thomas Porter and Tom Duff, 1984  (r, g, b, a) represents a pixel that is a covered by the color

C=(r/a, g/a, b/a)

 Store components premultiplied by a  We can display (r, g, b) values directly  Why? Closure in composition algebra  Note: Many images do not use premultiplied color

Premultiplied Color

slide-28
SLIDE 28

28

 What is the meaning of the following?

 (0, 1, 0, 1) =  (0, ½, 0, 1) =  (0, ½, 0, ½) =  (0, ½, 0, 0) =

Premultiplied Color

slide-29
SLIDE 29

29

 What is the meaning of the following?

 (0, 1, 0, 1) = full green, opaque  (0, ½, 0, 1) = half green, opaque  (0, ½, 0, ½) = full green, partially transparent  (0, ½, 0, 0) = transparent

Premultiplied Color

slide-30
SLIDE 30

30

 Suppose we put A over B over background G  How much of B is blocked by A ?

Semi-transparent Objects

A B G

slide-31
SLIDE 31

31

 Suppose we put A over B over background G  How much of B is blocked by A ?  aA

Semi-transparent Objects

A B G

slide-32
SLIDE 32

32

 How much of B is shown through A ?

Semi-transparent Objects

A B G

slide-33
SLIDE 33

33

 How much of G is shown through both A and B?

Semi-transparent Objects

A B G

slide-34
SLIDE 34

34

 How much of G is shown through both A and B?  (1-aA) (1-aB)

Semi-transparent Objects

A B G

slide-35
SLIDE 35

35

 How do we combine 2 partially covered objects?

 3 Possible colors (0, A, B)  4 Regions (0, A, B, AB)

Opaque Objects

slide-36
SLIDE 36

36

 12 reasonable combinations

Composition Algebra

slide-37
SLIDE 37

37

 For colors that are not premultiplied :

 C = AaA + (1 – aA) BaB  aC = aA + (1 – aA) aB

 For colors that are premultiplied :

 C = A + (1 – aA) B  aC = aA + (1 – aA) aB

Example: C = A over B

slide-38
SLIDE 38

38

 Photoshop masks and layers

Example: Masks

slide-39
SLIDE 39

39

 Multiple blend modes and transparency

Example: Transparency

slide-40
SLIDE 40

40

Example: Green Screen

slide-41
SLIDE 41

41

  • Image Filtering / Image Manipulation
  • Pixel operations
  • Filtering
  • Composition
  • Quantization
  • Warping and Morphing
  • Sampling, Reconstruction and Aliasing

Image Processing

slide-42
SLIDE 42

42

  • Image Filtering / Image Manipulation
  • Pixel operations
  • Filtering
  • Composition
  • Quantization
  • Halftoning
  • Dithering
  • Warping and Morphing
  • Sampling, Reconstruction and Aliasing

Image Processing

slide-43
SLIDE 43

43

 Artifact due to limited intensity resolution

 Frame buffers have limited number of bits per pixel  Physical devices have limited dynamic range

Quantization

slide-44
SLIDE 44

44

P(x,y) = trunc(I(x,y))

Uniform Quantization

slide-45
SLIDE 45

45

 Grayscale image with decreasing bits per pixel  How to prevent dramatic decrease in quality?

Quantization

slide-46
SLIDE 46

46

 Halftoning

 Classical halftoning  Halftoning Patterns

 Dithering

 Random dither  Ordered dither  Error diffusion dither

Reducing Effects of Quantization

slide-47
SLIDE 47

47

 Use dot size to represent intensity  Area of dots proportional to intensity in image

Classical Halftoning

slide-48
SLIDE 48

48

 Use cluster of pixels to represent intensity  Trade spatial resolution for intensity resolution  How many intensities are there for n x n?

Halftone Patterns

slide-49
SLIDE 49

49

 Distribute errors among pixels

 Exploit spatial integration in our eye  Display greater range of perceptible intensities

8bit 1bit 1bit dithered

Dithering

slide-50
SLIDE 50

50

 Randomize quantization errors  Errors will appear as noise  P(x,y) = trunc(I(x,y)+noise(x,y))

Random Dither

slide-51
SLIDE 51

51

 Results are ... Random 8 bit 1 bit 1bit random dither

Random Dither

slide-52
SLIDE 52

52

 Pseudo-random quantization errors  Matrix stores pattern of thresholds

n = 2 for each y for each x

  • ldpixel = I[x][y] + D[x mod n][y mod n]

P[x][y] = trunc(oldpixel)

Ordered Dither

slide-53
SLIDE 53

53

 Slightly better result

8 bit 1 bit 1 bit Ordered dither

Ordered Dither

slide-54
SLIDE 54

54

 Errors are distributed to pixels right and below  Robert W. Floyd and Louis Steinberg, 1976  for each y  for each x  P[x][y] = trunc(I[x][y])  # Pass error to other pixels  e = I[x][y] - P[x][ y]  I[x+1][y] = I[x+1][y]+7/16*e  I[x-1][y+1] = I[x-1][y+1]+3/16*e  I[x][y+1] = I[x][y+1]+5/16*e  I[x+1][y+1] = I[x+1][y+1]+1/16*e

Error Diffusion Dither

slide-55
SLIDE 55

55

8 bit 1 bit 1 bit 1 bit Original Random Ordered Floyd-Steinberg

Dither Comparison

slide-56
SLIDE 56

56

  • Image Filtering / Image Manipulation
  • Pixel operations
  • Filtering
  • Composition
  • Quantization
  • Warping and Morphing
  • Scale
  • Rotate
  • Arbitrary Warps
  • Sampling, Reconstruction and Aliasing

Image Processing

slide-57
SLIDE 57

57

  • Transform image pixels
  • Mapping
  • Forward
  • Inverse
  • Resampling

Warping

slide-58
SLIDE 58

58

  • Define image transformation
  • Describe the destination (x, y) for every location (u, v) in the source (or

vice-versa, if inversible)

v y u x

Mapping

slide-59
SLIDE 59

 Scale by factor

 x = factor * u  y = factor * v

v y Scale 0.8 u x

59

Example: Scaling

slide-60
SLIDE 60

 Rotate by angle θ

 x = u cos(θ) - v sin(θ)  y = u sin(θ) + v cos(θ)

v y Rotate 30 u x

60

Example: Rotate

slide-61
SLIDE 61

61

Example: Shear

 Shear in X by factor

 x = u + factor * v  y = v

 Shear in

Y by factor

 x = u  y = v + factor * u

slide-62
SLIDE 62

 Any function of u and v

 x = 𝒈𝒚(u,v)  y = 𝒈𝒛(u,v)

62

Other Mappings

slide-63
SLIDE 63

 Forward mapping

for(int u=0; u<umax; u++) { for(int v=0; v<vmax; v++) { float x = 𝒈𝒚 (u,v); float y = 𝒈𝒛(u,v); dst(x,y) = src(u,v); } }

63

Implementation

slide-64
SLIDE 64

 Iterate over source image  But ...

64

Forward Mapping

slide-65
SLIDE 65

 Iterate over source image  Many pixels map on the same destination!

65

Forward Mapping

slide-66
SLIDE 66

 Iterate over source image  Some pixels will not be covered!

66

Forward Mapping

slide-67
SLIDE 67

 Inverse mapping

for(int x=0; x<xmax; x++) { for(int y=0; y<ymax; y++) { float u = 𝒈𝒚

−𝟐 (x,y);

float v = 𝒈𝒛

−𝟐 (x,y);

dst(x,y) = resample_src(u,v); } }

67

Implementation 2

slide-68
SLIDE 68

 Iterate over destination image

 Must resample source  Much simpler but may oversample

68

Inverse Mapping

slide-69
SLIDE 69

69

  • Image Filtering / Image Manipulation
  • Pixel operations
  • Filtering
  • Composition
  • Quantization
  • Warping and Morphing
  • Resampling
  • Nearest
  • Bilinear
  • Others

Image Processing

slide-70
SLIDE 70

 Evaluate source image at arbitrary (u,v)  (u,v) coordinates are generally not integer

70

Resampling

slide-71
SLIDE 71

 Take value of closest pixel  Fast! Low quality

int iu = trunc(u+0.5du); int iv = trunc(v+0.5dv); dst(x, y) = src(iu, iv);

71

Nearest neighbor

slide-72
SLIDE 72

 Bilinearly interpolate four closest pixels

 a = linear interpolation of src(u1, v2) and src(u2, v2)  b = linear interpolation of src(u1, v1) and src(u2, v1)  dst(x, y) = linear interpolation of “a” and “b”

 Reasonably Fast. Good quality.

72

Bilinear Filtering

slide-73
SLIDE 73

 Bicubic Filtering

 Considers 4x4 pixels (16 pixels)  Smoother, less artifacts  Computationally expensive

 Gaussian Filtering

 Uses weighted sum of neighborhood  Weights are normalized using Gaussian function

73

Other Filters

slide-74
SLIDE 74

 Comparison of resampling quality Nearest Bilinear Bicubic

74

Filtering Comparison

slide-75
SLIDE 75

75

  • Ask questions, please!!!
  • Be communicative
  • www.slido.com #PPGSO10
  • More active you are, the better for you!

How the lectures should look like #2

slide-76
SLIDE 76

76

Raycasting

Next Week

slide-77
SLIDE 77

77

Acknowledgements

 Thanks to all the people, whose work is shown here and whose

slides were used as a material for creation of these slides:

Matej Novotný, GSVM lectures at FMFI UK Peter Drahoš, PPGSO lectures at FIIT STU

slide-78
SLIDE 78

78

www.slido.com #PPGSO10 martin.madaras@stuba.sk

Questions ?!