Image and Video Coding: Representation, Acquisition, Display ... - - PowerPoint PPT Presentation

Image and Video Coding: Representation, Acquisition, Display

encoder decoder ... 10011 ...

Representation Formats

Representation Formats

encoder decoder representation format bitstream ... 10011 ... representation format

Raw Data Formats for Exchanging Pictures and Videos Output of camera, input to video encoder, output of video decoder, input to display Examples: BT.601 (SD), BT.709 (HD), BT.2020 (UHD) Images and Videos: Sample arrays characterized by Spatio-temporal sampling, linear color space (color gamut) Non-linear encoding (transfer function), color representation format (RGB, YCbCr, ...) Quantization (bit depth)

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 2 / 43

Representation Formats / Spatio-Temporal Sampling

Spatio-Temporal Sampling

Discrete Representation of Continuous Irradiance Pattern on Image Sensor Each image is represented by a W ×H array of samples cn[ ℓ, m ] = ccont( ℓ · ∆x, m · ∆y, n · ∆t ) Spatial sampling is typically done by image sensor (photocells of finite size) Video: Multiple pictures taken per second (for example: 50 per second) Gray-Level Image 2D array of samples x y Color image Three 2D arrays of samples (one per color components)

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 3 / 43

Representation Formats / Spatio-Temporal Sampling

Spatial Sampling

Orthogonal Progressive Sampling Image width W and image height H Sample aspect ratio (SAR) SAR = ∆x ∆y Picture aspect ratio (PAR) PAR = W · ∆x H · ∆y = W H · SAR Interlaced Sampling (old special case) Top field: Even scan lines Bottom field: Odd scan lines Top and bottom fields are alternatively scanned at successive time instances

x y ∆x ∆y W · ∆x H · ∆y

top field top field top field bottom field bottom field bottom field

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 4 / 43

Representation Formats / Spatio-Temporal Sampling

Common Picture Formats

picture size sample aspect picture aspect (in samples) ratio (SAR) ratio (PAR) 720 × 576 12 : 11 4 : 3 standard 720 × 480 10 : 11 4 : 3 definition 720 × 576 16 : 11 16 : 9 720 × 480 40 : 33 16 : 9 1280 × 720 1 : 1 16 : 9 high 1440 × 1080 4 : 3 16 : 9 definition 1920 × 1080 1 : 1 16 : 9 ultra-high 3840 × 2160 1 : 1 16 : 9 definition 7680 × 4320 1 : 1 16 : 9

Note: In SD formats, only 704 samples are displayed per line (overscan)

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 5 / 43

Representation Formats / Spatio-Temporal Sampling

Illustration: Spatial Resolution, Sample Aspect Ratio, Picture Aspect Ratio

CIF (352×288), SAR 12 : 11 CIF (352×288), SAR 16 : 11 QCIF (176×144), SAR 16 : 11

• n display (512×288); PAR 4 : 3
• n display (512×288); PAR 16 : 9
• n display (512×288); PAR 16 : 9

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 6 / 43

Representation Formats / Color Space

Representation of Color Images

Color Components Trichromatic vision: Require 3 color components Representation formats are display-oriented: Linear RGB color space with real primaries Require conversion from camera color space to RGB color space of the representation format   R G B  

• rep. format

= M3×3 ·   R G B  

camera

Conversion matrix M3×3 typically includes white balancing

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 7 / 43

Representation Formats / Color Space

Color Gamut

RGB color space: Specified by chromaticity coordinates of primaries and the white point The chosen RGB color space determines the representable color gamut

BT.709 BT.2020 ProPhoto XYZ xr 0.6400 0.7080 0.7347 1.0000 red yr 0.3300 0.2920 0.2653 0.0000 xg 0.3000 0.1700 0.1596 0.0000 green yg 0.6000 0.7970 0.8404 1.0000 xb 0.1500 0.1310 0.0366 0.0000 blue yb 0.0600 0.0460 0.0001 0.0000 xw 0.3127 0.3127 0.3457 0.3333 white yw 0.3290 0.3290 0.3585 0.3333

D65 white BT.709 (HD) BT.2020 (UHD) [ wide color gamut ] human gamut ProPhoto RGB XYZ x y

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 8 / 43

Representation Formats / Color Space

Color Space Conversion

RGB color spaces are linearly related to XYZ color space (and cone excitation space LMS) XYZ color space is given by color-matching functions ¯ x(λ), ¯ y(λ), ¯ z(λ) Determination of Conversion Matrix to XYZ space

  X Y Z  =   Xr Xg Xb Yr Yg Yb Zr Zg Zb  ·   R G B   white point − − − − − − − − →   Xw/Yw 1 Zw/Yw  =   Xr Xg Xb Yr Yg Yb Zr Zg Zb  ·   1 1 1  

Use X/Y = x/y and Z/Y = (1 − x − y)/y  

xw yw

1

1−xw−yw yw

 =   

xr yr xg yg xb yb

1 1 1

1−xr −yr yr 1−xg−yg yg 1−xb−yb yb

     Yr Yg Yb   Solve linear equation system for unknown values Yr, Yg, and Yb Determine resulting matrix (using X = x/y · Y and Z = (1 − x − y)/y · Y )

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 9 / 43

Representation Formats / Color Space

Example: Conversion between XYZ and sRGB (BT.709)

red green blue white x 0.6400 0.3000 0.1500 0.3127 y 0.3300 0.6000 0.0600 0.3290

Linear equation system (only 8 digits of precision shown)

  0.95045593 1 1.08905775  =   1.93939394 0.5 2.5 1 1 1 0.09090909 0.16666667 13.16666667     Yr Yg Yb  

Solution (6 digits of precision shown)

Yr = 0.212673, Yg = 0.715152, Yb = 0.072175

Resulting conversion matrix (4 digits of precision)

  X Y Z  =   0.4125 0.3576 0.1804 0.2127 0.7152 0.0722 0.0193 0.1192 0.9503     R G B  

sRGB

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 10 / 43

Representation Formats / Non-Linear Encoding

Non-Linear Encoding / Gamma Encoding

Human Vision: Non-Linear Brightness Perception Weber-Fechner law: Perceivable difference in luminance depends on background luminance Certain amount of quantization noise is more visible in dark image regions Reduce effect by non-linear encoding components before quantization E ′ = fTC(E) Gamma Encoding / Gamma Decoding Encoder side: Approximation by power law Y ′ = fTC(Y ) = Y γe with encoding gamma γe ≈ 1/2.2 ≈ 0.45 with Y being the relative luminance in range [ 0; 1 ] Decoder side: Invert the gamma encoding Y = f −1

TC (Y ′) = (Y ′)γd

with decoding gamma γd ≈ 1/γe ≈ 2.2

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 11 / 43

Representation Formats / Non-Linear Encoding

Transfer Characteristics

linear increasing Y linear increasing Y ′ = fTC (Y )

0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 BT.709 BT.2020 γe = 1 / 2.2 linear encoding non-linear encoded signal E' linear component signal E

Representation Formats Piecewise-defined transfer function (linear function for very small values) E ′ = fTC(E) = κ · E : 0 ≤ E < b a · E γ − (a − 1) : b ≤ E ≤ 1 BT.709 / BT.2020 : γ = 0.45, κ = 4.5, a ≈ 1.0993, b ≈ 0.0181 High Dynamic Range (HDR): Modified transfer functions (PQ, HLG)

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 12 / 43

Representation Formats / YCC Color Formats

YCC Color Formats

Reasons for using YCC color formats Color television: Add color difference information to black & white television Decorrelation of color components (remember: opponent processes) Luminance-related signal L Two color difference signals C1, C2 (e.g., yellow-blue & red-green) Consider mapping LC1C2 → RGB → XYZ   X Y Z   =   Xr Xg Xb Yr Yg Yb Zr Zg Zb   ·   Rℓ Rc1 Rc2 Gℓ Gc1 Gc2 Bℓ Bc1 Bc2   ·   L C1 C2   Desirable properties

1 Achromatic signals (x = xw and y = yw) have C1 = C2 = 0 2 Changes in C1 and C2 do not have any impact on relative luminance Y

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 13 / 43

Representation Formats / YCC Color Formats / The YCbCr Format

YCbCr: Achromatic Signals have Zero Color Differences

  X Y Z   =   Xr Xg Xb Yr Yg Yb Zr Zg Zb   ·   Rℓ Rc1 Rc2 Gℓ Gc1 Gc2 Bℓ Bc1 Bc2   ·   L C1 C2   Fulfilling desirable properties

1 Achromatic signals (x = xw and y = yw) have C1 = C2 = 0

In the RGB format, this means C1 = C2 = 0 = ⇒ R = G = B (white/gray point) Hence, we require Rℓ = Gℓ = Bℓ This choice yields   Rℓ Rc1 Rc2 Gℓ Gc1 Gc2 Bℓ Bc1 Bc2   =   a ? ? a ? ? a ? ?  

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 14 / 43

Representation Formats / YCC Color Formats / The YCbCr Format

YCbCr: Color Differences have no Impact on Luminance

  X Y Z   =   Xr Xg Xb Yr Yg Yb Zr Zg Zb   ·   a Rc1 Rc2 a Gc1 Gc2 a Bc1 Bc2   ·   L C1 C2   Fulfilling desirable properties

2 Changes in C1 and C2 do not have any impact on relative luminance Y

Product of conversion matrices must have zero entries for YC1 and YC2 YC1 = Yr · Rc1 + Yg · Gc1 + Yb · Bc1 = 0 YC2 = Yr · Rc2 + Yg · Gc2 + Yb · Bc2 = 0 Early researchers additionally chose Rc1 = 0 and Bc2 = 0 These choices yield (with b = Bc1 and c = Rc2)   Rℓ Rc1 Rc2 Gℓ Gc1 Gc2 Bℓ Bc1 Bc2   =   a c a −b · Yb

Yg

−c · Yr

Yg

a b  

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 15 / 43

Representation Formats / YCC Color Formats / The YCbCr Format

YCbCr: Meaning of Components

  X Y Z   =   Xr Xg Xb Yr Yg Yb Zr Zg Zb   ·   a c a −b · Yb

Yg

−c · Yr

Yg

a b   ·   L C1 C2   Relative luminance Y (remember: Yr + Yg + Yb = 1) Y = Yr · R + Yg · G + Yb · B = Yr · (aL + cC2) + Yg ·

• aL − b Yb

Yg C1 − c Yr Yg C2

• + Yb · (aL + bC1)

= a · (Yr + Yg + Yb) · L = a · L L = Y /a Blue and red components R and B B = a · L + b · C1 C1 = (B − a · L)/b = (B − Y )/b R = a · L + c · C2 C2 = (R − a · L)/c = (R − Y )/c

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 16 / 43

Representation Formats / YCC Color Formats / The YCbCr Format

YCbCr: Scaling Factors

The choices yields: L = sℓ · Y C1 = sc1 · (B − Y ) C2 = sc2 · (R − Y ) with sℓ, sc1, sc2 being arbitrary non-zero scaling factors Interpretation of components L – Scaled version of relative luminance C1, C2 – Difference between a primary and the relative luminance Y Scaling factors are chosen so that L ∈ [ 0; 1 ] and C1, C2 ∈ [ −0.5; 0.5 ] Yields the following transform RGB → LC1C2 L = Yr · R + (1 − Yr − Yb) · G + Yb · B C1 = (B − Y ) / (2 − 2Yb) C2 = (R − Y ) / (2 − 2Yr)

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 17 / 43

Representation Formats / YCC Color Formats / The YCbCr Format

The Y’CbCr Color Format

Decision in early years of television: YCC transform after gamma encoding Transformation is given by Luma : Y ′ = KR · R′ + (1 − KR − KB) · G ′ + KB · B′ with KR = Yr Chroma : Cb = (B′ − Y ′) / (2 − 2KB) KB = Yb Cr = (R′ − Y ′) / (2 − 2KR) BT.709: KR = 0.2126, KB = 0.0722 BT.2020: KR = 0.2627, KB = 0.0593

color image luma comp. Y ′ chroma comp. Cb chroma comp. Cr

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 18 / 43

Representation Formats / YCC Color Formats / The YCbCr Format

Advantage of Y’CbCr Format

R’ Y’ G’ Cb B’ Cr

Transform into Y’CbCr domain Decorrelates RGB data (or cone responses) for typical natural images Color components can be independently quantized / coded Quantization noise is introduced in perceptually meaningful way

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 19 / 43

Representation Formats / YCC Color Formats / The YCoCg Format

Alternative YCC Format: The YCoCg Format

The YCoCg Transform Forward and inverse transform are given by   Y ′ Co Cg   =   0.25 0.50 0.25 0.50 0.00 −0.50 −0.25 0.50 −0.25     R′ G ′ B′     R′ G ′ B′   =   1 1 −1 1 1 1 −1 −1     Y ′ Co Cg   Lifting-based Reversible Variation: YCoCg-R Chroma components have twice the range as in YCbCr or the standard YCoCg RGB → YCoCg :

Co = R − B tmp = B + Co / 2 Cg = G − tmp Y = tmp + Cg / 2

YCoCg → RGB :

tmp = Y − Cg / 2 G = Cg + tmp B = tmp − Co / 2 R = B + Co

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 20 / 43

Representation Formats / YCC Color Formats / The YCoCg Format

Comparison: RGB, YCbCr, YCoCg

R’G’B’ Y’CbCr Y’CoCg

theoretical gain for Kodak set color format coding gain RGB (reference) 0.00 dB YCbCr (BT.709) 3.79 dB YCoCg 4.54 dB

[ Malvar 2003 ]

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 21 / 43

Representation Formats / YCC Color Formats / Chroma Subsampling

Chroma Subsampling

Property of Human Vision Contrast sensitivity: Human beings are more sensitive to high-frequency components in isochromati than isoluminant stimuli Chroma components are often downsampled for saving bit rate 4:4:4 – No downsampling 4:2:2 – Factor of two in horizontal direction 4:2:0 – Factor of two in both horizontal and vertical direction Chroma sample locations are specified in representation format or video bitstream

4:4:4 4:2:2 4:2:0

BT.2020

4:2:0

MPEG-1

4:2:0

MPEG-2

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 22 / 43

Representation Formats / YCC Color Formats / Chroma Subsampling

Color Sampling Formats

R’G’B’ Y’CbCr 4:4:4 Y’CbCr 4:2:2 Y’CbCr 4:2:0 most common chroma format

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 23 / 43

Representation Formats / YCC Color Formats / Chroma Subsampling

Demonstration: Difference in Luma and Chroma Perception

Compare luma and chroma perception Selective down-sampling for luma or chroma channels

1 Down-sample by factor 2 in both directions 2 Up-sample by factor 2 in both directions

Order of presentation

1 Original luma and chroma components 2 Down-sampled luma component but original chroma components 3 Original luma and chroma components (repeated) 4 Original luma component but down-sampled chroma components

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 24 / 43

Representation Formats / YCC Color Formats / Chroma Subsampling

Demonstration: Original Picture

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 25 / 43

Representation Formats / YCC Color Formats / Chroma Subsampling

Demonstration: Down-Sampled Luma Component

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 26 / 43

Representation Formats / YCC Color Formats / Chroma Subsampling

Demonstration: Original Picture (Repeated)

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 27 / 43

Representation Formats / YCC Color Formats / Chroma Subsampling

Demonstration: Down-Sampled Chroma Components

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 28 / 43

Representation Formats / Quantization of Sample Values

Quantization of Sample Values

Discrete-Amplitude Samples ITU-R Recommendations BT.601, BT.709, and BT.2020 specify Y = round

• (219 · E ′

Y + 16) · 2B−8

Cb = round

• (224 · E ′

Cb + 128) · 2B−8

Cr = round

• (224 · E ′

Cr + 128) · 2B−8

where B specifies the bit depth (in bits per sample) Typical bit depths: 8, 10, or 12 bits per sample Footroom and Headroom Unused values of the range of B-bit integer values [ 0; 2B − 1 ] Allow implementation of signal processing operations without clipping Example for other usage: xvYCC color space

Footroom/headroom is used for extending color gamut

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 29 / 43

Representation Formats / Quantization of Sample Values

Illustration: Impact of Quantization (in YCbCr)

8 bits per sample (16 Mio colors) 4 bits per sample (4096 colors) 3 bits per sample (512 colors) 2 bits per sample (64 colors)

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 30 / 43

Representation Formats / Summary

Intermediate Summary: Representation Formats

Spatio-temporal sampling

Each color component of a picture: W × H array of samples Sample aspect ratio, picture aspect ratio, frame rate

Linear RGB color space

Chromaticity coordinates of primaries and white point Specifies color gamut: Range of representable colors

Non-linear encoding

Gamma encoding approximates human brightness perception Quantization noise is introduced in a perceptually meaningful way

YCC color formats

Decorrelation of RGB data (and cone responses) Allows subsampling of chroma components

Quantization

Represent samples as discrete-amplitude values Uniform quantization specified by bit depth

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 31 / 43

Image Acquisition

Image Acquisition

Lens Projects the real-world scene

• nto the image plane

Image sensor Converts analog irradiance pattern into an array of image samples Image processor Analog-to-digital conversion Demosaicing Gamma encoding White balancing, color space conversion Denoising, sharpening, etc. Typically: Compression

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 32 / 43

Image Acquisition / Image Sensor

Image Sensor

saturation voltage saturation exposure level voltage exposure

Image sensor in digital cameras Array of light-sensitive photocells (photocell = sample) Photocells employ photoelectric effect Irradiance is converted into electric signal Filter: Remove unwanted wavelengths Exposure-voltage function approximately linear (below saturation level)

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 33 / 43

Image Acquisition / Color Images

Capturing of Color Images

Filter incoming light using three (or more) different color filters Capture filtered color components Convert into signals for color primaries of representation format

capture red- filtered image capture green- filtered image capture blue- filtered image red component green component blue component real image color filters

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 34 / 43

Image Acquisition / Color Images

Three-Sensor Systems

Cameras with three image sensors Light is split into three color components using a trichroic prism assembly (coatings for which reflection/transmission depends on wavelength) Three image sensors: One for each color component Main advantage: High light sensitivity (all photons are used) Disadvantage: Expensive, large, heavy

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 35 / 43

Image Acquisition / Color Images

Sensors with Color Filter Arrays

Single sensor cameras Separate color filter on top of each photocell Photocells have different spectral responses Requires demosaicing (interpolation of unknown sample values) Lower resolution / demosaicing artifacts Bayer pattern Most common type of color filter array Twice as many green than red/blue samples (humans more sensitive to middle wavelengths)

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 36 / 43

Image Acquisition / Demosaicing

Bayer Image Demosaicing

raw sensor data gamma encoding

demosaicing / interpolation

generated color image

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 37 / 43

Image Acquisition / Demosaicing

Demosaicing Artifacts

Only one color component is captured per pixel 67% of the samples have to be interpolated Interpolation can cause visible artifacts: Moiré patterns Interpolation artifacts can be reduced by optical low-pass filter Optical low-pass filter also reduces sharpness

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 38 / 43

Image Acquisition / Image Processor

The Image Processor

Converts Sensor Signal into Representation Format Demosaicing (for sensors with color filter arrays) Conversion into RGB space of representation format, including white balancing Gamma encoding of linear color components Transform into Y’CbCr format (if desired) Final quantization of sample values Additional Processing Steps Algorithms for improving image quality Denoising Sharpening Reduction of lens artifacts Compression (e.g. JPEG, H.264/AVC, or H.265/HEVC)

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 39 / 43

Image Display / Basic Design

Displays

Basic Design of Color Displays Mix (at least) three primary colors per image point Typically: A pixel (picture element) is formed by a red, a green, and a blue sub-pixel Signal Processing in Displays ? Y’CbCr to R’G’B’ conversion (depending on representation format) Gamma decoding (in most cases) Color conversion: Required if primaries of display and rep. format differ  

R G B

 

dis

= M(dis)

XYZ→RGB · M(rep) RGB→XYZ ·

 

R G B

 

rep

Resampling: May be required if resolution does not match

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 40 / 43

Image Display / Basic Design

Illustration of Display Signal Processing

R’G’B’ samples (BT.2020) BT.709 Display

(interpreted as linear luminance levels)

gamma decoding color space conversion

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 41 / 43

Image Display / Display Technologies

Display Technologies

Cathode Ray Tube (CRT)

electron beams shadow mask screen with phosphors electron guns deflection system (magnetic coils) screen

Liquid Crystal Display (LCD)

backlight V polarizer liquid crystals color filters polarizer

Plasma Displays

cell with red phosphor cell with green phosphor cell with blue phosphor

+

• +
• + -

+

• +
• +
• +
• OLED Displays

+ + + + + +

• +

emission of red light emission of green light emission of blue light Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 42 / 43

Summary

Summary of Lecture

Representation Formats Spatio-temporal sampling (image size and sample aspect ratio) Linear color space (specified by chromaticity coordinates of primaries and white point) Non-linear encoding (allows perceptual meaningful quantization) YCC color representation (e.g., Y’CbCr or Y’CoCg) and chroma downsampling Quantization of sample values (specified by bit depth) Image Acquisition Image sensor: Matrix of light-sensitive photocells (linear transfer characteristics) Color images: Three-sensor systems or sensors with color filter array (demosaicing) Image processor: Conversion of sensor data into representation format, additional processing Image Display Mixing of three primary lights for each pixel Technologies: CRT, LCD, Plasma, OLED

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding: Representation, Acquisition, Display 43 / 43