Exercise 4a: Padding of Input Image Main Goal : The image codec - - PowerPoint PPT Presentation

Exercise 4a: Padding of Input Image

Main Goal: The image codec should work with all combinations of sizes and block sizes Encoder Header: Original image size W ×H and block size B Calculate image dimension W ′×H′ used for coding

Both W ′ and H′ are multiples of block size B

Fill missing samples with suitable values

Suggestion: Constant border extension (see figure)

Decoder Read image size W ×H and block size B from header Calculate image dimension W ′×H′ used for coding Remove added samples before outputting the reconstructed image of size W ×H

s[ x, y ] = s[ W −1, y ] s[ x, y ] = s[ x, H−1 ]

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding

Exercise 4b: Discrete Cosine Transform (DCT-II)

Goal: Implement forward and inverse transform Separable DCT-II for specified block size Verify implementation:

Successive application of DCT and inverse DCT should yield original block of samples [ very small errors (PSNR > 80 dB) might occur ]

Encoder Apply separable DCT-II to each block of samples Block of transform coefficients (floating-point) Decoder Apply separable IDCT-II to block of coefficients Includes rounding to closest integer Reconstructed block of samples (integer)

# ----- discrete cosine transform

• class

Transform: # constructor : #

• create

transform matrices for # DCT -II of given block size def __init__( self , blockSize ): ... # Forward transform: #

• apply

separable DCT -II to given # block of samples (integers) #

• return

block of transform # coefficients (floating -point values) def dct( self , sampleBlock ): ... # Inverse transform: #

• apply

separable inverse DCT -II # to block of transform coefficients #

• round

result to closest integer #

• return

reconstructed block #

• f

samples (integer values) def idct( self , transformBlock ): ...

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding

Exercise 4c: Scalar Quantization (first version)

Goal: Implement scalar quantization Uniform Reconstruction Quantizer (URQ) with simple rounding at encoder side Quantization Parameter QP Integer value controlling strength of quantization Add as optional command line parameter

Option “-qp”, range [ 0; 31 ], default = 12

Write/read to/from header with 8 bits Quantization and Dequantization Quantization step size: ∆ = 2

QP 4

Quantization (encoder): q = round u ∆

• Dequantization (decoder):

u′ = ∆ · q

# ----- uniform reconstruction quantizer

• class

Quantization : # constructor : #

• calculate

quantization step size def __init__( self , QP ): ... # Quantization (simple version ): #

• divide

transform coefficients (float) # by quantization step size (float) #

• round

result to closest integer #

• return

block of quantization # indexes (integer values) def quant( self , transformBlock ): ... # Dequantization : #

• multiply

quantization indexes #

• f given

block (integer) with # quantization step size (float) #

• return

reconstructed block #

• f

coefficients (floating point) def dequant( self , quantIndexes ): ...

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding

Exercise 4d: Integration and Testing

Goal: First real image codec (with transform coding of sample blocks) Integration Integrate padding, transform, quantization Check header: Image size, block size, QP At end of block processing in decoder

Add clipping to 8-bit range [ 0; 255 ]

Testing and Evaluation Select at least 3 different test images Generate rate-PSNR curves by running codec

with different QP values {8, 12, 16, 20, 24} measure rate and PSNR (using PSNR tool)

Repeat with different block sizes {1, 2, 4, 8, 16, 32} Compare curves to JPEG (see git repository)

Encoder operation per block

1 Subtract middle gray value (128) 2 Forward transform (exercise 4b) 3 Quantization (exercise 4c) 4 Entropy coding (Exp-Golomb code)

Decoder operation per block

1 Entropy decding (Exp-Golomb code) 2 Dequantization (exercise 4c) 3 Inverse transform (exercise 4b) 4 Add middle gray value (128) 5 Clip to 8-bit range [ 0; 255 ] (add)

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding

Exercise 4e: Diagonal Scanning of Quantization Indexes

Goal: Prepare improved entropy coding by adding suitable scan (similar to zig-zag scan) Scanning of Quantization Indexes Implement diagonal scan (as used in HEVC) shown in figure Implementation should work for all square block sizes Suggestion: Store scan pattern in array Initialize array at start of encoder/decoder depending on block size used Test Implementation Use the diagonal scan for entropy encoding and decoding (i.e., quantization indexes are en-/decoded in scan order) Coding efficiency should not change (same bitstream size) (scan used in HEVC)

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding