Exercise 12: Adding Color Information Goal : Add color information to - - PowerPoint PPT Presentation

Exercise 12: Adding Color Information

Goal: Add color information to bitstream Integration with Minimum Changes Encoder: Upsample chroma to same resolution as luma Decoder: Downsample chroma before outputting pictures Coding of Chroma Information Use same coding mode and motion data for all channels Motion estimation for luma channel only Consider all three channels in mode decision

Y Cb Cb Y Cb Cb Y Cb Cb

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding

Exercise 12a: Prepare Encoder/Decoder Framework

Raw Data Format: Video sequences in YCbCr 4:2:0 format Y, Cb, and Cr are coded after each other (Cb and Cr have half horizontal and vertical resolution) Encoder Modification Add command line parameter that specifies if input video is luma only or YCbCr 4:2:0 Add flag to header that indicates if bitstream contains luma only or YCbCr 4:2:0 Read Cb and Cr components into separate arrays for each video frame (only if parameter specifies YCbCr 4:2:0 format) Decoder Modification Read header flag which indicates whether video is luma only or YCbCr For YCbCr video, write Cb and Cr components when outputting frames

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding

Exercise 12b: Chroma Upsampling and Downsampling

Goal: Implement upsampling and downsampling Encoder Upsample Cb and Cr arrays after reading to same resolution as Y Horizontal upsampling by factor of 2 Vertical upsampling by factor of 2 Use upsampling method of your choice (bi-linear, already implemented half-sample filters, ...) Decoder Downsample Cb and Cr arrays before writing out pictures Write out every second sample in horizontal and vertical direction ! Do not modified reference picture (need full resolution Cb and Cr for motion compensation)

Y Cb Cb Y Cb Cb Y Cb Cb

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding

Exercise 12c: Coding of Chroma Components

General Concept Coding order for a block: Mode and motion, Y residual, Cb residual, Cr residual Coding mode and motion data are only transmitted once per block (valid for all color channels) Residual (cbf + quantization indexes) are coded once per color component Encoder Perform prediction and residual coding for all three color arrays (per block) Consider all color components in mode decision J = SSDY + SSDCb + SSDCr + λ · (RY + RCb + RCr) Perform motion estimation with luma component only (i.e., no changes) Decoder Read coding mode and motion data (once per block) Read cbf and quantization indexes once per color component Perform prediction and reconstruction for all three color channels

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding

Exercise 12d: Integration and Testing

Integrate the Code of other Exercises Reading and writing of Cb and Cr channels [ Exercise 12a ] Upsampling and downsampling of Cb and Cr arrays [ Exercise 12b ] Actual coding of chroma data (including mode decision) [ Exercise 12c ] Ensure that components work correctly together Critically look at the code from other exercises Try to identify potential issues Testing Run simulations for the integrated code Compare coding efficiency with luma only version (using PSNR for luma) ! Remember to specify “420” (instead of “400”) when calculating PSNR for YCbCr 4:2:0 videos

Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding