In the name of Allah the compassionate, the merciful p , f D D - PowerPoint PPT Presentation

In the name of Allah the compassionate, the merciful p , f

D D IGITAL V IDEO P ROCESSING V P S. Kasaei Room: CE 307 Department of Computer Engineering Sharif University of Technology Sh if U i it f T h l E-Mail: skasaei@sharif.edu Webpage: http://sharif.edu/~skasaei Webpage: http://sharif.edu/ skasaei Lab. Website: http://ipl.ce.sharif.edu

A CKNOWLEDGMENT A CKNOWLEDGMENT Most of the slides used in this course have been provided by: Prof Yao Wang (Polytechnic provided by: Prof. Yao Wang (Polytechnic University, Brooklyn) based on the book: Video Processing & Communications g by: Yao Wang, Jom Ostermann, & Ya-Oin Zhang Prentice Hall, 1 st edition, 2001, ISBN: 0130175471. [SUT Code: TK 5105 2 W36 2001] [SUT Code: TK 5105 .2 .W36 2001].

C HAPTER 14 Error Control in Video Communications

O UTLINE O UTLINE � Necessity & challenge for error control y g Kas saei � Characteristics of typical applications & networks � Overview of techniques � Error resilient encoding � Error concealment decoding � Encoder-decoder-network interactive error control 6

V IDEO C OMMUNICATION S YSTEM V IDEO C OMMUNICATION S YSTEM Kasaei A typical video communication system. 7

C HALLENGE FOR V IDEO C OMMUNICATIONS C OMMUNICATIONS � Effective video communication includes: Kas saei � Reduction of video data rate. � Handling errors & losses in communication networks. � Data communications are not usually subject to � Data communications are not usually subject to strict delay constraints. � Can be handled using network protocols; that use resubmission to ensure error-free delivery. y � Transmission error categories include: � Random bit errors. � Caused by imperfections of physical channels; bit inversion/insertion/deletion. � Erasure errors. � Caused by packet lost in packet networks, long burst errors C d b k t l t i k t t k l b t 8 in storage media due to physical defects, system failures/link downs, random bit errors in VLC coded streams.

C HALLENGE FOR V IDEO C OMMUNICATIONS C OMMUNICATIONS � Real networks are unreliable � Kas saei � In wireless networks: random bit errors, long burst errors, & possibly link downs. � In the Internet: packet loss & variable delay due to In the Internet: packet loss & variable delay due to network congestion. � For real-time applications: � Excessive delay = loss � Excessive delay = loss. � Real networks are heterogeneous in bandwidth & reliability & reliability. � Video signals are delay-sensitive: � One cannot rely on retransmission for error O l i i f 9 control because of the stringent delay requirement!

D IFFICULTIES OF C ONVENTIONAL S OURCE C ODING T ECHNIQUE C ONVENTIONAL S OURCE C ODING T ECHNIQUE � Conventional source coding techniques difficulties: g q Kas saei � Optimal performance is obtained only for fixed rates & perfect channels. � It results in a poor reconstruction quality when parts of coded data are lost. 10

D IFFICULTIES OF C ONVENTIONAL S OURCE C ODING T ECHNIQUE C ONVENTIONAL S OURCE C ODING T ECHNIQUE � Compressed video data is very sensitive to p y Kas saei transmission errors because of utilizing: � Variable length coding (VLC). � Temporal predictive coding. p p g � Spatial predictive coding. � All contribute to error propagation within the � All contribute to error propagation within the same frame as well as the following frames: � One bit error or packet loss can render the following received data useless received data useless. 11

S PATIOTEMPORAL E RROR P ROPAGATION S PATIOTEMPORAL E RROR P ROPAGATION Kasaei Illustration of spatiotemporal error propagation Illustration of spatiotemporal error propagation. 12

D RIFT P ROBLEM (R EFERENCE M ISMATCH ) (R EFERENCE M ISMATCH ) Kas � Motion compensated temporal prediction � Motion compensated temporal prediction saei should be retained to preserve the coding efficiency. efficiency. � But, loss in a previous frame can cause mismatch between the reference frame used in the encoder & that in the decoder. � Encoder & decoder will be out of sync. 13

D RIFT P ROBLEM (R EFERENCE M ISMATCH ) (R EFERENCE M ISMATCH ) Kasaei Di t Distortion ti Satellite dish refere ref ence ference Transmission encoder decoder 14

E FFECT OF T RANSMISSION E RRORS E FFECT OF T RANSMISSION E RRORS Kasaei 3% loss coded, no loss 5% loss 10% loss 15 Reconstructed video frames using a H.263 coded sequence, subject to packet losses.

C HANNEL C ODING B ASICS C HANNEL C ODING B ASICS � Channel coding: forward error correction C g f Kas saei (FEC): � Adding redundancy bits on compressed source bit to e able e bits to enable error detection & correction. o detectio & co ectio � Simple example: � Adding a parity check bit at the end of a block of data stream can detect all single bit errors. � Channel coding rate: � For every k source bits, add l channel bits, to create n=k+l bits channel coding rate r=k/n . � Well designed code ( e.g., Reed-Solomon (RS) code) can correct t=l/2 error bits in each n -bit block. 16

C HANNEL C ODING B ASICS C HANNEL C ODING B ASICS � Classical Shannon information theory states that: Kas saei � One can separately design the source & channel coders to achieve error-free delivery of a compressed bit stream, as long as the source is represented by a rate below the channel capacity below the channel capacity. � Source coding minimizes the bitrate necessary to satisfy a distortion criterion (Shannon rate-distortion theory). � Channel coding adds just enough redundancy bits to reduce the raw channel error rate to the permitted level. d th h l t t th itt d l l � Such ideal error-free delivery requires infinite delays in implementing FEC delays in implementing FEC. � It is only valid for stationary source & channel. � Requires processing of infinitely long blocks of data. � Delay = infinity! � Delay = infinity! 17

FEC FOR V IDEO T RANSMISSION FEC FOR V IDEO T RANSMISSION � For wireless networks, FEC is necessary to reduce raw bit Kas error rates error rates. saei � For the Internet, errors are mainly due to congestion- caused packet losses. � FEC can be applied across packets to correct/detect packet losses losses. � Unequal error protection (UEP): � Using stronger channel codes ( r=k/n smaller) for more “important” bit streams (base layer). � Best implemented with rate compatible punctured � Best implemented with rate-compatible punctured 18 convolutional (RCPC) code.

T RANSMISSION E RROR R ESILIENCE T RANSMISSION E RROR R ESILIENCE � Joint source & channel coding is often a g Kas more riable scheme. saei � Which allocates a total amount of redundancy between the source & channel coding. g � All error resilient encoding methods work under this promise. d thi i � This is usually accomplished by carefully � This is usually accomplished by carefully designing both the predictive coding loop & the VLC, to limit the extend of error propagation propagation. 19

E RROR C ONCEALMENT E RROR C ONCEALMENT � When a part of image is missing due to Kas transmission errors, the decoder estimates them i i h d d i h saei based on surrounding received samples. � This is accomplished by using the inherent correlation among spatially & temporally adjacent samples among spatially & temporally adjacent samples. � It does not employ any additional bitrates. ☺ � But adds computational complexity at the decoder � � But, adds computational complexity at the decoder. � � To facilitate error concealment in the decoder, the codec & the network transmission protocol must p cooperate. For example: � Important parts should be assigned as a more stringent set of QoS parameters. � Network may provide a feedback channel. N t k id f db k h l 20 � The encoder does not use this part for prediction.

T YPICAL V IDEO A PPLICATIONS & C OMMUNICATION N ETWORKS & C OMMUNICATION N ETWORKS � Some basic definitions: Kas saei � Latency: maximally allowed average end-to-end delay. � Jitter: delay variation. � Real-time delivery: compressed data are transferred at R l ti d li d d t t f d t a speed that matches the coded video source rate. 21

Q O S R EQUIREMENTS OF T YPICAL V IDEO A PPLICATIONS OF T YPICAL V IDEO A PPLICATIONS � Video applications consist of: pp Kas saei � Interactive two-way visual communications. � One-way video streaming. � One-way video downloading. O id d l di � No difference from file downloading. 22

I NTERACTIVE T WO -W AY V ISUAL C OMMUNICATIONS C OMMUNICATIONS � Examples include: Kas � Teleconferencing, Video telephony, Virtual classrooms. T l f i Vid l h Vi l l saei � They have very stringent delay requirements. � The latency & jitter must be kept within a certain limit. � For intercontinental telephone conversations (& video conferencing), the ITU-T G.114 standard recommends: � <=150 ms (one way) desired � <=150 ms (one way) desired. � 150-400 ms can be acceptable. � >400 ms not acceptable. � Audio & video must be in sync to maintain lip synchronization. h i ti 23 � Both encoding & decoding must be completed in real-time.