In the name In the name of of Allah Allah the compassionate, the - - PowerPoint PPT Presentation

▶
in the name in the name of of allah allah
SMART_READER_LITE
LIVE PREVIEW

In the name In the name of of Allah Allah the compassionate, the - - PowerPoint PPT Presentation

Jul 09, 2023 413 likes •926 views

In the name In the name of of Allah Allah the compassionate, the merciful Digital Digital Video Processing Video Processing S. Kasaei S. Kasaei Room: CE 307 Department of Computer Engineering Sharif University of T echnology E-Mail:

slide-1
SLIDE 1
slide-2
SLIDE 2

In the name In the name of

  • f Allah

Allah

the compassionate, the merciful

slide-3
SLIDE 3

Digital Video Processing Digital Video Processing

  • S. Kasaei
  • S. Kasaei

Room: CE 307 Department of Computer Engineering Sharif University of T echnology E-Mail: skasaei@sharif edu E Mail: skasaei@sharif.edu Webpage: http://sharif.edu/~skasaei

  • Lab. Website: http://ipl.ce.sharif.edu
slide-4
SLIDE 4

Acknowledgment Acknowledgment

Most of the slides used in this lecture have been provided by: Dr. DapengWu (University of Florida, Department of Electrical & Computer Engineering) based on the book: Video Processing & Communications written by: Yao Wang, Jom Ostermann, & Ya-Oin Zhang Prentice Hall, 1st edition, 2001, ISBN: 0130175471. [SUT Code: TK 5105 .2 .W36 2001].

slide-5
SLIDE 5

Chapter 15 Chapter 15

Streaming Video over the Internet

slide-6
SLIDE 6

Outline Outline

Architecture for Video Streaming Systems

Architecture for Video Streaming Systems

Video Compression Application-Layer QoS Control for Streaming Application Layer QoS Control for Streaming

Video

Continuous Media Distribution Services

Continuous Media Distribution Services

Streaming Services Media Synchronization

Media Synchronization

Protocols for Streaming Video Streaming Video over Wireless IP Networks Streaming Video over Wireless IP Networks

Kasaei Kasaei 6

slide-7
SLIDE 7

What is Video Streaming? What is Video Streaming?

Recent developments in computing

p p g technology, compression technology, high- bandwidth storage devices, & high-speed g g p networks have made it feasible to provide real-time multimedia services over the Internet.

Video streaming refers to real-time

transmission of stored video.

Kasaei Kasaei 7

slide-8
SLIDE 8

Video Streaming Video Streaming

Video streaming implies that the video

content need to be downloaded in full, but it is being played out while parts of th t t b i i d & the content are being received & decoded.

Video streaming typically has bandwidth,

delay, & loss requirements. y q

Kasaei Kasaei 8

slide-9
SLIDE 9

Outline Outline

  • Challenges for quality video transport
  • An architecture for video streaming

– Video compression – Application-layer QoS control – Continuous media distribution services – Streaming server – Media synchronization mechanisms ed sy c

  • ec

s s – Protocols for streaming media

  • Summary

Kasaei Kasaei 9

  • Summary
slide-10
SLIDE 10

What is Video Streaming? g

Access Access SW SW

Receiver 1 Receiver 1

  • Download mode: no delay bound.
  • Streaming mode: delay bound.

Domain B

SW SW

g y

Domain A

Access Access SW SW

Data path

Access Access SW SW

Domain C

Internet

SW SW SW SW

Internet

Source Source

Kasaei Kasaei 10

Sou ce Sou ce Receiver 2 Receiver 2

cnn.com

RealPlayer

slide-11
SLIDE 11

Time-Varying Available Bandwidth

Access Access SW SW

Receiver Receiver

  • No bandwidth reservation.

Domain B

R>=56 kb/s

Data path

Domain A

Data path

Access Access SW SW

56 kb/ R<56 kb/s

Source Source

56 kb/s

RealPlayer

Kasaei Kasaei 11

cnn.com

slide-12
SLIDE 12

Time-Varying Delay

Access Access SW SW

Receiver Receiver

R lPl

Domain B

RealPlayer

Data path

Domain A

Data path

Access Access SW SW

56 kb/ Delayed packets regarded as lost.

Source Source

56 kb/s

Kasaei Kasaei 12

cnn.com

slide-13
SLIDE 13

Effect of Packet Loss

Access Access SW SW

Receiver Receiver No packet loss.

Domain B

Data path

Domain A

Data path

Access Access SW SW

Loss of packets. N i i Source Source No retransmission.

Kasaei Kasaei 13

slide-14
SLIDE 14

Unicast vs. Multicast Unicast vs. Multicast

Unicast

(email, file transfer, Web browsing)

Multicast

(streaming media) (email, file transfer, Web browsing) ( g )

Pros and cons?

Kasaei Kasaei 14

Pros and cons?

slide-15
SLIDE 15

Heterogeneity for Multicast Heterogeneity for Multicast

Receiver 2 Receiver 2

256 kb/s

  • Network heterogeneity.

Domain B

Access Access SW SW

256 kb/s

  • Receiver heterogeneity.

What Quality?

Domain A Domain C

Internet

Access Access SW SW Gateway Gateway

Ethernet Source Source Receiver 1 Receiver 1 Ethernet

Telephone networks

Receiver 3 Receiver 3

64 kb/s 1 Mb/s

Kasaei Kasaei 15

Receiver 3 Receiver 3

What Quality?

slide-16
SLIDE 16

Outline Outline

  • Challenges for quality video transport
  • An architecture for video streaming

– Video compression – Application-layer QoS control – Continuous media distribution services – Streaming server – Media synchronization mechanisms ed sy c

  • ec

s s – Protocols for streaming media

  • Summary

Kasaei Kasaei 16

  • Summary
slide-17
SLIDE 17

Architecture for Video Streaming

Adapts bit streams according to network status & QoS requirements. Packetizes compressed bit streams.

Kasaei Kasaei 17

slide-18
SLIDE 18

Outline Outline

  • Challenges for quality video transport
  • An architecture for video streaming

– Video compression – Application-layer QoS control – Continuous media distribution services – Streaming server – Media synchronization mechanisms ed sy c

  • ec

s s – Protocols for streaming media

  • Summary

Kasaei Kasaei 18

  • Summary
slide-19
SLIDE 19

Video Compression Schemes Video Compression Schemes

Scalable: Gracef ll copies ith the Scalable: Gracefully copies with the bandwidth fluctuations of the Internet. Non scalable Non-scalable.

Kasaei Kasaei 19

slide-20
SLIDE 20

Video Compression Video Compression

L L D D

+

Layer 0 Layer 0 Layer 1 Layer 1

256 kb/s 64 kb/s

D D

+ +

Layer 2 Layer 2

1 Mb/s

Layered video encoding/decoding. [D denotes the decoder.]

Kasaei Kasaei 20

[ ]

slide-21
SLIDE 21

Application of Layered Video Application of Layered Video

Receiver 2 Receiver 2

A

256 kb/s

  • IP multicast

Domain B

Access Access SW SW

  • IP multicast.

2 layers Domain A Domain C

Internet

Access Access SW SW Gateway Gateway

Ethernet

Telephone

Source Source Receiver 1 Receiver 1

Telephone networks

Receiver 3 Receiver 3

64 kb/s 1 Mb/s 3 layers

Kasaei Kasaei 21

1 layer

slide-22
SLIDE 22

Outline Outline

  • Challenges for quality video transport
  • An architecture for video streaming

– Video compression – Application-layer QoS control – Continuous media distribution services – Streaming server – Media synchronization mechanisms ed sy c

  • ec

s s – Protocols for streaming media

  • Summary

Kasaei Kasaei 22

  • Summary
slide-23
SLIDE 23

Application-Layer QoS Control Application-Layer QoS Control

T

  • cope with changing network

p g g conditions & changing presentation quality requested by users. T echniques include:

  • Congestion control is employed to preserve

packet loss & reduce delay packet loss & reduce delay.

  • Error control improves video presentation quality

in the presence of packet loss.

Th l d b h d These are employed by the end systems.

  • Do not require QoS support from routers or

networks. networks.

Kasaei Kasaei 23

slide-24
SLIDE 24

Application-Layer QoS Control Application-Layer QoS Control

Congestion control (using rate control):

  • Source-based requires:

rate-adaptive compression, or rate shaping (forces the source to send at the rate dictated by the rate control algorithm). g )

  • Receiver-based.
  • Hybrid.

Error control:

  • Forward error correction (FEC).
  • Retransmission.
  • Error resilient compression.
  • Error concealment.

Kasaei Kasaei 24

slide-25
SLIDE 25

Congestion Control Congestion Control

  • Window-based vs. rate control. (pros and cons?)

Wi d b d t l R t t l

Kasaei Kasaei 25

Window-based control Rate control

slide-26
SLIDE 26

Source-Based Rate Control Source-Based Rate Control

Kasaei Kasaei 26

slide-27
SLIDE 27

Video Multicast Video Multicast

  • How to extend source-based rate control to multicast?
  • Limitation of source-based rate control in multicast.
  • Trade-off between bandwidth efficiency & service

fl ibilit flexibility.

Kasaei Kasaei 27

Trade-off between efficiency and flexibility.

slide-28
SLIDE 28

Receiver-Based Rate Control Receiver-Based Rate Control

Receiver 2 Receiver 2

256 kb/s

  • IP multicast for layered video.

Domain B

Access Access SW SW

y

2 layers Domain A Domain C Domain C

Internet

Access Access SW SW Gateway Gateway

Ethernet

T l h

Source Source Receiver 1 Receiver 1

Telephone networks

Receiver 3 Receiver 3

64 kb/s 1 Mb/s

Kasaei Kasaei 28

1 layer 3 layers

slide-29
SLIDE 29

Hybrid Rate Control Hybrid Rate Control

Under hybrid rate control:

y

  • Receiver regulates the receiving rate by

adding & dropping channels, while

  • Sender also adjusts transmission rate of each

channel based on feedback from receivers.

Kasaei Kasaei 29

slide-30
SLIDE 30

Rate Shaping Rate Shaping

Is a technique through which the rate of

Is a technique through which the rate of precompressed video bit streams are adapted to a target rate constraint. p g

Rate shaper is an interface (or filter)

between:

  • compression layer & network transport layer,
  • r

k ( h h h h d

  • two network segments (with which the video

stream can be matched to the available network bandwidth). bandwidth).

Kasaei Kasaei 30

slide-31
SLIDE 31

Error Control Error Control

Is classified into four categories:

g

  • Transport-level (FEC & delay-constrained

retransmission).

  • Encoder error-resilient coding.
  • Decoder error concealment.
  • Encoder-decoder interactive error control.

Kasaei Kasaei 31

slide-32
SLIDE 32

Error Control Error Control

  • FEC adds redundant information in
  • riginal message to compensate packet

loss: – Channel coding. – Source coding-based FEC. – Joint source/channel coding.

  • Delay-constrained retransmission is used

y if one-way trip time is short (with respect to the max allowable delay).

Kasaei Kasaei 32

slide-33
SLIDE 33

Channel Coding Channel Coding

Kasaei Kasaei 33

slide-34
SLIDE 34

Delay-Constrained Retransmission Delay Constrained Retransmission

Kasaei Kasaei 34

slide-35
SLIDE 35

Error Control Error Control

  • Error resilient compression addresses loss

recovery from compressed perspective.

  • It attempts to prevent error propagation or

limit the scope of the damage in the compressed layer.

  • Error concealment is applied in the receiver

when packet loss is detected, using: – Spatial interpolation. – Temporal interpolation (motion

Kasaei Kasaei 35

compensated).

slide-36
SLIDE 36

Outline Outline

  • Challenges for quality video transport
  • An architecture for video streaming

– Video compression – Application-layer QoS control – Continuous media distribution services – Streaming server – Media synchronization mechanisms ed sy c

  • ec

s s – Protocols for streaming media

  • Summary

Kasaei Kasaei 36

  • Summary
slide-37
SLIDE 37

Kasaei Kasaei 37

slide-38
SLIDE 38

C i M di Di ib i S i Continuous Media Distribution Services

A d i d t id Q S & hi

  • Are designed to provide QoS & achieve

efficiency over the best-effort Internet. It i l d It includes: – Network filtering/shaping/thinning. – Application-level multicast (overlay networks). ) – Content replication (caching & mirroring).

Kasaei Kasaei 38

mirroring).

slide-39
SLIDE 39

Caching Caching

  • What is caching?
  • Why using caching? WWW means World Wide Wait?
  • Pros and cons?

Kasaei Kasaei 39

slide-40
SLIDE 40

Outline Outline

  • Challenges for quality video transport
  • An architecture for video streaming

– Video compression – Application-layer QoS control – Continuous media distribution services – Streaming server – Media synchronization mechanisms ed sy c

  • ec

s s – Protocols for streaming media

  • Summary

Kasaei Kasaei 40

  • Summary
slide-41
SLIDE 41

Streaming Server Streaming Server

  • Is different from a web server:

– Processes multimedia data under timing constraints. S t i t ti t l ti ( – Supports interactive control operations (e.g., fast forward/backward, random access, & pause/resume). p ) – Must retrieve media components in a synchronous fashion. – It is consist of:

– Cummunicator (transport protocol), O ti t &

Kasaei Kasaei 41

– Operating system, & – Storage system.

slide-42
SLIDE 42

Outline Outline

  • Challenges for quality video transport
  • An architecture for video streaming

– Video compression – Application-layer QoS control – Continuous media distribution services – Streaming server – Media synchronization mechanisms ed sy c

  • ec

s s – Protocols for streaming media

  • Summary

Kasaei Kasaei 42

  • Summary
slide-43
SLIDE 43

Media Synchronization Media Synchronization

  • Why media synchronization?
  • Example: lip-synchronization (video/audio).

Kasaei Kasaei 43

slide-44
SLIDE 44

Outline Outline

  • Challenges for quality video transport
  • An architecture for video streaming

– Video compression – Application-layer QoS control – Continuous media distribution services – Streaming server – Media synchronization mechanisms ed sy c

  • ec

s s – Protocols for streaming media

  • Summary

Kasaei Kasaei 44

  • Summary
slide-45
SLIDE 45

Protocols for Streaming Video Protocols for Streaming Video

  • Provides communication between clients

& streaming servers.

  • It is classified into:

It is classified into:

  • Network-layer protocol,

T t t l

  • Transport protocol,
  • Session control protocol.

Kasaei Kasaei 45

slide-46
SLIDE 46

Protocols for Streaming Video Protocols for Streaming Video

  • Network layer protocol: Internet protocol (IP).
  • Transport protocol:

– Lower layer: user datagram protocol (UDP) & t i i t l t l (TCP) & transmission control protocol (TCP). – Upper layer: Real-time transport protocol (RTP) & Real-time control protocol (RTCP) (RTP) & Real time control protocol (RTCP).

  • Session control protocol:

– Real-time streaming protocol (RTSP): Real time streaming protocol (RTSP): RealPlayer. – Session initiation protocol (SIP): Microsoft

Kasaei Kasaei 46

Windows MediaPlayer; Internet telephony.

slide-47
SLIDE 47

Protocol Stacks

Kasaei Kasaei 47

slide-48
SLIDE 48

Summary Summary

  • Challenges for quality video transport:

Ti i il bl b d idth – Time-varying available bandwidth. – Time-varying delay. – Packet loss – Packet loss.

  • An architecture for video streaming:

– Video compression Video compression. – Application-layer QoS control. – Continuous media distribution services. – Streaming server. – Media synchronization mechanisms.

Kasaei Kasaei 48

– Protocols for streaming media.

slide-49
SLIDE 49

Homework 12 Homework 12

  • Reading assignment:

– Chap. 15 p

  • Computer assignment:

– Course project Course project

Kasaei Kasaei 49

slide-50
SLIDE 50

The End The End