[PPT] - Step Response Metric for Video Encoder Rate Control PowerPoint Presentation

SLIDE 1

1

Step Response Metric for Video Encoder Rate Control Characterisation

NOSSDAV 2016

Ralf Globisch and Keith Ferguson (CSIR Meraka Institute) Yago Sanchez and Thomas Schierl (Fraunhofer HHI)

SLIDE 2

2

Globisch et al Step Response Metric For Video Encoder Rate Control Characterisation NOSSDAV2016

Real-Time Streaming Application

Live Source Encoder Cooperative Function Transmission RC Sender Network Feedback Target rate

SLIDE 4

4

Globisch et al Step Response Metric For Video Encoder Rate Control Characterisation NOSSDAV2016

Motivation

Real-Time Streaming Protocols

– RTP/RTCP , RTSP , SIP – Non HTTP-based adaptive streaming

Video Conferencing: WebRTC, Skype, WeChat

– Real-Time Communication in WEB-browsers RTCWeb1

Video and audio in the browser

– RTP Media Congestion Avoidance Techniques RMCAT2

Receiver feedback results in rate configuration
GCC, NADA, Scream
CC dependent on Codec Rate Control and access

network characteristics

[1] https://tools.ietf.org/wg/rtcweb/ [2] https://tools.ietf.org/wg/rmcat/

SLIDE 5

5

Globisch et al Step Response Metric For Video Encoder Rate Control Characterisation NOSSDAV2016

Goal

Develop a metric that allows characterisation of encoder

rate control algorithms

Metric should be resolution/framerate independent
Different configuration options of a single rate control

implementation can also be characterised.

The goal is *NOT* to compare and rank different rate

control implementations e.g. “x264 is better than OpenH264”

SLIDE 6

6

Globisch et al Step Response Metric For Video Encoder Rate Control Characterisation NOSSDAV2016

Rate Normalisation

Bits per pixel (bpp)
bitrate_kbps = width * height * fps * bpp
Define that an encoder is under stress when using 0.035

bpp, not under stress when using 0.140 bpp.

– For CIF at 30 fps = 106 – 426 kbps – For 720p HD at 25 fps = 806 – 3226 kbps – For 720p HD at 30 fps = 968 – 3870 kbps

SLIDE 7

7

Globisch et al Step Response Metric For Video Encoder Rate Control Characterisation NOSSDAV2016

Step Response Metric: Model

𝑠" # = 𝑆& + 𝑆( − 𝑆& 𝑓,(#,#.)/1

R0 R1 𝜐 1 𝑓 (𝑆( − 𝑆&) t0

SLIDE 8

8

Globisch et al Step Response Metric For Video Encoder Rate Control Characterisation NOSSDAV2016

Step Response Metric: Decay Function

𝑠" # = 𝑆& + 𝑆( − 𝑆& 𝑓,(#,#.)/1
𝜐 = −

∑ (#5 ,#. )6

7 5

∑ 89

|;< = >?@ | ?.>?@

.(#5,#.)

7 5

SLIDE 9

9

Globisch et al Step Response Metric For Video Encoder Rate Control Characterisation NOSSDAV2016

Step Response Metric: Rate Cost

𝑑 =

& 1 (∑

[𝑠

D 𝑢 − 𝑆&]) #.G 1 HI(

RCR=

(@

J ∑

[K< # ,L@])

=.M J 5N.

L@

SLIDE 10

10

Globisch et al Step Response Metric For Video Encoder Rate Control Characterisation NOSSDAV2016

Experimental Setup

YUV420P sequences

– CIF

Soccer (30 fps)
Foreman (30 fps)
Mobile (30 fps)
Four People (30 fps, cropped & scaled from original)

– Base for testing rate normalisation

– 720p

Four People (30fps)
Mobile Calendar (25fps)

SLIDE 11

11

Globisch et al Step Response Metric For Video Encoder Rate Control Characterisation NOSSDAV2016

Experimental Setup (Cntd.)

Codecs

– x264 (git clone http://git.videolan.org/git/x264.git)

preset=ultrafast, tune=zerolatency

– OpenH264 v1.5 (https://github.com/cisco/openh264)

CAMERA_VIDEO_REAL_TIME

– VPP H264v2 (https://sourceforge.net/projects/videoprocessing/ https://github.com/CSIR-RTVC/video-processing-project)

Log (mode=2)
Pow (mode=2)
Cbr (mode=1)

– Frame-skipping disabled! – Automatic I-frame generation disabled

Various switching points tested

SLIDE 12

12

Globisch et al Step Response Metric For Video Encoder Rate Control Characterisation NOSSDAV2016

Rate Plots: Four People (720p)

0.0000 0.0500 0.1000 0.1500 0.2000 0.2500 0.3000 0.000 1.000 2.000 3.000 4.000 5.000 6.000 7.000 8.000 9.000 10.000

Rate (bpp) Time (sec) VPP-Cbr x264 VPP-Pow VPP-Log OpenH264

SLIDE 13

13

Globisch et al Step Response Metric For Video Encoder Rate Control Characterisation NOSSDAV2016

Rate Model: Four People (720p)

0.0000 0.0200 0.0400 0.0600 0.0800 0.1000 0.1200 0.1400 0.1600 0.1800 0.2000 0.000 1.000 2.000 3.000 4.000 5.000 6.000 7.000 8.000 9.000 10.000

Rate (bpp) Time (sec) VPP-Cbr x264 VPP-Pow VPP-Log OpenH264

SLIDE 14

14

Globisch et al Step Response Metric For Video Encoder Rate Control Characterisation NOSSDAV2016

Rate Plot: Mobile (CIF)

0.0000 0.0500 0.1000 0.1500 0.2000 0.2500 0.3000 0.000 1.000 2.000 3.000 4.000 5.000 6.000 7.000 8.000 9.000 10.000

Rate (bpp) Time (sec) VPP-Cbr x264 VPP-Pow VPP-Log OpenH264

SLIDE 15

15

Globisch et al Step Response Metric For Video Encoder Rate Control Characterisation NOSSDAV2016

Rate Model: Mobile (CIF)

0.0000 0.0200 0.0400 0.0600 0.0800 0.1000 0.1200 0.1400 0.1600 0.1800 0.2000 0.000 1.000 2.000 3.000 4.000 5.000 6.000 7.000 8.000 9.000 10.000

Rate (bpp) Time (sec) VPP-Cbr x264 VPP-Pow VPP-Log OpenH264

SLIDE 16

16

Globisch et al Step Response Metric For Video Encoder Rate Control Characterisation NOSSDAV2016

Soccer Foreman Mobile Four People CIF Four People HD Calendar HD Encoder Tau (sec) Rate Cost Ratio Tau (sec) Rate Cost Ratio Tau (sec) Rate Cost Ratio Tau (sec) Rate Cost Ratio Tau (sec) Rate Cost Ratio Tau (sec) Rate Cost Ratio x264 0.29 3.21 0.26 3.14 0.16 2.95 0.32 2.71 0.28 2.68 0.27 2.41 OpenH264 11.07 2.52 0.35 2.21

35.89

3.55 0.33 0.42 0.04 0.08 0.04 0.18 VPP-Log 0.60 1.59 0.27 2.15 0.23 0.72 0.17 1.13 0.19 1.04 0.25 0.95 VPP-Pow 0.58 1.62 0.22 0.85 0.12 0.00 0.09

0.70

0.08

0.57

0.04

0.23

VPP-Cbr 0.01

0.06

0.02

0.02

0.01

0.01

0.02

0.02

0.01

0.01

0.01

0.01

Results: Summary of Encoder Time Constants and Total Rate Cost Ratio

Mid point switch
Third point switch

Soccer Foreman Mobile Four People HD Encoder Tau (sec) Rate Cost Ratio Tau (sec) Rate Cost Ratio Tau (sec) Rate Cost Ratio Tau (sec) Rate Cost Ratio x264 0.29 2.48 0.26 2.23 0.26 1.59 0.41 1.53 OpenH264 15.21 2.41 0.24 0.17

33.76

3.78 0.07

0.06

VPP-Log 1.96 2.29 0.27 0.41 0.31 0.70 0.42 0.57 VPP-Pow 0.78 1.30 0.19 0.02 0.26 0.12 0.19

0.53

VPP-Cbr 0.02

0.02

0.01

0.01

0.01

0.02

0.01

0.01

SLIDE 17

17

Globisch et al Step Response Metric For Video Encoder Rate Control Characterisation NOSSDAV2016

Results: Transition Encoder Response

All Encoder Tau (sec) Rate Cost Ratio x264 0.27 2.47 OpenH264 0.15 0.51 VPP-Log 0.50 1.16 VPP-Pow 0.27 0.29 VPP-Cbr 0.01

0.02

SLIDE 18

18

Globisch et al Step Response Metric For Video Encoder Rate Control Characterisation NOSSDAV2016

Steady State Encoder Rate-Distortion

High Rate
A
A
A
A
Low Rate

Soccer Foreman Mobile Four People HD Calendar HD Encoder Average Encoder Rate (bpp) PSNR (dB) Rate (bpp) PSNR (dB) Rate (bpp) PSNR (dB) Rate (bpp) PSNR (dB) Rate (bpp) PSNR (dB) Rate (bpp) PSNR (dB) x264 0.136 31.27 0.144 32.43 0.140 21.58 0.140 40.40 0.138 29.12 0.139 30.96 OpenH264 0.140 33.32 0.141 34.82 0.190 27.32 0.139 40.21 0.138 32.67 0.150 33.67 VPP-Log 0.149 30.26 0.145 34.27 0.141 25.34 0.141 40.82 0.141 32.00 0.143 32.54 VPP-Pow 0.146 30.21 0.148 34.38 0.142 25.26 0.142 40.98 0.145 32.29 0.144 32.63 VPP-Cbr 0.139 30.02 0.139 34.21 0.139 25.24 0.139 40.85 0.139 32.20 0.139 32.50 Soccer Foreman Mobile Four People HD Calendar HD Encoder Average Encoder Rate (bpp) PSNR (dB) Rate (bpp) PSNR (dB) Rate (bpp) PSNR (dB) Rate (bpp) PSNR (dB) Rate (bpp) PSNR (dB) Rate (bpp) PSNR (dB) x264 0.035 27.03 0.033 25.79 0.034 18.84 0.034 37.11 0.034 28.35 0.034 27.43 OpenH264 0.114 31.83 0.048 29.51 0.156 27.01 0.035 38.28 0.050 32.05 0.081 31.74 VPP-Log 0.073 26.59 0.043 23.78 0.045 19.90 0.033 37.47 0.065 25.71 0.052 26.69 VPP-Pow 0.071 26.59 0.043 23.77 0.045 19.82 0.035 37.69 0.065 25.68 0.052 26.71 VPP-Cbr 0.033 22.51 0.034 22.73 0.035 19.67 0.035 37.86 0.034 22.66 0.034 25.08

SLIDE 19

19

Globisch et al Step Response Metric For Video Encoder Rate Control Characterisation NOSSDAV2016

Results: Steady-state RD considerations

High Rate Low Rate Encoder Rate (bpp) PSNR (dB) Rate (bpp) PSNR (dB) x264 0.139 30.96 0.034 27.43 OpenH264 0.150 33.67 0.081 31.74 VPP-Log 0.143 32.54 0.052 26.69 VPP-Pow 0.144 32.63 0.052 26.71 VPP-Cbr 0.139 32.50 0.034 25.08 Table 6: Averaged Steady State Rate-Distortion

SLIDE 20

20

Globisch et al Step Response Metric For Video Encoder Rate Control Characterisation NOSSDAV2016

Conclusion: Summary

Binary composite metric
Allows characterisation and evaluation of encoder rate

control algorithms and/or configurations in terms of duration and rate cost.

Trade-off between quality and meeting rate constraints.

SLIDE 21

21

Globisch et al Step Response Metric For Video Encoder Rate Control Characterisation NOSSDAV2016

Conclusion: Why is this useful

Assists system designers/implementors to make

informed decisions on codec selection and configuration as well as access network

Allows determination of usefulness of switch in real-time

communication/stream: when/how often

Helpful in determining bugs or improving rate control

SLIDE 22

22

Globisch et al Step Response Metric For Video Encoder Rate Control Characterisation NOSSDAV2016

Conclusion & Future Work

Code & Scripts

– https://github.com/CSIR-RTVC/CodecStepResponse – Latest VPP H264v2 to be released

Evaluate x265 rate adaptation
Add VP8 & 9 codec support

SLIDE 23

23

Thank you

Name (rglobisch@csir.co.za)

SLIDE 24

24

Globisch et al Step Response Metric For Video Encoder Rate Control Characterisation NOSSDAV2016

Analysis Client 1 Client 2 Client 3 Client 4 Studio channel Satellite channel Playlist channel Activity database Transcoder Servers Media Switching Servers

INTERNET

Channel viewing

viewing stats

Remote channel

User interaction Media Web Server Media App Server

Remote Transcoder Server

rtsp://news.mm1

MEME-B Mobile IPTV Platform

Stripped messaging

social media

Step Response Metric for Video Encoder Rate Control Characterisation

NOSSDAV 2016

Contents

Real-Time Streaming Application

Motivation

network characteristics

Goal

rate control algorithms

implementation can also be characterised.

control implementations e.g. “x264 is better than OpenH264”

Rate Normalisation

bpp, not under stress when using 0.140 bpp.

Step Response Metric: Model

𝑠" # = 𝑆& + 𝑆( − 𝑆& 𝑓,(#,#.)/1

Step Response Metric: Decay Function

∑ (#5 ,#. )6

7 5

∑ 89

|;< = >?@ | ?.>?@

.(#5,#.)

7 5

Step Response Metric: Rate Cost

& 1 (∑

[𝑠

D 𝑢 − 𝑆&]) #.G 1 HI(

(@

J ∑

[K< # ,L@])

=.M J 5N.

L@

Experimental Setup

Experimental Setup (Cntd.)

Rate Plots: Four People (720p)

Rate Model: Four People (720p)

Rate Plot: Mobile (CIF)

Rate Model: Mobile (CIF)

Results: Summary of Encoder Time Constants and Total Rate Cost Ratio

Results: Transition Encoder Response

Steady State Encoder Rate-Distortion

Results: Steady-state RD considerations

Conclusion: Summary

control algorithms and/or configurations in terms of duration and rate cost.

Conclusion: Why is this useful

informed decisions on codec selection and configuration as well as access network

communication/stream: when/how often

Conclusion & Future Work

Thank you

MEME-B Mobile IPTV Platform