Make Streaming Great Again IEEE ComSoc Distinguished Lectures - - PowerPoint PPT Presentation

make streaming great again
SMART_READER_LITE
LIVE PREVIEW

Make Streaming Great Again IEEE ComSoc Distinguished Lectures - - PowerPoint PPT Presentation

Make Streaming Great Again IEEE ComSoc Distinguished Lectures Portland State University, Dec. 2019 Ali C. Begen, PhD http://ali.begen.net Ankara BS in EE Toronto, ON Istanbul San Jose, CA Adv. Res. and Dev. Prof. in CS Dept. Atlanta,


slide-1
SLIDE 1

IEEE ComSoc Distinguished Lectures – Portland State University, Dec. 2019

Make Streaming Great Again

Ali C. Begen, PhD

http://ali.begen.net

slide-2
SLIDE 2

Istanbul

  • Prof. in CS Dept.

Ankara

BS in EE

Atlanta, GA

PhD in ECE w/ CS

San Jose, CA

  • Adv. Res. and Dev.

Toronto, ON San Diego, CA

Intern

Konya

2200 guests

Konya

Born, raised and living

slide-3
SLIDE 3

3

AMAAAS*

We will Scratch the Surface on the Red Colored Items

Capture/ Ingest Encoding Encryption Packaging Storage Analytics

Consumption Reliable and Scalable Delivery Production

CDNs Access Core

IEEE ComSoc Distinguished Lecture - Dec. 2019

AMAAAS: Ask me almost anything about streaming

slide-4
SLIDE 4

First Things First

Download vs. Streaming, Why HTTP, Manifests, etc.

slide-5
SLIDE 5

One Request à One Response

Progressive Download

HTTP Request HTTP Response

IEEE ComSoc Distinguished Lecture - Dec. 2019 5

Playback starts only after there is several seconds of data in the playback buffer Download will continue as fast as possible Fetched content will be wasted if the viewer clicks away Can seek only throughout the fetched content

slide-6
SLIDE 6

IEEE ComSoc Distinguished Lecture - Dec. 2019 6

What is Streaming?

Streaming Sounds Cooler!

Client consumption rate is also limited by real-time constraints as

  • pposed to just bandwidth availability

That is, client cannot fetch content not available yet Server transmission rate (loosely or tightly) matches to client consumption rate That is, no buffer overrun or underrun is acceptable

Streaming is transmission of a continuous content from a server to a client and its simultaneous consumption by the client

slide-7
SLIDE 7

Streaming is More Viewer Friendly

IEEE ComSoc Distinguished Lecture - Dec. 2019 7

Playback starts when there is just few seconds of data Download rate will match the encoding bitrate and downloading pauses if the player pauses à Less waste Can seek to anywhere in the entire content

slide-8
SLIDE 8

Video Delivery over HTTP

  • Enables

playback while still downloading

  • Server sends the

file as fast as possible

Progressive Download

  • Enables seeking

via media indexing

  • Server paces

transmission based

  • n encoding rate

Pseudo Streaming

  • Content is divided

into short-duration chunks

  • Enables live

streaming and ad insertion

Chunked Streaming

  • Multiple versions
  • f the content are

created

  • Enables to adapt

to network and device conditions

Adaptive Streaming

IEEE ComSoc Distinguished Lecture - Dec. 2019 8

slide-9
SLIDE 9

Adaptive Streaming over HTTP

Decoding and Presentation Streaming Client Media Buffer Content Ingest (Live or Pre-captured) Multi-rate Encoder Packager Origin (HTTP) Server … … … … Server Storage HTTP GET Request Response

IEEE ComSoc Distinguished Lecture - Dec. 2019 9

slide-10
SLIDE 10

Adapt Video to Web rather than Changing the Web

HTTP Adaptive Streaming (HAS)

  • Why HTTP

Features well-understood naming/addressing and authentication/authorization infrastructure

Provides easy traversal for all kinds of middleboxes (e.g., NATs, firewalls)

Enables cloud access, leverages the existing (cheap) HTTP caching infrastructure in the CDNs

  • Imitation of streaming via short downloads

Downloads small chunks to minimize waste

Enables monitoring consumption and tracking streaming clients

  • Adaptation to varying conditions on the network and different device capabilities
  • Improved quality of experience

Enables shorter stream start time upon zapping or seeking

Reduces skips and freezes

IEEE ComSoc Distinguished Lecture - Dec. 2019 10

The cure may be worse than the disease if you are not careful enough

slide-11
SLIDE 11

Dead, Surviving, Maturing and Newborn Technologies

  • Move Adaptive Stream (Long gone, but some components are in Slingbox)

http://www.movenetworks.com

  • Microsoft Smooth Streaming (Legacy)

http://www.iis.net/expand/SmoothStreaming

  • Adobe Flash (Almost dead, thank God)

http://www.adobe.com/products/flashplayer.html

  • Adobe HTTP Dynamic Streaming (Legacy)

http://www.adobe.com/products/httpdynamicstreaming

  • Apple HTTP Live Streaming (The elephant in the room, the de facto format)

https://tools.ietf.org/html/rfc8216

https://datatracker.ietf.org/doc/draft-pantos-hls-rfc8216bis

  • MPEG DASH and CMAF (The de jure standards)

http://mpeg.chiariglione.org/standards/mpeg-dash

http://mpeg.chiariglione.org/standards/mpeg-a/common-media-application-format

IEEE ComSoc Distinguished Lecture - Dec. 2019 11

slide-12
SLIDE 12

DASH MPD: Template Based and Extensible

Manifests: List of Accessible Segments and Their Timings

MPD Period id = 1 start = 0 s Period id = 3 start = 300 s Period id = 4 start = 850 s Period id = 2 start = 100 s Adaptation Set 0 subtitle turkish Adaptation Set 2 audio english Adaptation Set 1

BaseURL=http://abr.rocks.com/

Representation 2 Rate = 1 Mbps Representation 4 Rate = 3 Mbps Representation 1 Rate = 500 Kbps

Representation 3

Rate = 2 Mbps Resolution = 720p Segment Info Duration = 10 s Template: 3/$Number$.mp4

Segment Access

Initialization Segment

http://abr.rocks.com/3/0.mp4

Media Segment 1 start = 0 s

http://abr.rocks.com/3/1.mp4

Media Segment 2 start = 10 s

http://abr.rocks.com/3/2.mp4

Adaptation Set 3 audio italian Adaptation Set 1 video Period id = 2 start = 100 s

Representation 3 Rate = 2 Mbps

Selection of components/tracks Well-defined media format Selection of representations Splicing of arbitrary content like ads Chunks with addresses and timing

IEEE ComSoc Distinguished Lecture - Dec. 2019 12

slide-13
SLIDE 13

IEEE ComSoc Distinguished Lecture - Dec. 2019 13

HLS Playlist: Text Based and Verbose

Manifests: List of Accessible Segments and Their Timings

#EXTM3U #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=232370,CODECS="mp4a.40.2, avc1.4d4015" gear1/prog_index.m3u8 #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=649879,CODECS="mp4a.40.2, avc1.4d401e" gear2/prog_index.m3u8 #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=41457,CODECS="mp4a.40.2" gear0/prog_index.m3u8

master.m3u8

Source: https://developer.apple.com/streaming/examples/ and https://www.gpac-licensing.com/2014/12/01/apple-hls-comparing-versions/

#EXTM3U #EXT-X-TARGETDURATION:10 #EXT-X-VERSION:3 #EXT-X-MEDIA-SEQUENCE:0 #EXT-X-PLAYLIST-TYPE:VOD #EXTINF:9.97667, fileSequence0.ts #EXTINF:9.97667, fileSequence1.ts #EXTINF:9.97667, fileSequence2.ts . . . #EXT-X-ENDLIST

gear1/prog_index.m3u8

slide-14
SLIDE 14

Example Representations

Encoding Bitrate Resolution

  • Rep. #1

3.45 Mbps 1280 x 720

  • Rep. #2

2.2 Mbps 960 x 540

  • Rep. #3

1.4 Mbps 960 x 540

  • Rep. #4

900 Kbps 512 x 288

  • Rep. #5

600 Kbps 512 x 288

  • Rep. #6

400 Kbps 340 x 192

  • Rep. #7

200 Kbps 340 x 192

Source: Vertigo MIX10, Alex Zambelli’s Streaming Media Blog, Akamai, Comcast

Vancouver 2010 Sochi 2014

Encoding Bitrate Resolution

  • Rep. #1

3.45 Mbps 1280 x 720

  • Rep. #2

1.95 Mbps 848 x 480

  • Rep. #3

1.25 Mbps 640 x 360

  • Rep. #4

900 Kbps 512 x 288

  • Rep. #5

600 Kbps 400 x 224

  • Rep. #6

400 Kbps 312 x 176 Encoding Bitrate Resolution

  • Rep. #1

18 Mbps 4K (60p)

  • Rep. #2

12.2 Mbps 2560x1440 (60p)

  • Rep. #3

4.7 Mbps 2K (60p)

  • Rep. #4

3.5 Mbps 1280x720 (60p)

  • Rep. #5

2 Mbps 1280 x 720

  • Rep. #6

1.2 Mbps 768 x 432

  • Rep. #7

750 Kbps 640 x 360

  • Rep. #8

500 Kbps 512 x 288

  • Rep. #9

300 Kbps 320 x 180

  • Rep. #10 200 Kbps

320 x 180

PyeongChang 2018

IEEE ComSoc Distinguished Lecture - Dec. 2019 14

slide-15
SLIDE 15

Smart and Selfish Clients

HAS Working Principle

  • Client fetches and parses the manifest
  • Client uses the OS-provided HTTP stack

(HTTP/1.1/2 runs over TCP, HTTP/2/3 runs over QUIC)

  • Client uses the required decryption tools for the

protected content Client monitors and measures

  • Size of the playout buffer (both in bytes and seconds)
  • Chunk download times and throughput
  • Local resources (CPU, memory, window size, etc.)
  • Dropped frames

Client performs adaptation

Request Response HTTP Server Client

Client measures and reports metrics for analytics

(One can also multicast media segments)

IEEE ComSoc Distinguished Lecture - Dec. 2019 15

slide-16
SLIDE 16

Tradeoffs in Adaptive Streaming

User experience Overall quality Quality stability Stalls Zapping/seeking time Live latency

IEEE ComSoc Distinguished Lecture - Dec. 2019 16

slide-17
SLIDE 17

Some Observations

Scaling Issues

slide-18
SLIDE 18

A Single Microsoft Smooth Streaming Client under a Controlled Environment

Demystifying a Streaming Client

1 2 3 4 5 50 100 150 200 250 300 350 400 450 500 Bitrate (Mbps) Time (s) Available Bandwidth Requests Chunk Tput Average Tput

Reading: “An experimental evaluation of rate-adaptation algorithms in adaptive streaming over HTTP,” ACM MMSys 2011

Buffer-filling State Back-to-back requests Steady State Periodic requests

IEEE ComSoc Distinguished Lecture - Dec. 2019 18

slide-19
SLIDE 19
  • Leverage tried-and-true Web

– Video is just ordinary Web content

  • Leverage tried-and-true TCP

– Congestion avoidance – Reliable transport – No need for special QoS for video

Does it just work?

Mostly yes, when streaming clients compete with other types of traffic Not really, when streaming clients compete with each

  • ther

Streaming clients form an “accidental” distributed control-feedback system, causing all kinds of issues in

  • House with multiple screens
  • Crowded public hotspots
  • ISP access/aggregation links

Streaming over HTTP – The Promise

IEEE ComSoc Distinguished Lecture - Dec. 2019 19

slide-20
SLIDE 20

10 Streaming Clients Sharing a 10 Mbps Link

Selfishness Hurts Everyone

400 800 1200 1600 100 200 300 400 500 Requested Bitrate (Kbps) Time (s) Alice Bob Lisa John Jane

IEEE ComSoc Distinguished Lecture - Dec. 2019 20

slide-21
SLIDE 21

Two Competing Clients

Understanding the Root Cause

  • Depending on the timing of the ON periods:

– Unfairness, underutilization and/or instability may occur – Clients may grossly overestimate their fair share of the available bandwidth

Clients cannot figure out how much bandwidth to use until they use too much (Just like TCP)

Reading: “What happens when HTTP adaptive streaming players compete for bandwidth?,” ACM NOSSDAV 2012

IEEE ComSoc Distinguished Lecture - Dec. 2019 21

slide-22
SLIDE 22

How to Solve the Issues?

  • Use a better adaptation algorithm like PANDA or BOLA
  • Use machine learning or deep learning like Pensieve
  • Improve the HTTP/TCP/QUIC stack, try out the new developments (HTTP/2/3)
  • Adopt ideas from game/consensus theory (GTA)

Fix the clients and/or the transport

  • QoS in the core/edge
  • SDN

Get support from the network

  • Assist the clients and network elements through metrics and analytics

Enable a control plane

IEEE ComSoc Distinguished Lecture - Dec. 2019 22

slide-23
SLIDE 23

Review of over 100 Adaptation Schemes

IEEE ComSoc Distinguished Lecture - Dec. 2019 23

Looking for a Permanent head Damage (Degree)?

Bitrate Adaptation Schemes Client- based Adaptation Bandwidth- based Buffer- based Mixed adaptation Proprietary solutions MDP-based Server- based Adaptation Network- assisted Adaptation Hybrid Adaptation SDN-based Server and network- assisted

Reading: “A survey on bitrate adaptation schemes for streaming media over HTTP,” IEEE Commun. Surveys Tuts., 2019

slide-24
SLIDE 24

Conflicting Goals in the Media Delivery Chain

  • I want to make sure that my content is protected and looks awesome on any device
  • I want to make sure that my ads are viewed, trackable and measurable
  • I want to make sure that my servers are properly used and latency is low
  • I want to control the QoE of my customers, make $ from and differentiate my own video services
  • I want to make sure that my device/app provides the best possible video quality
  • I want the best quality for minimal $

Consumer Content Provider ISP CDN Provider Advertiser Device/ App

IEEE ComSoc Distinguished Lecture - Dec. 2019 24

slide-25
SLIDE 25

A Control Plane Approach

IEEE ComSoc Distinguished Lecture - Dec. 2019 25

Origin (HTTP) Server Encoder/ Transcoder Packager CDN RG Clients

Media Parameters for enhancing reception (PER) Metrics and status messages Parameters for enhancing delivery (PED)

Analytics Server

  • CTA-2066: Streaming Quality of

Experience Metrics (almost done)

  • New proposal at CTA: Common

Media Client Data

slide-26
SLIDE 26

Service Provider: “Your video or CDN provider must be slow” Video/CDN Provider: “Your home network must suck” Consumer: “The device

  • r the app is

slow” Device/App Vendor: “It must be the OS” OS Vendor: “Your Internet connection must be bad”

IEEE ComSoc Distinguished Lecture - Dec. 2019 26

  • Use a common language (e.g., CTA-

2066) across players

  • Beware infobesity

“Overabundance of information implies a scarcity of user attention”

Yes, at the Source, Encoder, Packager, Origin, Cache, GW, Player, etc.

Do We Need Analytics?

slide-27
SLIDE 27

What about Latency

Linear Delivery under Three Seconds

Reading: “Bandwidth prediction in low-latency chunked streaming,” ACM NOSSDAV 2019

slide-28
SLIDE 28
  • Video encoding pipeline
  • Ingest and packaging operations
  • Network propagation
  • Server I/O, CDN buffering
  • Media segment duration
  • Player behavior

Buffering

Playhead positioning

Resilience

Contributors to the Latency

IEEE ComSoc Distinguished Lecture - Dec. 2019 28

slide-29
SLIDE 29

IEEE ComSoc Distinguished Lecture - Dec. 2019 29

Low Latency is Always a Trade-Off against Playback Robustness

Stream Start Time ≠ Latency

Time Live encoder producing 2-second segments iOS (3 segments) Last fully available segment Lowest latency 1 2 3 4 Start Now 2 3 4 4 5 Latency: 7 s Latency: 3 s Latency: 2 s 6 seconds of buffer ~0 seconds of startup* 2 seconds of buffer ~0 seconds of startup* 2 seconds of buffer 1 second of startup*

* Segment fetching time is assumed to be negligible in this example

5

slide-30
SLIDE 30

CMAF Chunks are One or More Frames

Refresher on CMAF

RAP

RAP

RAP

RAP

Fragment Fragment Fragment Fragment

Segment Segment

Track File

… … …

Encoding Packaging Encryption CMAF Header

Seamless switching can only happen at fragment boundaries

30 IEEE ComSoc Distinguished Lecture - Dec. 2019

slide-31
SLIDE 31

What about Quality

Encoding and Streaming in a Quality-Aware Fashion

Reading: “Spending quality time with the Web video,” IEEE Internet Comput., 2016, and “Quality-aware HTTP adaptive streaming,” IBC 2015

slide-32
SLIDE 32

Nomenclature of Rate Control

  • ABR: Adaptive bitrate

– Misnomer, refers to adaptive streaming over HTTP

  • CBR: Constant bitrate encoding

– The decoder’s buffer is filled at a constant rate

  • (True) VBR: Variable bitrate encoding

– The decoder’s buffer is filled at a non-constant rate

  • Capped VBR: How VBR is implemented in practice

– The decoder’s buffer is filled at a non-constant rate with strict min and max bounds

  • Title/content-based (or content-aware) encoding

– Choosing the bitrate ladder based on the content

  • Context-aware encoding

– Advanced optimizations based on viewer, display and viewing conditions

P a s t

CBR encoding with fixed bitrate ladders

Today

Slowly moving to cVBR and custom bitrate ladders Future

C

  • n

t e x t

  • a

w a r e e n c

  • d

i n g ?

IEEE ComSoc Distinguished Lecture - Dec. 2019 32

slide-33
SLIDE 33

Picking the Bitrate Ladder Based on the Content

Content-Based (Content-Aware) Encoding

Content-aware encoding gives us fairness in quality as

  • pposed to fairness in bitrate

IEEE ComSoc Distinguished Lecture - Dec. 2019 33

slide-34
SLIDE 34
  • If the following holds true

– Segments are ~CBR encoded – Client fetches segments based on bitrate

information only

  • Then, viewer experience will vary

because of

– Low-motion/complexity vs. high-

motion/complexity scenes

– Upshifts and downshifts dictated by the

adaptation logic

Segments Have Different Complexities

Bitrate Quality Video Segment #1 Equal Bitrate Allocation among Segments Consistent Quality Video Segment #2

IEEE ComSoc Distinguished Lecture - Dec. 2019 34

slide-35
SLIDE 35

Guidelines Limited Bitrate Variability to (Mostly) 10% So Far

Adaptation Feature Delivers Inconsistent Quality

Easy Moderate Easy Easy Easy Difficult Difficult Difficult Moderate Moderate Moderate Moderate

S Time (s) Segment Size 0 2 4 6 8 10 12 14 16 18 20 22 24 Segment Quality QCBR Small variation in encoding bitrate Large variation in quality

If there is something worse than having to watch a video at a lousy quality, it is to watch that video with varying quality

IEEE ComSoc Distinguished Lecture - Dec. 2019 35

slide-36
SLIDE 36

What If We Encode in a More Subtle Fashion?

Easy Moderate Easy Easy Easy Difficult Difficult Difficult Moderate Moderate Moderate Moderate

Time (s) Segment Size 2 4 6 8 10 12 14 16 18 20 22 24 Segment Quality QVBR

While we spend the same total amount of bits, we not only increase average quality but also reduce quality variation

Large variation in encoding bitrate Low variation in quality S

HLS authoring spec for ATV allows 2x capping rate for VoD. For linear content, variability is limited to 10-25% range.

IEEE ComSoc Distinguished Lecture - Dec. 2019 36

slide-37
SLIDE 37

Content-aware Encoding Content-aware Streaming

Generating VBR-encoded segments is easy, but streaming them is not!

IEEE ComSoc Distinguished Lecture - Dec. 2019 37

slide-38
SLIDE 38

We can Still Save Bandwidth and/or Improve Quality

What If the Content is Already CBR Encoded

Reading: “Streaming video over HTTP with consistent quality,” ACM MMSys 2014

2.8 Mbps Network HTTP Server

k+1 k+2

4 Mbps

k+3 k k+1 k+2 k+3 k

3 Mbps

2 Mbps

1 Mbps

Representations (4 bitrate levels) Smart Clients Naive (Most) Clients A Bit Smarter (Few) Clients

… … …

IEEE ComSoc Distinguished Lecture - Dec. 2019 38

slide-39
SLIDE 39

We can Still Save Bandwidth

What If There is No Smartness in the Client

3.2 Mbps Network Regular HTTP Server

k+1 k+2

4 Mbps

k+3 k k+1 k+2 k+3 k

3 Mbps

2 Mbps

1 Mbps

Representations (4 bitrate levels) Naive Clients 3.2 Mbps Network Quality-aware HTTP Server (or Packager) Naive Clients

… …

Reading: “More juice less bits: content aware streaming,” ACM MMSys 2016

Bandwidth Savings

The server/packager replaces some of the 3 Mbps segments with the 2 Mbps ones since delta quality is insignificant

IEEE ComSoc Distinguished Lecture - Dec. 2019 39

slide-40
SLIDE 40

What If the Content is VBR Encoded

2.8 Mbps Network HTTP Server

k+1 k+2 k+3 k

QL_3

QL_2

QL_1

Representations (3 quality levels) Smart Clients

The client streams the highest consistent-quality video without draining its buffer while respecting the available bandwidth (Solved using dynamic programming) The resolution stays the same but the encoding rate varies in a given representation (per quality level)

IEEE ComSoc Distinguished Lecture - Dec. 2019 40

slide-41
SLIDE 41

Dynamic Programming Solution

  • Brute-force search has exponential complexity

à Dynamic programming reduces processing time to polynomial time Time Buffer Size

B[0] BHIGH BLOW B[N] B[n]

IEEE ComSoc Distinguished Lecture - Dec. 2019 41

slide-42
SLIDE 42
  • If the goal is to maximize the

average quality (or total quality)

– Pick Q=1, then – Pick Q=4

  • If the goal is to maximize the worst-

case quality

– Pick Q=2, then – Pick Q=2

  • Picking Q=1 and then Q=2 is

suboptimal

  • Picking Q=2 and then Q=4 is

infeasible

A Toy Example

Buffer size (s) Increase in buffered data Decrease in buffered data Quality gain

Q

Steps Buffer (s) 1 1.5 0.5 1.9 0.9 0.8

  • 0.2

1 2 Q = 1 Q = 2 Q = 2 Q = 4 Q = 2 Q = 4

IEEE ComSoc Distinguished Lecture - Dec. 2019 42

Reading: “Streaming video over HTTP with consistent quality,” ACM MMSys 2014

slide-43
SLIDE 43

DASH Toolset for Content-Aware Streaming

  • At the encoder/packager side

– Set minBufferTime to allow for degree of VBRness – Provide the client information about effective segment bitrates and qualities

  • Index segment (`sidx`) maps byte offsets to durations
  • Quality metrics can be carried in a separate track (timed metadata track – ISO/IEC 23001-10:2015) or as an

aux file signaled in the manifest (ISO/IEC 23009-8 in progress)

– Decouple switching and random access

  • Longer “switching” segments used for delivery
  • Shorter “random access” segments used to improve seek granularity
  • At the client side

– Estimate download times

  • Look at minBufferTime / bandwidth model
  • Schedule downloads using index information, when possible

– Take segment size and quality values into account in the adaptation logic

IEEE ComSoc Distinguished Lecture - Dec. 2019 43

slide-44
SLIDE 44
  • Most common scenario

– CBR encoders and naive streaming clients – Capped VBR encoders (small caps) and

naive clients

  • Emerging scenario

– Capped VBR encoders (small cap) and a

bit smarter clients

  • Best possible scenario

– Capped VBR encoders (large cap) and

smart clients

Deployment Scenarios

What If We Have Both CBR and VBR Encoded Content

If you do not control both ends, follow Postel’s Law (RFC 1122) Be liberal in what you accept, conservative in what you send If there are unknown clients:

  • Send CBR encoded segments
  • Set minBufferTime sufficiently

large

Otherwise, the clients might get confused, break, stall or even crash

IEEE ComSoc Distinguished Lecture - Dec. 2019 44

slide-45
SLIDE 45
  • Same principle applies to both:

– In-stream: Temporal bit shifting between segments – Across-streams: Bit shifting between streams sharing a bottleneck link

Extending the Idea to Optimization across Streams

Bitrate Quality Stream 1 (News) Equal Bitrate Allocation among Streams Consistent Quality Stream 2 (Sports)

Reading: “Spending quality time with the Web video,” IEEE Internet Comput., 2016, and “Quality-aware HTTP adaptive streaming,” IBC 2015

IEEE ComSoc Distinguished Lecture - Dec. 2019 45

Temporal Pooling Viewers react differently to glitches for different content types, and they forgive in different time scales

  • A young viewer (likely to have longer-term memory)

watching sports on a big screen vs.

  • An elder viewer (likely to have short-term memory)

watching news on a smaller screen Spatial Pooling We want controlled unfairness (which is fairness in quality not bitrate) among viewers

  • Example: You are watching football on a 65” screen

and your wife is watching a food show on a tablet

  • Example: You are watching sports on a phone and

two others are watching news on a 48” screen

slide-46
SLIDE 46
  • Content formatting

– Each asset is copied multiple times

  • Different audio/video codecs
  • Different (regional) frame rates
  • Different HDR formats
  • Different container formats, encryption modes

– Huge cost for encoding/packaging/storage – Inefficiencies in CDN caching/distribution

  • Across platforms

– Lack of consistent app behavior – Varying video features, APIs and semantics

  • Playback

– Partial profile support – Switching glitches – Audio discontinuities – Ad splicing problems – Long-term playback instability – Request protocol deficiencies – Memory problems, CPU weaknesses – Scaling (display) issues – Variable HDR support – Unknown capabilities, …

  • Security

– Piracy and restreaming – Account sharing, use of VPNs/proxies, …

Houston, We Have So Many Problems!

IEEE ComSoc Distinguished Lecture - Dec. 2019 46

slide-47
SLIDE 47

IEEE ComSoc Distinguished Lecture - Dec. 2019 47

Recommendations

  • A common media segment format for all devices

– Package in CMAF/DASH segments (fmp4 only, no TS) à Less packaging/storage capacity

  • Use HEVC and newer codecs, newer HDR formats à Less bandwidth requirement at the same quality
  • Package in 6 second segments, 2 second fragments
  • Further use <500 millisecond chunks for low latency

– Use common encryption for multi-DRM support à Simpler packaging, less storage, flexibility to

switch DRM in case of a breach

  • Two manifest formats:

– For iOS devices, create an HLS playlist (version 7+) – For everything else, create a DASH MPD

  • One (possibly two) encryption mode:

– Encrypt using the CBCS mode

  • FairPlay is already CBCS
  • Widevine/PlayReady to support CBCS soon à May halve packager instances and storage needs

– If Widevine and PlayReady do not support CBCS

  • Either create another copy with the CTR mode or consider doing just-in-time re-encryption
slide-48
SLIDE 48
  • Two grand challenges

Improving open-source HEVC encoding

Low-latency live streaming

  • Focus areas in 2020

Machine learning and statistical modeling for video streaming

Volumetric media: from capture to consumption

Fake media and tools for preventing illegal broadcasts

  • A challenge dedicated to high-school students
  • Three confirmed keynotes from Google, MIT and Tsinghua
  • Expecting reduced registration fees thanks to strong support

Important Dates Submit by Research Track

  • Jan. 10 (firm)

Demo Track

  • Feb. 29

Open Source/Dataset

  • Feb. 29

Workshops

  • Mar. 27

Conference June 8-11

NEW

Visit http://acmmmsys.org today!

NEW

slide-49
SLIDE 49

Visit ht http://ali li.begen.net fo for tutorials and papers Also AM AMAAAS AAAS al ali.begen@ozyegin.edu.tr

DO DON’T T DRI DRINK K AN AND D DRI DRIVE, JU JUST STREAM AN AND D FLY!