Make Streaming Great Again IEEE ComSoc Distinguished Lectures - PowerPoint PPT Presentation

Make Streaming Great Again IEEE ComSoc Distinguished Lectures – Portland State University, Dec. 2019 Ali C. Begen, PhD http://ali.begen.net

Ankara BS in EE Toronto, ON Istanbul San Jose, CA Adv. Res. and Dev. Prof. in CS Dept. Atlanta, GA Konya Konya PhD in ECE w/ CS 2200 guests Born, raised and living San Diego, CA Intern

We will Scratch the Surface on the Red Colored Items AMAAAS* Production Reliable and Scalable Delivery Consumption Capture/ Ingest Encryption Packaging Access Storage Encoding Analytics Core CDNs AMAAAS: Ask me almost anything about streaming IEEE ComSoc Distinguished Lecture - Dec. 2019 3

First Things First Download vs. Streaming, Why HTTP, Manifests, etc.

Progressive Download One Request à One Response HTTP Request HTTP Response Download will continue as fast Playback starts only after there as possible is several seconds of data in the Fetched content will be wasted playback buffer if the viewer clicks away Can seek only throughout the fetched content IEEE ComSoc Distinguished Lecture - Dec. 2019 5

Streaming Sounds Cooler! What is Streaming? Client consumption rate is also limited by real-time constraints as opposed to just bandwidth availability That is, client cannot fetch content not available yet Streaming is transmission of a continuous content from a server to a client and its simultaneous consumption by the client Server transmission rate (loosely or tightly) matches to client consumption rate That is, no buffer overrun or underrun is acceptable IEEE ComSoc Distinguished Lecture - Dec. 2019 6

Streaming is More Viewer Friendly Playback starts when there is just few seconds of data Download rate will match the encoding bitrate and downloading pauses if the player pauses à Less waste Can seek to anywhere in the entire content IEEE ComSoc Distinguished Lecture - Dec. 2019 7

Video Delivery over HTTP Adaptive Pseudo Streaming Streaming • Multiple versions • Enables seeking of the content are • Enables • Content is divided via media indexing created playback while into short-duration • Server paces still downloading • Enables to adapt chunks transmission based to network and • Server sends the • Enables live on encoding rate device conditions file as fast as streaming and ad possible insertion Progressive Chunked Download Streaming IEEE ComSoc Distinguished Lecture - Dec. 2019 8

Adaptive Streaming over HTTP … … Server Storage … … Multi-rate Packager Origin (HTTP) Encoder Server Content Ingest (Live or Pre-captured) Response HTTP GET Request Media Buffer Decoding and Presentation Streaming Client IEEE ComSoc Distinguished Lecture - Dec. 2019 9

HTTP Adaptive Streaming (HAS) Adapt Video to Web rather than Changing the Web • Why HTTP Features well-understood naming/addressing and authentication/authorization infrastructure – Provides easy traversal for all kinds of middleboxes (e.g., NATs, firewalls) – Enables cloud access, leverages the existing (cheap) HTTP caching infrastructure in the CDNs – • Imitation of streaming via short downloads Downloads small chunks to minimize waste – Enables monitoring consumption and tracking streaming clients – • Adaptation to varying conditions on the network and different device capabilities • Improved quality of experience Enables shorter stream start time upon zapping or seeking – Reduces skips and freezes – The cure may be worse than the disease if you are not careful enough IEEE ComSoc Distinguished Lecture - Dec. 2019 10

Dead, Surviving, Maturing and Newborn Technologies • Move Adaptive Stream (Long gone, but some components are in Slingbox) http://www.movenetworks.com – • Microsoft Smooth Streaming (Legacy) http://www.iis.net/expand/SmoothStreaming – • Adobe Flash (Almost dead, thank God) http://www.adobe.com/products/flashplayer.html – • Adobe HTTP Dynamic Streaming (Legacy) http://www.adobe.com/products/httpdynamicstreaming – • Apple HTTP Live Streaming (The elephant in the room, the de facto format) https://tools.ietf.org/html/rfc8216 – https://datatracker.ietf.org/doc/draft-pantos-hls-rfc8216bis – • MPEG DASH and CMAF (The de jure standards) http://mpeg.chiariglione.org/standards/mpeg-dash – http://mpeg.chiariglione.org/standards/mpeg-a/common-media-application-format – IEEE ComSoc Distinguished Lecture - Dec. 2019 11

Manifests: List of Accessible Segments and Their Timings DASH MPD: Template Based and Extensible Segment Access MPD Adaptation Set 1 Period id = 2 BaseURL=http://abr.rocks.com/ start = 100 s Period id = 1 Initialization Segment start = 0 s Representation 3 http://abr.rocks.com/3/0.mp4 Representation 1 Adaptation Set 0 Rate = 2 Mbps Rate = 500 Kbps subtitle turkish Resolution = 720p Period id = 2 Media Segment 1 Representation 2 start = 100 s start = 0 s Adaptation Set 1 Rate = 1 Mbps http://abr.rocks.com/3/1.mp4 Segment Info video Duration = 10 s Period id = 3 Representation 3 Media Segment 2 Rate = 2 Mbps Template: start = 300 s start = 10 s Adaptation Set 2 3/$Number$.mp4 http://abr.rocks.com/3/2.mp4 audio english Representation 4 Period id = 4 Rate = 3 Mbps start = 850 s Adaptation Set 3 audio italian Splicing of arbitrary Selection of Selection of Well-defined Chunks with addresses content like ads components/tracks representations and timing media format IEEE ComSoc Distinguished Lecture - Dec. 2019 12

Manifests: List of Accessible Segments and Their Timings HLS Playlist: Text Based and Verbose gear1/prog_index.m3u8 #EXTM3U #EXT-X-TARGETDURATION:10 master.m3u8 #EXT-X-VERSION:3 #EXT-X-MEDIA-SEQUENCE:0 #EXTM3U #EXT-X-PLAYLIST-TYPE:VOD #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=232370,CODECS="mp4a.40.2, avc1.4d4015" #EXTINF:9.97667, gear1/prog_index.m3u8 fileSequence0.ts #EXTINF:9.97667, #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=649879,CODECS="mp4a.40.2, avc1.4d401e" fileSequence1.ts gear2/prog_index.m3u8 #EXTINF:9.97667, fileSequence2.ts #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=41457,CODECS="mp4a.40.2" . gear0/prog_index.m3u8 . . #EXT-X-ENDLIST Source: https://developer.apple.com/streaming/examples/ and https://www.gpac-licensing.com/2014/12/01/apple-hls-comparing-versions/ IEEE ComSoc Distinguished Lecture - Dec. 2019 13

Example Representations Vancouver 2010 Sochi 2014 PyeongChang 2018 Encoding Resolution Encoding Resolution Encoding Resolution Bitrate Bitrate Bitrate Rep. #1 3.45 Mbps 1280 x 720 Rep. #1 3.45 Mbps 1280 x 720 Rep. #1 18 Mbps 4K (60p) Rep. #2 2.2 Mbps 960 x 540 Rep. #2 12.2 Mbps 2560x1440 (60p) Rep. #2 1.95 Mbps 848 x 480 Rep. #3 1.25 Mbps 640 x 360 Rep. #3 1.4 Mbps 960 x 540 Rep. #3 4.7 Mbps 2K (60p) Rep. #4 900 Kbps 512 x 288 Rep. #4 3.5 Mbps 1280x720 (60p) Rep. #4 900 Kbps 512 x 288 Rep. #5 600 Kbps 512 x 288 Rep. #5 2 Mbps 1280 x 720 Rep. #5 600 Kbps 400 x 224 Rep. #6 400 Kbps 340 x 192 Rep. #6 1.2 Mbps 768 x 432 Rep. #6 400 Kbps 312 x 176 Rep. #7 200 Kbps 340 x 192 Rep. #7 750 Kbps 640 x 360 Rep. #8 500 Kbps 512 x 288 Rep. #9 300 Kbps 320 x 180 Rep. #10 200 Kbps 320 x 180 Source: Vertigo MIX10, Alex Zambelli’s Streaming Media Blog, Akamai, Comcast IEEE ComSoc Distinguished Lecture - Dec. 2019 14

HAS Working Principle - Client fetches and parses the manifest Smart and Selfish Clients - Client uses the OS-provided HTTP stack (HTTP/1.1/2 runs over TCP, HTTP/2/3 runs over QUIC) - Client uses the required decryption tools for the protected content Client monitors and measures - Size of the playout buffer (both in bytes and seconds) - Chunk download times and throughput HTTP Server - Local resources (CPU, memory, window size, etc.) Request - Dropped frames Response Client performs adaptation (One can also multicast Client media segments) Client measures and reports metrics for analytics IEEE ComSoc Distinguished Lecture - Dec. 2019 15

Tradeoffs in Adaptive Streaming User experience Overall quality Quality stability Stalls Zapping/seeking time Live latency IEEE ComSoc Distinguished Lecture - Dec. 2019 16

Some Observations Scaling Issues

Demystifying a Streaming Client A Single Microsoft Smooth Streaming Client under a Controlled Environment 5 4 Steady State Bitrate (Mbps) Periodic requests 3 2 1 Buffer-filling State 0 Back-to-back requests 0 50 100 150 200 250 300 350 400 450 500 Time (s) Available Bandwidth Requests Chunk Tput Average Tput Reading: “An experimental evaluation of rate-adaptation algorithms in adaptive streaming over HTTP,” ACM MMSys 2011 IEEE ComSoc Distinguished Lecture - Dec. 2019 18

Streaming over HTTP – The Promise • Leverage tried-and-true Web • Leverage tried-and-true TCP – Video is just ordinary Web content – Congestion avoidance – Reliable transport – No need for special QoS for video Mostly yes, when streaming Does it just work? clients compete with other types of traffic Streaming clients form an “accidental” distributed control-feedback system, causing all kinds of issues in • House with multiple screens Not really, when streaming clients compete with each • Crowded public hotspots other • ISP access/aggregation links IEEE ComSoc Distinguished Lecture - Dec. 2019 19