Deep Encode: Machine Learning for Per-Title Encoding Daniel - PowerPoint PPT Presentation

Fraunhofer FOKUS Institut für Offene Kommunikationssysteme Deep Encode: Machine Learning for Per-Title Encoding Daniel Silhavy| IBC20|

Per-Title Encoding – What & Why & 1 How?

Benefits of Per-Title Encoding ü Storage and bitrate Conventional/Static savings while Encoding Ladder preserving Per-Title Encoding ü Quality increase at optimal quality Ladder identical bitrates 3 Deep Encode –Machine Learning for Per-Title Encoding

Benefits of Per-Title Encoding: the Numbers Average values Bitrate (kbit/s) VMAF PSNR (dB) Storage (MB) Conventional 7648.18 94.92 44.37 1397.7 Per-Title 4941.75 93.06 42.41 675.2 Difference +2706.43 -1.86 -1.96 +722.5 MB Abs Difference +36% -1% -4% +52% (%) * Based on a streaming session with a 100Mbit/s connection in dash.js 3.1.1 5 Deep Encode –Machine Learning for Per-Title Encoding

How does Per-Title Encoding work? Test Encodes Convex Hull Estimation Production encoding • Perform test encodings with • Select bitrate-resolution pairs • Perform the production different settings and calculate that are close to the convex encoding using the optimal corresponding VMAF values. hull. encoding ladder. l l u H x e v n o C ！ A large amount of test encodes is required to derive a sufficient amount of data points. 6 Deep Encode –Machine Learning for Per-Title Encoding

Machine Learning for Per-Title 3 Encoding

How to avoid the computationally heavy test encodes ML-based predictions Convex Hull Production encoding • Predict mandatory Bitrate • Select bitrate-resolution • Perform production CBR / / VMAF pairs using pairs that are close to the Two-pass constrained machine learning. convex hull VBR encoding using the optimal encoding ladder ！ Use ML-based predictions to avoid test encodes and still derive a sufficient amount data points. Deep Encode –Machine Learning for Per-Title Encoding 8

Deep Encode – ML for Video Metric Prediction • Bitrate • Resolution Regression • Scene Changes Predicted [Bitrate, Resolution, Quality] • Temporal & Metadata spatial extraction complexities Content Training Data Type 1 Convex Hull Categories & Labels Encoding Ladder Estimation Content Classification Type 2 Database [Bitrate,Resolution,Quality] Content Type 3 Test Encodes 9 Deep Encode –Machine Learning for Per-Title Encoding

Deep Encode: Hands-on UI 10 | Deep Encode –Machine Learning for Per-Title Encoding

4 Summary, Outlook & Next Steps

Deep Encode: Towards Context-Aware Encoding Deep Encode Conventional Static Encoding • Same encoding ladder for all types of ü No computationally heavy test encodes content ü Metadata extraction and AI-based image • Increased storage and delivery Conventional & Per-Title Encoding processing for content analysis costs • Content categorization and labeling • “Waste” of quality • Automatic scene detection Per-Scene Encoding • Lack of optimization for complex • Metadata extraction content ü Deep Learning for optimal encoding ladders Conventional Per-Title Encoding • Prediction of [PSNR|VMAF, • Computationally heavy Bitrate] pairs test encodes • Dynamic prediction of the optimal • No dynamic reaction to encoding ladder complex scenes within a ü Enhancements movie • Live-stream support • Per-scene and context-aware Encoding 12 | Deep Encode –Machine Learning for Per-Title Encoding

Daniel Silhavy Project Manager Future Applications and Media (FAME) daniel.silhavy@fokus.fraunhofer.de Fraunhofer FOKUS Berlin, Germany FAME Video Development Blog: https://websites.fraunhofer.de/video-dev/