Reconstructing proprietary video streaming algorithms Maximilian - - PowerPoint PPT Presentation

▶
reconstructing proprietary video streaming algorithms
SMART_READER_LITE
LIVE PREVIEW

Reconstructing proprietary video streaming algorithms Maximilian - - PowerPoint PPT Presentation

Apr 08, 2023 470 likes •1.9k views

Reconstructing proprietary video streaming algorithms Maximilian Grner , Melissa Licciardello, Ankit Singla Photo: ETH Zrich / Gian Marco Castelberg Roadmap 2 Roadmap Introduction 2 Roadmap Introduction Reconstruction 2

slide-1
SLIDE 1

Reconstructing proprietary video streaming algorithms

Photo: ETH Zürich / Gian Marco Castelberg

Maximilian Grüner, Melissa Licciardello, Ankit Singla

slide-2
SLIDE 2

Roadmap

2

slide-3
SLIDE 3

Roadmap

  • Introduction

2

slide-4
SLIDE 4

Roadmap

  • Introduction
  • Reconstruction

2

slide-5
SLIDE 5

Roadmap

  • Introduction
  • Reconstruction
  • Experiments

2

slide-6
SLIDE 6

Roadmap

  • Introduction
  • Reconstruction
  • Experiments
  • Limitations and future work

2

slide-7
SLIDE 7

Roadmap

  • Introduction
  • Reconstruction
  • Experiments
  • Limitations and future work
  • Takeaways

2

slide-8
SLIDE 8

3

Streaming in Industry

Categories Share of global internet traffic %

[Sandvine 2019]

slide-9
SLIDE 9

3

Streaming in Industry

Categories Share of global internet traffic %

[Sandvine 2019]

slide-10
SLIDE 10

3

Streaming in Industry

Categories Share of global internet traffic %

[Sandvine 2019]

23.8 % Various, 23.1 % Netflix, 12.7 % YouTube, 4.2 % Twitch…

slide-11
SLIDE 11

4

Streaming in academia

Year # Google Scholar citations

slide-12
SLIDE 12

4

Streaming in academia

Year # Google Scholar citations

slide-13
SLIDE 13

Applications of reconstruction

5

slide-14
SLIDE 14

Applications of reconstruction

5

slide-15
SLIDE 15

Applications of reconstruction

5

slide-16
SLIDE 16

Applications of reconstruction

5

slide-17
SLIDE 17

6

slide-18
SLIDE 18

6

slide-19
SLIDE 19

6

slide-20
SLIDE 20

6

slide-21
SLIDE 21

6

slide-22
SLIDE 22

7

Introduction into ABRs

Resolution Played Throughput

[Video from YouTube]

slide-23
SLIDE 23

7

Introduction into ABRs

Resolution Played Throughput

[Video from YouTube]

slide-24
SLIDE 24

7

Introduction into ABRs

Resolution Played Throughput

[Video from YouTube]

slide-25
SLIDE 25

7

Introduction into ABRs

Resolution Played Throughput

[Video from YouTube]

slide-26
SLIDE 26

7

Introduction into ABRs

Resolution Played Throughput

[Video from YouTube]

slide-27
SLIDE 27

7

Introduction into ABRs

Resolution Played Throughput

[Video from YouTube]

slide-28
SLIDE 28

Simple ABR in code

8

def choose_next_quality(throughput_history): throughput_estimate = throughput_history[-1] if throughput_estimate == low: next_quality = ⬇⬇ else: ## throughput_estimate == high next_quality = ⬆⬆ return next_quality

slide-29
SLIDE 29

Simple ABR in code

8

def choose_next_quality(throughput_history): throughput_estimate = throughput_history[-1] if throughput_estimate == low: next_quality = ⬇⬇ else: ## throughput_estimate == high next_quality = ⬆⬆ return next_quality

slide-30
SLIDE 30

Simple ABR in code

8

def choose_next_quality(throughput_history): throughput_estimate = throughput_history[-1] if throughput_estimate == low: next_quality = ⬇⬇ else: ## throughput_estimate == high next_quality = ⬆⬆ return next_quality

slide-31
SLIDE 31

Simple ABR in code

8

def choose_next_quality(throughput_history): throughput_estimate = throughput_history[-1] if throughput_estimate == low: next_quality = ⬇⬇ else: ## throughput_estimate == high next_quality = ⬆⬆ return next_quality

slide-32
SLIDE 32

Simple ABR in code

8

def choose_next_quality(throughput_history): throughput_estimate = throughput_history[-1] if throughput_estimate == low: next_quality = ⬇⬇ else: ## throughput_estimate == high next_quality = ⬆⬆ return next_quality

slide-33
SLIDE 33

Simple ABR as a tree

9

throughput_estimate == low next_quality = ⬆⬆ next_quality = ⬇⬇

False True

slide-34
SLIDE 34

(Slightly more) complex ABR in code

10

def choose_next_quality(throughput_history): throughput_estimate = np.mean(throughput_history[-2:]) if throughput_estimate == low: next_quality = ⬇⬇ else: ## throughput_estimate == high next_quality = ⬆⬆ return next_quality

slide-35
SLIDE 35

(Slightly more) complex ABR in code

10

def choose_next_quality(throughput_history): throughput_estimate = np.mean(throughput_history[-2:]) if throughput_estimate == low: next_quality = ⬇⬇ else: ## throughput_estimate == high next_quality = ⬆⬆ return next_quality

slide-36
SLIDE 36

Why do we need primitives

11

slide-37
SLIDE 37

Why do we need primitives

11

slide-38
SLIDE 38

Why do we need primitives

11

2 Rules

slide-39
SLIDE 39

Why do we need primitives

11

5 Rules

slide-40
SLIDE 40

Why do we need primitives

11

25 Rules

slide-41
SLIDE 41

Why do we need primitives

11

100 Rules

slide-42
SLIDE 42

Complex ABR in code

12

def choose_next_quality(throughput_history, buffer_size): throughput_estimate = np.mean(throughput_history[-2:]) if throughput_estimate == low: download_time = chunk_size_high / throughput_estimate if download_time < buffer_size: next_quality = ⬇⬇ else: next_quality = ⬆⬆ else: ## throughput_estimate == high next_quality = ⬆⬆ return next_quality

slide-43
SLIDE 43

Complex ABR in code

12

def choose_next_quality(throughput_history, buffer_size): throughput_estimate = np.mean(throughput_history[-2:]) if throughput_estimate == low: download_time = chunk_size_high / throughput_estimate if download_time < buffer_size: next_quality = ⬇⬇ else: next_quality = ⬆⬆ else: ## throughput_estimate == high next_quality = ⬆⬆ return next_quality

slide-44
SLIDE 44

Complex ABR in code

12

def choose_next_quality(throughput_history, buffer_size): throughput_estimate = np.mean(throughput_history[-2:]) if throughput_estimate == low: download_time = chunk_size_high / throughput_estimate if download_time < buffer_size: next_quality = ⬇⬇ else: next_quality = ⬆⬆ else: ## throughput_estimate == high next_quality = ⬆⬆ return next_quality

slide-45
SLIDE 45

Number of primitives

13

slide-46
SLIDE 46

Number of primitives

13

Features

slide-47
SLIDE 47

Number of primitives

13

Features Lookback X

slide-48
SLIDE 48

Number of primitives

13

Features Lookback Actions X X

slide-49
SLIDE 49

Number of primitives

13

~ 1300 Primitives

Features Lookback Actions X X

slide-50
SLIDE 50

14

slide-51
SLIDE 51

14

slide-52
SLIDE 52

15

Services of interest

Provider Description Alexa Rank

slide-53
SLIDE 53

15

Services of interest

Provider Description Alexa Rank YouTube Broad coverage 2, Global

slide-54
SLIDE 54

15

Services of interest

Provider Description Alexa Rank YouTube Broad coverage 2, Global ZDF German Public Service 47, Germany

slide-55
SLIDE 55

15

Services of interest

Provider Description Alexa Rank YouTube Broad coverage 2, Global ZDF German Public Service 47, Germany Pornhub Pornographic video sharing website 46, Global

slide-56
SLIDE 56

15

Services of interest

Provider Description Alexa Rank YouTube Broad coverage 2, Global ZDF German Public Service 47, Germany Pornhub Pornographic video sharing website 46, Global Provider Description Alexa Rank YouTube Broad coverage 2, Global ZDF German Public Service 47, Germany Pornhub Pornographic video sharing website 46, Global Arte French-German,cultural 270,France Fandom Gaming, pop-culture 91, Global SRF Swiss Public Service 45, Switzerland TubiTV Movies and series of all genres 1330, USA Twitch Live and VoD streaming service, gaming 39, Global Vimeo Artistic content 188, Global XVideos Pornographic video sharing website 67, Global

slide-57
SLIDE 57

Inference features

16

slide-58
SLIDE 58

Inference features

16

Providers

slide-59
SLIDE 59

Inference features

16

Providers Net-traces X

slide-60
SLIDE 60

Inference features

16

Providers Videos Net-traces X X

slide-61
SLIDE 61

Inference features

16

Providers Videos Net-traces Length X X X

slide-62
SLIDE 62

Inference features

16

~ 9 hours of training/testing streaming time

Providers Videos Net-traces Length X X X

slide-63
SLIDE 63

17

slide-64
SLIDE 64

17

slide-65
SLIDE 65

18

Why is agreement not sufficient

slide-66
SLIDE 66

18

Why is agreement not sufficient

Resolution Played Throughput

[Video from YouTube]

slide-67
SLIDE 67

18

Why is agreement not sufficient

Resolution Played Throughput

[Video from YouTube]

slide-68
SLIDE 68

18

Why is agreement not sufficient

Resolution Played Throughput

[Video from YouTube]

slide-69
SLIDE 69

18

Why is agreement not sufficient

Resolution Played Throughput

[Video from YouTube]

slide-70
SLIDE 70

18

Why is agreement not sufficient

Resolution Played Throughput

[Video from YouTube]

slide-71
SLIDE 71

Agreement

19

slide-72
SLIDE 72

Agreement

19

[Wikipedia]

slide-73
SLIDE 73

Agreement

19

[Wikipedia]

slide-74
SLIDE 74

Agreement

19

[Wikipedia]

slide-75
SLIDE 75

Agreement

19

[Wikipedia]

slide-76
SLIDE 76

Agreement

19

[Wikipedia]

slide-77
SLIDE 77

Agreement

19

[Wikipedia]

slide-78
SLIDE 78

Agreement

19

[Wikipedia]

slide-79
SLIDE 79

Agreement

19

[Wikipedia]

F1 = 2 ⋅ precision ⋅ recall precision + recall

slide-80
SLIDE 80

Similarity

20

slide-81
SLIDE 81

Similarity

20

[Scikit-learn]

slide-82
SLIDE 82

Similarity

20

[Scikit-learn]

slide-83
SLIDE 83

Similarity

20

[Scikit-learn]

slide-84
SLIDE 84

Similarity

20

[Scikit-learn]

slide-85
SLIDE 85

Similarity

20

[Scikit-learn]

ScoreSimilarity = 𝒬(Decision from reference distribution)

slide-86
SLIDE 86

General metric analysis

21

slide-87
SLIDE 87

General metric analysis

21

slide-88
SLIDE 88

General metric analysis

21

Best

slide-89
SLIDE 89

General metric analysis

21

Worst Best

slide-90
SLIDE 90

General metric analysis

21

Worst Best

slide-91
SLIDE 91

General metric analysis

21

Worst Best

slide-92
SLIDE 92

General metric analysis

21

Worst Best

slide-93
SLIDE 93

General metric analysis

21

Worst Best

slide-94
SLIDE 94

General metric analysis

21

Worst Best

slide-95
SLIDE 95

22

slide-96
SLIDE 96

22

slide-97
SLIDE 97

22

slide-98
SLIDE 98

22

slide-99
SLIDE 99

22

slide-100
SLIDE 100

Perceptual metric analysis

23

slide-101
SLIDE 101

Perceptual metric analysis

23

slide-102
SLIDE 102

Perceptual metric analysis

23

slide-103
SLIDE 103

Perceptual metric analysis

23

slide-104
SLIDE 104

Perceptual metric analysis

23

slide-105
SLIDE 105

Perceptual metric analysis

23

slide-106
SLIDE 106

Perceptual metric analysis

23

slide-107
SLIDE 107

Perceptual metric analysis

23

slide-108
SLIDE 108

Perceptual metric analysis

23

slide-109
SLIDE 109

Perceptual metric analysis

23

slide-110
SLIDE 110

Perceptual metric analysis

23

slide-111
SLIDE 111

24

slide-112
SLIDE 112

24

slide-113
SLIDE 113

Comparing industry with academia

25

slide-114
SLIDE 114

Comparing industry with academia

25

slide-115
SLIDE 115

Comparing industry with academia

25

slide-116
SLIDE 116

26

slide-117
SLIDE 117

26

slide-118
SLIDE 118

Analysing the simplified SRF tree

27

slide-119
SLIDE 119

Analysing the simplified SRF tree

27

Startup Completed

slide-120
SLIDE 120

Analysing the simplified SRF tree

27

( , 𝕌 +, 𝒬+, 𝒳~) < 0.6

ℛrelative

Startup Completed True False

slide-121
SLIDE 121

Analysing the simplified SRF tree

27

( , 𝕌 +, 𝒬+, 𝒳~) < 0.6

ℛrelative

Startup Completed Buffer ( , 𝕌 - ) < 0.847 True False

slide-122
SLIDE 122

Analysing the simplified SRF tree

27

( , 𝕌 +, 𝒬+, 𝒳~) < -0.049

ℛabsolute

( , 𝕌 +, 𝒬+, 𝒳~) < 0.6

ℛrelative

Fetch-time ( , 𝕌+) < 4.5 Startup Completed Buffer ( , 𝕌 - ) < 0.847 Startup beyond 25 % Startup beyond 50% True False

slide-123
SLIDE 123

Analysing the simplified SRF tree

27

( , 𝕌 +, 𝒬+, 𝒳~) < -0.049

ℛabsolute

( , 𝕌 +, 𝒬+, 𝒳~) < 0.6

ℛrelative

Fetch-time ( , 𝕌+) < 4.5 Startup Completed Buffer ( , 𝕌 - ) < 0.847 Startup beyond 25 % Startup beyond 50% True False

slide-124
SLIDE 124

What usage has interpretability

28

slide-125
SLIDE 125

What usage has interpretability

28

  • Identify potential issues
slide-126
SLIDE 126

What usage has interpretability

28

  • Identify potential issues
  • Tracing the input-output mapping
slide-127
SLIDE 127

Limitations & Future Work

29

slide-128
SLIDE 128

Limitations & Future Work

29

  • Primitives
slide-129
SLIDE 129

Limitations & Future Work

29

  • Primitives
  • Decision space
slide-130
SLIDE 130

Limitations & Future Work

29

  • Primitives
  • Decision space
  • Interpretability is subjective
slide-131
SLIDE 131

Limitations & Future Work

29

  • Primitives
  • Decision space
  • Interpretability is subjective
  • Region
slide-132
SLIDE 132

Takeaways

30

slide-133
SLIDE 133

Takeaways

30

  • Decision trees are a great versatile tool
slide-134
SLIDE 134

Takeaways

30

  • Decision trees are a great versatile tool
  • Interpretability is useful
slide-135
SLIDE 135

Takeaways

30

  • Decision trees are a great versatile tool
  • Interpretability is useful
  • Interpretability is subjective
slide-136
SLIDE 136

Takeaways

30

  • Decision trees are a great versatile tool
  • Interpretability is useful
  • Interpretability is subjective
  • Limited by domain knowledge
slide-137
SLIDE 137
slide-138
SLIDE 138

Melissa Licciardello melissa.licciardello@inf.ethz.ch

slide-139
SLIDE 139

Ankit Singla Melissa Licciardello melissa.licciardello@inf.ethz.ch ankit.singla@inf.ethz.ch

slide-140
SLIDE 140

Ankit Singla Melissa Licciardello Maximilian Grüner mgruener@inf.ethz.ch melissa.licciardello@inf.ethz.ch ankit.singla@inf.ethz.ch

slide-141
SLIDE 141

Ankit Singla Melissa Licciardello Maximilian Grüner mgruener@inf.ethz.ch melissa.licciardello@inf.ethz.ch ankit.singla@inf.ethz.ch

https://github.com/magruener/reconstructing-proprietary-video-streaming-algorithms