Camera Networks Dimensioning and Scheduling with Quasi Worst-Case - - PowerPoint PPT Presentation

▶
camera networks dimensioning and scheduling with quasi
SMART_READER_LITE
LIVE PREVIEW

Camera Networks Dimensioning and Scheduling with Quasi Worst-Case - - PowerPoint PPT Presentation

Oct 28, 2023 344 likes •1.07k views

Camera Networks Dimensioning and Scheduling with Quasi Worst-Case Transmission Time Viktor Edpalm Alexandre Martins Martina Maggio Karl-Erik rzn Axis Communications Axis Communications | Lund University Lund University Lund University

slide-1
SLIDE 1

Camera Networks Dimensioning and Scheduling with Quasi Worst-Case Transmission Time

Viktor Edpalm

Axis Communications

Alexandre Martins

Axis Communications | Lund University

Martina Maggio

Lund University

Karl-Erik Årzén

Lund University

slide-2
SLIDE 2

Who are we ?

  • Swedish company
  • Since 1996 / Canon since 2015
  • ~3000 employees & 179 countries
  • Development offices:

○ Lund ○ Linköping ○ Paris ○ Shanghai

  • ~835 million € sales revenue
  • We do (mainly) network cameras !

(and we are the world's number 2 at it)

slide-3
SLIDE 3

Network dimensioning ?

How to design a system of cameras ? How to estimate ?

  • Storage space ?
  • Network bandwidth ?
  • Peak capacity ?

Knowing that:

  • Each camera is unique

… but somewhat similar

  • Each set-up/time period is unique

… but belongs to a limited category list

Dimensioning <> "long term"

slide-4
SLIDE 4

Network scheduling ?

How to estimate ?

  • Maximum bandwidth ?
  • Delay ?
  • Peak capacity ?

Knowing that:

  • Each camera is unique

… but somewhat similar

  • Each set-up/time period is unique

… but belongs to a limited category list

Scheduling <> "short term"

slide-5
SLIDE 5

Video

slide-6
SLIDE 6

H.264

Group of picture

I-frame P

...

P ... P P I-frame Frame Frame Frame

...

Frame Frame Frame

... MJPEG H.26x

* We consider B-frames equivalent to P-frames * We consider I-frames as IDR frames

slide-7
SLIDE 7

H.264

slide-8
SLIDE 8

H.264

slide-9
SLIDE 9

H.264

I-frame P P ... P

slide-10
SLIDE 10

H.264

MJPEG H.264

slide-11
SLIDE 11

H.264 and scheduling

I-frame P

...

P ... P P I-frame Multiframe model

  • I-frames
  • B/P-frames

+ Many cameras: each generates a separate load

→ Paradigm: non-preemptive scheduling with multi-frame tasks

slide-12
SLIDE 12

How do we dimension ?

Dimensioning Steps

  • Characterize a reference camera model: in lab, in real scenarios
  • Characterize other camera models: in lab, (in real scenarios)
  • Characterize scene types: in real scenarios for different cameras
  • Gather scene/video parameters (motion, light level, zoom, GOP, fps…)

○ Measurable in real time ○ Estimated from usage/placement

  • Extrapolate expected frame sizes and bandwidth
slide-13
SLIDE 13

How do we dimension ?

  • 1. Parameters
slide-14
SLIDE 14

measured in laboratory measured at runtime

How do we dimension ?

  • 1. Parameters
slide-15
SLIDE 15

measured in laboratory measured at runtime

Camera

How do we dimension ?

  • 1. Parameters
slide-16
SLIDE 16

measured in laboratory measured at runtime

Camera Scene

How do we dimension ?

  • 1. Parameters
slide-17
SLIDE 17
  • 2. I-frames

Compression scaling (I-frame) Image width & height Camera noise factor (how much noise from the camera) Size of average object (lens, zoom…) Dynamic range Nature factor (amount of nature) Camera detail capability (how much does the camera retains details) Scene light level (how much light) Scene detail level (how complex is the scene)

How do we dimension ?

slide-18
SLIDE 18
  • 2. I-frames

How do we dimension ?

Compression scaling (I-frame) Image width & height Camera noise factor (how much noise from the camera) Size of average object (lens, zoom…) Dynamic range Nature factor (amount of nature) Camera detail capability (how much does the camera retains details) Scene light level (how much light) Scene detail level (how complex is the scene)

slide-19
SLIDE 19
  • 2. I-frames

How do we dimension ?

Compression scaling (I-frame) Image width & height Camera noise factor (how much noise from the camera) Size of average object (lens, zoom…) Dynamic range Nature factor (amount of nature) Camera detail capability (how much does the camera retains details) Scene light level (how much light) Scene detail level (how complex is the scene)

slide-20
SLIDE 20
  • 2. I-frames

How do we dimension ?

Compression scaling (I-frame) Image width & height Camera noise factor (how much noise from the camera) Size of average object (lens, zoom…) Dynamic range Nature factor (amount of nature) Camera detail capability (how much does the camera retains details) Scene light level (how much light) Scene detail level (how complex is the scene)

slide-21
SLIDE 21
  • 2. I-frames

How do we dimension ?

Compression scaling (I-frame) Image width & height Camera noise factor (how much noise from the camera) Size of average object (lens, zoom…) Dynamic range Nature factor (amount of nature) Camera detail capability (how much does the camera retains details) Scene light level (how much light) Scene detail level (how complex is the scene)

slide-22
SLIDE 22
  • 2. I-frames

How do we dimension ?

Compression scaling (I-frame) Image width & height Camera noise factor (how much noise from the camera) Size of average object (lens, zoom…) Dynamic range Nature factor (amount of nature) Camera detail capability (how much does the camera retains details) Scene light level (how much light) Scene detail level (how complex is the scene)

slide-23
SLIDE 23
  • 2. I-frames

Compression scaling (I-frame) Image width & height Camera noise factor (how much noise from the camera) Size of average object (lens, zoom…) Dynamic range Nature factor (amount of nature) Camera detail capability (how much does the camera retains details) Scene light level (how much light) Scene detail level (how complex is the scene)

How do we dimension ?

slide-24
SLIDE 24
  • 2. I-frames

Compression scaling (I-frame) Image width & height Camera noise factor (how much noise from the camera) Size of average object (lens, zoom…) Dynamic range Nature factor (amount of nature) Camera detail capability (how much does the camera retains details) Scene light level (how much light) Scene detail level (how complex is the scene)

How do we dimension ?

slide-25
SLIDE 25
  • 2. I-frames

Compression scaling (I-frame) Image width & height Camera noise factor (how much noise from the camera) Size of average object (lens, zoom…) Dynamic range Nature factor (amount of nature) Camera detail capability (how much does the camera retains details) Scene light level (how much light) Scene detail level (how complex is the scene)

How do we dimension ?

slide-26
SLIDE 26

Compression scaling (I-frame) Image width & height Camera noise factor (how much noise from the camera) Size of average object (lens, zoom…) Dynamic range Nature factor (amount of nature) Camera detail capability (how much does the camera retains details) Scene light level (how much light) Scene detail level (how complex is the scene)

  • 2. I-frames

How do we dimension ?

slide-27
SLIDE 27

Compression scaling (I-frame) Image width & height Camera noise factor (how much noise from the camera) Size of average object (lens, zoom…) Dynamic range Nature factor (amount of nature) Camera detail capability (how much does the camera retains details) Scene light level (how much light) Scene detail level (how complex is the scene)

  • 2. I-frames

How do we dimension ?

slide-28
SLIDE 28

Compression scaling (I-frame) Image width & height Camera noise factor (how much noise from the camera) Size of average object (lens, zoom…) Dynamic range Nature factor (amount of nature) Camera detail capability (how much does the camera retains details) Scene light level (how much light) Scene detail level (how complex is the scene)

  • 2. I-frames

+

How do we dimension ?

slide-29
SLIDE 29

Compression scaling (I-frame) Image width & height Camera noise factor (how much noise from the camera) Size of average object (lens, zoom…) Dynamic range Nature factor (amount of nature) Camera detail capability (how much does the camera retains details) Scene light level (how much light) Scene detail level (how complex is the scene)

  • 2. I-frames

+ =

How do we dimension ?

slide-30
SLIDE 30
  • 2. I-frames

IC <> size of each I pixel

How do we dimension ?

Compression scaling (I-frame) Image width & height Camera noise factor (how much noise from the camera) Size of average object (lens, zoom…) Dynamic range Nature factor (amount of nature) Camera detail capability (how much does the camera retains details) Scene light level (how much light) Scene detail level (how complex is the scene)

slide-31
SLIDE 31
  • 2. I-frames

Compression scaling (I-frame) Image width & height Camera noise factor (how much noise from the camera) Size of average object (lens, zoom…) Dynamic range Nature factor (amount of nature) Camera detail capability (how much does the camera retains details) Scene light level (how much light) Scene detail level (how complex is the scene)

I-frame IC

QP

QP +1 <> quantization step + 12%

How do we dimension ?

slide-32
SLIDE 32

Compression scaling (P-frame) Image width & height Camera noise factor (how much noise from the camera) Motion encoder efficiency (how good to find motion) Parenthesis part of I Frame rate & reference frame rate (how much changed) Motion level

  • 3. P-frames

How do we dimension ?

slide-33
SLIDE 33
  • 3. P-frames

Compression scaling (P-frame) Image width & height Camera noise factor (how much noise from the camera) Motion encoder efficiency (how good to find motion) Parenthesis part of I Frame rate & reference frame rate (how much changed) Motion level

How do we dimension ?

slide-34
SLIDE 34
  • 3. P-frames

Compression scaling (P-frame) Image width & height Camera noise factor (how much noise from the camera) Motion encoder efficiency (how good to find motion) Parenthesis part of I Frame rate & reference frame rate (how much changed) Motion level

How do we dimension ?

slide-35
SLIDE 35
  • 3. P-frames

Compression scaling (P-frame) Image width & height Camera noise factor (how much noise from the camera) Motion encoder efficiency (how good to find motion) Parenthesis part of I Frame rate & reference frame rate (how much changed) Motion level

How do we dimension ?

slide-36
SLIDE 36
  • 3. P-frames

Compression scaling (P-frame) Image width & height Camera noise factor (how much noise from the camera) Motion encoder efficiency (how good to find motion) Parenthesis part of I Frame rate & reference frame rate (how much changed) Motion level

How do we dimension ?

slide-37
SLIDE 37
  • 3. P-frames

Compression scaling (P-frame) Image width & height Camera noise factor (how much noise from the camera) Motion encoder efficiency (how good to find motion) Parenthesis part of I Frame rate & reference frame rate (how much changed) Motion level

How do we dimension ?

slide-38
SLIDE 38
  • 3. P-frames

Compression scaling (P-frame) Image width & height Camera noise factor (how much noise from the camera) Motion encoder efficiency (how good to find motion) Parenthesis part of I Frame rate & reference frame rate (how much changed) Motion level

How do we dimension ?

slide-39
SLIDE 39
  • 3. P-frames

Compression scaling (P-frame) Image width & height Camera noise factor (how much noise from the camera) Motion encoder efficiency (how good to find motion) Parenthesis part of I Frame rate & reference frame rate (how much changed) Motion level

How do we dimension ?

slide-40
SLIDE 40
  • 3. P-frames

Compression scaling (P-frame) Image width & height Camera noise factor (how much noise from the camera) Motion encoder efficiency (how good to find motion) Parenthesis part of I Frame rate & reference frame rate (how much changed) Motion level

How do we dimension ?

slide-41
SLIDE 41
  • 3. P-frames

Compression scaling (P-frame) Image width & height Camera noise factor (how much noise from the camera) Motion encoder efficiency (how good to find motion) Parenthesis part of I Frame rate & reference frame rate (how much changed) Motion level

How do we dimension ?

P I

slide-42
SLIDE 42
  • 3. P-frames

Compression scaling (P-frame) Image width & height Camera noise factor (how much noise from the camera) Motion encoder efficiency (how good to find motion) Parenthesis part of I Frame rate & reference frame rate (how much changed) Motion level

How do we dimension ?

...1s ...10s

slide-43
SLIDE 43
  • 3. P-frames

Compression scaling (P-frame) Image width & height Camera noise factor (how much noise from the camera) Motion encoder efficiency (how good to find motion) Parenthesis part of I Frame rate & reference frame rate (how much changed) Motion level

How do we dimension ?

...1s ...10s

Keep "close" to reference: limit to [2 … 0.5]

7 … 30 … 120 fps

slide-44
SLIDE 44
  • 3. P-frames

Compression scaling (P-frame) Image width & height Camera noise factor (how much noise from the camera) Motion encoder efficiency (how good to find motion) Parenthesis part of I Frame rate & reference frame rate (how much changed) Motion level

How do we dimension ?

slide-45
SLIDE 45
  • 3. P-frames

Compression scaling (P-frame) Image width & height Camera noise factor (how much noise from the camera) Motion encoder efficiency (how good to find motion) Parenthesis part of I Frame rate & reference frame rate (how much changed) Motion level

How do we dimension ?

PC <> part of each " pixel"

slide-46
SLIDE 46
  • 3. P-frames

Compression scaling (P-frame) Image width & height Camera noise factor (how much noise from the camera) Motion encoder efficiency (how good to find motion) Parenthesis part of I Frame rate & reference frame rate (how much changed) Motion level

How do we dimension ?

PC <> part of each "I pixel"

slide-47
SLIDE 47
  • 3. P-frames

Compression scaling (P-frame) Image width & height Camera noise factor (how much noise from the camera) Motion encoder efficiency (how good to find motion) Parenthesis part of I Frame rate & reference frame rate (how much changed) Motion level

How do we dimension ?

P-frame PC

QP

QP +1 <> quantization step + 12%

slide-48
SLIDE 48
  • 3. P-frames

Compression scaling (P-frame) Image width & height Camera noise factor (how much noise from the camera) Motion encoder efficiency (how good to find motion) Parenthesis part of I Frame rate & reference frame rate (how much changed) Motion level

How do we dimension ?

P-frame PC

QP

QP +1 <> quantization step + 12%

slide-49
SLIDE 49
  • 3. P-frames

Compression scaling (P-frame) Image width & height Camera noise factor (how much noise from the camera) Motion encoder efficiency (how good to find motion) Parenthesis part of I Frame rate & reference frame rate (how much changed) Motion level

How do we dimension ?

P-frame PC

QP

QP +1 <> quantization step + 12%

slide-50
SLIDE 50
  • 3. P-frames

Compression scaling (P-frame) Image width & height Camera noise factor (how much noise from the camera) Motion encoder efficiency (how good to find motion) Parenthesis part of I Frame rate & reference frame rate (how much changed) Motion level

How do we dimension ?

P-frame PC

QP

QP +1 <> quantization step + 12%

slide-51
SLIDE 51
  • 4. Video bandwidth

How do we dimension ?

slide-52
SLIDE 52
  • 4. Video bandwidth

How do we dimension ?

slide-53
SLIDE 53
  • 4. Video bandwidth

How do we dimension ?

slide-54
SLIDE 54
  • 4. Video bandwidth

How do we dimension ?

slide-55
SLIDE 55
  • 4. Video bandwidth

How do we dimension ?

slide-56
SLIDE 56
  • 4. Video bandwidth

Voilà !

How do we dimension ?

slide-57
SLIDE 57

What is already out there ?

Two documented algorithms:

  • RQM: Ding, W. and B. Liu (1996). “Rate control of mpeg video coding and recording

by rate-quantization modeling”

  • SOTA: Seetanadi, G. N., L. Oliveira, L. Almeida, K.-E. Arzen, and M. Maggio (2017).

“Game-theoretic network bandwidth distribution for self-adaptive cameras.

  • + multiple closed source security industry solutions (H.264?)

RQM (MPEG)

s(qr) = α + β · 1 / qrγ

SOTA (MJPEG)

s(ql) = ql· smax

s: frame size qr: compression level (1..31) α: constant (overhead bits) β: constant (resolution & motion) γ: constant (frametype) s: frame size ql: compression level (0.01..1) ; smax: "raw" max frame size

slide-58
SLIDE 58

How do we perform ?

Bandwidth prediction Tests

  • 1. Verification of the prediction using recorded material

available at Axis communications

  • 2. On-site test in a hotel complex
slide-59
SLIDE 59

How do we perform ?

Bandwidth prediction

Relative error %

Scenario numbers come from H.264 Video Frame Size Estimation: http://bit.ly/2LvcWtS

Lower = better

Parking lot no motion Parking lot low motion Parking lot high motion Parking lot no motion Parking lot low motion Parking lot high motion Highway Highway Fence Fence 4k street 2k fence Road crossing

slide-60
SLIDE 60

How do we perform ?

Bandwidth prediction

Industrial field test at a hotel complex, against 5 commercial bitrate estimations (obfuscated). 11 scenarios, ~5 days recordings. Relative error %

Scenario numbers come from H.264 Video Frame Size Estimation: http://bit.ly/2LvcWtS

Lower = better

Reception Exit door 2k fence Road crossing Office Street corner Reception Mall Elevator Exit door Parking lot Parking lot Parking lot

slide-61
SLIDE 61

How do we perform ?

Bandwidth prediction Average relative errors Our model: 29% SOTA: 2100% Commercial models: 336%

slide-62
SLIDE 62

How do we perform ?

Scheduling prediction Tests

  • 1. Verification using recorded material with triggered

motion amount

slide-63
SLIDE 63

How do we perform ?

Low Motion Scheduling prediction

slide-64
SLIDE 64

How do we perform ?

Low Motion Scheduling prediction

I part of the frame P part of the frame

slide-65
SLIDE 65

How do we perform ?

High Motion Scheduling prediction

slide-66
SLIDE 66

How do we perform ?

High Motion Scheduling prediction

I part of the frame P part of the frame

slide-67
SLIDE 67

How do we perform ?

Scheduling prediction

SOTA model

slide-68
SLIDE 68

How do we perform ?

Scheduling prediction

RQM performs sometimes better

… but is a posteriori … underestimates sometimes

SOTA over-estimates A LOT

(as expected)

SOTA model

slide-69
SLIDE 69

Take away

  • Model set of cameras with a taskset model.

○ allow reuse scheduling results from multiframe model. ○ better model for worst case transmission time. ○ simple enough to be run on camera.

  • Perform better (by a magnitude) than competitors.

○ both academic and industrial. ○ tested in real life scenario. ○ deployed and running for year(s).

slide-70
SLIDE 70

QUESTIONS ? REMARKS ?

Did you find the cookie monster ?

slide-71
SLIDE 71

Thank you !

http://bit.ly/2HtAVrb alexandre.martins@axis.com gitlab.com/MaralAfris /in/martinsalexandre

Did you find the cookie monster ?