Video scene analysis for a configurable hardware accelerator - - PowerPoint PPT Presentation

▶
video scene analysis for a configurable hardware
SMART_READER_LITE
LIVE PREVIEW

Video scene analysis for a configurable hardware accelerator - - PowerPoint PPT Presentation

Jul 01, 2023 458 likes •805 views

Video scene analysis for a configurable hardware accelerator dedicated to Smart-camera Imen Charfi, Wajdi Elhamzi, Julien Dubois, Mohamed Atri, Johel Mitran Le2i lab, Burgundy University (France) EE lab, Monastir University (Tunisia) 1

slide-1
SLIDE 1

1

Video scene analysis for a configurable hardware accelerator dedicated to Smart-camera

Imen Charfi, Wajdi Elhamzi, Julien Dubois, Mohamed Atri, Johel Mitéran

Le2i lab, Burgundy University (France) EµE lab, Monastir University (Tunisia)

slide-2
SLIDE 2

2

Video coding diversity

DVB-T/S/C/H UMTS, GPRS, cable, ADSL, dial-up, ...

Diversity in client devices

♫♪

Diversity of content form ats / standards Different types of core and access netw orks

♪♫♪ ♪♫ ♪♫

How to handle w ith such diversity ? How to provide a video codec able to support such diversity?

slide-3
SLIDE 3

3

Video codec embedded in a Smart camera

A significant contribution to handle w ith the diversity ! More challenging : flexibility to deal w ith environm ent constraints

slide-4
SLIDE 4

4

Dynamic adaptation

Scene analysis : event detection QoS analysis (network, user configuration)

slide-5
SLIDE 5

5

Current investigations

Scene analysis : PhD on automatic Fall detection QoS analysis (network, user configuration) Project with industrial partner (Re)Configurable Codec PhD on motion estimation

slide-6
SLIDE 6

6

Automatic Fall detection

Imen Charfi’s PhD

Data base available (60 videos, extension at 130 videos done) 1 vector of 3584 features / frame

slide-7
SLIDE 7

7

Detection Results

After final filtering and Bounding box Manually annotated SVM : Classification error rate 3% After final filtering and Bounding box automatically annotated error < 0.02 % per frame error < 0.05 % per frame

slide-8
SLIDE 8

8

Detection Results

Two preliminary evaluations show similar performances achieved :

  • on the extended video-data set (130 videos)
  • using boosting instead of SVM (from 10 to 100 time faster)

 The regularity and the complexity of boosting method enables a FPGA hardware implementation to be investigated. Technical lock : the real-time processing of all features

Johel MITERAN, Jiri MATAS, Elbey BOURENNANE, Michel PAINDAVOINE, Julien DUBOIS "Autom atic Hardw are I m plem entation Tool for a Discrete Adaboost-Based Decision Algorithm ", EURASIP Journal on Applied Signal Processing, Hindawi, 2005 (7), pp. 1035-1046, 2005 Fethi SMACH, Johel MITERAN, Mohamed ATRI, Julien DUBOIS, Mohamed ABID, Jean Paul GAUTHIER, "An FPGA-based accelerator for Fourier Descriptors com puting for color

  • bject recognition using SVM", Journal of Real-Time Image Processing (JRTIP), Springer, vol.2,
  • pp. 249-258, 2007.

Khalil Khattab, Julien Dubois and Johel Miteran, "Cascade Boosting Based Object Detection from High Level Description to Hardw are I m plem entation", EURASIP Journal of Embedded Systems, Special Issue "Design and Architectures for Signal Image Processing", Hindawi, 12 pages, 2009.

slide-9
SLIDE 9

9

Current investigations

Scene analysis : PhD on automatic Fall detection QoS analysis (network, user configration) Project with industrial partner (Re)Configurable Codec PhD on motion estimation

More than 60 % of the compression More than 60 % of the processing time How to use the detection results to adjust the video coding performances ?

slide-10
SLIDE 10

10

ME principle

Image split in 16x16 macro-block Each macro-block = > one motion vector

slide-11
SLIDE 11

11

How ME be flexible ?

Impact on bit-rate and on PSNR

+ 4 dB (also 45 % of overall processing time) Fast search algorithms: Processing time can be reduced up to 99% Several search phases required

slide-12
SLIDE 12

12

Architecture Overview

Wajdi Elhamzi’s PhD

slide-13
SLIDE 13

13

Integer ME architecture

slide-14
SLIDE 14

14

Halfpel & Quaterpel Co-processors

slide-15
SLIDE 15

15

Halfpel & Quaterpel Co-processor

1080 HD video streams at frame rate of 29 fps !!!! 610 cycles!!!!

slide-16
SLIDE 16

16

Overview of the proposed implementation

In IME : 752x576 video streams at 67 fps (in FS mode) In FME: 1080 HD (1920x1088) video streams at frame rate of 29 fps (around 232K Macroblocks/ s)

slide-17
SLIDE 17

17

Future works :

 Improve FME (preliminary results very promising)  Dynamic reconfiguration of the motion estimator  Investigation on other configurable parts of the codec (DCT, quantification… )  Design

  • f

a hardware video codec with the configurable motion estimation

slide-18
SLIDE 18

18

Conclusion

Our contributions for a smart camera design with adaptive video coding:  Configurable low-cost motion estimator based

  • n

FPGA component with competitive performances  Fall detection algorithm defined and hardware implementation currently investigated

slide-19
SLIDE 19

19

Questions ?

slide-20
SLIDE 20

20

Event detection

Imen Charfi’s Thesis

slide-21
SLIDE 21

21

Temporal Filtering of classification results

slide-22
SLIDE 22

22

Commun parts in H.264

Motion estimator architecture which support any search strategy! Must support H.264 features!

slide-23
SLIDE 23

23

Fast search strategy ?

Others reduced search: Three Steps… . H.264 features supported + Different search strategies supported SDSP LDSP Example

slide-24
SLIDE 24

24

Random search

Several phases requested ! One solution : prediction + random search More complex nevertheless dynamic modifications can be considered

slide-25
SLIDE 25

25

Integer ME architecture

slide-26
SLIDE 26

26

Smart Camera

CMOS sensor : 500 images/s, 1280 * 1024 pixels sur 10 bits Bandwidth required 6.55Gb/s. USB2 Bandwidth : Peak 340Mb/s upto 480Mb/s Average 200Mb/s Compression ration 30 : 6.55 / .20 ≈ 30 Others pre-processing implented

  • R. Mosqueron
  • J. Dubois
  • M. Paindavoine
slide-27
SLIDE 27

27

Smart camera with heterogeneous architecture

System Controleur Processeur Trimedia COP Ethernet Controleur VGA Controler PCI Extension SDRAM RS-232 RS-485 USB Dig I/O PCI interfaces CMOS Sensor

slide-28
SLIDE 28

28

Heterogeneous smart camera prototype

Contrôleur système Trimedia COP Contrôleur Ethernet Contrôleur VGA Extension PCI SDRAM RS-232 RS-485 USB Dig I/O PCI interfaces Capteur CMOS

slide-29
SLIDE 29

29

Application : postal sorting

slide-30
SLIDE 30

30

Application : postal sorting

slide-31
SLIDE 31

31

SystemC Modelisation

Face Detection and localisation based Viola-Jones method

  • J. Dubois J. Mitéran
  • W. Elhamzi I. Charfi
  • K. Khattab

No object Object 2 1 3 4

Rejected Sub-windows All Sub-windows Further processing

Cascade detector

A B C D +1 +1 +1 +1 +1 +1

  • 1
  • 1
  • 1
  • 1
  • 1

Haar Features

slide-32
SLIDE 32

32

Architecture model

BLOCK 1 :

  • 4 to 8

features

  • 100% Sub

windows

  • 35% total

memory access

Positives Sub windows addresses

Features

i

II

Image

Decision Shift&Scale Mem.1 Mem.2 Features

1 i

II 

Decision Mem.4 Features

2 i

II 

Decision Mem.3 BLOCK 2 :

  • 8 to 20

features

  • <50% sub

windows

  • 35% total

memory access BLOCK 3 :

  • up to 2000

features

  • <15 % sub

windows

  • 30% total

memory access

1 i

II 

Block 1 Block 2 Block 3

2 i

II 

3 i

II 

Sub windows addresses Sub windows addresses Sub windows addresses Sub windows addresses Decision

slide-33
SLIDE 33

33

Implementation

Tool : SystemCrafter