A Novel Event Based Image Sensor Architecture M. AKRARAI 1 L. FESQUET - - PowerPoint PPT Presentation

▶

Feb 27, 2023 105 likes •313 views

State of the art Proposed novel architecture Simulation results A Novel Event Based Image Sensor Architecture M. AKRARAI 1 L. FESQUET 2 G. SICARD 3 1 University Grenoble Alpes, TIMA, Grenoble, France 2 CNRS, TIMA, Grenoble, France 3 CEA LETI,

SLIDE 1

State of the art Proposed novel architecture Simulation results

A Novel Event Based Image Sensor Architecture

M. AKRARAI1
L. FESQUET2
G. SICARD3

1University Grenoble Alpes, TIMA, Grenoble, France 2CNRS, TIMA, Grenoble, France 3CEA LETI, Grenoble, France

IPSOC 2019

1 / 19

SLIDE 2

State of the art Proposed novel architecture Simulation results

Image Sensors

Image sensors are present in various aspects of our life. Today we face the following challenges: Power consumption, increasing resolution and frame rate. The last two, increase the resources required for image and video processing, a good example is Automotive. How can we solve these engineering challenges?

2 / 19

SLIDE 3

State of the art Proposed novel architecture Simulation results

Outline

1

State of the art

2

Proposed novel architecture

3

Simulation results

3 / 19

SLIDE 4

State of the art Proposed novel architecture Simulation results

Introduction of frameless event based image sensors and processing architectures: Compression of temporal redundancy(Tobi Delbruck [2]). Compression of spacial redundancy (Amani Darwish[1]). Event-based Object Classification (Prophesee[3]).

4 / 19

SLIDE 5

State of the art Proposed novel architecture Simulation results

In both [2] and [1], the output of the image sensors is a frameless series of row and column addresses of active pixels:

(a) Block schematic of the IS in (Tobi Delbruck [1]) (b) Block schematic of the IS

in ( Amani Darwish [2])

(c) Event based image sensor

utput

5 / 19

SLIDE 6

State of the art Proposed novel architecture Simulation results

(a) Principale of a Dynamic Vision Sensor pixel in [1] (b) Principale of a Time to First Spike pixel in [2]

DVS TFS

6 / 19

SLIDE 7

State of the art Proposed novel architecture Simulation results

Can we combine spatial and temporal redundancies suppression and generate an event based processable output?

7 / 19

SLIDE 8

State of the art Proposed novel architecture Simulation results

The combination of the two pixel (DVS, TFS) to harvest the benefits of both.

The TFS will measure absolute luminance only if the DVS pixel detects a change.

8 / 19

SLIDE 9

State of the art Proposed novel architecture Simulation results

(a) Architecture A

depicting 1 DVS pixel per 1 TFS

(b) Architecture B

depicting 1 DVS per 3 TFS

(c) Architecture C

depicting 1 DVS per 5 TFS

(d) Architecture D depicting 1

DVS per 8 TFS

(e) Architecture E depicting 1

DVS per 24 TFS

Figure : Suggested image sensor kernels

9 / 19

SLIDE 10

State of the art Proposed novel architecture Simulation results . . . . . . . . . . . . . . . . . . . . . ... ... ... ... ... ...

(a) Architecture A

. . . . . . . . . . . . . . . . . . . . . ... ... ... ... ... ...

(b) Architecture B Figure : Example of pixel matrix of architecture A and B

To form the image sensor matrix, the kernel is repeated until we reach the desired resolution.

10 / 19

SLIDE 11

State of the art Proposed novel architecture Simulation results

Scenario Dimensions Frame rate DVS thresholds Highway 1200 × 600 30 fps 1,5,10 % Parking 1200 × 600 30 fps 1,5,10 % Worst case, will be detecting all the events from the input videos for 10 seconds: 1200 × 600 × 30 × 10 = 216000000Events

(a) Highway scenario: car on the road (b) Parking scenario: a man walks in the front of a car Figure : The two simulation scenarios

11 / 19

SLIDE 12

State of the art Proposed novel architecture Simulation results

Highway scene (a) Original highway scene Parking scene (b) Original parking scene Highway.mp4 (c) Simulation output video for architecture B, 5 % DVS threshold, Highway scenario parking.mp4 (d) Simulation output video for architecture B, 5 % DVS threshold, Parking scenario

12 / 19

SLIDE 13

State of the art Proposed novel architecture Simulation results

(a) Highway test case (b) Parking test case Figure : The number of generated events per architecture and DVS threshold

13 / 19

SLIDE 14

State of the art Proposed novel architecture Simulation results

Figure : Comparison of architectures activity per test case for 5 % DVS threshold

14 / 19

SLIDE 15

State of the art Proposed novel architecture Simulation results

Conclusion The simulation results confirm that achieving complete redundancies suppression is possible, while maintaining the relevant information of the scene.

15 / 19

SLIDE 16

State of the art Proposed novel architecture Simulation results

Upcoming work One of the presented architectures will be implemented in 28 nm FDSOI technology to provide the first proof of concept.

16 / 19

SLIDE 17

State of the art Proposed novel architecture Simulation results

References I

”A 128×128 120 dB 15µs Latency Asynchronous Temporal Contrast Vision Sensor” P. Lichtsteiner and C. Posch and T. Delbruck, Journal, IEEE Journal of Solid-State Circuits, 2008

A. Darwish, L. M. G. Rocha, L. Fesquet and G. Sicard,

”Design of a fully asynchronous image sensor reading system,” 2015 Conference on Design of Circuits and Integrated Systems (DCIS), Estoril, 2015, pp. 1-5. Sironi, Amos & Brambilla, Manuele & Bourdis, Nicolas & Lagorce, Xavier & Benosman, Ryad. (2018). HATS: Histograms of Averaged Time Surfaces for Robust Event-based Object Classification.

17 / 19

SLIDE 18

State of the art Proposed novel architecture Simulation results

This work has been partially supported by the OCEAN12 project (H2020-EU.2.1.1.7 no78327)

18 / 19

SLIDE 19

State of the art Proposed novel architecture Simulation results

THANK YOU

19 / 19