Event-based color camera Alexandre Marcireau 2019-03-04 PhD - PowerPoint PPT Presentation

Event-based color camera Alexandre Marcireau 2019-03-04 PhD advisor: Pr. R. Benosman � 1

Frames and events t t y y x x synchronous, global shutter asynchronous, independent pixels � 2

ATIS (Asynchronous time-based image sensor) DVS + event-based absolute exposure measurements I ∝ 1 / Δ t � 3

Color sensor Three-chip event-based camera � 4

Generating color events ATIS ATIS ATIS � 5

Validation: color-based tracking Task: find the mean position of wooden pieces over time, using only color information � 6

Color spaces Using CIEL*a*b* to approximate human perception RGB, red + green + blue = 1 CIEL*a*b*, L = 50 a* 1 a b* b green c 0 0 red 1 Perceptually uniform with respect to a c b the Euclidean distance � 7

Color calibration Macbeth color checker Expected values as a function of the camera measurements. The red line is a mean-square errors fit. The green line is a non-linear optimisation fit with CIEL*a*b* distances as heuristic. � 8

Color calibration - results L* L* a* b* a* b* Measured points in CIEL*a*b* space Mean distance in CIEL*a*b* between measurements and expected values, expressed as a fraction the largest distance between expected values. � 9

Example optimized channel-wise linear tone mapping calibration � 10

Color signatures Bivariate gaussian distributions are used to describe the color measurements � 11

Event-based moving mean m i = λ m i-1 + (1 - λ ) e i λ ∈ [0, 1] is the inertia parameter Moving mean example with λ = 0.25 � 12

Event-based pipeline ATIS ATIS ATIS update the merge the convert the find the object’s event streams color to closest mean CIEL*a*b* signature position � 13

Results - controlled scene � 14

Outdoor scenes fixed objects, moving camera fixed camera, moving objects � 15

Results - outdoor scenes Time Time Time Time y y x x 200 140 120 250 150 x (pixels) y (pixels) x (pixels) y (pixels) 100 120 150 100 80 50 100 50 3.0 3.5 4.0 4.5 3.0 3.5 4.0 4.5 0 1 2 3 4 5 0 1 2 3 4 5 Time (s) Time (s) Time (s) Time (s) � 16

Discussion and future work 1 2 pixels / analog photoreceptor continuous spatial discrete spatial processing cells analog signal analog signal combine color channels? or 3 4 analog-to-digital digital discrete spatial conversion processing digital signal mechanical feedback � 17

Thank you for your attention � 18