Low-power smart imagers for vision-enabled Low-power smart imagers - - PowerPoint PPT Presentation

▶
low power smart imagers for vision enabled low power
SMART_READER_LITE
LIVE PREVIEW

Low-power smart imagers for vision-enabled Low-power smart imagers - - PowerPoint PPT Presentation

Dec 27, 2023 21 likes •323 views

Low-power smart imagers for vision-enabled Low-power smart imagers for vision-enabled wireless sensor networks and a case study wireless sensor networks and a case study J. Fernndez-Berni, R. Carmona-Galn, . Rodrguez-Vzquez Institute

slide-1
SLIDE 1

First ACM/ I EEE I nternational Workshop On Architecture of Smart Camera

Clermont Ferrand, France April 5-6, 2012

  • J. Fernández-Berni, R. Carmona-Galán, Á. Rodríguez-Vázquez

Institute of Microelectronics of Seville (IMSE-CNM), CSIC - Universidad de Sevilla (Spain)

Low-power smart imagers for vision-enabled wireless sensor networks and a case study Low-power smart imagers for vision-enabled wireless sensor networks and a case study

slide-2
SLIDE 2

WASC April 2012, Clermont-Ferrand (France)

Low-level image processing Low-level image processing

INSTRUCTION FLOW, COMPUTATIONAL LOAD

  • Potential parallel operation
  • Moderate accuracy required

Sobel Operators

slide-3
SLIDE 3

WASC April 2012, Clermont-Ferrand (France)

Conventional approach Conventional approach

F Imager

Low-level tasks Mid-level tasks High-level tasks

I nformation Flow: F F >> >> f > f > f

ADC

Digital Signal Processor F

f f

ARRAY OF SENSORS

 Analytic issues are mostly software issues

Brute force pattern matching used by many system developers

Extremely inefficient in terms of speed and power

slide-4
SLIDE 4

WASC April 2012, Clermont-Ferrand (France)

Focal-plane array computing Focal-plane array computing

f

Smart Imager

Mid-level tasks High-level tasks

I nformation Flow: f > f > f

ADC

Digital Signal Processor

f

f

ARRAY OF SENSOR – PROCESSORS

 Content-aware sensing-processing  Progressive extraction of relevant information  Parallel and distributed processing  Distributed memory

slide-5
SLIDE 5

WASC April 2012, Clermont-Ferrand (France)

Focal-plane array computing Focal-plane array computing

Sensor Memory Mixed‐signal processor

PER PIXEL:

Optical sensing

Neighbor Connectivity

Local Processing

Local Memory

Single Instruction Multiple Data (SIMD) Architecture

slide-6
SLIDE 6

WASC April 2012, Clermont-Ferrand (France)

Focal-plane array computing Focal-plane array computing

 Several

generation

  • f

chips designed

 With fully programmable features  Covering

a large variety

  • f

functional targets

 Image-to-Decision

at >1,000F/s with 60nW per pixel

 Spatio-temporal

filtering with 22nJ/cycle

 Content-aware

HDR acquisition with >145dB intra-frame DR

 Etc.

FLIP-Q

slide-7
SLIDE 7

WASC April 2012, Clermont-Ferrand (France)

FLI P-Q: floorplan FLI P-Q: floorplan

  • J. Fernández Berni, R. Carmona Galán and L.

Carranza González, “FLIP-Q: A QCIF Resolution Focal-Plane Array for Low-Power Image Processing,” in IEEE J. Solid-State Circuits, vol. 46, no. 3, pp. 669–680, March 2011

slide-8
SLIDE 8

WASC April 2012, Clermont-Ferrand (France)

FLI P-Q: elementary processing cell FLI P-Q: elementary processing cell

  • Reset transistor
  • n-well/p-substrate photodiode
  • Electronic global shutter
  • Programmable block-wise image

filtering and averaging

  • Programmable block-wise image

energy computation

  • Readout circuitry
slide-9
SLIDE 9

WASC April 2012, Clermont-Ferrand (France)

Physical design Physical design

34µm 29µm

Crucial aspect affecting the

total area, the fill factor and the pixel pitch

The electrical design must be

realized bearing in mind the subsequent physical design

Relevant issues: Metal layers available Full-custom routing Make the most of the

design rules

slide-10
SLIDE 10

WASC April 2012, Clermont-Ferrand (France)

Physical design Physical design

60%

+2% per additional µm in the elementary cell

slide-11
SLIDE 11

WASC April 2012, Clermont-Ferrand (France)

FLI P-Q: A prototype smart imager FLI P-Q: A prototype smart imager

slide-12
SLIDE 12

WASC April 2012, Clermont-Ferrand (France)

FLI P-Q: on-chip early vision FLI P-Q: on-chip early vision

ideal chip error

Programmable Gaussian filtering

slide-13
SLIDE 13

WASC April 2012, Clermont-Ferrand (France)

Fully-programmable multi-resolution scene representation

On-chip images

FLI P-Q: on-chip early vision FLI P-Q: on-chip early vision

slide-14
SLIDE 14

WASC April 2012, Clermont-Ferrand (France)

Image pre-distortion for reduced kernel filtering

FLI P-Q: on-chip early vision FLI P-Q: on-chip early vision

Original kernels Reduced kernels

slide-15
SLIDE 15

WASC April 2012, Clermont-Ferrand (France)

Wi-FLI P: a vision-enabled WSN node Wi-FLI P: a vision-enabled WSN node

f

Smart Imager

Mid-level tasks High-level tasks

I nformation Flow: f > f > f

ADC

Digital Signal Processor

f

f

ARRAY OF SENSOR – PROCESSORS

slide-16
SLIDE 16

WASC April 2012, Clermont-Ferrand (France)

Wi-FLI P: a vision-enabled WSN node Wi-FLI P: a vision-enabled WSN node

Imote2 (MEMSIC Inc.)

slide-17
SLIDE 17

WASC April 2012, Clermont-Ferrand (France)

Wi-FLI P: a vision-enabled WSN node Wi-FLI P: a vision-enabled WSN node

slide-18
SLIDE 18

WASC April 2012, Clermont-Ferrand (France)

Wi-FLI P: a vision-enabled WSN node Wi-FLI P: a vision-enabled WSN node

slide-19
SLIDE 19

WASC April 2012, Clermont-Ferrand (France)

Wi-FLI P: a vision-enabled WSN node Wi-FLI P: a vision-enabled WSN node

DoG-based edge detection

slide-20
SLIDE 20

WASC April 2012, Clermont-Ferrand (France)

Wi-FLI P: a vision-enabled WSN node Wi-FLI P: a vision-enabled WSN node

Very low throughput due to slow GPIO ports and TinyOS latency

slide-21
SLIDE 21

27

Case study: early detection of forest fires Case study: early detection of forest fires

  • High economic cost
  • Short maintenance cycles
  • Coarse grain coverage
  • Exact location must be inferred
slide-22
SLIDE 22

28

  • Vision-enabled Wireless Sensor Network

 Robustness  Scalability  Reliability  Better temporal resolution  Simpler smoke location ADVANTAGES  Ultra low power consumption required DRAWBACKS

Case study: early detection of forest fires Case study: early detection of forest fires

slide-23
SLIDE 23

29

Case study: early detection of forest fires Case study: early detection of forest fires

  • A power-efficient vision algorithm for smoke detection
  • Reconfigurable focal plane
  • Multiresolution scene representation
  • Clustering ratio
  • Growth rate
  • Propagation speed

Candidate regions SMOKE!

slide-24
SLIDE 24

30

  • Preliminary field tests

http://www.imse-cnm.csic.es/vmote/ Original sequence Motion detector Our algorithm

Case study: early detection of forest fires Case study: early detection of forest fires

slide-25
SLIDE 25

31

  • On-site smoke detection with Eye-RI S v1.2

Case study: early detection of forest fires Case study: early detection of forest fires

slide-26
SLIDE 26

32

  • Field tests with Wi-FLI P

Case study: early detection of forest fires Case study: early detection of forest fires

slide-27
SLIDE 27

33

  • Prescribed burning of a 95m x 20m shrub plot

Case study: early detection of forest fires Case study: early detection of forest fires

  • Wi-FLIP monitored all the activity for over two hours
  • No false alarm triggered
  • Successful smoke detection for two of the three vegetation areas explored
  • Thin smoke generated from a very sparse vegetation area was not detected
slide-28
SLIDE 28

34

CONCLUSI ONS CONCLUSI ONS

  • Early vision tasks represent a considerably heavy computational

load.

  • SIMD-based massively parallel mixed-signal processing takes

advantage of their intrinsic characteristics to achieve high power

efficiency and computational power.

  • FLI P-Q: A prototype vision chip tailored for ultra low-power
  • applications. Very competitive in the state of the art.
  • Wi-FLI P: A vision-enabled Wireless Sensor Network node

supported by I mote2. Current drawback: low throughput.

  • Case study: Early detection of forest fires, with very good results

in terms of reliability.

slide-29
SLIDE 29

35

Thank you very much for your attention

berni@imse-cnm.csic.es

Thank you very much for your attention

berni@imse-cnm.csic.es

Publication Date: May 31, 2012

slide-30
SLIDE 30

36

Acknowledgments

This work is financially supported by Andalusian Regional Government, through project 2006-TIC-2352, the Spanish Ministry of Economy and Competitiveness, through projects TEC 2009-11812 and IPT-2011-1625-430000, both co-funded by the EU-ERDF and by the Office of Naval Research (USA), through grant N000141110312.