Home
This Title All WIREs
WIREs RSS Feed
How to cite this WIREs title:
WIREs Data Mining Knowl Discov
Impact Factor: 2.541

Neuromorphic vision: From sensors to event‐based algorithms

Full article on Wiley Online Library:   HTML PDF

Can't access this content? Tell your librarian.

Regardless of the marvels brought by the conventional frame‐based cameras, they have significant drawbacks due to their redundancy in data and temporal latency. This causes problem in applications where low‐latency transmission and high‐speed processing are mandatory. Proceeding along this line of thought, the neurobiological principles of the biological retina have been adapted to accomplish data sparsity and high dynamic range at the pixel level. These bio‐inspired neuromorphic vision sensors alleviate the more serious bottleneck of data redundancy by responding to changes in illumination rather than to illumination itself. This paper reviews in brief one such representative of neuromorphic sensors, the activity‐driven event‐based vision sensor, which mimics human eyes. Spatio‐temporal encoding of event data permits incorporation of time correlation in addition to spatial correlation in vision processing, which enables more robustness. Henceforth, the conventional vision algorithms have to be reformulated to adapt to this new generation vision sensor data. It involves design of algorithms for sparse, asynchronous, and accurately timed information. Theories and new researches have begun emerging recently in the domain of event‐based vision. The necessity to compile the vision research carried out in this sensor domain has turned out to be considerably more essential. Towards this, this paper reviews the state‐of‐the‐art event‐based vision algorithms by categorizing them into three major vision applications, object detection/recognition, object tracking, localization and mapping. This article is categorized under: Technologies > Machine Learning
Inter device communication using AER. Figure courtesy (Gotarredona, Andreou, & Linarese, )
[ Normal View | Magnified View ]
The architecture has a pose tracking module and disparity space image (DSI) generation module. DSI is generated every key‐frame, which in turn gets created based on the movement of the camera
[ Normal View | Magnified View ]
The network consists of a feature extraction CNN, a fully connected layer, followed by drop‐out and reshaping and SP‐LSTM. Finally it gives the six dimensional pose as regressed output
[ Normal View | Magnified View ]
Motion compensated time image of a moving object drone. Blue represents event at time t0 and green represents event at time t0 + Δt. Though the moving object was occupying more space, motion compensation works well. Figure courtesy (Mitrokhin et al., )
[ Normal View | Magnified View ]
Left: image before motion compensation, right: image after motion compensation. Figure courtesy (Mitrokhin et al., )
[ Normal View | Magnified View ]
Object tracking for a moving camera. The image (available in the internet) is captured from DVS camera. As the camera was also moving, background generates a lot of clutters. Hence, tracking becomes a nontrivial task
[ Normal View | Magnified View ]
Gaussian blob tracker of face. Each part of the face such as eye, nose, mouth (Gaussian blobs of outline of the face is not shown here for clarity) is individually modeled as Gaussian cluster/blobs and a spring connectivity is maintained between them
[ Normal View | Magnified View ]
Image sequence captured from a static camera. As the background is clean, tracking becomes an easier task
[ Normal View | Magnified View ]
Top plot: HATS generation architecture, bottom plot: An example simulated local memory time surface. The pixel plane is divided into C cells. Local memory time surface is generated for each event in each cell. Histogram of each cell is generated by considering the events in the memory surface of that particular cell
[ Normal View | Magnified View ]
Various temporal weights that could be applied to events to generate memory surfaces. Figure courtesy (Afshar, Cohen, Hamilton, Tapson, & van Schaik )
[ Normal View | Magnified View ]
HFIRST architecture. S1 layer gives Gabor filter response, C1 layer is a MAX pooling layer. A separate S2 neuron for each class
[ Normal View | Magnified View ]
Framework of frame‐based feature extraction module followed by spiking classifier for object recognition
[ Normal View | Magnified View ]
Framework of ROI generation followed by conventional CNN for object recognition. Top plot shows the overall architecture. Bottom plot shows the details of the CNN layers
[ Normal View | Magnified View ]
Moving object recorded by an ATIS camera
[ Normal View | Magnified View ]
Left: image from conventional camera, right: event image from event‐based vision camera. Figure courtesy (Dynamic Vision Object Recog and Action Recog and Tracking Simulated Dataset, )
[ Normal View | Magnified View ]
Single pixel architecture of ATIS. Figure courtesy (Posch et al., )
[ Normal View | Magnified View ]
Architecture of a pixel of DVS. Figure courtesy (Lichtsteiner et al., )
[ Normal View | Magnified View ]

Browse by Topic

Technologies > Machine Learning

Access to this WIREs title is by subscription only.

Recommend to Your
Librarian Now!

The latest WIREs articles in your inbox

Sign Up for Article Alerts