Abstract:
Event cameras have advantages such as high temporal resolution, high dynamic range, and low latency, and low power consumption, while pulse neural networks (SNNs) inspired by neurodynamics have event driven characteristics that can simulate the information processing mechanism of the brain. Their combination provides a good foundation for mimicking biological vision. Therefore, a novel single step inference SNN is proposed to achieve dense depth estimation from sparse binocular event streams, focusing on how to utilize the high temporal resolution of event data for high-precision stereo matching. Specifically, an event data expression method more suitable for SNN excitation transfer is designed, and a feature correlation module is introduced to accurately match the spatiotemporal features in the binocular event stream. A U-shaped encoder-decoder and a multi-prediction-head architecture are used to extract and decode binocular event features, and the deep prediction task is directly completed through single step inference. The experimental results on indoor and outdoor sequences of the open source dataset MVSEC show that the proposed method outperforms the current advanced ANN (artificial neural network), SNN, and ANN-SNN hybrid baseline methods in terms of accuracy and energy consumption. The ablation experiment further verifies the effectiveness of each module.