Progress of Materials and Devices for Neuromorphic Vision Sensors

The latest developments in bio-inspired neuromorphic vision sensors can be summarized in 3 keywords:smaller,faster,and smarter.(1)Smaller:Devices are becoming more compact by integrating previously separated components such as sensors,memory,and processing units.As a prime example,the transition from traditional sensory vision computing to in-sensor vision computing has shown clear benefits,such as simpler circuitry,lower power consumption,and less data redundancy.(2)Swifter:Owing to the nature of physics,smaller and more integrated devices can detect,process,and react to input more quickly.In addition,the methods for sensing and processing optical information using various materials(such as oxide semiconductors)are evolving.(3)Smarter:Owing to these two main research directions,we can expect advanced applications such as adaptive vision sensors,collision sensors,and nociceptive sensors.This review mainly focuses on the recent progress,working mechanisms,image pre-processing techniques,and advanced features of two types of neuromorphic vision sensors based on near-sensor and in-sensor vision computing methodologies.

Neuromorphic vision sensors: Principle, progress and perspectives

Conventional frame-based image sensors suffer greatly from high energy consumption and latency.Mimicking neurobiological structures and functionalities of the retina provides a promising way to build a neuromorphic vision sensor with highly efficient image processing.In this review article,we will start with a brief introduction to explain the working mechanism and the challenges of conventional frame-based image sensors,and introduce the structure and functions of biological retina.In the main section,we will overview recent developments in neuromorphic vision sensors,including the silicon retina based on conventional Si CMOS digital technologies,and the neuromorphic vision sensors with the implementation of emerging devices.Finally,we will provide a brief outline of the prospects and outlook for the development of this field.

Recent advances in imaging devices:image sensors and neuromorphic vision sensors

Remarkable developments in image recognition technology trigger demands for more advanced imaging devices.In recent years,traditional image sensors,as the go-to imaging devices,have made substantial progress in their optoelectronic characteristics and functionality.Moreover,a new breed of imaging device with information processing capability,known as neuromorphic vision sensors,is developed by mimicking biological vision.In this review,we delve into the recent progress of imaging devices,specifically image sensors and neuromorphic vision sensors.This review starts by introducing their core components,namely photodetectors and photonic synapses,while placing a strong emphasis on device structures,working mechanisms and key performance parameters.Then it proceeds to summarize the noteworthy achievements in both image sensors and neuromorphic vision sensors,including advancements in large-scale and highresolution imaging,filter-free multispectral recognition,polarization sensitivity,flexibility,hemispherical designs,and self-power supply of image sensors,as well as in neuromorphic imaging and data processing,environmental adaptation,and ultra-low power consumption of neuromorphic vision sensors.Finally,the challenges and prospects that lie ahead in the ongoing development of imaging devices are addressed.

Low-dimensional optoelectronic synaptic devices for neuromorphic vision sensors

Neuromorphic systems represent a promising avenue for the development of the next generation of artificial intelligence hardware.Machine vision,one of the cores in artificial intelligence,requires system-level support with low power consumption,low latency,and parallel computing.Neuromorphic vision sensors provide an efficient solution for machine vision by simulating the structure and function of the biological retina.Optoelectronic synapses,which use light as the main means to achieve the dual functions of photosensitivity and synapse,are the basic units of the neuromorphic vision sensor.Therefore,it is necessary to develop various optoelectronic synaptic devices to expand the application scenarios of neuromorphic vision systems.This review compares the structure and function for both biological and artificial retina systems,and introduces various optoelectronic synaptic devices based on low-dimensional materials and working mechanisms.In addition,advanced applications of optoelectronic synapses as neuromorphic vision sensors are comprehensively summarized.Finally,the challenges and prospects in this field are briefly discussed.