LOEN:Lensless opto-electronic neural network empowered machine vision

Machine vision faces bottlenecks in computing power consumption and large amounts of data.Although opto-electronic hybrid neural networks can provide assistance,they usually have complex structures and are highly dependent on a coherent light source;therefore,they are not suitable for natural lighting environment applications.In this paper,we propose a novel lensless opto-electronic neural network architecture for machine vision applications.The architecture optimizes a passive optical mask by means of a task-oriented neural network design,performs the optical convolution calculation operation using the lensless architecture,and reduces the device size and amount of calculation required.We demonstrate the performance of handwritten digit classification tasks with a multiple-kernel mask in which accuracies of as much as 97.21%were achieved.Furthermore,we optimize a large-kernel mask to perform optical encryption for privacy-protecting face recognition,thereby obtaining the same recognition accuracy performance as no-encryption methods.Compared with the random MLS pattern,the recognition accuracy is improved by more than 6%.

FourierCam:a camera for video spectrum acquisition in a single shot

The novel camera architecture facilitates the development of machine vision. Instead of capturing frame sequences in the temporal domain as traditional video cameras, Fourier Cam directly measures the pixel-wise temporal spectrum of the video in a single shot through optical coding. Compared to the classic video cameras and timefrequency transformation pipeline, this programmable frequency-domain sampling strategy has an attractive combination of characteristics for low detection bandwidth, low computational burden, and low data volume.Based on the various temporal filter kernel designed by Fourier Cam, we demonstrated a series of exciting machine vision functions, such as video compression, background subtraction, object extraction, and trajectory tracking.

Single-shot 3D tracking based on polarization multiplexed Fourier-phase camera

For moving objects,3D mapping and tracking has found important applications in the 3D reconstruction for vision odometry or simultaneous localization and mapping.This paper presents a novel camera architecture to locate the fast-moving objects in four-dimensional(4D)space(x,y,z,t)through a single-shot image.Our 3D tracking system records two orthogonal fields-of-view(FoVs)with different polarization states on one polarization sensor.An optical spatial modulator is applied to build up temporal Fourier-phase coding channels,and the integration is performed in the corresponding CMOS pixels during the exposure time.With the 8 bit grayscale modulation,each coding channel can achieve 256 times temporal resolution improvement.A fast single-shot 3D tracking system with 0.78 ms temporal resolution in 200 ms exposure is experimentally demonstrated.Furthermore,it provides a new image format,Fourier-phase map,which has a compact data volume.The latent spatio-temporal information in one 2D image can be efficiently reconstructed at relatively low computation cost through the straightforward phase matching algorithm.Cooperated with scene-driven exposure as well as reasonable Fourier-phase prediction,one could acquire 4D data(x,y,z,t)of the moving objects,segment 3D motion based on temporal cues,and track targets in a complicated environment.

Atomistic Investigation of the Influence of Hydrogen on Mechanical Response during Nanoindentation in Pure Iron

The effects of hydrogen on the mechanical response of pure iron with randomly distributed hydrogen atoms under nanoindentation were systematically investigated by molecular dynamics simulations with the aim to further understand hydrogen embrittlement mechanism in the steels.The simulations results revealed that,for the three models with[001],[110]and[111]surface normal,hydrogen reduced the critical load of the pop-in event,promoted the dislocation slipping and reduced the plastic region size and dislocation density around the indenter compared to the hydrogen free model.Meanwhile,the different mechanical responses of the three models with different surface normal were further explained in the perspective of Schmid factor.

Single-shot 3D tracking based on polarization multiplexed Fourier-phase camera:erratum

Figures 5(b)and 5(c)in the original article[1]are not consistent with their captions.Correct images are shown as follows.The article[1]was corrected online on 29 March 2022.