Sophisticated deep learning with on-chip optical diffractive tensor processing

Ever-growing deep-learning technologies are making revolutionary changes for modern life.However,conventional computing architectures are designed to process sequential and digital programs but are burdened with performing massive parallel and adaptive deep-learning applications.Photonic integrated circuits provide an efficient approach to mitigate bandwidth limitations and the power-wall brought on by its electronic counterparts,showing great potential in ultrafast and energy-free high-performance computation.Here,we propose an optical computing architecture enabled by on-chip diffraction to implement convolutional acceleration,termed“optical convolution unit”(OCU).We demonstrate that any real-valued convolution kernels can be exploited by the OCU with a prominent computational throughput boosting via the concept of structral reparameterization.With the OCU as the fundamental unit,we build an optical convolutional neural network(oCNN)to implement two popular deep learning tasks:classification and regression.For classification,Fashion Modified National Institute of Standards and Technology(Fashion-MNIST)and Canadian Institute for Advanced Research(CIFAR-4)data sets are tested with accuracies of 91.63%and 86.25%,respectively.For regression,we build an optical denoising convolutional neural network to handle Gaussian noise in gray-scale images with noise levelσ=10,15,and 20,resulting in clean images with an average peak signal-to-noise ratio(PSNR)of 31.70,29.39,and 27.72 dB,respectively.The proposed OCU presents remarkable performance of low energy consumption and high information density due to its fully passive nature and compact footprint,providing a parallel while lightweight solution for future compute-in-memory architecture to handle high dimensional tensors in deep learning.