Photonic neuromorphic architecture for tens-of-task lifelong learning

Scalable,high-capacity,and low-power computing architecture is the primary assurance for increasingly manifold and large-scale machine learning tasks.Traditional electronic artificial agents by conventional power-hungry processors have faced the issues of energy and scaling walls,hindering them from the sustainable performance improvement and iterative multi-task learning.Referring to another modality of light,photonic computing has been progressively applied in high-efficient neuromorphic systems.Here,we innovate a reconfigurable lifelong-learning optical neural network(L2 ONN),for highly-integrated tens-of-task machine intelligence with elaborated algorithm-hardware codesign.Benefiting from the inherent sparsity and parallelism in massive photonic connections,L2 ONN learns each single task by adaptively activating sparse photonic neuron connections in the coherent light field,while incrementally acquiring expertise on various tasks by gradually enlarging the activation.The multi-task optical features are parallelly processed by multi-spectrum representations allocated with different wavelengths.Extensive evaluations on freespace and on-chip architectures confirm that for the first time,L2 ONN avoided the catastrophic forgetting issue of photonic computing,owning versatile skills on challenging tens-of-tasks(vision classification,voice recognition,medical diagnosis,etc.)with a single model.Particularly,L2 ONN achieves more than an order of magnitude higher efficiency than the representative electronic artificial neural networks,and 14×larger capacity than existing optical neural networks while maintaining competitive performance on each individual task.The proposed photonic neuromorphic architecture points out a new form of lifelong learning scheme,permitting terminal/edge AI systems with light-speed efficiency and unprecedented scalability.

In situ optical backpropagation training of diffractive optical neural networks

Training an artificial neural network with backpropagation algorithms to perform advanced machine learning tasks requires an extensive computational process.This paper proposes to implement the backpropagation algorithm optically for in situ training of both linear and nonlinear diffractive optical neural networlks,which enables the acceleration of training speed and improvement in energy efficiency on core computing modules.We demonstrate that the gradient of a loss function with respect to the weights of diffractive layers can be accurately calculated by measuring the forward and backward propagated optical fields based on light reciprocity and phase conjunction principles.The diffractive modulation weights are updated by programming a high-speed spatial light modulator to minimize the error between prediction and target output and perform inference tasks at the speed of light.We numerically validate the effectiveness of our approach on simulated networks for various applications.The proposed in situ optical learning architecture achieves accuracy comparable to in silico training with an electronic computer on the tasks of object dlassification and matrix-vector multiplication,which further allows the diffractive optical neural network to adapt to system imperfections.Also,the self-adaptive property of our approach facilitates the novel application of the network for all-optical imaging through scattering media.The proposed approach paves the way for robust implementation of large-scale difractive neural networks to perform distinctive tasks all-optically.

A multichannel optical computing architecture for advanced machine vision

Endowed with the superior computing speed and energy efficiency,optical neural networks(ONNs)have attracted ever-growing attention in recent years.Existing optical computing architectures are mainly single-channel due to the lack of advanced optical connection and interaction operators,solving simple tasks such as hand-written digit classification,saliency detection,etc.The limited computing capacity and scalability of single-channel ONNs restrict the optical implementation of advanced machine vision.Herein,we develop Monet:a multichannel optical neural network architecture for a universal multiple-input multiple-channel optical computing based on a novel projection-interference-prediction framework where the inter-and intra-channel connections are mapped to optical interference and diffraction.In our Monet,optical interference patterns are generated by projecting and interfering the multichannel inputs in a shared domain.These patterns encoding the correspondences together with feature embeddings are iteratively produced through the projection-interference process to predict the final output optically.For the first time,Monet validates that multichannel processing properties can be optically implemented with high-efficiency,enabling real-world intelligent multichannel-processing tasks solved via optical computing,including 3D/motion detections.Extensive experiments on different scenarios demonstrate the effectiveness of Monet in handling advanced machine vision tasks with comparative accuracy as the electronic counterparts yet achieving a ten-fold improvement in computing efficiency.For intelligent computing,the trends of dealing with real-world advanced tasks are irreversible.Breaking the capacity and scalability limitations of single-channel ONN and further exploring the multichannel processing potential of wave optics,we anticipate that the proposed technique will accelerate the development of more powerful optical Al as critical support for modern advanced machine vision.

In situ optical backpropagation training of diffractive optical neural networks:publisher’s note

This publisher’s note corrects the authors’affiliations in Photon.Res.8,940(2020).