Analog Optical Computing for Artificial Intelligence

The rapid development of artificial intelligence(AI)facilitates various applications from all areas but also poses great challenges in its hardware implementation in terms of speed and energy because of the explosive growth of data.Optical computing provides a distinctive perspective to address this bottleneck by harnessing the unique properties of photons including broad bandwidth,low latency,and high energy efficiency.In this review,we introduce the latest developments of optical computing for different AI models,including feedforward neural networks,reservoir computing,and spiking neural networks(SNNs).Recent progress in integrated photonic devices,combined with the rise of AI,provides a great opportunity for the renaissance of optical computing in practical applications.This effort requires multidisciplinary efforts from a broad community.This review provides an overview of the state-of-the-art accomplishments in recent years,discusses the availability of current technologies,and points out various remaining challenges in different aspects to push the frontier.We anticipate that the era of large-scale integrated photonics processors will soon arrive for practical AI applications in the form of hybrid optoelectronic frameworks.

From Brain Science to Artificial Intelligence

Reviewing the history of the development of artificial intelligence(AI)clearly reveals that brain science has resulted in breakthroughs in AI,such as deep learning.At present,although the developmental trend in AI and its applications has surpassed expectations,an insurmountable gap remains between AI and human intelligence.It is urgent to establish a bridge between brain science and AI research,including a link from brain science to AI,and a connection from knowing the brain to simulating the brain.The first steps toward this goal are to explore the secrets of brain science by studying new brain-imaging technology;to establish a dynamic connection diagram of the brain;and to integrate neuroscience experiments with theory,models,and statistics.Based on these steps,a new generation of AI theory and methods can be studied,and a subversive model and working mode from machine perception and learning to machine thinking and decision-making can be established.This article discusses the opportunities and challenges of adapting brain science to AI.

Selective Laser Melting under Variable Ambient Pressure: A Mesoscopic Model and Transport Phenomena

Recent reports on the selective laser melting(SLM)process under a vacuum or low ambient pressure have shown fewer defects and better surface quality of the as-printed products.Although the physical process of SLM in a vacuum has been investigated by high-speed imaging,the underlying mechanisms governing the heat transfer and molten flow are still not well understood.Herein,we first developed a mesoscopic model of SLM under variable ambient pressure based on our recent laser-welding studies.We simulated the transport phenomena of SLM 316L stainless steel powders under atmospheric and 100 Pa ambient pressure.For typical process parameters(laser power:200W;scanning speed:2m∙s^(-1);powder diameter:27 lm),the average surface temperature of the cavity approached 2800 K under atmospheric pressure,while it came close to 2300 K under 100 Pa pressure.More vigorous fluid flow(average speed:4m∙s^(-1))was observed under 100 Pa ambient pressure,because the pressure difference between the evaporation-induced surface pressure and the ambient pressure was relatively larger and drives the flow under lower pressure.It was also shown that there are periodical ripple flows(period:14ls)affecting the surface roughness of the as-printed track.Moreover,the molten flow was shown to be laminar because the Reynolds number is less than 400 and is far below the critical value of turbulence;thus,the viscous dissipation is significant.It was demonstrated that under a vacuum or lower ambient pressure,the ripple flow can be dissipated more easily by the viscous effect because the trajectory length of the ripple is longer;thus,the surface quality of the tracks is improved.To summarize,our model elucidates the physical mechanisms of the interesting transport phenomena that have been observed in independent experimental studies of the SLM process under variable ambient pressure,which could be a powerful tool for optimizing the SLM process in the future.

In situ optical backpropagation training of diffractive optical neural networks

Training an artificial neural network with backpropagation algorithms to perform advanced machine learning tasks requires an extensive computational process.This paper proposes to implement the backpropagation algorithm optically for in situ training of both linear and nonlinear diffractive optical neural networlks,which enables the acceleration of training speed and improvement in energy efficiency on core computing modules.We demonstrate that the gradient of a loss function with respect to the weights of diffractive layers can be accurately calculated by measuring the forward and backward propagated optical fields based on light reciprocity and phase conjunction principles.The diffractive modulation weights are updated by programming a high-speed spatial light modulator to minimize the error between prediction and target output and perform inference tasks at the speed of light.We numerically validate the effectiveness of our approach on simulated networks for various applications.The proposed in situ optical learning architecture achieves accuracy comparable to in silico training with an electronic computer on the tasks of object dlassification and matrix-vector multiplication,which further allows the diffractive optical neural network to adapt to system imperfections.Also,the self-adaptive property of our approach facilitates the novel application of the network for all-optical imaging through scattering media.The proposed approach paves the way for robust implementation of large-scale difractive neural networks to perform distinctive tasks all-optically.

Three-level all-fiber laser at 915 nm based on polarization-maintaining Nd^3+-doped silica fiber

Nd3+-doped fiber lasers at around 900 nm based on the 4F3/2→4I9/2 transition have obtained much research attention since they can be used as the laser sources for generating pure blue fiber lasers through the frequency doubling.Here,an all-fiber laser at 915 nm was realized by polarization-maintaining Nd3+-doped silica fiber.A net gain per unit length of up to 1.0 dB/cm at 915 nm was obtained from a 4.5 cm fiber,which to our best knowledge is the highest gain coefficient reported in this kind of silica fiber.The optical-to-optical conversion efficiency varies with the active fiber length and the reflectivity of the output fiber Bragg grating(FBG),presenting an optimal value of 5.3%at 5.1 cm fiber length and 70%reflectivity of the low reflection FBG.Additionally,the linear distributed Bragg reflector short cavity was constructed to explore its potential in realizing single-frequency 915 nm fiber laser.The measurement result of longitudinal-mode properties shows it is still multi-longitudinal mode laser operation with 40 mm laser cavity.These results indicate that the Nd3+-doped silica fiber could be used to realize all-fiber laser at 915 nm,which presents potential to be the seed source of high-power fiber laser.

DiLFM:an artifact-suppressed and noise-robust light-field microscopy through dictionary learning

Light field microscopy(LFM)has been widely used for recording 3D biological dynamics at camera frame rate.However,LFM suffers from artifact contaminations due to the illness of the reconstruction problem via naive Richardson-Lucy(RL)deconvolution.Moreover,the performance of LFM significantly dropped in low-light conditions due to the absence of sample priors.In this paper,we thoroughly analyze different kinds of artifacts and present a new LFM technique termed dictionary LFM(DiLFM)that substantially suppresses various kinds of reconstruction artifacts and improves the noise robustness with an over-complete dictionary.We demonstrate artifact-suppressed reconstructions in scattering samples such as Drosophila embryos and brains.Furthermore,we show our DiLFM can achieve robust blood cell counting in noisy conditions by imaging blood cell dynamic at 100 Hz and unveil more neurons in whole-brain calcium recording of zebrafish with low illumination power in vivo.

In situ optical backpropagation training of diffractive optical neural networks:publisher’s note

This publisher’s note corrects the authors’affiliations in Photon.Res.8,940(2020).

Frequency-domain diagonal extension imaging

The pixel size of a charge-coupled device(CCD)camera plays a major role in the image resolution,and the square pixels are attributed to the physical anisotropy of the sampling frequency.We synthesize the high sampling frequency directions from multiple frames acquired with different angles to enhance the resolution by 1.4×over conventional CCD orthogonal sampling.To directly demonstrate the improvement of frequency-domain diagonal extension(FDDE)microscopy,lens-free microscopy is used,as its resolution is dominantly determined by the pixel size.We demonstrate the resolution enhancement with a mouse skin histological specimen and a clinical blood smear sample.Further,FDDE is extended to lens-based photography with an ISO 12233 resolution target.This method paves a new way for enhancing the image resolution for a variety of imaging techniques in which the resolution is primarily limited by the sampling pixel size,for example,microscopy,photography,and spectroscopy.

Photo-responsive shape memory polymer composites enabled by doping with biomass-derived carbon nanomaterials

As photothermal conversion agents,carbon nanomaterials are widely applied in polymers for light-triggered shape memory behaviors on account of their excellent light absorption.However,they are usually derived from non-renewable fossil resources,which go against the demand for sustainable development.Biomass-derived carbon nanomaterials are expected as alternatives if they are designed with good dispersibility as well as splendid photothermal properties.Up to date,very few researches focused on this area.Herein,we report a novel light-triggered shape memory composite by incorporating renewable biomass-derived carbon nanomaterials into acrylate polymers without deep purification and processing.These functionalized carbon nanomaterials not only have stable dispersion in polymers as fillers,but also can endow the polymers with excellent and stable thermal and photothermal responsive properties in biological friendly environment.With the introduction of biomass-derived carbon nanomaterials,the mechanical properties of the composites are also further enhanced with the formation of hydrogen bonding between the carbon nanomaterials and the polymers.Notably,the doping of 1%carbon nanomaterials endows the polymer with sufficient hydrogen bonds that not only exhibit excellent thermal and photothermal responsive properties,but also with enough space for the motion of chains.These properties make such composite a promising and safe candidate for shape memory applications,which provide a new avenue in smart fabrics or intelligent soft robotics.

Mirror-enhanced scanning light-field microscopy for long-term high-speed 3D imaging with isotropic resolution

Various biological behaviors can only be observed in 3D at high speed over the long term with low phototoxicity.Light-field microscopy(LFM)provides an elegant compact solution to record 3D information in a tomographic manner simultaneously,which can facilitate high photon efficiency.However,LFM still suffers from the missing-cone problem,leading to degraded axial resolution and ringing effects after deconvolution.Here,we propose a mirrorenhanced scanning LFM(MiSLFM)to achieve long-term high-speed 3D imaging at super-resolved axial resolution with a single objective,by fully exploiting the extended depth of field of LFM with a tilted mirror placed below samples.To establish the unique capabilities of MiSLFM,we performed extensive experiments,we observed various organelle interactions and intercellular interactions in different types of photosensitive cells under extremely low light conditions.Moreover,we demonstrated that superior axial resolution facilitates more robust blood cell tracking in zebrafish larvae at high speed.

Unsupervised content-preserving transformation for optical microscopy

The development of deep learning and open access to a substantial collection of imaging data together provide a potential solution for computational image transformation,which is gradually changing the landscape of optical imaging and biomedical research.However,current implementations of deep learning usually operate in a supervised manner,and their reliance on laborious and error-prone data annotation procedures remains a barrier to more general applicability.Here,we propose an unsupervised image transformation to facilitate the utilization of deep learning for optical microscopy,even in some cases in which supervised models cannot be applied.Through the introduction of a saliency constraint,the unsupervised model,named Unsupervised content-preserving Transformation for Optical Microscopy(UTOM);can learn the mapping between two image domains without requiring paired training data while avoiding distortions of the image content.UTOM shows promising performance in a wide range of biomedical image transformation tasks,including in silico histological staining,fluorescence image restoration,and virtual fluorescence labeling.Quantitative evaluations reveal that UTOM achieves stable and high-fidelity image transformations across different imaging conditions and modalities.We anticipate that our framework will encourage a paradigm shift in training neural networks and enable more applications of artificial intelligence in biomedical imaging.

A modular hierarchical array camera

Array cameras removed the optical limitations of a single camera and paved the way for high-performance imaging via the combination of micro-cameras and computation to fuse multiple aperture images.However,existing solutions use dense arrays of cameras that require laborious calibration and lack flexibility and practicality.Inspired by the cognition function principle of the human brain,we develop an unstructured array camera system that adopts a hierarchical modular design with multiscale hybrid cameras composing different modules.Intelligent computations are designed to collaboratively operate along both intra-and intermodule pathways.This system can adaptively allocate imagery resources to dramatically reduce the hardware cost and possesses unprecedented flexibility,robustness,and versatility.Large scenes of real-world data were acquired to perform human-centric studies for the assessment of human behaviours at the individual level and crowd behaviours at the population level requiring highresolution long-term monitoring of dynamic wide-area scenes.

A Through-Intact-Skull(TIS)chronic window technique for cortical structure and function observation in mice

Modern optical imaging techniques provide powerful tools for observing cortical structure and functions at high resolutions.Various skull windows have been established for different applications of cortical imaging,and each has its advantages and limitations.Most critical of the limitations,none of the current skull windows is suitable for observing the responses to some acute craniocerebral injuries on a large scale and at high resolution.Here,we developed a“Through-Intact-Skull(TIS)window”that enables the observation of an immune response on a bilateral cortical scale and at single-cell resolution after traumatic brain injury without affecting the pathological environment of the brain.The TIS window also has the advantages of craniotomy-freeness,centimeter-field of view,synaptic resolution,large imaging depth,long-term observation capability,and suitability for awake mice.Therefore,the TIS window is a promising new approach for intravital cortical microscopy in basic research in neuroscience.

Artificial Intelligence for Metaverse:A Framework

The metaverse is attracting considerable attention recently.It aims to build a virtual environment that people can interact with the world and cooperate with each other.In this survey paper,we re-introduce metaverse in a new framework based on a broad range of technologies,including perception which enables us to precisely capture the characteristics of the real world,computation which supports the large computation requirement over large-scale data,reconstruction which builds the virtual world from the real one,cooperation which facilitates long-distance communication and teamwork between users,and interaction which bridges users and the virtual world.Despite its popularity,the fundamental techniques in this framework are still immature.Innovating new techniques to facilitate the applications of metaverse is necessary.In recent years,artificial intelligence(AI),especially deep learning,has shown promising results for empowering various areas,from science to industry.It is reasonable to imagine how we can combine AI with the framework in order to promote the development of metaverse.In this survey,we present the recent achievement by AI for metaverse in the proposed framework,including perception,computation,reconstruction,cooperation,and interaction.We also discuss some future works that AI can contribute to metaverse.