Artificial intelligence assisted light control and computational imaging through scattering media

Coherent optical control within or through scattering media via wavefront shaping has seen broad applications since its invention around 2007.Wavefront shaping is aimed at overcoming the strong scattering,featured by random interference,namely speckle patterns.This randomness occurs due to the refractive index inhomogeneity in complex media like biological tissue or the modal dispersion in multimode fiber,yet this randomness is actually deterministic and potentially can be time reversal or precompensated.Various wavefront shaping approaches,such as optical phase conjugation,iterative optimization,and transmission matrix measurement,have been developed to generate tight and intense optical delivery or high-resolution image of an optical object behind or within a scattering medium.The performance of these modula-tions,however,is far from satisfaction.Most recently,artifcial intelligence has brought new inspirations to this field,providing exciting hopes to tackle the challenges by mapping the input and output optical patterns and building a neuron network that inherently links them.In this paper,we survey the developments to date on this topic and briefly discuss our views on how to harness machine learning(deep learning in particular)for further advancements in the field.

Special issue “Photoacoustic imaging: microscopy, tomography, and their recent applications in biomedicine” in visual computation for industry, biomedicine, and art

Photoacoustic(PA)imaging is a promising non-invasive and non-ionizing biomedical imaging modality that emerged in recent years.The articles presented in this special issue describe some of newest progress in this field.We are extremely grateful to all contributing authors.The first part of the issue covers new laser source devel-opment,including fiber lasers and laser diodes.The sec-ond part is dedicated to improving the image resolution through chronic cranial window techniques,virtual-point concept,fast polygon scanning,and Fabry Perot sensing.The third part shows the basic principles of photoacous-tic/ultrasound imaging and its applications.

Active wavefront shaping for controlling and improving multimode fi ber sensor

Wavefront shaping(WFS)techniques have been used as a powerful tool to control light propagation in complex media,including multimode fibers.In this paper,we propose a new application of WFS for multimode fber-based sensors.The use of a single multimode fiber alone,without any special fabrication,as a sensor based on the light intensity variations is not an easy task.The twist effect on multimode fiber is used as an example herein.Experimental results show that light intensity through the multimode fiber shows no direct relationship with the twist angle,but the correlation coefficient(CC)of speckle patterns does.Moreover,if WFS is applied to transform the spatially seemingly random light pattern at the exit of the multimode fiber into an optical focus.The focal pattern correlation and intensity both can serve to gauge the twist angle,with doubled measurement range and allowance of using a fast point detector to provide the feedback.With further development,WFS may find potentials to facilitate the development of multimode fber-based sensors in a variety of scenarios.

Tunable absorption characteristics in multilayered structures with graphene for biosensing

Graphene derivatives,possessing strong Raman scattering and near-infrared absorption intrin-sically,have boosted many exciting biosensing applications.The tunability of the absorption characteristics,however,remains largely unexplored to date.Here,we proposed a multilayer configuration constructed by a graphene monolayer sandwiched between a buffer layer and one-dimensional photonic crystal(1DPC)to achieve tunable graphene absorption under total in-ternal reflection(TIR).It is interesting that the unique optical properties of the buffer-graphene-1DPC multilayer structure,the electromagnetically induced transparency(EIT)-like and Fano-like absorptions,can be achieved with pre-determined resonance wavelengths,and furtherly be tuned by adjusting either the structure parameters or the incident angle of light.Theoretical analyses demonstrate that such EIT-and Fano-like absorptions are due to the interference of light in the multilayer structure and the complete transmission produced by the evanescent wave resonance in the configuration.The enhanced absorptions and the huge electrical field en-hancement effect exhibit potentials for broad applications,such as photoacoustic imaging and Raman imaging.

Introduction to the special issue on high-resolution optical focusing and imaging within or through thick scattering media

Optical technologies have been increasingly utilized in biomedicine,including diagnosis,therapy,and surgery.In almost all of these applications,photons need to propagate some distance in tissue.Therefore,the capability of focusing or demodulating light information plays an essential role,largely determining the sensitivity and spatial resolution of these techniques.This has always been desired yet considered challenging within or through thick biological tissues due to the strong scattering of light.

Long distance all-optical logic operations through a single multimode fiber empowered by wavefront shaping

Multimode fibers(MMFs)are a promising solution for high-throughput signal transmission in the time domain.However,crosstalk among different optical modes within the MMF scrambles input information and creates seemingly random speckle patterns at the output.To characterize this process,a transmission matrix(TM)can be used to relate input and output fields.Recent innovations use TMs to manipulate the output field by shaping the input wavefront for exciting advances in deep-brain imaging,neuron stimulation,quantum networks,and analog operators.However,these approaches consider input/output segments as independent,limiting their use for separate signal processing,such as logic operations.Our proposed method,which makes input/output segments as interdependent,adjusts the phase of corresponding output fields using phase bias maps superimposed on input segments.Coherent superposition enables signal logic operations through a 15-m-long MMF.In experiments,a single optical logic gate containing three basic logic functions and cascading multiple logic gates to handle binary operands is demonstrated.Bitwise operations are performed for multi-bit logic operations,and multiple optical logic gates are reconstructed simultaneously in a single logic gate with polarization multiplexing.The proposed method may open new avenues for long-range logic signal processing and transmission via MMFs.

Learning-based super-resolution interpolation for sub-Nyquist sampled laser speckles

Information retrieval from visually random optical speckle patterns is desired in many scenarios yet considered challenging.It requires accurate understanding or mapping of the multiple scattering process,or reliable capability to reverse or compensate for the scattering-induced phase distortions.In whatever situation,effective resolving and digitization of speckle patterns are necessary.Nevertheless,on some occasions,to increase the acquisition speed and/or signal-to-noise ratio(SNR),speckles captured by cameras are inevitably sampled in the sub-Nyquist domain via pixel binning(one camera pixel contains multiple speckle grains)due to finite size or limited bandwidth of photosensors.Such a down-sampling process is irreversible;it undermines the fine structures of speckle grains and hence the encoded information,preventing successful information extraction.To retrace the lost information,super-resolution interpolation for such sub-Nyquist sampled speckles is needed.In this work,a deep neural network,namely SpkSRNet,is proposed to effectively up sample speckles that are sampled below 1/10 of the Nyquist criterion to well-resolved ones that not only resemble the comprehensive morphology of original speckles(decompose multiple speckle grains from one camera pixel)but also recover the lost complex information(human face in this study)with high fidelity under normal-and low-light conditions,which is impossible with classic interpolation methods.These successful speckle super-resolution interpolation demonstrations are essentially enabled by the strong implicit correlation among speckle grains,which is non-quantifiable but could be discovered by the well-trained network.With further engineering,the proposed learning platform may benefit many scenarios that are physically inaccessible,enabling fast acquisition of speckles with sufficient SNR and opening up new avenues for seeing big and seeing clearly simultaneously in complex scenarios.

Physical origin and boundary of scalable imaging through scattering media:a deep learning-based exploration

Imaging through scattering media is valuable for many areas,such as biomedicine and communication.Recent progress enabled by deep learning(DL)has shown superiority especially in the model generalization.However,there is a lack of research to physically reveal the origin or define the boundary for such model scalability,which is important for utilizing DL approaches for scalable imaging despite scattering with high confidence.In this paper,we find the amount of the ballistic light component in the output field is the prerequisite for endowing a DL model with generalization capability by using a“one-to-all”training strategy,which offers a physical meaning invariance among the multisource data.The findings are supported by both experimental and simulated tests in which the roles of scattered and ballistic components are revealed in contributing to the origin and physical boundary of the model scalability.Experimentally,the generalization performance of the network is enhanced by increasing the portion of ballistic photons in detection.The mechanism understanding and practical guidance by our research are beneficial for developing DL methods for descattering with high adaptivity.

Towards ideal focusing of diffused light via optical wavefront shaping

Multiple scattering can significantly scramble the amplitude and phase profile of an optical field.It obscures subtle observations but only speckle patterns can be seen,unlike the ballistic regime where the information or the optical field can be identified with limited distortions.Efficient optical manipulation including information transmission and precise focusing is therefore obstructed as light travels deep into turbidmedia such as fog,turbid fluids,and biological tissues.

Wearable ultrasound bioelectronics for healthcare monitoring

Customized and personalized healthcare,being self-administered by patients,is highly sought after in light of the increasing burdens of chronic illnesses and aging populations.Outside clinics that empower individuals to have a more prominent role in managing their health are becoming a trend due to many benefits.For example,they can reduce hospital visits and infection risk during a pandemic like COVID-19.

Adaptive optical focusing through perturbed scattering media with a dynamic mutation algorithm

Optical imaging through or inside scattering media, such as multimode fiber and biological tissues, has a significant impact in biomedicine yet is considered challenging due to the strong scattering nature of light. In the past decade, promising progress has been made in the field, largely benefiting from the invention of iterative optical wavefront shaping, with which deep-tissue high-resolution optical focusing and hence imaging becomes possible. Most of the reported iterative algorithms can overcome small perturbations on the noise level but fail to effectively adapt beyond the noise level, e.g., sudden strong perturbations. Reoptimizations are usually needed for significant decorrelation to the medium since these algorithms heavily rely on the optimization performance in the previous iterations. Such ineffectiveness is probably due to the absence of a metric that can gauge the deviation of the instant wavefront from the optimum compensation based on the concurrently measured optical focusing.In this study, a square rule of binary-amplitude modulation, directly relating the measured focusing performance with the error in the optimized wavefront, is theoretically proved and experimentally validated. With this simple rule, it is feasible to quantify how many pixels on the spatial light modulator incorrectly modulate the wavefront for the instant status of the medium or the whole system. As an example of application, we propose a novel algorithm, the dynamic mutation algorithm, which has high adaptability against perturbations by probing how far the optimization has gone toward the theoretically optimal performance. The diminished focus of scattered light can be effectively recovered when perturbations to the medium cause a significant drop in the focusing performance, which no existing algorithms can achieve due to their inherent strong dependence on previous optimizations. With further improvement, the square rule and the new algorithm may boost or inspire many applications, such as high-resolution optical imaging and stimulation, in instable or dynamic scattering environments.

Towards smart optical focusing:deep learningempowered dynamic wavefront shaping through nonstationary scattering media

Optical focusing through scattering media is of great significance yet challenging in lots of scenarios,including biomedical imaging,optical communication,cybersecurity,three-dimensional displays,etc.Wavefront shaping is a promising approach to solve this problem,but most implementations thus far have only dealt with static media,which,however,deviates from realistic applications.Herein,we put forward a deep learning-empowered adaptive framework,which is specifically implemented by a proposed Timely-Focusing-Optical-Transformation-Net(TFOTNet),and it effectively tackles the grand challenge of real-time light focusing and refocusing through time-variant media without complicated computation.The introduction of recursive fine-tuning allows timely focusing recovery,and the adaptive adjustment of hyperparameters of TFOTNet on the basis of medium changing speed efficiently handles the spatiotemporal non-stationarity of the medium.Simulation and experimental results demonstrate that the adaptive recursive algorithm with the proposed network significantly improves light focusing and tracking performance over traditional methods,permitting rapid recovery of an optical focus from degradation.It is believed that the proposed deep learning-empowered framework delivers a promising platform towards smart optical focusing implementations requiring dynamic wavefront control.

Alternative Interpretation of Speckle Autocorrelation Imaging Through Scattering Media

High-resolution optical imaging through or within thick scattering media is a long sought after yet unreached goal.In the past decade,the thriving technique developments in wavefront measurement and manipulation do not significantly push the boundary forward.The optical diffusion limit is still a ceiling.In this work,we propose that a scattering medium can be conceptualized as an assembly of randomly packed pinhole cameras and the corresponding speckle pattern as a superposition of randomly shifted pinhole images.The concept is demonstrated through both simulation and experiments,confirming the new perspective to interpret the mechanism of information transmission through scattering media under incoherent illumination.We also analyze the efficiency of single-pinhole and dual-pinhole channels.While in infancy,the proposed method reveals a new perspective to understand imaging and information transmission through scattering media.

Enhancing spatiotemporal focusing of light deep inside scattering media with Time-Gated Reflection Matrix

Time-gated reflection matrix(RM)has been successfully used for optical imaging deep inside Scattering media.Recently,this method was extended to enhance the spatiotemporal focusing of light ultra-deep inside scattering media.This is achieved by calibrating the decomposition of the RM with the Tikhonov regularization parameter to convert mutiply scattered photons that share the same time of flight with the singly scattered photons into singly scattered photons.Such a capability suggests a reshaping to the interaction mechanism between light and scattering media,which may beneft or inspire wide optical applications that desire enhanced spatiotemporal focusing of light at depths inside scattering media.

Fabrication of multifunctional polydopamine-coated gold nanobones for PA/CT imaging and enhanced synergistic chemo-photothermal therapy

Multimodal imaging-guided chemo-photothermal therapy is an excellent cancer treatment,which can not only efficiently against tumor,but also can offer precise treatment window and real-time monitoring of the treatment efficiency.In our work,polydopamine(PDA)-coated gold nanobones(AuNBs@PDA nanocomplexes)were designed for this approach.The AuNBs@PDA nanocomplexes have strong absorbance in the near infrared(NIR)region and higher photothermal conversion efficiency(75.48%)than gold nanobones alone,which was facilitated for photoacoustic imaging and photothermal therapy.Besides,the loading efficiency of doxorubicin(DOX)by AuNBs@PDA nanocomplexes could be up to about 70%and DOX release from AuNBs@PDA/DOX nanocomplexes sensitively response to the lower pH environment and NIR laser irradiation,which makes them become the excellent nano-carrier for the delivery of chemotherapy drug.In vitro and in vivo studies showed significant cytotoxicity and antitumor efficacy by the AuNBs@PDA/DOX nanoplatform with negligible side effects.Meanwhile,the nanoplatform was also successfully employed for computed tomography(CT)imaging,attributing to the high atomic number and high X-ray attenuation coefficient of gold.Therefore,we believed that the proposed PDA-coated gold nanobones would be a novel multifunctional theranostic nanoagent to realize the PA/CT imaging-guided chemo-photothermal therapy of cancer.

Wavefront shaping: A versatile tool to conquer multiple scattering in multidisciplinary fields

Optical techniques offer a wide variety of applications as light-matter interactions provide extremely sensitive mechanisms to probe or treat target media.Most of these implementations rely on the usage of ballistic or quasi-ballistic photons to achieve high spatial resolution.However,the inherent scattering nature of light in biological tissues or tissue-like scattering media constitutes a critical obstacle that has restricted the penetration depth of non-scattered photons and hence limited the implementation of most optical techniques for wider applications.In addition,the components of an optical system are usually designed and manufactured for a fixed function or performance.Recent advances in wavefront shaping have demonstrated that scattering-or component-induced phase distortions can be compensated by optimizing the wavefront of the input light pattern through iteration or by conjugating the transmission matrix of the scattering medium.

Multifunctional layered black phosphorene-based nanoplatform for disease diagnosis and treatment:a review

As an outstanding two-dimensional material,black phosphorene,has attracted significant attention in the biomedicine field due to its large surface area,strong optical absorption,distinct bioactivity,excellent biocompatibility,and high biodegradability.In this review,the preparation and properties of black phosphorene are summarized first.Thereafter,black phosphorene-based multifunctional platforms employed for the diagnosis and treatment of diseases,including cancer,bone injuries,brain diseases,progressive oxidative diseases,and kidney injury,are reviewed in detail.This review provides a better understanding of the exciting properties of black phosphorene,such as its high drug-loading efficiency,photothermal conversion capability,high'O2 generation efficiency,and high electrical conductivity,as well as how these properties can be exploited in biomedicine.Finally,the research perspectives of black phosphorene are discussed.

Accelerating deep learning with high energy efficiency:From microchip to physical systems

In the era of digits and intemet,massive data have been continuously gener-ated from a variety of sources,including video,photo,audio,text,intemet of things,etc.It is intuitive that more accurate pattems can be obtained by feeding more data for effective analysis;despite the data redundancy,a clearer picture can be delineated for better decision-making.However,traditional methods,even in machine learning,do not benefit from the expanding amount of data,whose perfomance nearty saturates when the data collection is large enough(Figure 1A).Such a dilemma emerges due to their limited capability and insuff cient supply of computation power in the past.

Cartilage-inspired hydrogel lubrication strategy

In nature,when two solid interfaces contact and slide with each other,adverse friction and wear ensue.Lubrication is an important means to reduce them.Fluid film lubrication and boundary lubrication are the most typical lubrication paradigms that are different in the principle of formation.1 More generally,in fluid film lubrication mode,the two sliding surfaces are completely separated by a fluid film with a thickness much larger than the surface asperities.At this time,the energy dissipation is mainly caused by the fluid being sheared.Boundary lubrication occurs when two surfaces are in molecular contact,and the energy dissipation mainly comes from van der Waals force,Coulomb force,and dis-entanglement of surface polymers.Therefore,boundary lubrication is directly related to the physical and chemical properties of the surface.