Artificial intelligence assisted light control and computational imaging through scattering media

Coherent optical control within or through scattering media via wavefront shaping has seen broad applications since its invention around 2007.Wavefront shaping is aimed at overcoming the strong scattering,featured by random interference,namely speckle patterns.This randomness occurs due to the refractive index inhomogeneity in complex media like biological tissue or the modal dispersion in multimode fiber,yet this randomness is actually deterministic and potentially can be time reversal or precompensated.Various wavefront shaping approaches,such as optical phase conjugation,iterative optimization,and transmission matrix measurement,have been developed to generate tight and intense optical delivery or high-resolution image of an optical object behind or within a scattering medium.The performance of these modula-tions,however,is far from satisfaction.Most recently,artifcial intelligence has brought new inspirations to this field,providing exciting hopes to tackle the challenges by mapping the input and output optical patterns and building a neuron network that inherently links them.In this paper,we survey the developments to date on this topic and briefly discuss our views on how to harness machine learning(deep learning in particular)for further advancements in the field.

An All‑In‑One Multifunctional Touch Sensor with Carbon‑Based Gradient Resistance Elements

Human–machine interactions using deep-learning methods are important in the research of virtual reality,augmented reality,and metaverse.Such research remains challenging as current interactive sensing interfaces for single-point or multipoint touch input are trapped by massive crossover electrodes,signal crosstalk,propagation delay,and demanding configuration requirements.Here,an all-inone multipoint touch sensor(AIOM touch sensor)with only two electrodes is reported.The AIOM touch sensor is efficiently constructed by gradient resistance elements,which can highly adapt to diverse application-dependent configurations.Combined with deep learning method,the AIOM touch sensor can be utilized to recognize,learn,and memorize human–machine interactions.A biometric verification system is built based on the AIOM touch sensor,which achieves a high identification accuracy of over 98%and offers a promising hybrid cyber security against password leaking.Diversiform human–machine interactions,including freely playing piano music and programmatically controlling a drone,demonstrate the high stability,rapid response time,and excellent spatiotemporally dynamic resolution of the AIOM touch sensor,which will promote significant development of interactive sensing interfaces between fingertips and virtual objects.

Conformal Human–Machine Integration Using Highly Bending‑Insensitive,Unpixelated,and Waterproof Epidermal Electronics Toward Metaverse

Efficient and flexible interactions require precisely converting human intentions into computer-recognizable signals,which is critical to the breakthrough development of metaverse.Interactive electronics face common dilemmas,which realize highprecision and stable touch detection but are rigid,bulky,and thick or achieve high flexibility to wear but lose precision.Here,we construct highly bending-insensitive,unpixelated,and waterproof epidermal interfaces(BUW epidermal interfaces)and demonstrate their interactive applications of conformal human–machine integration.The BUW epidermal interface based on the addressable electrical contact structure exhibits high-precision and stable touch detection,high flexibility,rapid response time,excellent stability,and versatile“cut-and-paste”character.Regardless of whether being flat or bent,the BUW epidermal interface can be conformally attached to the human skin for real-time,comfortable,and unrestrained interactions.This research provides promising insight into the functional composite and structural design strategies for developing epidermal electronics,which offers a new technology route and may further broaden human–machine interactions toward metaverse.

Three-dimensional image authentication using binarized images in double random phase integral imaging

We proposed a three-dimensional (3D) image authentication method using binarized phase images in double random phase integral imaging (Ini). Two-dimensional (2D) element images obtained from Ini are encoded using a double random phase encryption (DRPE) algorithm. Only part of the phase information is used in the proposed method rather than using all of the amplitude and phase information, which can make the final data sparse and beneficial to data compression, storage, and transmission. Experimental results verified the method and successfully proved the developed 3D authentication process using a nonlinear cross correlation method.

Towards smart optical focusing:deep learningempowered dynamic wavefront shaping through nonstationary scattering media

Optical focusing through scattering media is of great significance yet challenging in lots of scenarios,including biomedical imaging,optical communication,cybersecurity,three-dimensional displays,etc.Wavefront shaping is a promising approach to solve this problem,but most implementations thus far have only dealt with static media,which,however,deviates from realistic applications.Herein,we put forward a deep learning-empowered adaptive framework,which is specifically implemented by a proposed Timely-Focusing-Optical-Transformation-Net(TFOTNet),and it effectively tackles the grand challenge of real-time light focusing and refocusing through time-variant media without complicated computation.The introduction of recursive fine-tuning allows timely focusing recovery,and the adaptive adjustment of hyperparameters of TFOTNet on the basis of medium changing speed efficiently handles the spatiotemporal non-stationarity of the medium.Simulation and experimental results demonstrate that the adaptive recursive algorithm with the proposed network significantly improves light focusing and tracking performance over traditional methods,permitting rapid recovery of an optical focus from degradation.It is believed that the proposed deep learning-empowered framework delivers a promising platform towards smart optical focusing implementations requiring dynamic wavefront control.

Adaptive optical focusing through perturbed scattering media with a dynamic mutation algorithm

Optical imaging through or inside scattering media, such as multimode fiber and biological tissues, has a significant impact in biomedicine yet is considered challenging due to the strong scattering nature of light. In the past decade, promising progress has been made in the field, largely benefiting from the invention of iterative optical wavefront shaping, with which deep-tissue high-resolution optical focusing and hence imaging becomes possible. Most of the reported iterative algorithms can overcome small perturbations on the noise level but fail to effectively adapt beyond the noise level, e.g., sudden strong perturbations. Reoptimizations are usually needed for significant decorrelation to the medium since these algorithms heavily rely on the optimization performance in the previous iterations. Such ineffectiveness is probably due to the absence of a metric that can gauge the deviation of the instant wavefront from the optimum compensation based on the concurrently measured optical focusing.In this study, a square rule of binary-amplitude modulation, directly relating the measured focusing performance with the error in the optimized wavefront, is theoretically proved and experimentally validated. With this simple rule, it is feasible to quantify how many pixels on the spatial light modulator incorrectly modulate the wavefront for the instant status of the medium or the whole system. As an example of application, we propose a novel algorithm, the dynamic mutation algorithm, which has high adaptability against perturbations by probing how far the optimization has gone toward the theoretically optimal performance. The diminished focus of scattered light can be effectively recovered when perturbations to the medium cause a significant drop in the focusing performance, which no existing algorithms can achieve due to their inherent strong dependence on previous optimizations. With further improvement, the square rule and the new algorithm may boost or inspire many applications, such as high-resolution optical imaging and stimulation, in instable or dynamic scattering environments.

Wavefront shaping: A versatile tool to conquer multiple scattering in multidisciplinary fields

Optical techniques offer a wide variety of applications as light-matter interactions provide extremely sensitive mechanisms to probe or treat target media.Most of these implementations rely on the usage of ballistic or quasi-ballistic photons to achieve high spatial resolution.However,the inherent scattering nature of light in biological tissues or tissue-like scattering media constitutes a critical obstacle that has restricted the penetration depth of non-scattered photons and hence limited the implementation of most optical techniques for wider applications.In addition,the components of an optical system are usually designed and manufactured for a fixed function or performance.Recent advances in wavefront shaping have demonstrated that scattering-or component-induced phase distortions can be compensated by optimizing the wavefront of the input light pattern through iteration or by conjugating the transmission matrix of the scattering medium.

Learning-based super-resolution interpolation for sub-Nyquist sampled laser speckles

Information retrieval from visually random optical speckle patterns is desired in many scenarios yet considered challenging.It requires accurate understanding or mapping of the multiple scattering process,or reliable capability to reverse or compensate for the scattering-induced phase distortions.In whatever situation,effective resolving and digitization of speckle patterns are necessary.Nevertheless,on some occasions,to increase the acquisition speed and/or signal-to-noise ratio(SNR),speckles captured by cameras are inevitably sampled in the sub-Nyquist domain via pixel binning(one camera pixel contains multiple speckle grains)due to finite size or limited bandwidth of photosensors.Such a down-sampling process is irreversible;it undermines the fine structures of speckle grains and hence the encoded information,preventing successful information extraction.To retrace the lost information,super-resolution interpolation for such sub-Nyquist sampled speckles is needed.In this work,a deep neural network,namely SpkSRNet,is proposed to effectively up sample speckles that are sampled below 1/10 of the Nyquist criterion to well-resolved ones that not only resemble the comprehensive morphology of original speckles(decompose multiple speckle grains from one camera pixel)but also recover the lost complex information(human face in this study)with high fidelity under normal-and low-light conditions,which is impossible with classic interpolation methods.These successful speckle super-resolution interpolation demonstrations are essentially enabled by the strong implicit correlation among speckle grains,which is non-quantifiable but could be discovered by the well-trained network.With further engineering,the proposed learning platform may benefit many scenarios that are physically inaccessible,enabling fast acquisition of speckles with sufficient SNR and opening up new avenues for seeing big and seeing clearly simultaneously in complex scenarios.

Strategy to minimize bending strain interference for flexible acoustic wave sensing platform

There are great concerns for sensing using flexible acoustic wave sensors and lab-on-a-chip,as mechanical strains will dramatically change the sensing signals(e.g.,frequency)when they are bent during measurements.These strain-induced signal changes cannot be easily separated from those of real sensing signals(e.g.,humidity,ultraviolet,or gas/biological molecules).Herein,we proposed a new strategy to minimize/eliminate the effects of mechanical bending strains by optimizing off-axis angles between the direction of bending deformation and propagation of acoustic waves on curved surfaces of layered piezoelectric film/flexible glass structure.This strategy has theoretically been proved by optimization of bending designs of off-axis angles and acoustically elastic effect.Proof-of-concept for humidity and ultraviolet-light sensing using flexible SAW devices with negligible interferences are achieved within a wide range of bending strains.This work provides the best solution for achieving high-performance flexible acoustic wave sensors under deformed/bending conditions.