Physics-informed deep learning for fringe pattern analysis

Recently,deep learning has yielded transformative success across optics and photonics,especially in optical metrology.Deep neural networks (DNNs) with a fully convolutional architecture (e.g.,U-Net and its derivatives) have been widely implemented in an end-to-end manner to accomplish various optical metrology tasks,such as fringe denoising,phase unwrapping,and fringe analysis.However,the task of training a DNN to accurately identify an image-to-image transform from massive input and output data pairs seems at best naive,as the physical laws governing the image formation or other domain expertise pertaining to the measurement have not yet been fully exploited in current deep learning practice.To this end,we introduce a physics-informed deep learning method for fringe pattern analysis (PI-FPA) to overcome this limit by integrating a lightweight DNN with a learning-enhanced Fourier transform profilometry (Le FTP) module.By parameterizing conventional phase retrieval methods,the Le FTP module embeds the prior knowledge in the network structure and the loss function to directly provide reliable phase results for new types of samples,while circumventing the requirement of collecting a large amount of high-quality data in supervised learning methods.Guided by the initial phase from Le FTP,the phase recovery ability of the lightweight DNN is enhanced to further improve the phase accuracy at a low computational cost compared with existing end-to-end networks.Experimental results demonstrate that PI-FPA enables more accurate and computationally efficient single-shot phase retrieval,exhibiting its excellent generalization to various unseen objects during training.The proposed PI-FPA presents that challenging issues in optical metrology can be potentially overcome through the synergy of physics-priors-based traditional tools and data-driven learning approaches,opening new avenues to achieve fast and accurate single-shot 3D imaging.

On the use of deep learning for phase recovery

Phase recovery(PR)refers to calculating the phase of the light field from its intensity measurements.As exemplified from quantitative phase imaging and coherent diffraction imaging to adaptive optics,PR is essential for reconstructing the refractive index distribution or topography of an object and correcting the aberration of an imaging system.In recent years,deep learning(DL),often implemented through deep neural networks,has provided unprecedented support for computational imaging,leading to more efficient solutions for various PR problems.In this review,we first briefly introduce conventional methods for PR.Then,we review how DL provides support for PR from the following three stages,namely,pre-processing,in-processing,and post-processing.We also review how DL is used in phase image processing.Finally,we summarize the work in DL for PR and provide an outlook on how to better use DL to improve the reliability and efficiency of PR.Furthermore,we present a live-updating resource(https://github.com/kqwang/phase-recovery)for readers to learn more about PR.

Hybrid brightfield and darkfield transport of intensity approach for high-throughput quantitative phase microscopy

Transport of intensity equation(TIE)is a well-established non-interferometric phase retrieval approach that enables quantitative phase imaging(QPI)by simply measuring intensity images at multiple axially displaced planes.The advantage of a TIE-based QPI system is its compatibility with partially coherent illumination,which provides speckle-free imaging with resolution beyond the coherent diffraction limit.However,TIE is generally implemented with a brightfield(BF)configuration,and the maximum achievable imaging resolution is still limited to the incoherent diffraction limit(twice the coherent diffraction limit).It is desirable that TIE-related approaches can surpass this limit and achieve high-throughput[high-resolution and wide field of view(FOV)]QPI.We propose a hybrid BF and darkfield transport of intensity(HBDTI)approach for highthroughput quantitative phase microscopy.Two through-focus intensity stacks corresponding to BF and darkfield illuminations are acquired through a low-numerical-aperture(NA)objective lens.The high-resolution and large-FOV complex amplitude(both quantitative absorption and phase distributions)can then be synthesized based on an iterative phase retrieval algorithm taking the coherence model decomposition into account.The effectiveness of the proposed method is experimentally verified by the retrieval of the USAF resolution target and different types of biological cells.The experimental results demonstrate that the half-width imaging resolution can be improved from 1230 nm to 488 nm with 2.5×expansion across a 4×FOV of 7.19 mm2,corresponding to a 6.25×increase in space-bandwidth product from∼5 to∼30.2 megapixels.In contrast to conventional TIE-based QPI methods where only BF illumination is used,the synthetic aperture process of HBDTI further incorporates darkfield illuminations to expand the accessible object frequency,thereby significantly extending the maximum available resolution from 2NA to∼5NA with a∼5×promotion of the coherent diffraction limit.Given its capability for high-throughput QPI,the proposed HBDTI approach is expected to be adopted in biomedical fields,such as personalized genomics and cancer diagnostics.

Fast and robust phase retrieval for masked coherent diffractive imaging

Conventional phase retrieval algorithms for coherent diffractive imaging(CDI)require many iterations to deliver reasonable results,even using a known mask as a strong constraint in the imaging setup,an approach known as masked CDI.This paper proposes a fast and robust phase retrieval method for masked CDI based on the alternating direction method of multipliers(ADMM).We propose a plug-and-play ADMM to incorporate the prior knowledge of the mask,but note that commonly used denoisers are not suitable as regularizers for complex-valued latent images directly.Therefore,we develop a regularizer based on the structure tensor and Harris corner detector.Compared with conventional phase retrieval methods,our technique can achieve comparable reconstruction results with less time for the masked CDI.Moreover,validation experiments on real in situ CDI data for both intensity and phase objects show that our approach is more than 100 times faster than the baseline method to reconstruct one complex-valued image,making it possible to be used in challenging situations,such as imaging dynamic objects.Furthermore,phase retrieval results for single diffraction patterns show the robustness of the proposed ADMM.

Enhanced depth resolution in optical scanning holography using a configurable pupil

The optical scanning holography(OSH)technique can capture all the three-dimensional volume information of an object in a hologram via a single raster scan.The digital hologram can then be processed to reconstruct individual sectional images of the object.In this paper,we present a scheme to reconstruct sectional images in OSH with enhanced depth resolution,where a spatial light modulator(SLM)is adopted as a configurable point pupil.By switching the SLM between two states,different Fresnel zone plates(FZPs)are generated based on the same optical system.With extra information provided by different FZPs,a depth resolution at 0.7μm can be achieved.

Unsupervised learning on scientific ocean drilling datasets from the South China Sea

Unsupervised learning methods were applied to explore data patterns in multivariate geophysical datasets collected from ocean floor sediment core samples coming from scientific ocean drilling in the South China Sea.Compared to studies on similar datasets,but using supervised learning methods which are designed to make predictions based on sample training data,unsupervised learning methods require no a priori information and focus only on the input data.In this study,popular unsupervised learning methods including K-means,self-organizing maps,hierarchical clustering and random forest were coupled with different distance metrics to form exploratory data clusters.The resulting data clusters were externally validated with lithologic units and geologic time scales assigned to the datasets by conventional methods. Compact and connected data clusters displayed varying degrees of correspondence with existing classification by lithologic units and geologic time scales.K-means and self-organizing maps were observed to perform better with lithologic units while random forest corresponded best with geologic time scales.This study sets a pioneering example of how unsupervised machine learning methods can be used as an automatic processing tool for the increasingly high volume of scientific ocean drilling data.

Resolution enhancement of optical scanning holography with a spiral modulated point spread function

In optical scanning holography, one pupil produces a spherical wave and another produces a plane wave. They interfere with each other and result in a fringe pattern for scanning a three-dimensional object. The resolution of the hologram reconstruction is affected by the point spread function(PSF) of the optical system. In this paper, we modulate the PSF by a spiral phase plate, which significantly enhances the lateral and depth resolution. We explain the theory for such resolution enhancement and show simulation results to verify the efficacy of the approach.