Deep learning assisted variational Hilbert quantitative phase imaging

We propose a high-accuracy artifacts-free single-frame digital holographic phase demodulation scheme for relatively lowcarrier frequency holograms-deep learning assisted variational Hilbert quantitative phase imaging(DL-VHQPI).The method,incorporating a conventional deep neural network into a complete physical model utilizing the idea of residual compensation,reliably and robustly recovers the quantitative phase information of the test objects.It can significantly alleviate spectrum-overlapping-caused phase artifacts under the slightly off-axis digital holographic system.Compared to the conventional end-to-end networks(without a physical model),the proposed method can reduce the dataset size dramatically while maintaining the imaging quality and model generalization.The DL-VHQPI is quantitatively studied by numerical simulation.The live-cell experiment is designed to demonstrate the method's practicality in biological research.The proposed idea of the deep learning-assisted physical model might be extended to diverse computational imaging techniques.

Physics-informed deep learning for fringe pattern analysis

Recently,deep learning has yielded transformative success across optics and photonics,especially in optical metrology.Deep neural networks (DNNs) with a fully convolutional architecture (e.g.,U-Net and its derivatives) have been widely implemented in an end-to-end manner to accomplish various optical metrology tasks,such as fringe denoising,phase unwrapping,and fringe analysis.However,the task of training a DNN to accurately identify an image-to-image transform from massive input and output data pairs seems at best naive,as the physical laws governing the image formation or other domain expertise pertaining to the measurement have not yet been fully exploited in current deep learning practice.To this end,we introduce a physics-informed deep learning method for fringe pattern analysis (PI-FPA) to overcome this limit by integrating a lightweight DNN with a learning-enhanced Fourier transform profilometry (Le FTP) module.By parameterizing conventional phase retrieval methods,the Le FTP module embeds the prior knowledge in the network structure and the loss function to directly provide reliable phase results for new types of samples,while circumventing the requirement of collecting a large amount of high-quality data in supervised learning methods.Guided by the initial phase from Le FTP,the phase recovery ability of the lightweight DNN is enhanced to further improve the phase accuracy at a low computational cost compared with existing end-to-end networks.Experimental results demonstrate that PI-FPA enables more accurate and computationally efficient single-shot phase retrieval,exhibiting its excellent generalization to various unseen objects during training.The proposed PI-FPA presents that challenging issues in optical metrology can be potentially overcome through the synergy of physics-priors-based traditional tools and data-driven learning approaches,opening new avenues to achieve fast and accurate single-shot 3D imaging.

Single-Shot 360-Degree Cranial Deformity Detection System Using Digital Image Correlation

In this paper,a single-shot 360-degree cranial deformity detection system using digital image correlation(DIC)is presented to quickly obtain and detect accurate 3D data of infants’cra-nium.By introducing plane mirrors into a stereo 3D DIC measurement system,a multi-view 3D imaging model is established to convert 3D data from real and virtual perspectives into 360-degree 3D data of the tested infant cranium,achieving single-shot and panoramic 3D measurement.Exper-imental results showed that the performance and measurement accuracy of the proposed system can meet the requirements for cranial deformity detection,which provides a fast,accurate,and low-cost solution medically.

Neural-field-assisted transport-of-intensity phase microscopy: partially coherent quantitative phase imaging under unknown defocus distance

The transport-of-intensity equation(TIE) enables quantitative phase imaging(QPI) under partially coherent illumination by measuring the through-focus intensities combined with a linearized inverse reconstruction algorithm. However, overcoming its sensitivity to imaging settings remains a challenging problem because of the difficulty in tuning the optical parameters of the imaging system accurately and because of the instability to long-time measurements. To address these limitations, we propose and experimentally validate a solution called neural-field-assisted transport-of-intensity phase microscopy(NFTPM) by introducing a tunable defocus parameter into neural field. Without weak object approximation, NFTPM incorporates the physical prior of partially coherent image formation to constrain the neural field and learns the continuous representation of phase object without the need for training. Simulation and experimental results of He La cells demonstrate that NFTPM can achieve accurate, partially coherent QPI under unknown defocus distances, providing new possibilities for extending applications in live cell biology.

Fringe pattern analysis using deep learning

In many optical metrology techniques,fringe pattern analysis is the central algorithm for recovering the underlying phase distribution from the recorded fringe patterns.Despite extensive research efforts for decades,how to extract the desired phase information,with the highest possible accuracy,from the minimum number of fringe patterns remains one of the most challenging open problems.Inspired by recent successes of deep learning techniques for computer vision and other applications,we demonstrate for the first time,to our knowledge,that the deep neural networks can be trained to perform fringe analysis,which substantially enhances the accuracy of phase demodulation from a single fringe pattern.The effectiveness of the proposed method is experimentally verified using carrier fringe patterns under the scenario of fringe projection profilometry.Experimental results demonstrate its superior performance,in terms of high accuracy and edge-preserving,over two representative single-frame techniques:Fourier transform profilometry and windowed Fourier transform profilometry.

Transport of intensity diffraction tomography with non-interferometric synthetic aperture for three-dimensional label-free microscopy

We present a new label-free three-dimensional(3D)microscopy technique,termed transport of intensity diffraction tomography with non-interferometric synthetic aperture(TIDT-NSA).Without resorting to interferometric detection,TIDT-NSA retrieves the 3D refractive index(RI)distribution of biological specimens from 3D intensity-only measurements at various illumination angles,allowing incoherent-diffraction-limited quantitative 3D phase-contrast imaging.The unique combination of z-scanning the sample with illumination angle diversity in TIDT-NSA provides strong defocus phase contrast and better optical sectioning capabilities suitable for high-resolution tomography of thick biological samples.Based on an off-the-shelf bright-field microscope with a programmable light-emitting-diode(LED)illumination source,TIDT-NSA achieves an imaging resolution of 206 nm laterally and 520 nm axially with a high-NA oil immersion objective.We validate the 3D RI tomographic imaging performance on various unlabeled fixed and live samples,including human breast cancer cell lines MCF-7,human hepatocyte carcinoma cell lines HepG2,mouse macrophage cell lines RAW 264.7,Caenorhabditis elegans(C.elegans),and live Henrietta Lacks(HeLa)cells.These results establish TIDT-NSA as a new non-interferometric approach to optical diffraction tomography and 3D label-free microscopy,permitting quantitative characterization of cell morphology and time-dependent subcellular changes for widespread biological and medical applications.

Resolution-enhanced intensity diffraction tomography in high numerical aperture label-free microscopy

We propose label-free and motion-free resolution-enhanced intensity diffraction tomography(reIDT)recovering the 3D complex refractive index distribution of an object.By combining an annular illumination strategy with a high numerical aperture(NA)condenser,we achieve near-diffraction-limited lateral resolution of 346 nm and axial resolution of 1.2μm over 130μm×130μm×8μm volume.Our annular pattern matches the system’s maximum NA to reduce the data requirement to 48 intensity frames.The reIDT system is directly built on a standard commercial microscope with a simple LED array source and condenser lens adds-on,and promises broad applications for natural biological imaging with minimal hardware modifications.To test the capabilities of our technique,we present the 3D complex refractive index reconstructions on an absorptive USAF resolution target and Henrietta Lacks(HeLa)and HT29 human cancer cells.Our work provides an important step in intensity-based diffraction tomography toward high-resolution imaging applications.

Hybrid brightfield and darkfield transport of intensity approach for high-throughput quantitative phase microscopy

Transport of intensity equation(TIE)is a well-established non-interferometric phase retrieval approach that enables quantitative phase imaging(QPI)by simply measuring intensity images at multiple axially displaced planes.The advantage of a TIE-based QPI system is its compatibility with partially coherent illumination,which provides speckle-free imaging with resolution beyond the coherent diffraction limit.However,TIE is generally implemented with a brightfield(BF)configuration,and the maximum achievable imaging resolution is still limited to the incoherent diffraction limit(twice the coherent diffraction limit).It is desirable that TIE-related approaches can surpass this limit and achieve high-throughput[high-resolution and wide field of view(FOV)]QPI.We propose a hybrid BF and darkfield transport of intensity(HBDTI)approach for highthroughput quantitative phase microscopy.Two through-focus intensity stacks corresponding to BF and darkfield illuminations are acquired through a low-numerical-aperture(NA)objective lens.The high-resolution and large-FOV complex amplitude(both quantitative absorption and phase distributions)can then be synthesized based on an iterative phase retrieval algorithm taking the coherence model decomposition into account.The effectiveness of the proposed method is experimentally verified by the retrieval of the USAF resolution target and different types of biological cells.The experimental results demonstrate that the half-width imaging resolution can be improved from 1230 nm to 488 nm with 2.5×expansion across a 4×FOV of 7.19 mm2,corresponding to a 6.25×increase in space-bandwidth product from∼5 to∼30.2 megapixels.In contrast to conventional TIE-based QPI methods where only BF illumination is used,the synthetic aperture process of HBDTI further incorporates darkfield illuminations to expand the accessible object frequency,thereby significantly extending the maximum available resolution from 2NA to∼5NA with a∼5×promotion of the coherent diffraction limit.Given its capability for high-throughput QPI,the proposed HBDTI approach is expected to be adopted in biomedical fields,such as personalized genomics and cancer diagnostics.

Exploiting optical degrees of freedom for information multiplexing in diffractive neural networks

Exploiting internal degrees of freedom of light, such as polarization, provides efficient ways to scale the capacity of optical diffractive computing, which may ultimately lead to high-throughput, multifunctional all-optical diffractive processors that can execute a diverse range of tasks in parallel.

Upconversion nanoparticles for super-resolution quantification of single small extracellular vesicles

Although small EVs(sEVs)have been used widely as biomarkers in disease diagnosis,their heterogeneity at single EV level has rarely been revealed.This is because high-resolution characterization of sEV presents a major challenge,as their sizes are below the optical diffraction limit.Here,we report that upconversion nanoparticles(UCNPs)can be used for super-resolution profiling the molecular heterogeneity of sEVs.We show that Er3+-doped UCNPs has better brightness and Tm3+-doped UCNPs resulting in better resolution beyond diffraction limit.Through an orthogonal experimental design,the specific targeting of UCNPs to the tumour epitope on single EV has been cross validated,resulting in the Pearson’s R-value of 0.83 for large EVs and~65%co-localization double-positive spots for sEVs.Furthermore,super-resolution nanoscopy can distinguish adjacent UCNPs on single sEV with a resolution of as high as 41.9 nm.When decreasing the size of UCNPs from 40 to 27 nm and 18 nm,we observed that the maximum UCNPs number on single sEV increased from 3 to 9 and 21,respectively.This work suggests the great potentials of UCNPs approach“digitally”quantify the surface antigens on single EVs,therefore providing a solution to monitor the EV heterogeneity changes along with the tumour progression progress.