Multi-prior physics-enhanced neural network enables pixel super-resolution and twin-imagefree phase retrieval from single-shot hologram

Digital in-line holographic microscopy(DIHM)is a widely used interference technique for real-time reconstruction of living cells’morphological information with large space-bandwidth product and compact setup.However,the need for a larger pixel size of detector to improve imaging photosensitivity,field-of-view,and signal-to-noise ratio often leads to the loss of sub-pixel information and limited pixel resolution.Additionally,the twin-image appearing in the reconstruction severely degrades the quality of the reconstructed image.The deep learning(DL)approach has emerged as a powerful tool for phase retrieval in DIHM,effectively addressing these challenges.However,most DL-based strategies are datadriven or end-to-end net approaches,suffering from excessive data dependency and limited generalization ability.Herein,a novel multi-prior physics-enhanced neural network with pixel super-resolution(MPPN-PSR)for phase retrieval of DIHM is proposed.It encapsulates the physical model prior,sparsity prior and deep image prior in an untrained deep neural network.The effectiveness and feasibility of MPPN-PSR are demonstrated by comparing it with other traditional and learning-based phase retrieval methods.With the capabilities of pixel super-resolution,twin-image elimination and high-throughput jointly from a single-shot intensity measurement,the proposed DIHM approach is expected to be widely adopted in biomedical workflow and industrial measurement.

Structured illumination microscopy for super-resolution and optical sectioning

Optical microscopy plays an essential role in biological studies due to its capability and compatibility of non-contact,minimally invasive observation and measurement of live specimens.However,the conventional optical microscopy only has a spatial resolution about200 nm due to the Abbe diffraction limit,and also lacks the ability of three-dimensional imaging.Super-resolution farfield optical microscopy based on special illumination schemes has been dramatically developed over the last decade.Among them,only the structured illumination microscopy(SIM)is of wide-field geometry that enables it easily compatible with the conventional optical microscope.In this article,the principle of SIM was introduced in terms of point spread function(PSF)and optical transform function(OTF)of the optical system.The SIM for super-resolution(SIM-SR)proposed by Gustafsson et al.and the SIM for optical sectioning(SIM-OS)proposed by Neil et al.are the most popular ones in the research community of microscopy.They have the same optical configuration,but with different data postprocessing algorithms.We mathematically described the basic theories for both of the SIMs,respectively,and presented some numerical simulations to show the effects of super-resolution and optical sectioning.Various approaches to generation of structured illumination patterns were reviewed.As an example,a SIM system based on DMDmodulation and LED-illumination was demonstrated.A lateral resolution of 90 nm was achieved with gold nanoparticles.The optical sectioning capability of the microscope was demonstrated with Golgi-stained mouse brain neurons,and the sectioning strength of 930 nm was obtained.

Dual-wavelength in-line digital holography with untrained deep neural networks

Dual-wavelength in-line digital holography(DIDH)is one of the popular methods for quantitative phase imaging of objects with non-contact and high-accuracy features.Two technical challenges in the reconstruction of these objects include suppressing the amplified noise and the twin-image that respectively originate from the phase difference and the phase-conjugated wavefronts.In contrast to the conventional methods,the deep learning network has become a powerful tool for estimating phase information in DIDH with the assistance of noise suppressing or twin-image removing ability.However,most of the current deep learning-based methods rely on supervised learning and training instances,thereby resulting in weakness when it comes to applying this training to practical imaging settings.In this paper,a new DIDH network(DIDH-Net)is proposed,which encapsulates the prior image information and the physical imaging process in an untrained deep neural network.The DIDHNet can effectively suppress the amplified noise and the twin-image of the DIDH simultaneously by automatically adjusting the weights of the network.The obtained results demonstrate that the proposed method with robust phase reconstruction is well suited to improve the imaging performance of DIDH.

Erratum to:High-throughput fast full-color digital pathology based on Fourier ptychographic microscopy via color transfer [Sci. China-Phys. Mech. Astron. 64(11), 114211 (2021)]

In this paper[1],Figure 5(a)is mistakenly used with the figure of ground truth,which is taken and stitched by 10×/0.3NA objective and the operations have been detailed in the main text.The correction is as follows(Figure 1).This error does not affect the conclusion of this paper.The authors have also checked other figures such as Figure 6,and they are used correctly.

Digital holographic microscopy with phase-shift-free structured illumination

When structured illumination is used in digital holographic microscopy(DHM),each direction of the illumination fringe is required to be shifted at least three times to perform the phase-shifting reconstruction.In this paper,we propose a scheme for spatial resolution enhancement of DHM by using the structured illumination but without phase shifting.The structured illuminations of different directions,which are generated by a spatial light modulator,illuminate the sample sequentially in the object plane.The formed object waves interfere with a reference wave in an off-axis configuration,and a CCD camera records the generated hologram.After the object waves are reconstructed numerically,a synthetic aperture is performed by an iterative algorithm to enhance the spatial resolution.The resolution improvement of the proposed method is proved and demonstrated by both simulation and experiment.

Rapid full-color Fourier ptychographic microscopy via spatially filtered color transfer

Full-color imaging is of critical importance in digital pathology for analyzing labeled tissue sections.In our previous cover story[Sci.China:Phys.,Mech.Astron.64,114211(2021)],a color transfer approach was implemented on Fourier ptychographic microscopy(FPM)for achieving high-throughput full-color whole slide imaging without mechanical scanning.The approach was able to reduce both acquisition and reconstruction time of FPM by three-fold with negligible trade-off on color accuracy.However,the method cannot properly stain samples with two or more dyes due to the lack of spatial constraints in the color transfer process.It also requires a high computation cost in histogram matching of individual patches.Here we report a modified full-color imaging algorithm for FPM,termed color-transfer filtering FPM(CFFPM).In CFFPM,we replace the original histogram matching process with a combination of block processing and trilateral spatial filtering.The former step reduces the search of the solution space for colorization,and the latter introduces spatial constraints that match the low-resolution measurement.We further adopt an iterative process to refine the results.We show that this method can perform accurate and fast color transfer for various specimens,including those with multiple stains.The statistical results of 26 samples show that the average root mean square error is only 1.26%higher than that of the red-green-blue sequential acquisition method.For some cases,CFFPM outperforms the sequential method because of the coherent artifacts introduced by dust particles.The reported CFFPM strategy provides a turnkey solution for digital pathology via computational optical imaging.

High-throughput fast full-color digital pathology based on Fourier ptychographic microscopy via color transfer

The usage of full-color imaging in digital pathology produces significant results.Compared with a grayscale image or a pseudocolor image containing contrast information,a full-color image can identify and detect the target object better with color texture information.Fourier ptychographic microscopy(FPM)is a high-throughput computational imaging technique that breaks the tradeoff between high resolution(HR)and a large field of view.It also eliminates the artifacts of scanning and stitching in digital pathology and improves its imaging efficiency.However,the conventional full-color digital pathology based on FPM is still time-consuming because of the repeated experiments with tri-wavelengths.A color transfer FPM approach termed“CFPM”was reported.The color texture information of a low-resolution full-color pathologic image is directly transferred to the HR grayscale FPM image captured by only a single wavelength.Both of the color space of FPM based on the standard CIE-XYZ color model and the display based on the standard RGB color space were established.Different FPM colorization schemes were analyzed and compared with 30 biological samples.Three types of evaluation approaches were provided,including the root-mean-square error(RMSE),the difference maps,and the image histogram cosine similarity.The average RMSE values of the conventional method and CFPM compared with the ground truth were 5.3%and 5.7%,respectively.Therefore,the reconstruction time is significantly reduced by 2/3 with the sacrifice of precision of only 0.4%.The CFPM method is also compatible with advanced fast FPM approaches to further reduce computation time.

Off-axis optical levitation and transverse spinning of metallic microparticles

Optical manipulation of metallic microparticles remains a significant challenge because of the strong scattering forces arising from the high extinction coefficient of the particles.This paper reports a new mechanism for stable confinement of metallic microparticles using a tightly focused linearly polarized Gaussian beam.Theoretical and experimental results demonstrate that metallic microparticles can be captured off the optical axis in such a beam.Meanwhile,the three-dimensionally confined particles are observed spinning transversely as a response to the asymmetric force field.The off-axis levitation and transverse spinning of metallic microparticles may provide a new way for effective manipulation of metallic microparticles.

Digital holographic shape measurement using Fizeau microscopy

We present a Fizeau interferometer using a microscopic objective as a tool for surface contouring without the need for a numerical lens for reconstruction. The interferometer is associated with a telescope system to feature the object with collimated light. The experiment is conducted on two objects possessing different step heights.The phase maps from the captured off-axis holograms are calculated numerically, which allows us to deduce the contours of the objects. The great advantages of the presented technique are that it can be done in real time and there is no need for numerical lenses for micro-objects reconstruction.