Learning the imaging mechanism directly from optical microscopy observations

The optical microscopy image plays an important role in scientific research through the direct visualization of the nanoworld,where the imaging mechanism is described as the convolution of the point spread function(PSF)and emitters.Based on a priori knowledge of the PSF or equivalent PSF,it is possible to achieve more precise exploration of the nanoworld.However,it is an outstanding challenge to directly extract the PSF from microscopy images.Here,with the help of self-supervised learning,we propose a physics-informed masked autoencoder(PiMAE)that enables a learnable estimation of the PSF and emitters directly from the raw microscopy images.We demonstrate our method in synthetic data and real-world experiments with significant accuracy and noise robustness.PiMAE outperforms DeepSTORM and the Richardson–Lucy algorithm in synthetic data tasks with an average improvement of 19.6%and 50.7%(35 tasks),respectively,as measured by the normalized root mean square error(NRMSE)metric.This is achieved without prior knowledge of the PSF,in contrast to the supervised approach used by DeepSTORM and the known PSF assumption in the Richardson–Lucy algorithm.Our method,PiMAE,provides a feasible scheme for achieving the hidden imaging mechanism in optical microscopy and has the potential to learn hidden mechanisms in many more systems.

SSL Depth: self-supervised learning enables 16× speedup in confocal microscopy-based 3D surface imaging [Invited]

In scientific and industrial research, three-dimensional (3D) imaging, or depth measurement, is a critical tool that provides detailed insight into surface properties. Confocal microscopy, known for its precision in surface measurements, plays a key role in this field. However, 3D imaging based on confocal microscopy is often challenged by significant data requirements and slow measurement speeds. In this paper, we present a novel self-supervised learning algorithm called SSL Depth that overcomes these challenges. Specifically, our method exploits the feature learning capabilities of neural networks while avoiding the need for labeled data sets typically associated with supervised learning approaches. Through practical demonstrations on a commercially available confocal microscope, we find that our method not only maintains higher quality, but also significantly reduces the frequency of the z-axis sampling required for 3D imaging. This reduction results in a remarkable 16×measurement speed, with the potential for further acceleration in the future. Our methodological advance enables highly efficient and accurate 3D surface reconstructions, thereby expanding the potential applications of confocal microscopy in various scientific and industrial fields.