Estimation-free spatial-domain image reconstruction of structured illumination microscopy

Structured illumination microscopy(SIM)achieves super-resolution(SR)by modulating the high-frequency information of the sample into the passband of the optical system and subsequent image reconstruction.The traditional Wiener-filtering-based reconstruction algorithm operates in the Fourier domain,it requires prior knowledge of the sinusoidal illumination patterns which makes the time-consuming procedure of parameter estimation to raw datasets necessary,besides,the parameter estimation is sensitive to noise or aberration-induced pattern distortion which leads to reconstruction artifacts.Here,we propose a spatial-domain image reconstruction method that does not require parameter estimation but calculates patterns from raw datasets,and a reconstructed image can be obtained just by calculating the spatial covariance of differential calculated patterns and differential filtered datasets(the notch filtering operation is performed to the raw datasets for attenuating and compensating the optical transfer function(OTF)).Experiments on reconstructing raw datasets including nonbiological,biological,and simulated samples demonstrate that our method has SR capability,high reconstruction speed,and high robustness to aberration and noise.

扫描电镜中透射-反射式STEM明场成像装置的研制及应用

本文研制了一套透射-反射式扫描透射(STEM)明场成像装置,该装置使用背散射电子探头接收经过铂片反射的透射电子信号,可安装在FEI Quanta FEG系列扫描电镜中实现STEM明场成像。分别以埃洛石、Au@ZIF-8纳米颗粒和喷金乳胶标样验证自制STEM明场成像装置的实用性。结果表明:埃洛石STEM明场像的衬度与透射电镜中STEM明场像一致,管腔和管壁具有明显的衬度差异,可以清晰辨识管腔结构;在Au@ZIF-8纳米颗粒的STEM明场像中,Au颗粒衬度较暗,ZIF-8衬度较亮,通过两者之间的衬度差异可以观察到核壳结构;喷金乳胶标样上纳米金颗粒之间可识别的最小距离为2.03 nm,进一步验证了自制STEM明场成像装置具有较高的分辨率。

Single-cell volumetric imaging with light field microscopy: Advances in systems and algorithms

Single-cell volumetric imaging is essential for researching individual characteristics of cells.As a nonscanning imaging technique,lighteld microscopy(LFM)is a critical tool to achieve realtime three-dimensional imaging with the advantage of single-shot.To address the inherent limits including nonuniform resolution and block-wise artifacts,various modied LFM strategies have been developed to provide new insights into the structural and functional information of cells.This review will introduce the principle and development of LFM,discuss the improved approaches based on hardware designs and 3D reconstruction algorithms,and present the applications in single-cell imaging.

Classification of Human Protein in Multiple Cells Microscopy Images Using CNN

The subcellular localization of human proteins is vital for understanding the structure of human cells.Proteins play a significant role within human cells,as many different groups of proteins are located in a specific location to perform a particular function.Understanding these functions will help in discoveringmany diseases and developing their treatments.The importance of imaging analysis techniques,specifically in proteomics research,is becoming more prevalent.Despite recent advances in deep learning techniques for analyzing microscopy images,classification models have faced critical challenges in achieving high performance.Most protein subcellular images have a significant class imbalance.We use oversampling and under sampling techniques in this research to overcome this issue.We have used a Convolutional Neural Network(CNN)model called GapNet-PL for the multi-label classification task on the Human Protein Atlas Classification(HPA)Dataset.Authors have found that the ParametricRectified LinearUnit(PreLU)activation function is better than the Scaled Exponential LinearUnit(SeLU)activation function in the GapNet-PL model in most classification metrics.The results showed that the GapNet-PL model with the PReLU activation function achieved an area under the ROC curve(AUC)equal to 0.896,an F1 score of 0.541,and a recall of 0.473.

Multi-prior physics-enhanced neural network enables pixel super-resolution and twin-imagefree phase retrieval from single-shot hologram

Digital in-line holographic microscopy(DIHM)is a widely used interference technique for real-time reconstruction of living cells’morphological information with large space-bandwidth product and compact setup.However,the need for a larger pixel size of detector to improve imaging photosensitivity,field-of-view,and signal-to-noise ratio often leads to the loss of sub-pixel information and limited pixel resolution.Additionally,the twin-image appearing in the reconstruction severely degrades the quality of the reconstructed image.The deep learning(DL)approach has emerged as a powerful tool for phase retrieval in DIHM,effectively addressing these challenges.However,most DL-based strategies are datadriven or end-to-end net approaches,suffering from excessive data dependency and limited generalization ability.Herein,a novel multi-prior physics-enhanced neural network with pixel super-resolution(MPPN-PSR)for phase retrieval of DIHM is proposed.It encapsulates the physical model prior,sparsity prior and deep image prior in an untrained deep neural network.The effectiveness and feasibility of MPPN-PSR are demonstrated by comparing it with other traditional and learning-based phase retrieval methods.With the capabilities of pixel super-resolution,twin-image elimination and high-throughput jointly from a single-shot intensity measurement,the proposed DIHM approach is expected to be widely adopted in biomedical workflow and industrial measurement.

Cryogenic transmission electron microscopy on beam-sensitive materials and quantum science

Transmission electron microscopy(TEM)offers unparalleled atomic-resolution imaging of complex materials and heterogeneous structures.However,high-energy imaging electrons can induce structural damage,posing a challenge for electron-beam-sensitive materials.Cryogenic TEM(Cryo-TEM)has revolutionized structural biology,enabling the visualization of biomolecules in their near-native states at unprecedented detail.The low electron dose imaging and stable cryogenic environment in Cryo-TEM are now being harnessed for the investigation of electron-beam-sensitive materials and low-temperature quantum phenomena.Here,we present a systematic review of the interaction mechanisms between imaging electrons and atomic structures,illustrating the electron beam-induced damage and the mitigating role of Cryo-TEM.This review then explores the advancements in low-dose Cryo-TEM imaging for elucidating the structures of organic-based materials.Furthermore,we showcase the application of Cryo-TEM in the study of strongly correlated quantum materials,including the detection of charge order and novel topological spin textures.Finally,we discuss the future prospects of Cryo-TEM,emphasizing its transformative potential in unraveling the complexities of materials and phenomena across diverse scientific disciplines.

Advancing respirable coal mine dust source apportionment:a preliminary laboratory exploration of optical microscopy as a novel monitoring tool

Exposure to respirable coal mine dust(RCMD)can cause chronic and debilitating lung diseases.Real-time monitoring capabilities are sought which can enable a better understanding of dust components and sources.In many underground mines,RCMD includes three primary components which can be loosely associated with three major dust sources:coal dust from the coal seam itself,silicates from the surrounding rock strata,and carbonates from the inert‘rock dust’products that are applied to mitigate explosion hazards.A monitor which can reliably partition RCMD between these three components could thus allow source apportionment.And tracking silicates,specifically,could be valuable since the most serious health risks are typically associated with this component-particularly if abundant in crystalline silica.Envisioning a monitoring concept based on field microscopy,and following up on prior research using polarized light,the aim of the current study was to build and test a model to classify respirable-sized particles as either coal,silicates,or carbonates.For model development,composite dust samples were generated in the laboratory by successively depositing dust from high-purity materials onto a sticky transparent substrate,and imaging after each deposition event such that the identity of each particle was known a priori.Model testing followed a similar approach,except that real geologic materials were used as the source for each dust component.Results showed that the model had an overall accuracy of 86.5%,indicating that a field-microscopy based moni-tor could support RCMD source apportionment and silicates tracking in some coal mines.

Cryo-EM combined with image deconvolution to determine ZIF-8 crystal structure

Metal–organic frameworks(MOFs) are crystalline porous materials with tunable properties, exhibiting great potential in gas adsorption, separation and catalysis.[1,2]It is challenging to visualize MOFs with transmission electron microscopy(TEM) due to their inherent instability under electron beam irradiation. Here, we employ cryo-electron microscopy(cryoEM) to capture images of MOF ZIF-8, revealing inverted-space structural information at a resolution of up to about 1.7A and enhancing its critical electron dose to around 20 e^(-)/A^(2). In addition, it is confirmed by electron-beam irradiation experiments that the high voltage could effectively mitigate the radiolysis, and the structure of ZIF-8 is more stable along the [100] direction under electron beam irradiation. Meanwhile, since the high-resolution electron microscope images are modulated by contrast transfer function(CTF) and it is difficult to determine the positions corresponding to the atomic columns directly from the images. We employ image deconvolution to eliminate the impact of CTF and obtain the structural images of ZIF-8. As a result, the heavy atom Zn and the organic imidazole ring within the organic framework can be distinguished from structural images.

A promising approach for quantifying focal stroke modeling and assessing stroke progression:optical resolution photoacoustic microscopy photothrombosis

To investigate the mechanisms underlying the onset and progression of ischemic stroke,some methods have been proposed that can simultaneously monitor and create embolisms in the animal cerebral cortex.However,these methods often require complex systems and the effect of age on cerebral embolism has not been adequately studied,although ischemic stroke is strongly age-related.In this study,we propose an optical-resolution photoacoustic microscopy-based visualized photothrombosis methodology to create and monitor ischemic stroke in mice simultaneously using a 532 nm pulsed laser.We observed the molding process in mice of different ages and presented age-dependent vascular embolism differentiation.Moreover,we integrated optical coherence tomography angiography to investigate age-associated trends in cerebrovascular variability following a stroke.Our imaging data and quantitative analyses underscore the differential cerebrovascular responses to stroke in mice of different ages,thereby highlighting the technique's potential for evaluating cerebrovascular health and unraveling age-related mechanisms involved in ischemic strokes.

Numerical Simulation of Super-Resolution Structured Illumination Microscopy (SIM) Using Heintzmann-Cremer Algorithm with Non-Continuous Spatial Frequency Support

We report a comprehensive numerical study of super resolution (SR) structured illumination microscopy (SIM) utilizing the classic Heintzmann-Cremer SIM process and algorithm. In particular, we investigated the impact of the diffraction limit of the underlying imaging system on the optimal SIM grating frequency that can be used to obtain the highest SR enhancement with non-continuous spatial frequency support. Besides confirming the previous theoretical and experimental work that SR-SIM can achieve an enhancement close to 3 times the diffraction limit with grating pattern illuminations, we also observe and report a series of more subtle effects of SR-SIM with non-continuous spatial frequency support. Our simulations show that when the SIM grating frequency exceeds twice that of the diffraction limit, the higher SIM grating frequency can help achieve a higher SR enhancement for the underlying imaging systems whose diffraction limit is low, though this enhancement is obtained at the cost of losing resolution at some lower resolution targets. Our simulations also show that, for underlying imaging systems with high diffraction limits, however, SR-SIM grating frequencies above twice the diffraction limits tend to bring no significant extra enhancement. Furthermore, we observed that there exists a limit grating frequency above which the SR enhancement effect is lost, and the reconstructed images essentially have the same resolution as the one obtained directly from the underlying imaging system without using the SIM process.

用TEM分析材料在实验教学中的实践

透射电子显微镜的快速发展使人类对客观世界的认识深入到了原子尺度。目前,透射电子显微镜的分辨率已经达到了亚埃级别,是材料研究领域十分重要的测试表征手段。针对材料类本科专业实验教学的特点,以分析金纳米颗粒的微观结构为实例,从理论讲解、实验观摩到结果分析与讨论三个方面开展了透射电子显微分析的实验教学工作,使学生理解透射电子显微镜的基本原理、功能和应用。

Method of lateral image reconstruction in structured illumination microscopy with super resolution

The image reconstruction process in super-resolution structured illumination microscopy(SIM)is investigated.The structured pattern is generated by the interference of two Gaussian beams to encode undetectable spectra into detectable region of microscope.After parameters estimation of the structured pattern,the encoded spectra are computationally decoded and recombined in Fourier domain to equivalently increase the cut-off frequency of microscope,resulting in the extension of detectable spectra and a reconstructed image with about two-fold enhanced resolution.Three di®erent methods to estimate the initial phase of structured pattern are compared,verifying the auto-correlation algorithm a®ords the fast,most precise and robust measurement.The artifacts sources and detailed reconstruction°owchart for both linear and nonlinear SIM are also presented.

A microvascular image analysis method for optical-resolution photoacoustic microscopy

Optical-resolution photoacoustic microscopy(OR-PAM)has been shown to be an excellent tool for high-resolution imaging of microvasculature,and quantitative analysis of the microvascula-ture can provide valuable information for the early diagnosis and treatment of various vascular-related diseases.In order to address the characteristics of weak signals,discontinuity and small diameters in photoacoustic microvascular images,we propose a method adaptive to the micro-vascular segmentation in photoacoustic images,including Hessian matrix enhancement and the morphological connection operators.The accuracy of our vascular segmentation method is quantitatively evaluated by the multiple criteria.To obtain more precise and continuous mi-crovascular skeletons,an improved skeleton extraction framework based on the multistencil fast marching(MSFM)method is developed.We carried out in vivo OR-PAM microvascular imaging in mouse ears and subcutaneous hepatoma tumor model to verify the correctness and superiority of our proposed method.Compared with the previous methods,our proposed method can extract the microvascular network more completely,continuously and accurately,and provide an ef-fective solution for the quantitative analysis of photoacoustic microvascular images with many small branches.

U-Net-based deep learning for tracking and quantitative analysis of intracellular vesicles in time-lapse microscopy images

Fluorescence microscopy has become an essential tool for biologists,to visualize the dynamics of intracellular structures with specific labeling.Quantitatively measuring the dynamics of moving objects inside the cell is pivotal for understanding of the underlying regulatory mechanism.Protein-containing vesicles are involved in various biological processes such as material transportation,organelle interaction,and hormonal regulation,whose dynamic characteristics are signi¯cant to disease diagnosis and drug screening.Although some algorithms have been developed for vesicle tracking,most of them have limited performance when dealing with images with low resolution,poor signal-to-noise ratio(SNR)and complicated motion.Here,we proposed a novel deep learning-based method for intracellular vesicle tracking.We trained the U-Net for vesicle localization and motion classification,with demonstrates great performance in both simulated datasets and real biological samples.By combination with fan-shaped tracker(FsT)we have previously developed,this hybrid new algorithm significantly improved the performance of particle tracking with the function of subsequently automated vesicle motion classification.Furthermore,its performance was further demonstrated in analyzing with vesicle dynamics in different temperature,which achieved reasonable outcomes.Thus,we anticipate that this novel method would have vast applications in analyzing the vesicle dynamics in living cells.

High-speed all-optic optical coherence tomography and photoacoustic microscopy dual-modal system for microcirculation evaluation

We propose a high-speed all-optic dual-modal system that integrates spectral domain optical coherence tomography and photoacoustic microscopy(PAM).A 3*3 coupler-based interfer-ometer is used to remotely detect the surface vibration caused by photoacoustic(PA)waves.Three outputs of the interferometer are acquired simultaneously with a multi-channel data ac-quisition card.One channel data with the highest PA signal detection sensitivity is selected for sensitivity compensation.Experiment on the phantom demonstrates that the proposed method can sucessfully compensate for the loss of intensity caused by sensitivity variation.The imaging speed of the PAM is improved compared to our previous system.The total time to image a sample with 256×256 pixels is~20s.Using the proposed system,the microvasculature in the mouse auricle is visualized and the blood flow state is accessed.

A mathematical morphological approach for region of interest coding of microscopy image compression

A novel mathematical morphological approach for region of interest(ROI) automatic determination and JPEG2000-based coding of microscopy image compression is presented.The algorithm is very fast and requires lower computing power,which is particularly suitable for some irregular region-based cell microscopy images with poor qualities.Firstly,an active threshold-based method is discussed to create a rough mask of regions of interest(cells).And then some morphological operations are designed and applied to achieve the segmentation of cells.In addition,an extra morphological operation,dilation,is applied to create the final mask with some redundancies to avoid the"edge effect"after removing false cells.Finally,ROI and region of background(ROB) are obtained and encoded individually in different compression ratio flexibly based on the JPEG2000,which can adjust the quality between ROI and ROB without coding for ROI shape.The experimental results certify the effectiveness of the proposed algorithm,and compared with JPEG2000,the proposed algorithm has better performance in both subjective quality and objective quality at the same compression ratios.

SDOCT IMAGE RECONSTRUCTION BY INTERFEROMETRIC SYNTHETIC APERTURE MICROSCOPY

Spectral domain optical coherence tomography(SDOCT)is a noninvasive,cross-sectional imaging technique that measures depth resolved reflectance of tissue by Fourier transforming the spectral interferogram with the scanning of the reference avoided.Interferometric synthetic aperture microscopy(ISAM)is an optical microscopy computed-imaging technique for measuring the optical properties of biological tissues,which can overcome the compromise between depth of focus and transverse resolution.This paper describes the principle of SDOCT and ISAM,which multiplexes raw acquisitions to provide quantitatively meaningful data with reliable spatially invariant resolution at all depths.A mathematical model for a coherent microscope with a planar scanning geometry and spectral detection was described.The two-dimensional fast Fourier transform(FFT)of spectral data in the transverse directions was calculated.Then the nonuniform ISAM resampling and filtering was implemented to yield the scattering potential within the scalar model.Inverse FFT was used to obtain the ISAM reconstruction.One scatterer,multiple scatterers,and noisy simulations were implemented by use of ISAM to catch spatially invariant resolution.ISAM images were compared to those obtained using standard optical coherence tomography(OCT)methods.The high quality of the results validates the rationality of the founded model and that diffraction limited resolution can be achieved outside the focal plane.

Stimulated Raman scattering microscopy on biological cellular machinery

Beneting from the developments of advanced optical microscopy techniques,the mysteries of biological functions at the cellular and subcellular levels have been continuously revealed.Stimulated Raman scattering(SRS)microscopy is a rapidly growing technique that has attracted broad attentions and become a powerful tool for biology and biomedicine,largely thanks to its chemical specicity,high sensitivity and fast image speed.This review paper introduces the principles of SRS,discusses the technical developments and implementations of SRS microscopy,then highlights and summarizes its applications on biological cellular machinery andnally shares our visions of potential breakthroughs in the future.

Deep-learning-based methods for super-resolution fluorescence microscopy

The algorithm used for reconstruction or resolution enhancement is one of the factors affectingthe quality of super-resolution images obtained by fluorescence microscopy.Deep-learning-basedalgorithms have achieved stateof-the-art performance in super-resolution fluorescence micros-copy and are becoming increasingly attractive.We firstly introduce commonly-used deep learningmodels,and then review the latest applications in terms of the net work architectures,the trainingdata and the loss functions.Additionally,we discuss the challenges and limits when using deeplearning to analyze the fluorescence microscopic data,and suggest ways to improve the reliability and robustness of deep learning applications.

Detection of Omicron Caused Pneumonia from Radiology Images Using Convolution Neural Network(CNN)

COVID-19 disease caused by the SARS-CoV-2 virus has created social and economic disruption across theworld.The ability of the COVID-19 virus to quickly mutate and transfer has created serious concerns across the world.It is essential to detectCOVID-19 infection caused by different variants to take preventive measures accordingly.The existing method of detection of infections caused by COVID-19 and its variants is costly and time-consuming.The impacts of theCOVID-19 pandemic in developing countries are very drastic due to the unavailability of medical facilities and infrastructure to handle the pandemic.Pneumonia is the major symptom of COVID-19 infection.The radiology of the lungs in varies in the case of bacterial pneumonia as compared to COVID-19-caused pneumonia.The pattern of pneumonia in lungs in radiology images can also be used to identify the cause associated with pneumonia.In this paper,we propose the methodology of identifying the cause(either due to COVID-19 or other types of infections)of pneumonia from radiology images.Furthermore,because different variants of COVID-19 lead to different patterns of pneumonia,the proposed methodology identifies pneumonia,the COVID-19 caused pneumonia,and Omicron caused pneumonia from the radiology images.To fulfill the above-mentioned tasks,we have used three Convolution Neural Networks(CNNs)at each stage of the proposed methodology.The results unveil that the proposed step-by-step solution enhances the accuracy of pneumonia detection along with finding its cause,despite having a limited dataset.