Estimation-free spatial-domain image reconstruction of structured illumination microscopy

Structured illumination microscopy(SIM)achieves super-resolution(SR)by modulating the high-frequency information of the sample into the passband of the optical system and subsequent image reconstruction.The traditional Wiener-filtering-based reconstruction algorithm operates in the Fourier domain,it requires prior knowledge of the sinusoidal illumination patterns which makes the time-consuming procedure of parameter estimation to raw datasets necessary,besides,the parameter estimation is sensitive to noise or aberration-induced pattern distortion which leads to reconstruction artifacts.Here,we propose a spatial-domain image reconstruction method that does not require parameter estimation but calculates patterns from raw datasets,and a reconstructed image can be obtained just by calculating the spatial covariance of differential calculated patterns and differential filtered datasets(the notch filtering operation is performed to the raw datasets for attenuating and compensating the optical transfer function(OTF)).Experiments on reconstructing raw datasets including nonbiological,biological,and simulated samples demonstrate that our method has SR capability,high reconstruction speed,and high robustness to aberration and noise.

Numerical Simulation of Super-Resolution Structured Illumination Microscopy (SIM) Using Heintzmann-Cremer Algorithm with Non-Continuous Spatial Frequency Support

We report a comprehensive numerical study of super resolution (SR) structured illumination microscopy (SIM) utilizing the classic Heintzmann-Cremer SIM process and algorithm. In particular, we investigated the impact of the diffraction limit of the underlying imaging system on the optimal SIM grating frequency that can be used to obtain the highest SR enhancement with non-continuous spatial frequency support. Besides confirming the previous theoretical and experimental work that SR-SIM can achieve an enhancement close to 3 times the diffraction limit with grating pattern illuminations, we also observe and report a series of more subtle effects of SR-SIM with non-continuous spatial frequency support. Our simulations show that when the SIM grating frequency exceeds twice that of the diffraction limit, the higher SIM grating frequency can help achieve a higher SR enhancement for the underlying imaging systems whose diffraction limit is low, though this enhancement is obtained at the cost of losing resolution at some lower resolution targets. Our simulations also show that, for underlying imaging systems with high diffraction limits, however, SR-SIM grating frequencies above twice the diffraction limits tend to bring no significant extra enhancement. Furthermore, we observed that there exists a limit grating frequency above which the SR enhancement effect is lost, and the reconstructed images essentially have the same resolution as the one obtained directly from the underlying imaging system without using the SIM process.

Comparison of point detection and area detection for point-scanning structured illumination microscopy

Structured illumination microscopy(SIM)is suitable for biological samples because of its relatively low-peak illumination intensity requirement and high imaging speed.The system resolution is affected by two typical detection modes:Point detection and area detection.However,a systematic analysis of the imaging performance of the different detection modes of the system has rarely been conducted.In this study,we compared laser point scanning point detection(PS-PD)and point scanning area detection(PS-AD)imaging in nonconfocal microscopy through theoretical analysis and simulated imaging.The results revealed that the imaging resolutions of PSPD and PS-AD depend on excitation and emission point spread functions(PSFs),respectively.Especially,we combined the second harmonic generation(SHG)of point detection(P-SHG)and area detection(A-SHG)with SIM to realize a nonlinear SIM-imaging technique that improves the imaging resolution.Moreover,we analytically and experimentally compared the nonlinear SIM performance of P-SHG with that of A-SHG.

Nonlinear scanning structured illumination microscopy based on nonsinusoidal modulation

Structured illumination microscopy(SIM)is an essential super-resolution microscopy technique that enhances resolution.Several images are required to reconstruct a super-resolution image.However,linear SIM resolution enhancement can only increase the spatial resolution of micros-copy by a factor of two at most because the frequency of the structured illumination pattern is limited by the cutoff frequency of the excitation point spread function.The frequency of the pattern generated by the nonlinear response in samples is not limited;therefore,nonlinear SIM(NL-SIM),in theory,has no inherent limit to the resolution.In the present study,we describe a two-photon nonlinear SIM(2P-SIM)technique using a multiple harmonics scanning pattern that employs a composite structured illumination pattern,which can produce a higher order harmonic pattern based on the fluorescence nonlinear response in a 2P process.The theoretical models of super-resolution imaging were established through our simulation,which describes the working mechanism of the multi-frequency structure of the nonsinusoidal function to improve the reso-lution.The simulation results predict that a 5-fold improvement in resolution in the 2P-SIM is possible.

Structured illumination microscopy and its new developments

Optical microscopy allows us to observe the biological structures and processes within living cells.However,the spatial resolution of the optical microscopy is limited to about half of the wavelength by the light di®raction.Structured illumination microscopy(SIM),a type of new emerging super-resolution microscopy,doubles the spatial resolution by illuminating the specimen with a patterned light,and the sample and light source requirements of SIM are not as strict as the other super-resolution microscopy.In addition,SIM is easier to combine with the other imaging techniques to improve their imaging resolution,leading to the developments of diverse types of SIM.SIM has great potential to meet the various requirements of living cells imaging.Here,we review the recent developments of SIM and its combination with other imaging techniques.

Improvement in Resolution of Multiphoton Scanning Structured Illumination Microscopy via Harmonics

We describe a multiphoton(mP)-structured illumination microscopy(SIM)technique,which demonstrates substantial improvement in image resolution compared with linear SIM due to the nonlinear response of fluorescence.This nonlinear response is caused by the effect of nonsinusoidal structured illumination created by scanning a sinusoidally modulated illumination to excite an mP fluorescence signal.The harmonics of the structured fluorescence illumination are utilised to improve resolution.We present an mP-SIM theory for reconstructing the super-resolution image of the system.Theoretically,the resolution of our m P-SIM is unlimited if all the high-order harmonics of the nonlinear response of fluorescence are considered.Experimentally,we demonstrate an 86 nm lateral resolution for two-photon(2P)-SIM and a 72 nm lateral resolution for second-harmonic-generation(SHG)-SIM.We further demonstrate their application by imaging cells stained with F-actin and collagen fibres in mouse-tail tendon.Our method can be directly used in commercial mP microscopes and requires no specific fluorophores or high-intensity laser.

Rapid bacteria identification using structured illumination microscopy and machine learning

Traditionally,optical microscopy is used to visualize the morphological features of pathogenic bacteria,of which the features are further used for the detection and ident ification of the bacteria.However,due to the resolution limitation of conventional optical microscopy as well as the lack of standard pattern library for bacteria identification,the ffectiveness of this optical microscopy-based method is limited.Here,we reported a pilot study on a combined use of Structured Illumination Microscopy(SIM)with machine learning for rapid bacteria identification.After applying machine learning to the SIM image datasets from three model bacteria(including Escherichia coli,Mycobacterium smegmatis,and Pseudomonas aeruginosa),we obtained a classifcation accuracy of up to 98%.This study points out a promising possibility for rapid bacterial identification by morphological features.

Graphene/Rh(111) Structure Studied Using In-Situ Scanning Tunneling Microscopy

Scanning tunnel microscopy （STM） is performed to verify if an Rh ＇nails＇ structure is formed accompanying the graphene growing during chemical vapor deposition. A structure of a graphene island in an Rh vacancy island is used as the start. While the graphene island is removed by oxygenation, the variations of the Rh vacancy island are imaged with an in-situ high-temperature STM. By fitting with our model and calculations, we conclude that the best fit is obtained for 0% Rh, i.e., for the complete absence of nails below graphene on Rh（111）. That is, when graphene is formed on Rh（111）, the substrate remains fiat and does not develop a SUPPorting nail structure.

TRANSMITTED ELECTRON MICROSCOPY STUDY OF ION-CONTAINING BIOMEDICAL SEGMENTED POLYETHERURETHANE FINE STRUCTURES

The microphase separation extent of biomedical segmented polyetherurethanes were greatly enhanced due to the presence of sulfoalkyl pendant groups contained in the hard segments,and the hard segments were more orderly aggregated through ionic interaction.

Stimulated emission-depletion-based point-scanning structured illumination microscopy

Wide-field linear structured illumination microscopy(LSIM)extends resolution beyond the diffraction limit by moving unresolvable high-frequency information into the passband of the microscopy in the form of moiréfringes.However,due to the diffraction limit,the spatial frequency of the structured illumination pattern cannot be larger than the microscopy cutoff frequency,which results in a twofold resolution improvement over wide-field microscopes.This Letter presents a novel approach in point-scanning LSIM,aimed at achieving higher-resolution improvement by combining stimulated emission depletion(STED)with point-scanning structured illumination microscopy(ps SIM)(STED-ps SIM).The according structured illumination pattern whose frequency exceeds the microscopy cutoff frequency is produced by scanning the focus of the sinusoidally modulated excitation beam of STED microscopy.The experimental results showed a 1.58-fold resolution improvement over conventional STED microscopy with the same depletion laser power.

Bacterial entombment by intratubular mineralization following orthograde mineral trioxide aggregate obturation: a scanning electron microscopy study

The time domain entombment of bacteria by intratubular mineralization following orthograde canal obturation with mineral trioxide aggregate（MTA） was studied by scanning electron microscopy（SEM）. Single-rooted human premolars（n560） were instrumented to an apical size #50/0.06 using ProF ile and treated as follows： Group 1（n510） was filled with phosphate buffered saline（PBS）; Group 2（n510） was incubated with Enterococcus faecalis for 3 weeks, and then filled with PBS; Group 3（n520） was obturated orthograde with a paste of OrthoM TA（BioM TA, Seoul, Korea） and PBS; and Group 4（n520） was incubated with E. faecalis for 3 weeks and then obturated with OrthoM TA–PBS paste. Following their treatments, the coronal openings were sealed with PBS-soaked cotton and intermediate restorative material（IRM）, and the roots were then stored in PBS for 1, 2, 4, 8 or 16 weeks. After each incubation period, the roots were split and their dentin/MTA interfaces examined in both longitudinal and horizontal directions by SEM. There appeared to be an increase in intratubular mineralization over time in the OrthoM TA-filled roots（Groups 3 and 4）. Furthermore, there was a gradual entombment of bacteria within the dentinal tubules in the E. faecalis inoculated MTA-filled roots（Group 4）. Therefore, the orthograde obturation of root canals with OrthoM TA mixed with PBS may create a favorable environment for bacterial entombment by intratubular mineralization.

Application of super-resolution fluorescence microscopy in hematologic malignancies

Hematologic malignancies are one of the most common malignant tumors caused by the clonal proliferation and differentiation of hematopoietic and lymphoid stem cells.The examination of bone marrow cells combined with immunodeficiency typing is of great significance to the diagnostic type,treatment and prognosis of hematologic malignancies.Super-resolution fluorescence microscopy(SRM)is a special kind of optical microscopy technology,which breaks the resolution limit and was awarded the Nobel Prize in Chemistry in 2014.With the development of SRM,many related technologies have been applied to the diagnosis and treatment of clinical diseases.It was reported that a major type of SRM technique,single molecule localization microscopy(SMLM),is more sensitive than flow cytometry(FC)in detecting cell membrane antigens'expression,thus enabling better chances in detecting antigens on hematopoietic cells than traditional analytic tools.Furthermore,SRM may be applied to clinical pathology and may guide precision medicine and personalized medicine for clone hematopoietic cell diseases.In this paper,we mainly discuss the application of SRM in clone hematological malignancies.

Topography Structure and Scanning Tunneling Spectrum of Nickel( Ⅱ )-tetraphenylporphyrin Molecules on Au(111)

Scanning tunneling microscopy （STM） and scanning tunneling spectroscopy （STS） were performed on monolayer film of NiTPP supported on Au（111） under ultrahigh vacuum （UHV） conditions. The constant current STM images show remarkable bias dependence. High resolution STM data clearly show the individual NiTPP molecules and allow easy differentiation between NiTPP and CoTPP reported before. Scanning tunneling spectra, as a function of molecule-tip separation, were acquired over a range of tip motion of 0.42 nm. Spectra do not show the variation in band splitting with tip distance. It appears for molecules such as NiTPP that the average potential at the molecule is essentially the same at the same metal substrate. For molecules of the height of NiTPP, the scanning tunneling spectra should give reliable occupied and unoccupied orbital energies over a wide range of tip-molecule distances.

High Resolution Electron Microscopy Observations of Structural Changes in Iron Nitride Films Annealed in Vacuum

Iron-nitride films were prepared by reactive sputtering, and the effect of annealing treatment on the structures was investigated by means of in-situ electron microscopy and high resolution electron microscopy (HREM). As-deposited films were observed to be a mixed structure of a few ultrafine epsilon-Fe2-3N particles existing in the amorphous matrix. it was found that the structure-relaxation in the amorphous occurred at 473 K, and the ultrafine grains began to grow at the higher annealing temperatures. The transition of the amorphous to epsilon-Fe2-3N was almost completed at 673 K. It is considered that the formation of the ideal epsilon-Fe3N is originated from the ordering of the nitrogen atoms during the annealing in vacuum. On the other hand, gamma'-phase (Fe4N) was seen to precipitation of epsilon-phase at 723 K. Two possible modes are proposed in the precipitation of gamma'-phase, depending on the heating rate and crystallographic orientation relationships, i.e. [121](epsilon)//[001](gamma), (2(1) over bar0$)(epsilon)//(110)(gamma) and [100](epsilon)//[110](gamma), (001)(epsilon)//(111)(gamma). In addition, alpha-Fe particles were observed to form from the gamma'-phase at high temperatures. We assumed that these structural changes are due to the diffusion of nitrogen and iron atoms during the annealing, except for the case of the precipitation of the gamma'-phase as depicted above. The results obtained in this work are in a good agreement with the assumption.

Super-resolution microscopy and its applications in neuroscience

Optical microscopy promises researchers to soe most tiny substances directly.However,the resolution of conventional microscopy is resticted by the diffraction limit.This makes it a challenge to observe subcellular processes happened in nanoscale.The development of super-resolution microscopy provides a solution to this challenge.Here,we briefly review several commonly used super-resolution techniques,explicating their basic principles and applications in biological science,especially in neuroscience.In addition,characteristics and limitations of each techrique are compared to provide a guidance for biologists to choose the most suitable tool.

DEEP-UV CONFOCAL FLUORESCENCE IMAGING AND SUPER-RESOLUTION OPTICAL MICROSCOPY OF BIOLOGICAL SAMPLES

A wide range of techniques has been developed to image biological samples at high spatial and temporal resolution.In this paper,we report recent results from deep-UV confocal fAuorescence microscopy to image inherent emission from fuorophores such as tryptophan,and structured ilumination microscopy(SIM)of biological materials.One motivation for developing deep-UV fhuorescence imaging and SIM is to provide methods to complement our measurements in the emerging field of X-ray coherent diffractive imaging.

Fluorescence interference structured illumination microscopy for 3D morphology imaging with high axial resolution

Imaging three-dimensional,subcellular structures with high axial resolution has always been the core purpose of fluorescence microscopy.However,trade-offs exist between axial resolution and other important technical indicators,such as temporal resolution,optical power density,and imaging process complexity.We report a new imaging modality,fluorescence interference structured illumination microscopy(FI-SIM),which is based on three-dimensional structured illumination microscopy for wide-field lateral imaging and fluorescence interference for axial reconstruction.FI-SIM can acquire images quickly within the order of hundreds of milliseconds and exhibit even 30 nm axial resolution in half the wavelength depth range without z-axis scanning.Moreover,the relatively low laser power density relaxes the requirements for dyes and enables a wide range of applications for observing fixed and live subcellular structures.

Structured illumination microscopy based on principal component analysis

Structured illumination microscopy(SIM)is one of the powerful super-resolution modalities in bioscience with the advantages of full-field imaging and high photon efficiency.However,artifact-free super-resolution image reconstruction requires precise knowledge about the illumination parameters.The sample-and environment-dependent on-the-fly experimental parameters need to be retrieved a posteriori from the acquired data,posing a major challenge for real-time,long-term live-cell imaging,where low photobleaching,phototoxicity,and light dose are a must.In this work,we present an efficient and robust SIM algorithm based on principal component analysis(PCA-SIM).PCA-SIM is based on the observation that the ideal phasor matrix of a SIM pattern is of rank one,leading to the low complexity,precise identification of noninteger pixel wave vector and pattern phase while rejecting components that are unrelated to the parameter estimation.We demonstrate that PCA-SIM achieves non-iteratively fast,accurate(below 0.01-pixel wave vector and 0.1%of 2relative phase under typical noise level),and robust parameter estimation at low SNRs,which allows real-time super-resolution imaging of live cells in complicated experimental scenarios where other state-of-the-art methods inevitably fail.In particular,we provide the open-source MATLAB toolbox of our PCA-SIM algorithm and associated datasets.The combination of iteration-free reconstruction,robustness to noise,and limited computational complexity makes PCA-SIM a promising method for high-speed,long-term,artifact-free super-resolution imaging of live cells.

High-speed image reconstruction for optically sectioned,super-resolution structured illumination microscopy

Super-resolution structured illumination microscopy(SR-SIM)is an outstanding method for visualizing the subcellular dynamics in living cells.To date,by using elaborately designed systems and algorithms,SR-SIM can achieve rapid,optically sectioned,SR observation with hundreds to thousands of time points.However,real-time observation is still out of reach for most SIM setups as conventional algorithms for image reconstruction involve a heavy computing burden.To address this limitation,an accelerated reconstruction algorithm was developed by implementing a simplified workflow for SR-SIM,termed joint space and frequency reconstruction.This algorithm results in an 80-fold improvement in reconstruction speed relative to the widely used Wiener-SIM.Critically,the increased processing speed does not come at the expense of spatial resolution or sectioning capability,as demonstrated by live imaging of microtubule dynamics and mitochondrial tubulation.

Woofer–tweeter adaptive optical structured illumination microscopy

A woofer–tweeter adaptive optical structured illumination microscope(AOSIM) is presented. By combining a low-spatial-frequency large-stroke deformable mirror(woofer) with a high-spatial-frequency low-stroke deformable mirror(tweeter), we are able to remove both large-amplitude and high-order aberrations. In addition, using the structured illumination method, as compared to widefield microscopy, the AOSIM can accomplish highresolution imaging and possesses better sectioning capability. The AOSIM was tested by correcting a large aberration from a trial lens in the conjugate plane of the microscope objective aperture. The experimental results show that the AOSIM has a point spread function with an FWHM that is 140 nm wide(using a water immersion objective lens with NA=1.1) after correcting a large aberration(5.9 μm peak-to-valley wavefront error with 2.05 μm RMS aberration). After structured light illumination is applied, the results show that we are able to resolve two beads that are separated by 145 nm, 1.62× below the diffraction limit of 235 nm. Furthermore, we demonstrate the application of the AOSIM in the field of bioimaging. The sample under investigation was a green-fluorescentprotein-labeled Drosophila embryo. The aberrations from the refractive index mismatch between the microscope objective, the immersion fluid, the cover slip, and the sample itself are well corrected. Using AOSIM we were able to increase the SNR for our Drosophila embryo sample by 5×.