Estimation-free spatial-domain image reconstruction of structured illumination microscopy

Structured illumination microscopy(SIM)achieves super-resolution(SR)by modulating the high-frequency information of the sample into the passband of the optical system and subsequent image reconstruction.The traditional Wiener-filtering-based reconstruction algorithm operates in the Fourier domain,it requires prior knowledge of the sinusoidal illumination patterns which makes the time-consuming procedure of parameter estimation to raw datasets necessary,besides,the parameter estimation is sensitive to noise or aberration-induced pattern distortion which leads to reconstruction artifacts.Here,we propose a spatial-domain image reconstruction method that does not require parameter estimation but calculates patterns from raw datasets,and a reconstructed image can be obtained just by calculating the spatial covariance of differential calculated patterns and differential filtered datasets(the notch filtering operation is performed to the raw datasets for attenuating and compensating the optical transfer function(OTF)).Experiments on reconstructing raw datasets including nonbiological,biological,and simulated samples demonstrate that our method has SR capability,high reconstruction speed,and high robustness to aberration and noise.

Numerical Simulation of Super-Resolution Structured Illumination Microscopy (SIM) Using Heintzmann-Cremer Algorithm with Non-Continuous Spatial Frequency Support

We report a comprehensive numerical study of super resolution (SR) structured illumination microscopy (SIM) utilizing the classic Heintzmann-Cremer SIM process and algorithm. In particular, we investigated the impact of the diffraction limit of the underlying imaging system on the optimal SIM grating frequency that can be used to obtain the highest SR enhancement with non-continuous spatial frequency support. Besides confirming the previous theoretical and experimental work that SR-SIM can achieve an enhancement close to 3 times the diffraction limit with grating pattern illuminations, we also observe and report a series of more subtle effects of SR-SIM with non-continuous spatial frequency support. Our simulations show that when the SIM grating frequency exceeds twice that of the diffraction limit, the higher SIM grating frequency can help achieve a higher SR enhancement for the underlying imaging systems whose diffraction limit is low, though this enhancement is obtained at the cost of losing resolution at some lower resolution targets. Our simulations also show that, for underlying imaging systems with high diffraction limits, however, SR-SIM grating frequencies above twice the diffraction limits tend to bring no significant extra enhancement. Furthermore, we observed that there exists a limit grating frequency above which the SR enhancement effect is lost, and the reconstructed images essentially have the same resolution as the one obtained directly from the underlying imaging system without using the SIM process.

Speckle structured illumination endoscopy with enhanced resolution at wide field of view and depth of field

Structured illumination microscopy(SIM)is one of the most widely applied wide field super resolution imaging techniques with high temporal resolution and low phototoxicity.The spatial resolution of SIM is typically limited to two times of the diffraction limit and the depth of field is small.In this work,we propose and experimentally demonstrate a low cost,easy to implement,novel technique called speckle structured illumination endoscopy(SSIE)to enhance the resolution of a wide field endoscope with large depth of field.Here,speckle patterns are used to excite objects on the sample which is then followed by a blind-SIM algorithm for super resolution image reconstruction.Our approach is insensitive to the 3D morphology of the specimen,or the deformation of illuminations used.It greatly simplifies the experimental setup as there are no calibration protocols and no stringent control of illumination patterns nor focusing optics.We demonstrate that the SSIE can enhance the resolution 2–4.5 times that of a standard white light endoscopic(WLE)system.The SSIE presents a unique route to super resolution in endoscopic imaging at wide field of view and depth of field,which might be beneficial to the practice of clinical endoscopy.

Complex-amplitude Fourier single-pixel imaging via coherent structured illumination

We propose a method of complex-amplitude Fourier single-pixel imaging(CFSI)with coherent structured illumination to acquire both the amplitude and phase of an object.In the proposed method,an object is illustrated by a series of coherent structured light fields,which are generated by a phase-only spatial light modulator,the complex Fourier spectrum of the object can be acquired sequentially by a single-pixel photodetector.Then the desired complex-amplitude image can be retrieved directly by applying an inverse Fourier transform.We experimentally implemented this CFSI with several different types of objects.The experimental results show that the proposed method provides a promising complex-amplitude imaging approach with high quality and a stable configuration.Thus,it might find broad applications in optical metrology and biomedical science.

Comparison of point detection and area detection for point-scanning structured illumination microscopy

Structured illumination microscopy(SIM)is suitable for biological samples because of its relatively low-peak illumination intensity requirement and high imaging speed.The system resolution is affected by two typical detection modes:Point detection and area detection.However,a systematic analysis of the imaging performance of the different detection modes of the system has rarely been conducted.In this study,we compared laser point scanning point detection(PS-PD)and point scanning area detection(PS-AD)imaging in nonconfocal microscopy through theoretical analysis and simulated imaging.The results revealed that the imaging resolutions of PSPD and PS-AD depend on excitation and emission point spread functions(PSFs),respectively.Especially,we combined the second harmonic generation(SHG)of point detection(P-SHG)and area detection(A-SHG)with SIM to realize a nonlinear SIM-imaging technique that improves the imaging resolution.Moreover,we analytically and experimentally compared the nonlinear SIM performance of P-SHG with that of A-SHG.

Improvement in Resolution of Multiphoton Scanning Structured Illumination Microscopy via Harmonics

We describe a multiphoton(mP)-structured illumination microscopy(SIM)technique,which demonstrates substantial improvement in image resolution compared with linear SIM due to the nonlinear response of fluorescence.This nonlinear response is caused by the effect of nonsinusoidal structured illumination created by scanning a sinusoidally modulated illumination to excite an mP fluorescence signal.The harmonics of the structured fluorescence illumination are utilised to improve resolution.We present an mP-SIM theory for reconstructing the super-resolution image of the system.Theoretically,the resolution of our m P-SIM is unlimited if all the high-order harmonics of the nonlinear response of fluorescence are considered.Experimentally,we demonstrate an 86 nm lateral resolution for two-photon(2P)-SIM and a 72 nm lateral resolution for second-harmonic-generation(SHG)-SIM.We further demonstrate their application by imaging cells stained with F-actin and collagen fibres in mouse-tail tendon.Our method can be directly used in commercial mP microscopes and requires no specific fluorophores or high-intensity laser.

Nonlinear scanning structured illumination microscopy based on nonsinusoidal modulation

Structured illumination microscopy(SIM)is an essential super-resolution microscopy technique that enhances resolution.Several images are required to reconstruct a super-resolution image.However,linear SIM resolution enhancement can only increase the spatial resolution of micros-copy by a factor of two at most because the frequency of the structured illumination pattern is limited by the cutoff frequency of the excitation point spread function.The frequency of the pattern generated by the nonlinear response in samples is not limited;therefore,nonlinear SIM(NL-SIM),in theory,has no inherent limit to the resolution.In the present study,we describe a two-photon nonlinear SIM(2P-SIM)technique using a multiple harmonics scanning pattern that employs a composite structured illumination pattern,which can produce a higher order harmonic pattern based on the fluorescence nonlinear response in a 2P process.The theoretical models of super-resolution imaging were established through our simulation,which describes the working mechanism of the multi-frequency structure of the nonsinusoidal function to improve the reso-lution.The simulation results predict that a 5-fold improvement in resolution in the 2P-SIM is possible.

Structured illumination microscopy based on asymmetric three-beam interference

Structured illumination microscopy(SIM)is a rapidly developing super-resolution technology.It has been widely used in various application fields of biomedicine due to its excellent two-and three-dimensional imaging capabilities.Furthermore,faster three-dimensional imaging methods are required to help enable more research-oriented living cell imaging.In this paper,a fast and sensitive three-dimensional structured illumination microscopy based on asymmetric three-beam interference is proposed.An innovative time-series acquisition method is employed to halve the time required to obtain each raw image.A segmented half-wave plate as a substantial linear polarization modulation method is applied to the three-dimensional SIM system for the first time.Although it needs to acquire 21 raw images instead of 15 to reconstruct one super-resolution image,the SIM setup proposed in this paper is 30%faster than the traditional spatial light modulator-SIM(SLM-SIM)in imaging each super-resolution image.The related theoretical derivation,hardware system,and verification experiment are elaborated in this paper.The stable and fast 3D super-resolution imaging method proposed in this paper is of great significance to the research of organelle interaction,intercellular communication,and other biomedical fields.

Structured illumination microscopy and its new developments

Optical microscopy allows us to observe the biological structures and processes within living cells.However,the spatial resolution of the optical microscopy is limited to about half of the wavelength by the light di®raction.Structured illumination microscopy(SIM),a type of new emerging super-resolution microscopy,doubles the spatial resolution by illuminating the specimen with a patterned light,and the sample and light source requirements of SIM are not as strict as the other super-resolution microscopy.In addition,SIM is easier to combine with the other imaging techniques to improve their imaging resolution,leading to the developments of diverse types of SIM.SIM has great potential to meet the various requirements of living cells imaging.Here,we review the recent developments of SIM and its combination with other imaging techniques.

Structured Illumination Chip Based on Integrated Optics

A compact structured illumination chip based on integrated optics is proposed and fabricated on a silicon-on- insulator platform. Based on the simulation of Caussian beam interference, we adopt a chirped diffraction grating to achieve a specific interference pattern. The experimental results match well with the simulations. The portability and flexibility of the structured illumination chip can be increased greatly through horizontal encapsulation. High levels of integration, compared with the conventional structured illumination approach, make this chip very compact, with a footprint of only around 1 mm2. The chip has no optical lenses and can be easily combined with a microfluidic system. These properties would make the chip very suitable for portable 3D scanner and compact super-resolution microscopy applications.

Fringe optimization for structured illumination super-resolution microscope with digital micromirror device

Structured illumination microscopy(SIM)is a promising super-resolution technique for imaging subcellular structures and dynamics due to its compatibility with most commonly usedffuorescent labeling methods.Structured illumination can be obtained by either laser interference or projection of fringe patterns.Here,we proposed a fringe projector composed of a compact multiwavelength LEDs module and a digital micromirror device(DMD)which can be directly attached to most commercial invertedffuorescent microscopes and update it into a SIM system.The effects of the period and duty cycle of fringe patterns on the modulation depth of the structured lightfield were studied.With the optimized fringe pattern,1:6×resolution improvement could be obtained with high-end oil objectives.Multicolor imaging and dynamics of subcellular organelles in live cells were also demonstrated.Our method provides a low-cost solution for SIM setup to expand its wide range of applications to most research labs in thefield of life science and medicine.

A lateral super-resolution imaging method using structured illumination without phase shift

Structured illumination microscopy has been a useful method for achieving lateral super-resolution,but it typically requires at least three precise phase shifts per orientation.In this paper,we propose a super-resolution method that utilizes structured illumination without phase shift.The reconstruction process requires only a conventionally illuminated image and an image with structured illumination.This method achieves the same effect as the traditional phase shift method,and more than doubles the resolution by synthesizing a few reconstructions at different illumination frequencies.We verify the resolution improvement process using a combination of theoretical derivations and diagrams,and demonstrate its effectiveness with numerical simulations.

Method of lateral image reconstruction in structured illumination microscopy with super resolution

The image reconstruction process in super-resolution structured illumination microscopy(SIM)is investigated.The structured pattern is generated by the interference of two Gaussian beams to encode undetectable spectra into detectable region of microscope.After parameters estimation of the structured pattern,the encoded spectra are computationally decoded and recombined in Fourier domain to equivalently increase the cut-off frequency of microscope,resulting in the extension of detectable spectra and a reconstructed image with about two-fold enhanced resolution.Three di®erent methods to estimate the initial phase of structured pattern are compared,verifying the auto-correlation algorithm a®ords the fast,most precise and robust measurement.The artifacts sources and detailed reconstruction°owchart for both linear and nonlinear SIM are also presented.

Stimulated emission-depletion-based point-scanning structured illumination microscopy

Wide-field linear structured illumination microscopy(LSIM)extends resolution beyond the diffraction limit by moving unresolvable high-frequency information into the passband of the microscopy in the form of moiréfringes.However,due to the diffraction limit,the spatial frequency of the structured illumination pattern cannot be larger than the microscopy cutoff frequency,which results in a twofold resolution improvement over wide-field microscopes.This Letter presents a novel approach in point-scanning LSIM,aimed at achieving higher-resolution improvement by combining stimulated emission depletion(STED)with point-scanning structured illumination microscopy(ps SIM)(STED-ps SIM).The according structured illumination pattern whose frequency exceeds the microscopy cutoff frequency is produced by scanning the focus of the sinusoidally modulated excitation beam of STED microscopy.The experimental results showed a 1.58-fold resolution improvement over conventional STED microscopy with the same depletion laser power.

Structured illumination microscopy based on principal component analysis

Structured illumination microscopy(SIM)is one of the powerful super-resolution modalities in bioscience with the advantages of full-field imaging and high photon efficiency.However,artifact-free super-resolution image reconstruction requires precise knowledge about the illumination parameters.The sample-and environment-dependent on-the-fly experimental parameters need to be retrieved a posteriori from the acquired data,posing a major challenge for real-time,long-term live-cell imaging,where low photobleaching,phototoxicity,and light dose are a must.In this work,we present an efficient and robust SIM algorithm based on principal component analysis(PCA-SIM).PCA-SIM is based on the observation that the ideal phasor matrix of a SIM pattern is of rank one,leading to the low complexity,precise identification of noninteger pixel wave vector and pattern phase while rejecting components that are unrelated to the parameter estimation.We demonstrate that PCA-SIM achieves non-iteratively fast,accurate(below 0.01-pixel wave vector and 0.1%of 2relative phase under typical noise level),and robust parameter estimation at low SNRs,which allows real-time super-resolution imaging of live cells in complicated experimental scenarios where other state-of-the-art methods inevitably fail.In particular,we provide the open-source MATLAB toolbox of our PCA-SIM algorithm and associated datasets.The combination of iteration-free reconstruction,robustness to noise,and limited computational complexity makes PCA-SIM a promising method for high-speed,long-term,artifact-free super-resolution imaging of live cells.

Fluorescence interference structured illumination microscopy for 3D morphology imaging with high axial resolution

Imaging three-dimensional,subcellular structures with high axial resolution has always been the core purpose of fluorescence microscopy.However,trade-offs exist between axial resolution and other important technical indicators,such as temporal resolution,optical power density,and imaging process complexity.We report a new imaging modality,fluorescence interference structured illumination microscopy(FI-SIM),which is based on three-dimensional structured illumination microscopy for wide-field lateral imaging and fluorescence interference for axial reconstruction.FI-SIM can acquire images quickly within the order of hundreds of milliseconds and exhibit even 30 nm axial resolution in half the wavelength depth range without z-axis scanning.Moreover,the relatively low laser power density relaxes the requirements for dyes and enables a wide range of applications for observing fixed and live subcellular structures.

High-speed image reconstruction for optically sectioned,super-resolution structured illumination microscopy

Super-resolution structured illumination microscopy(SR-SIM)is an outstanding method for visualizing the subcellular dynamics in living cells.To date,by using elaborately designed systems and algorithms,SR-SIM can achieve rapid,optically sectioned,SR observation with hundreds to thousands of time points.However,real-time observation is still out of reach for most SIM setups as conventional algorithms for image reconstruction involve a heavy computing burden.To address this limitation,an accelerated reconstruction algorithm was developed by implementing a simplified workflow for SR-SIM,termed joint space and frequency reconstruction.This algorithm results in an 80-fold improvement in reconstruction speed relative to the widely used Wiener-SIM.Critically,the increased processing speed does not come at the expense of spatial resolution or sectioning capability,as demonstrated by live imaging of microtubule dynamics and mitochondrial tubulation.

Woofer–tweeter adaptive optical structured illumination microscopy

A woofer–tweeter adaptive optical structured illumination microscope(AOSIM) is presented. By combining a low-spatial-frequency large-stroke deformable mirror(woofer) with a high-spatial-frequency low-stroke deformable mirror(tweeter), we are able to remove both large-amplitude and high-order aberrations. In addition, using the structured illumination method, as compared to widefield microscopy, the AOSIM can accomplish highresolution imaging and possesses better sectioning capability. The AOSIM was tested by correcting a large aberration from a trial lens in the conjugate plane of the microscope objective aperture. The experimental results show that the AOSIM has a point spread function with an FWHM that is 140 nm wide(using a water immersion objective lens with NA=1.1) after correcting a large aberration(5.9 μm peak-to-valley wavefront error with 2.05 μm RMS aberration). After structured light illumination is applied, the results show that we are able to resolve two beads that are separated by 145 nm, 1.62× below the diffraction limit of 235 nm. Furthermore, we demonstrate the application of the AOSIM in the field of bioimaging. The sample under investigation was a green-fluorescentprotein-labeled Drosophila embryo. The aberrations from the refractive index mismatch between the microscope objective, the immersion fluid, the cover slip, and the sample itself are well corrected. Using AOSIM we were able to increase the SNR for our Drosophila embryo sample by 5×.

High-speed 3D imaging based on structured illumination and electrically tunable lens

In this Letter, we present a high-speed volumetric imaging system based on structured illumination and an electrically tunable lens（ETL）, where the ETL performs fast axial scanning at hundreds of Hz. In the system,a digital micro-mirror device（DMD） is utilized to rapidly generate structured images at the focal plane in synchronization with the axial scanning unit. The scanning characteristics of the ETL are investigated theoretically and experimentally. Imaging experiments on pollen samples are performed to verify the optical cross-sectioning and fast axial scanning capabilities. The results show that our system can perform fast axial scanning and threedimensional（3D） imaging when paired with a high-speed camera, presenting an economic solution for advanced biological imaging applications.

Miniaturization of a coherent monocular structured illumination system for future combination with digital holography

Holographic and 3D-measurement processes are an often-used tool in industry,medicine,and scientific applications.While small deviations of objects can be visualized by holographic means with high accuracy,optical systems with active structured illumination are a reliable source of absolute 3D-information in these fields.The combination of digital holography with structured illumination allows to simultaneously measure deformations and absolute 3D coordinates but also requires coherent light and has already been demonstrated in principle with a stereo camera setup.Multi-camera systems are limited to certain setup sizes given by the volume and distance of the detectors.Reducing the system to a one-camera(monocular)setup reduces space and acquisition costs.By using a multi-aperture illumination source an extremely high projection rate could be realized and reduced to a monocular approach with a novel voxel-calibration technique,while the projection system itself still requires a large amount of space.In this paper we present a miniaturized,monocular 3D-measurement system that works with repeatable,coherent speckles,generated by a fiber-coupled laser whose light was distributed by a fiber-switch to a diffuser plate connected with a measurement-head,also including a camera.By addressing different fibers through the switch,varying but repeatable patterns are generated.The size of the device(diameter<3 cm)is now mainly limited by the volume of the camera.A first 3D-reconstruction of an object and an outlook for a combination of this system with digital holography is given,allowing absolute 3D-coordinates and relative deviations of object points to be measured simultaneously.