Fringe optimization for structured illumination super-resolution microscope with digital micromirror device

Structured illumination microscopy(SIM)is a promising super-resolution technique for imaging subcellular structures and dynamics due to its compatibility with most commonly usedffuorescent labeling methods.Structured illumination can be obtained by either laser interference or projection of fringe patterns.Here,we proposed a fringe projector composed of a compact multiwavelength LEDs module and a digital micromirror device(DMD)which can be directly attached to most commercial invertedffuorescent microscopes and update it into a SIM system.The effects of the period and duty cycle of fringe patterns on the modulation depth of the structured lightfield were studied.With the optimized fringe pattern,1:6×resolution improvement could be obtained with high-end oil objectives.Multicolor imaging and dynamics of subcellular organelles in live cells were also demonstrated.Our method provides a low-cost solution for SIM setup to expand its wide range of applications to most research labs in thefield of life science and medicine.

Shear Let Transform Residual Learning Approach for Single-Image Super-Resolution

Super-resolution techniques are employed to enhance image resolution by reconstructing high-resolution images from one or more low-resolution inputs.Super-resolution is of paramount importance in the context of remote sensing,satellite,aerial,security and surveillance imaging.Super-resolution remote sensing imagery is essential for surveillance and security purposes,enabling authorities to monitor remote or sensitive areas with greater clarity.This study introduces a single-image super-resolution approach for remote sensing images,utilizing deep shearlet residual learning in the shearlet transform domain,and incorporating the Enhanced Deep Super-Resolution network(EDSR).Unlike conventional approaches that estimate residuals between high and low-resolution images,the proposed approach calculates the shearlet coefficients for the desired high-resolution image using the provided low-resolution image instead of estimating a residual image between the high-and low-resolution image.The shearlet transform is chosen for its excellent sparse approximation capabilities.Initially,remote sensing images are transformed into the shearlet domain,which divides the input image into low and high frequencies.The shearlet coefficients are fed into the EDSR network.The high-resolution image is subsequently reconstructed using the inverse shearlet transform.The incorporation of the EDSR network enhances training stability,leading to improved generated images.The experimental results from the Deep Shearlet Residual Learning approach demonstrate its superior performance in remote sensing image recovery,effectively restoring both global topology and local edge detail information,thereby enhancing image quality.Compared to other networks,our proposed approach outperforms the state-of-the-art in terms of image quality,achieving an average peak signal-to-noise ratio of 35 and a structural similarity index measure of approximately 0.9.

AFBNet: A Lightweight Adaptive Feature Fusion Module for Super-Resolution Algorithms

At present,super-resolution algorithms are employed to tackle the challenge of low image resolution,but it is difficult to extract differentiated feature details based on various inputs,resulting in poor generalization ability.Given this situation,this study first analyzes the features of some feature extraction modules of the current super-resolution algorithm and then proposes an adaptive feature fusion block(AFB)for feature extraction.This module mainly comprises dynamic convolution,attention mechanism,and pixel-based gating mechanism.Combined with dynamic convolution with scale information,the network can extract more differentiated feature information.The introduction of a channel spatial attention mechanism combined with multi-feature fusion further enables the network to retain more important feature information.Dynamic convolution and pixel-based gating mechanisms enhance the module’s adaptability.Finally,a comparative experiment of a super-resolution algorithm based on the AFB module is designed to substantiate the efficiency of the AFB module.The results revealed that the network combined with the AFB module has stronger generalization ability and expression ability.

PSMFNet:Lightweight Partial Separation and Multiscale Fusion Network for Image Super-Resolution

The employment of deep convolutional neural networks has recently contributed to significant progress in single image super-resolution(SISR)research.However,the high computational demands of most SR techniques hinder their applicability to edge devices,despite their satisfactory reconstruction performance.These methods commonly use standard convolutions,which increase the convolutional operation cost of the model.In this paper,a lightweight Partial Separation and Multiscale Fusion Network(PSMFNet)is proposed to alleviate this problem.Specifically,this paper introduces partial convolution(PConv),which reduces the redundant convolution operations throughout the model by separating some of the features of an image while retaining features useful for image reconstruction.Additionally,it is worth noting that the existing methods have not fully utilized the rich feature information,leading to information loss,which reduces the ability to learn feature representations.Inspired by self-attention,this paper develops a multiscale feature fusion block(MFFB),which can better utilize the non-local features of an image.MFFB can learn long-range dependencies from the spatial dimension and extract features from the channel dimension,thereby obtaining more comprehensive and rich feature information.As the role of the MFFB is to capture rich global features,this paper further introduces an efficient inverted residual block(EIRB)to supplement the local feature extraction ability of PSMFNet.A comprehensive analysis of the experimental results shows that PSMFNet maintains a better performance with fewer parameters than the state-of-the-art models.

Efficient 2-D MUSIC algorithm for super-resolution moving target tracking based on an FMCW radar

Frequency modulated continuous wave(FMCW)radar is an advantageous sensor scheme for target estimation and environmental perception.However,existing algorithms based on discrete Fourier transform(DFT),multiple signal classification(MUSIC)and compressed sensing,etc.,cannot achieve both low complexity and high resolution simultaneously.This paper proposes an efficient 2-D MUSIC algorithm for super-resolution target estimation/tracking based on FMCW radar.Firstly,we enhance the efficiency of 2-D MUSIC azimuth-range spectrum estimation by incorporating 2-D DFT and multi-level resolution searching strategy.Secondly,we apply the gradient descent method to tightly integrate the spatial continuity of object motion into spectrum estimation when processing multi-epoch radar data,which improves the efficiency of continuous target tracking.These two approaches have improved the algorithm efficiency by nearly 2-4 orders of magnitude without losing accuracy and resolution.Simulation experiments are conducted to validate the effectiveness of the algorithm in both single-epoch estimation and multi-epoch tracking scenarios.

Pyramid Separable Channel Attention Network for Single Image Super-Resolution

Single Image Super-Resolution(SISR)technology aims to reconstruct a clear,high-resolution image with more information from an input low-resolution image that is blurry and contains less information.This technology has significant research value and is widely used in fields such as medical imaging,satellite image processing,and security surveillance.Despite significant progress in existing research,challenges remain in reconstructing clear and complex texture details,with issues such as edge blurring and artifacts still present.The visual perception effect still needs further enhancement.Therefore,this study proposes a Pyramid Separable Channel Attention Network(PSCAN)for the SISR task.Thismethod designs a convolutional backbone network composed of Pyramid Separable Channel Attention blocks to effectively extract and fuse multi-scale features.This expands the model’s receptive field,reduces resolution loss,and enhances the model’s ability to reconstruct texture details.Additionally,an innovative artifact loss function is designed to better distinguish between artifacts and real edge details,reducing artifacts in the reconstructed images.We conducted comprehensive ablation and comparative experiments on the Arabidopsis root image dataset and several public datasets.The experimental results show that the proposed PSCAN method achieves the best-known performance in both subjective visual effects and objective evaluation metrics,with improvements of 0.84 in Peak Signal-to-Noise Ratio(PSNR)and 0.017 in Structural Similarity Index(SSIM).This demonstrates that the method can effectively preserve high-frequency texture details,reduce artifacts,and have good generalization performance.

Multi-prior physics-enhanced neural network enables pixel super-resolution and twin-imagefree phase retrieval from single-shot hologram

Digital in-line holographic microscopy(DIHM)is a widely used interference technique for real-time reconstruction of living cells’morphological information with large space-bandwidth product and compact setup.However,the need for a larger pixel size of detector to improve imaging photosensitivity,field-of-view,and signal-to-noise ratio often leads to the loss of sub-pixel information and limited pixel resolution.Additionally,the twin-image appearing in the reconstruction severely degrades the quality of the reconstructed image.The deep learning(DL)approach has emerged as a powerful tool for phase retrieval in DIHM,effectively addressing these challenges.However,most DL-based strategies are datadriven or end-to-end net approaches,suffering from excessive data dependency and limited generalization ability.Herein,a novel multi-prior physics-enhanced neural network with pixel super-resolution(MPPN-PSR)for phase retrieval of DIHM is proposed.It encapsulates the physical model prior,sparsity prior and deep image prior in an untrained deep neural network.The effectiveness and feasibility of MPPN-PSR are demonstrated by comparing it with other traditional and learning-based phase retrieval methods.With the capabilities of pixel super-resolution,twin-image elimination and high-throughput jointly from a single-shot intensity measurement,the proposed DIHM approach is expected to be widely adopted in biomedical workflow and industrial measurement.

Three-dimensional morphological and fluorescent imaging of zebrafish with a continuous-rotational light-sheet microscope

Light-sheet fluorescence microscopy(LSFM)has been widely used to image the three-dimensional(3D)structures and functions of various millimeter-size bio-specimen such as zebrafish.However,the sample adsorption and scattering cause shading of the light-sheet illumination,preventing the even 3D image of thick samples.Herein,we report a continuous-rotational light-sheet microscope(CR-LSM)that enables simultaneous 3D bright-field and fluorescence imaging.With a high-accuracy rotational stage,CR-LSM records the outline projections and the fluorescent images of the sample at multiple rotation angles.Then,3D morphology and fluorescent structure were reconstructed with a developed algorithm.Using CR-LSM,zebrafish’s whole-fish contour and blood vessel structures were obtained simultaneously.

Improved spatiotemporal resolution of anti-scattering super-resolution label-free microscopy via synthetic wave 3D metalens imaging

Super-resolution(SR)microscopy has dramatically enhanced our understanding of biological processes.However,scattering media in thick specimens severely limits the spatial resolution,often rendering the images unclear or indistinguishable.Additionally,live-cell imaging faces challenges in achieving high temporal resolution for fast-moving subcellular structures.Here,we present the principles of a synthetic wave microscopy(SWM)to extract three-dimensional information from thick unlabeled specimens,where photobleaching and phototoxicity are avoided.SWM exploits multiple-wave interferometry to reveal the specimen’s phase information in the area of interest,which is not affected by the scattering media in the optical path.SWM achieves~0.42λ/NA resolution at an imaging speed of up to 106 pixels/s.SWM proves better temporal resolution and sensitivity than the most conventional microscopes currently available while maintaining exceptional SR and anti-scattering capabilities.Penetrating through the scattering media is challenging for conventional imaging techniques.Remarkably,SWM retains its efficacy even in conditions of low signal-to-noise ratios.It facilitates the visualization of dynamic subcellular structures in live cells,encompassing tubular endoplasmic reticulum(ER),lipid droplets,mitochondria,and lysosomes.

Faster split-based feedback network for image super-resolution

Although most of the existing image super-resolution(SR)methods have achieved superior performance,contrastive learning for high-level tasks has not been fully utilized in the existing image SR methods based on deep learning.This work focuses on two well-known strategies developed for lightweight and robust SR,i.e.,contrastive learning and feedback mechanism,and proposes an integrated solution called a split-based feedback network(SPFBN).The proposed SPFBN is based on a feedback mechanism to learn abstract representations and uses contrastive learning to explore high information in the representation space.Specifically,this work first uses hidden states and constraints in recurrent neural network(RNN)to implement a feedback mechanism.Then,use contrastive learning to perform representation learning to obtain high-level information by pushing the final image to the intermediate images and pulling the final SR image to the high-resolution image.Besides,a split-based feedback block(SPFB)is proposed to reduce model redundancy,which tolerates features with similar patterns but requires fewer parameters.Extensive experimental results demonstrate the superiority of the proposed method in comparison with the state-of-the-art methods.Moreover,this work extends the experiment to prove the effectiveness of this method and shows better overall reconstruction quality.

Constructing Electron Microscope Labs: Challenges and Solutions

The construction of advanced laboratories for precision instruments, such as electron microscopes, involves unique challenges that are influenced by the specific environmental conditions required for optimal functionality. These include mitigating interference from magnetic fields and vibrations, which are critical for maintaining the precision and accuracy of the instruments used. This study aims to offer enhanced project management strategies and detailed construction solutions that address the environmental and technical needs specific to electron microscopy labs, thereby facilitating effective lab operations and extending the lifecycle of high-end precision instruments. Case studies of existing laboratory constructions, onsite investigations, and comprehensive reviews of the technical and environmental requirements provide the basis for a best practice for constructing sophisticated electron microscopy labs. The approach integrates both pre-construction planning and post-construction adjustments to create optimal operational environments. The findings suggest that successful lab constructions are those that incorporate thorough onsite assessments, strategic location choices, and the use of advanced construction materials and techniques specifically designed to counteract environmental challenges like magnetic and vibration interferences. Actionable guidelines for both planning and executing the construction of electron microscope labs highlighted in this tutorial are intended as an important resource to troubleshoot or upgrade existing lab facilities and to consult in preparation of future lab construction projects.

SOFFLFM:Super-resolution optical fluctuation Fourierlight-field microscopy

Fourier light-field microscopy(FLFM)uses a microlens aray(MLA)to segment the Fourierplane of the microscopic objective lens to generate multiple two-dimensional perspective views,thereby reconstructing the threedimensional(3D)structure of the sample using 3D deconvo-lution calculation without scanning.However,the resolution of FLFM is stil limited by dif-fraction,and furthermore,it is dependent on the aperture division.In order to improve itsresolution,a super-resolution opticai fuctuation Fourier light-field microscopy(SOFFLFM)wasproposed here,in which the super-resolution optical fluctuation imaging(SOFI)with the abilityof super-resolution was introduced into FLFM.SOFFLFM uses higher-order cumulants statis-tical analysis on an image sequence collected by FLFM,and then carries out 3D deconvolutioncalculation to reconstruct the 3D structure of the sample.The theoretical basis of SOFFLFM onimproving resolution was explained and then verified with the simulations.Simulation resultsdemonstrated that SOFFLFM improved the lateral and axial resolution by more than V2 and 2times in the second-and fourth-order accumulations,compared with that of FLFM.

Deep-learning-based methods for super-resolution fluorescence microscopy

The algorithm used for reconstruction or resolution enhancement is one of the factors affectingthe quality of super-resolution images obtained by fluorescence microscopy.Deep-learning-basedalgorithms have achieved stateof-the-art performance in super-resolution fluorescence micros-copy and are becoming increasingly attractive.We firstly introduce commonly-used deep learningmodels,and then review the latest applications in terms of the net work architectures,the trainingdata and the loss functions.Additionally,we discuss the challenges and limits when using deeplearning to analyze the fluorescence microscopic data,and suggest ways to improve the reliability and robustness of deep learning applications.

Sub-diffraction-limit cell imaging using a super-resolution microscope with simplified pulse synchronization

Stimulated emission depletion(STED) microscope is one of the most prominent super-resolution bio-imaging instruments, which holds great promise for ultrahigh-resolution imaging of cells. To construct a STED microscope, it is challenging to realize temporal synchronization between the excitation pulses and the depletion pulses. In this study, we present a simple and low-cost method to achieve pulse synchronization by using a condensed fluorescent dye as a depletion indicator. By using this method, almost all the confocal microscopes can be upgraded to a STED system without losing its original functions. After the pulse synchronization,our STED system achieved sub-100-nm resolution for fluorescent nanospheres and single-cell imaging.

Hyperspectral Image Super-Resolution Meets Deep Learning:A Survey and Perspective

Hyperspectral image super-resolution,which refers to reconstructing the high-resolution hyperspectral image from the input low-resolution observation,aims to improve the spatial resolution of the hyperspectral image,which is beneficial for subsequent applications.The development of deep learning has promoted significant progress in hyperspectral image super-resolution,and the powerful expression capabilities of deep neural networks make the predicted results more reliable.Recently,several latest deep learning technologies have made the hyperspectral image super-resolution method explode.However,a comprehensive review and analysis of the latest deep learning methods from the hyperspectral image super-resolution perspective is absent.To this end,in this survey,we first introduce the concept of hyperspectral image super-resolution and classify the methods from the perspectives with or without auxiliary information.Then,we review the learning-based methods in three categories,including single hyperspectral image super-resolution,panchromatic-based hyperspectral image super-resolution,and multispectral-based hyperspectral image super-resolution.Subsequently,we summarize the commonly used hyperspectral dataset,and the evaluations for some representative methods in three categories are performed qualitatively and quantitatively.Moreover,we briefly introduce several typical applications of hyperspectral image super-resolution,including ground object classification,urban change detection,and ecosystem monitoring.Finally,we provide the conclusion and challenges in existing learning-based methods,looking forward to potential future research directions.

Contrastive Learning for Blind Super-Resolution via A Distortion-Specific Network

Previous deep learning-based super-resolution(SR)methods rely on the assumption that the degradation process is predefined(e.g.,bicubic downsampling).Thus,their performance would suffer from deterioration if the real degradation is not consistent with the assumption.To deal with real-world scenarios,existing blind SR methods are committed to estimating both the degradation and the super-resolved image with an extra loss or iterative scheme.However,degradation estimation that requires more computation would result in limited SR performance due to the accumulated estimation errors.In this paper,we propose a contrastive regularization built upon contrastive learning to exploit both the information of blurry images and clear images as negative and positive samples,respectively.Contrastive regularization ensures that the restored image is pulled closer to the clear image and pushed far away from the blurry image in the representation space.Furthermore,instead of estimating the degradation,we extract global statistical prior information to capture the character of the distortion.Considering the coupling between the degradation and the low-resolution image,we embed the global prior into the distortion-specific SR network to make our method adaptive to the changes of distortions.We term our distortion-specific network with contrastive regularization as CRDNet.The extensive experiments on synthetic and realworld scenes demonstrate that our lightweight CRDNet surpasses state-of-the-art blind super-resolution approaches.

Residual Feature Attentional Fusion Network for Lightweight Chest CT Image Super-Resolution

The diagnosis of COVID-19 requires chest computed tomography(CT).High-resolution CT images can provide more diagnostic information to help doctors better diagnose the disease,so it is of clinical importance to study super-resolution(SR)algorithms applied to CT images to improve the reso-lution of CT images.However,most of the existing SR algorithms are studied based on natural images,which are not suitable for medical images;and most of these algorithms improve the reconstruction quality by increasing the network depth,which is not suitable for machines with limited resources.To alleviate these issues,we propose a residual feature attentional fusion network for lightweight chest CT image super-resolution(RFAFN).Specifically,we design a contextual feature extraction block(CFEB)that can extract CT image features more efficiently and accurately than ordinary residual blocks.In addition,we propose a feature-weighted cascading strategy(FWCS)based on attentional feature fusion blocks(AFFB)to utilize the high-frequency detail information extracted by CFEB as much as possible via selectively fusing adjacent level feature information.Finally,we suggest a global hierarchical feature fusion strategy(GHFFS),which can utilize the hierarchical features more effectively than dense concatenation by progressively aggregating the feature information at various levels.Numerous experiments show that our method performs better than most of the state-of-the-art(SOTA)methods on the COVID-19 chest CT dataset.In detail,the peak signal-to-noise ratio(PSNR)is 0.11 dB and 0.47 dB higher on CTtest1 and CTtest2 at×3 SR compared to the suboptimal method,but the number of parameters and multi-adds are reduced by 22K and 0.43G,respectively.Our method can better recover chest CT image quality with fewer computational resources and effectively assist in COVID-19.

A Hybrid Regularization-Based Multi-Frame Super-Resolution Using Bayesian Framework

The prime purpose for the image reconstruction of a multi-frame super-resolution is to reconstruct a higher-resolution image through incorporating the knowledge obtained from a series of relevant low-resolution images,which is useful in numerousfields.Nevertheless,super-resolution image reconstruction methods are usually damaged by undesirable restorative artifacts,which include blurring distortion,noises,and stair-casing effects.Consequently,it is always challenging to achieve balancing between image smoothness and preservation of the edges inside the image.In this research work,we seek to increase the effectiveness of multi-frame super-resolution image reconstruction by increasing the visual information and improving the automated machine perception,which improves human analysis and interpretation processes.Accordingly,we propose a new approach to the image reconstruction of multi-frame super-resolution,so that it is created through the use of the regularization framework.In the proposed approach,the bilateral edge preserving and bilateral total variation regularizations are employed to approximate a high-resolution image generated from a sequence of corresponding images with low-resolution to protect significant features of an image,including sharp image edges and texture details while preventing artifacts.The experimental results of the synthesized image demonstrate that the new proposed approach has improved efficacy both visually and numerically more than other approaches.

IRMIRS:Inception-ResNet-Based Network for MRI Image Super-Resolution

Medical image super-resolution is a fundamental challenge due to absorption and scattering in tissues.These challenges are increasing the interest in the quality of medical images.Recent research has proven that the rapid progress in convolutional neural networks(CNNs)has achieved superior performance in the area of medical image super-resolution.However,the traditional CNN approaches use interpolation techniques as a preprocessing stage to enlarge low-resolution magnetic resonance(MR)images,adding extra noise in the models and more memory consumption.Furthermore,conventional deep CNN approaches used layers in series-wise connection to create the deeper mode,because this later end layer cannot receive complete information and work as a dead layer.In this paper,we propose Inception-ResNet-based Network for MRI Image Super-Resolution known as IRMRIS.In our proposed approach,a bicubic interpolation is replaced with a deconvolution layer to learn the upsampling filters.Furthermore,a residual skip connection with the Inception block is used to reconstruct a high-resolution output image from a low-quality input image.Quantitative and qualitative evaluations of the proposed method are supported through extensive experiments in reconstructing sharper and clean texture details as compared to the state-of-the-art methods.

Deep learning method for cell count from transmitted-light microscope

Automatic cell counting provides an effective tool for medical research and diagnosis.Currently,cell counting can be completed by transmitted-light microscope,however,it requires expert knowledge and the counting accuracy which is unsatisfied for overlapped cells.Further,the image-translation-based detection method has been proposed and the potential has been shown to accomplish cell counting from transmitted-light microscope,automatically and effectively.In this work,a new deep-learning(DL)-based two-stage detection method(cGAN-YOLO)is designed to further enhance the performance of cell counting,which is achieved by combining a DL-based fluorescent image translation model and a DL-based cell detection model.The various results show that cGAN-YOLO can effectively detect and count some different types of cells from the acquired transmitted-light microscope images.Compared with the previously reported YOLO-based one-stage detection method,high recognition accuracy(RA)is achieved by the cGAN-YOLO method,with an improvement of 29.80%.Furthermore,we can also observe that cGAN-YOLO obtains an improvement of 12.11%in RA compared with the previously reported image-translation-based detection method.In a word,cGAN-YOLO makes it possible to implement cell counting directly from the experimental acquired transmitted-light microscopy images with high flexibility and performance,which extends the applicability in clinical research.