PSMFNet:Lightweight Partial Separation and Multiscale Fusion Network for Image Super-Resolution

The employment of deep convolutional neural networks has recently contributed to significant progress in single image super-resolution(SISR)research.However,the high computational demands of most SR techniques hinder their applicability to edge devices,despite their satisfactory reconstruction performance.These methods commonly use standard convolutions,which increase the convolutional operation cost of the model.In this paper,a lightweight Partial Separation and Multiscale Fusion Network(PSMFNet)is proposed to alleviate this problem.Specifically,this paper introduces partial convolution(PConv),which reduces the redundant convolution operations throughout the model by separating some of the features of an image while retaining features useful for image reconstruction.Additionally,it is worth noting that the existing methods have not fully utilized the rich feature information,leading to information loss,which reduces the ability to learn feature representations.Inspired by self-attention,this paper develops a multiscale feature fusion block(MFFB),which can better utilize the non-local features of an image.MFFB can learn long-range dependencies from the spatial dimension and extract features from the channel dimension,thereby obtaining more comprehensive and rich feature information.As the role of the MFFB is to capture rich global features,this paper further introduces an efficient inverted residual block(EIRB)to supplement the local feature extraction ability of PSMFNet.A comprehensive analysis of the experimental results shows that PSMFNet maintains a better performance with fewer parameters than the state-of-the-art models.

Multi-prior physics-enhanced neural network enables pixel super-resolution and twin-imagefree phase retrieval from single-shot hologram

Digital in-line holographic microscopy(DIHM)is a widely used interference technique for real-time reconstruction of living cells’morphological information with large space-bandwidth product and compact setup.However,the need for a larger pixel size of detector to improve imaging photosensitivity,field-of-view,and signal-to-noise ratio often leads to the loss of sub-pixel information and limited pixel resolution.Additionally,the twin-image appearing in the reconstruction severely degrades the quality of the reconstructed image.The deep learning(DL)approach has emerged as a powerful tool for phase retrieval in DIHM,effectively addressing these challenges.However,most DL-based strategies are datadriven or end-to-end net approaches,suffering from excessive data dependency and limited generalization ability.Herein,a novel multi-prior physics-enhanced neural network with pixel super-resolution(MPPN-PSR)for phase retrieval of DIHM is proposed.It encapsulates the physical model prior,sparsity prior and deep image prior in an untrained deep neural network.The effectiveness and feasibility of MPPN-PSR are demonstrated by comparing it with other traditional and learning-based phase retrieval methods.With the capabilities of pixel super-resolution,twin-image elimination and high-throughput jointly from a single-shot intensity measurement,the proposed DIHM approach is expected to be widely adopted in biomedical workflow and industrial measurement.

Hyperspectral Image Super-Resolution Meets Deep Learning:A Survey and Perspective

Hyperspectral image super-resolution,which refers to reconstructing the high-resolution hyperspectral image from the input low-resolution observation,aims to improve the spatial resolution of the hyperspectral image,which is beneficial for subsequent applications.The development of deep learning has promoted significant progress in hyperspectral image super-resolution,and the powerful expression capabilities of deep neural networks make the predicted results more reliable.Recently,several latest deep learning technologies have made the hyperspectral image super-resolution method explode.However,a comprehensive review and analysis of the latest deep learning methods from the hyperspectral image super-resolution perspective is absent.To this end,in this survey,we first introduce the concept of hyperspectral image super-resolution and classify the methods from the perspectives with or without auxiliary information.Then,we review the learning-based methods in three categories,including single hyperspectral image super-resolution,panchromatic-based hyperspectral image super-resolution,and multispectral-based hyperspectral image super-resolution.Subsequently,we summarize the commonly used hyperspectral dataset,and the evaluations for some representative methods in three categories are performed qualitatively and quantitatively.Moreover,we briefly introduce several typical applications of hyperspectral image super-resolution,including ground object classification,urban change detection,and ecosystem monitoring.Finally,we provide the conclusion and challenges in existing learning-based methods,looking forward to potential future research directions.

IRMIRS:Inception-ResNet-Based Network for MRI Image Super-Resolution

Medical image super-resolution is a fundamental challenge due to absorption and scattering in tissues.These challenges are increasing the interest in the quality of medical images.Recent research has proven that the rapid progress in convolutional neural networks(CNNs)has achieved superior performance in the area of medical image super-resolution.However,the traditional CNN approaches use interpolation techniques as a preprocessing stage to enlarge low-resolution magnetic resonance(MR)images,adding extra noise in the models and more memory consumption.Furthermore,conventional deep CNN approaches used layers in series-wise connection to create the deeper mode,because this later end layer cannot receive complete information and work as a dead layer.In this paper,we propose Inception-ResNet-based Network for MRI Image Super-Resolution known as IRMRIS.In our proposed approach,a bicubic interpolation is replaced with a deconvolution layer to learn the upsampling filters.Furthermore,a residual skip connection with the Inception block is used to reconstruct a high-resolution output image from a low-quality input image.Quantitative and qualitative evaluations of the proposed method are supported through extensive experiments in reconstructing sharper and clean texture details as compared to the state-of-the-art methods.

Super-Resolution Imaging of Mammograms Based on the Super-Resolution Convolutional Neural Network

Purpose: To apply and evaluate a super-resolution scheme based on the super-resolution convolutional neural network (SRCNN) for enhancing image resolution in digital mammograms. Materials and Methods: A total of 711 mediolateral oblique (MLO) images including breast lesions were sampled from the Curated Breast Imaging Subset of the Digital Database for Screening Mammography (CBIS-DDSM). We first trained the super-resolution convolutional neural network (SRCNN), which is a deep-learning based super-resolution method. Using this trained SRCNN, high-resolution images were reconstructed from low-resolution images. We compared the image quality of the super-resolution method and that obtained using the linear interpolation methods (nearest neighbor and bilinear interpolations). To investigate the relationship between the image quality of the SRCNN-processed images and the clinical features of the mammographic lesions, we compared the image quality yielded by implementing the SRCNN, in terms of the breast density, the Breast Imaging-Reporting and Data System (BI-RADS) assessment, and the verified pathology information. For quantitative evaluation, peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) were measured to assess the image restoration quality and the perceived image quality. Results: The super-resolution image quality yielded by the SRCNN was significantly higher than that obtained using linear interpolation methods (p p Conclusion: SRCNN can significantly outperform conventional interpolation methods for enhancing image resolution in digital mammography. SRCNN can significantly improve the image quality of magnified mammograms, especially in dense breasts, high-risk BI-RADS assessment groups, and pathology-verified malignant cases.

Polarimetric super-resolution algorithm for radar range imaging via spatial smoothing processing

A full-polarimetric super-resolution algorithm with spatial smoothing processing is presented for one-dimensional（1-D）radar imaging.The coherence between scattering centers is minimized by using spatial smoothing processing（SSP）.Then the range and polarimetric scattering matrix of the scattering centers are estimated.The impact of different lengths of the smoothing window on the imaging quality is mainly analyzed with different signal-to-noise ratios（SNR）.Simulation and experimental results show that an improved radar super-resolution range profile and more precise estimation can be obtained by adjusting the length of the smoothing window under different SNR conditions.

Near-field multiple super-resolution imaging from Mikaelian lens to generalized Maxwell's fish-eye lens

Super-resolution imaging is vital for optical applications, such as high capacity information transmission, real-time bio-molecular imaging, and nanolithography. In recent years, technologies and methods of super-resolution imaging have attracted much attention. Different kinds of novel lenses, from the superlens to the super-oscillatory lens, have been designed and fabricated to break through the diffraction limit. However, the effect of the super-resolution imaging in these lenses is not satisfactory due to intrinsic loss, aberration, large sidebands, and so on. Moreover, these lenses also cannot realize multiple super-resolution imaging. In this research, we introduce the solid immersion mechanism to Mikaelian lens(ML) for multiple super-resolution imaging. The effect is robust and valid for broadband frequencies. Based on conformal transformation optics as a bridge linking the solid immersion ML and generalized Maxwell's fish-eye lens(GMFEL), we also discovered the effect of multiple super-resolution imaging in the solid immersion GMFEL.

Super-resolution imaging in glycoscience: New developments and challenges

Carbohydrates on cell surfaces play a crucial role in a wide variety of biological processes,including cell adhesion,recognition and signaling,viral and bacterial infection,in°ammation and metastasis.However,owing to the large diversity and complexity of carbohydrate structure and nongenetically synthesis,glycoscience is the least understood¯eld compared with genomics and proteomics.Although the structures and functions of carbohydrates have been investigated by various conventional analysis methods,the distribution and role of carbohydrates in cell membranes remain elusive.This review focuses on the developments and challenges of super-resolution imaging in glycoscience through introduction of imaging principle and the available°uorescent probes for super-resolution imaging,the labeling strategies of carbohydrates,and the recent applications of super-resolution imaging in glycoscience,which will promote the super-resolution imaging technology as a promising tool to provide new insights into the study of glycoscience.

Contrastive Learning for Blind Super-Resolution via A Distortion-Specific Network

Previous deep learning-based super-resolution(SR)methods rely on the assumption that the degradation process is predefined(e.g.,bicubic downsampling).Thus,their performance would suffer from deterioration if the real degradation is not consistent with the assumption.To deal with real-world scenarios,existing blind SR methods are committed to estimating both the degradation and the super-resolved image with an extra loss or iterative scheme.However,degradation estimation that requires more computation would result in limited SR performance due to the accumulated estimation errors.In this paper,we propose a contrastive regularization built upon contrastive learning to exploit both the information of blurry images and clear images as negative and positive samples,respectively.Contrastive regularization ensures that the restored image is pulled closer to the clear image and pushed far away from the blurry image in the representation space.Furthermore,instead of estimating the degradation,we extract global statistical prior information to capture the character of the distortion.Considering the coupling between the degradation and the low-resolution image,we embed the global prior into the distortion-specific SR network to make our method adaptive to the changes of distortions.We term our distortion-specific network with contrastive regularization as CRDNet.The extensive experiments on synthetic and realworld scenes demonstrate that our lightweight CRDNet surpasses state-of-the-art blind super-resolution approaches.

A probability theory for filtered ghost imaging

Based on probability density functions,we present a theoretical model to explain filtered ghost imaging(FGI)we first proposed and experimentally demonstrated in 2017[Opt.Lett.425290(2017)].An analytic expression for the joint intensity probability density functions of filtered random speckle fields is derived according to their probability distributions.Moreover,the normalized second-order intensity correlation functions are calculated for the three cases of low-pass,bandpass and high-pass filterings to study the resolution and visibility in the FGI system.Numerical simulations show that the resolution and visibility predicted by our model agree well with the experimental results,which also explains why FGI can achieve a super-resolution image and better visibility than traditional ghost imaging.

Deep-learning-based methods for super-resolution fluorescence microscopy

The algorithm used for reconstruction or resolution enhancement is one of the factors affectingthe quality of super-resolution images obtained by fluorescence microscopy.Deep-learning-basedalgorithms have achieved stateof-the-art performance in super-resolution fluorescence micros-copy and are becoming increasingly attractive.We firstly introduce commonly-used deep learningmodels,and then review the latest applications in terms of the net work architectures,the trainingdata and the loss functions.Additionally,we discuss the challenges and limits when using deeplearning to analyze the fluorescence microscopic data,and suggest ways to improve the reliability and robustness of deep learning applications.

Super-resolution reconstruction of synthetic-aperture radar image using adaptive-threshold singular value decomposition technique

A super-resolution reconstruction approach of （SVD） technique was presented, and its performance was radar image using an adaptive-threshold singular value decomposition analyzed, compared and assessed detailedly. First, radar imaging model and super-resolution reconstruction mechanism were outlined. Then, the adaptive-threshold SVD super-resolution algorithm, and its two key aspects, namely the determination method of point spread function （PSF） matrix T and the selection scheme of singular value threshold, were presented. Finally, the super-resolution algorithm was demonstrated successfully using the measured synthetic-aperture radar （SAR） images, and a Monte Carlo assessment was carried out to evaluate the performance of the algorithm by using the input/output signal-to-noise ratio （SNR）. Five versions of SVD algorithms, namely 1 ） using all singular values, 2） using the top 80% singular values, 3） using the top 50% singular values, 4） using the top 20% singular values and 5） using singular values s such that S2≥/max（s2）/rinsNR were tested. The experimental results indicate that when the singular value threshold is set as Smax/（rinSNR）1/2, the super-resolution algorithm provides a good compromise between too much noise and too much bias and has good reconstruction results.

Image Super-Resolution Based on Generative Adversarial Networks: A Brief Review

Single image super resolution(SISR)is an important research content in the field of computer vision and image processing.With the rapid development of deep neural networks,different image super-resolution models have emerged.Compared to some traditional SISR methods,deep learning-based methods can complete the super-resolution tasks through a single image.In addition,compared with the SISR methods using traditional convolutional neural networks,SISR based on generative adversarial networks(GAN)has achieved the most advanced visual performance.In this review,we first explore the challenges faced by SISR and introduce some common datasets and evaluation metrics.Then,we review the improved network structures and loss functions of GAN-based perceptual SISR.Subsequently,the advantages and disadvantages of different networks are analyzed by multiple comparative experiments.Finally,we summarize the paper and look forward to the future development trends of GAN-based perceptual SISR.

Single color image super-resolution using sparse representation and color constraint

Color image super-resolution reconstruction based on the sparse representation model usually adopts the regularization norm(e.g.,L1 or L2).These methods have limited ability to keep image texture detail to some extent and are easy to cause the problem of blurring details and color artifacts in color reconstructed images.This paper presents a color super-resolution reconstruction method combining the L2/3 sparse regularization model with color channel constraints.The method converts the low-resolution color image from RGB to YCbCr.The L2/3 sparse regularization model is designed to reconstruct the brightness channel of the input low-resolution color image.Then the color channel-constraint method is adopted to remove artifacts of the reconstructed highresolution image.The method not only ensures the reconstruction quality of the color image details,but also improves the removal ability of color artifacts.The experimental results on natural images validate that our method has improved both subjective and objective evaluation.

Better Visual Image Super-Resolution with Laplacian Pyramid of Generative Adversarial Networks

Although there has been a great breakthrough in the accuracy and speed of super-resolution(SR)reconstruction of a single image by using a convolutional neural network,an important problem remains unresolved:how to restore finer texture details during image super-resolution reconstruction?This paper proposes an Enhanced Laplacian Pyramid Generative Adversarial Network(ELSRGAN),based on the Laplacian pyramid to capture the high-frequency details of the image.By combining Laplacian pyramids and generative adversarial networks,progressive reconstruction of super-resolution images can be made,making model applications more flexible.In order to solve the problem of gradient disappearance,we introduce the Residual-in-Residual Dense Block(RRDB)as the basic network unit.Network capacity benefits more from dense connections,is able to capture more visual features with better reconstruction effects,and removes BN layers to increase calculation speed and reduce calculation complexity.In addition,a loss of content driven by perceived similarity is used instead of content loss driven by spatial similarity,thereby enhancing the visual effect of the super-resolution image,making it more consistent with human visual perception.Extensive qualitative and quantitative evaluation of the baseline datasets shows that the proposed algorithm has higher mean-sort-score(MSS)than any state-of-the-art method and has better visual perception.

Fast image super-resolution algorithm based on multi-resolution dictionary learning and sparse representation

Sparse representation has attracted extensive attention and performed well on image super-resolution（SR） in the last decade. However, many current image SR methods face the contradiction of detail recovery and artifact suppression. We propose a multi-resolution dictionary learning（MRDL） model to solve this contradiction, and give a fast single image SR method based on the MRDL model. To obtain the MRDL model, we first extract multi-scale patches by using our proposed adaptive patch partition method（APPM）. The APPM divides images into patches of different sizes according to their detail richness. Then, the multiresolution dictionary pairs, which contain structural primitives of various resolutions, can be trained from these multi-scale patches.Owing to the MRDL strategy, our SR algorithm not only recovers details well, with less jag and noise, but also significantly improves the computational efficiency. Experimental results validate that our algorithm performs better than other SR methods in evaluation metrics and visual perception.

Super-resolution processing of passive millimeter-wave images based on conjugate-gradient algorithm

This paper designs a 3 mm radiometer and validate with experiments based on the principle of passive millimeter wave （PMMW） imaging. The poor spatial resolution, which is limited by antenna size, should be improved by post data processing. A conjugate-gradient （CG） algorithm is adopted to circumvent this drawback. Simulation and real data collected in laboratory environment are given, and the results show that the CG algorithm improves the spatial resolution and convergent rate. Further, it can reduce the ringing effects which are caused by regularizing the image restoration. Thus, the CG algorithm is easily implemented for PMMW imaging.

Image super-resolution reconstruction based on sparse representation and residual compensation

A super-resolution reconstruction algorithm is proposed. The algorithm is based on the idea of the sparse representation of signals, by using the fact that the sparsest representation of a sig- nal is unique as the constraint of the patched-based reconstruction, and compensating residual errors of the reconstruction results both locally and globally to solve the distortion problem in patch-based reconstruction algorithms. Three reconstruction algorithms are compared. The results show that the images reconstructed with the new algorithm have the best quality.

Super-resolution processing of passive millimeter-wave images based on adaptive projected Landweber algorithm

Passive millimeter wave （PMMW） images inherently have the problem of poor resolution owing to limited aperture dimension. Thus, efficient post-processing is necessary to achieve resolution improvement. An adaptive projected Landweber （APL） super-resolution algorithm using a spectral correction procedure, which attempts to combine the strong points of all of the projected Landweber （PL） iteration and the adaptive relaxation parameter adjustment and the spectral correction method, is proposed. In the algorithm, the PL iterations are implemented as the main image restoration scheme and a spectral correction method is included in which the calculated spectrum within the passband is replaced by the known low frequency component. Then, the algorithm updates the relaxation parameter adaptively at each iteration. A qualitative evaluation of this algorithm is performed with simulated data as well as actual radiometer image captured by 91.5 GHz mechanically scanned radiometer. From experiments, it is found that the super-resolution algorithm obtains better results and enhances the resolution and has lower mean square error （MSE）. These constraints and adaptive character and spectral correction procedures speed up the convergence of the Landweber algorithm and reduce the ringing effects that are caused by regularizing the image restoration problem.

A brief survey on deep learning based image super-resolution

Image super-resolution(SR)is an important technique for improving the resolution and quality of images.With the great progress of deep learning,image super-resolution achieves remarkable improvements recently.In this work,a brief survey on recent advances of deep learning based single image super-resolution methods is systematically described.The existing studies of SR techniques are roughly grouped into ten major categories.Besides,some other important issues are also introduced,such as publicly available benchmark datasets and performance evaluation metrics.Finally,this survey is concluded by highlighting four future trends.