Faster split-based feedback network for image super-resolution

Although most of the existing image super-resolution(SR)methods have achieved superior performance,contrastive learning for high-level tasks has not been fully utilized in the existing image SR methods based on deep learning.This work focuses on two well-known strategies developed for lightweight and robust SR,i.e.,contrastive learning and feedback mechanism,and proposes an integrated solution called a split-based feedback network(SPFBN).The proposed SPFBN is based on a feedback mechanism to learn abstract representations and uses contrastive learning to explore high information in the representation space.Specifically,this work first uses hidden states and constraints in recurrent neural network(RNN)to implement a feedback mechanism.Then,use contrastive learning to perform representation learning to obtain high-level information by pushing the final image to the intermediate images and pulling the final SR image to the high-resolution image.Besides,a split-based feedback block(SPFB)is proposed to reduce model redundancy,which tolerates features with similar patterns but requires fewer parameters.Extensive experimental results demonstrate the superiority of the proposed method in comparison with the state-of-the-art methods.Moreover,this work extends the experiment to prove the effectiveness of this method and shows better overall reconstruction quality.

Fast image super-resolution algorithm based on multi-resolution dictionary learning and sparse representation

Sparse representation has attracted extensive attention and performed well on image super-resolution（SR） in the last decade. However, many current image SR methods face the contradiction of detail recovery and artifact suppression. We propose a multi-resolution dictionary learning（MRDL） model to solve this contradiction, and give a fast single image SR method based on the MRDL model. To obtain the MRDL model, we first extract multi-scale patches by using our proposed adaptive patch partition method（APPM）. The APPM divides images into patches of different sizes according to their detail richness. Then, the multiresolution dictionary pairs, which contain structural primitives of various resolutions, can be trained from these multi-scale patches.Owing to the MRDL strategy, our SR algorithm not only recovers details well, with less jag and noise, but also significantly improves the computational efficiency. Experimental results validate that our algorithm performs better than other SR methods in evaluation metrics and visual perception.

Contrastive Learning for Blind Super-Resolution via A Distortion-Specific Network

Previous deep learning-based super-resolution(SR)methods rely on the assumption that the degradation process is predefined(e.g.,bicubic downsampling).Thus,their performance would suffer from deterioration if the real degradation is not consistent with the assumption.To deal with real-world scenarios,existing blind SR methods are committed to estimating both the degradation and the super-resolved image with an extra loss or iterative scheme.However,degradation estimation that requires more computation would result in limited SR performance due to the accumulated estimation errors.In this paper,we propose a contrastive regularization built upon contrastive learning to exploit both the information of blurry images and clear images as negative and positive samples,respectively.Contrastive regularization ensures that the restored image is pulled closer to the clear image and pushed far away from the blurry image in the representation space.Furthermore,instead of estimating the degradation,we extract global statistical prior information to capture the character of the distortion.Considering the coupling between the degradation and the low-resolution image,we embed the global prior into the distortion-specific SR network to make our method adaptive to the changes of distortions.We term our distortion-specific network with contrastive regularization as CRDNet.The extensive experiments on synthetic and realworld scenes demonstrate that our lightweight CRDNet surpasses state-of-the-art blind super-resolution approaches.

A brief survey on deep learning based image super-resolution

Image super-resolution(SR)is an important technique for improving the resolution and quality of images.With the great progress of deep learning,image super-resolution achieves remarkable improvements recently.In this work,a brief survey on recent advances of deep learning based single image super-resolution methods is systematically described.The existing studies of SR techniques are roughly grouped into ten major categories.Besides,some other important issues are also introduced,such as publicly available benchmark datasets and performance evaluation metrics.Finally,this survey is concluded by highlighting four future trends.

A NOVEL ALGORITHM OF SUPER-RESOLUTION RECONSTRUCTION FOR COMPRESSED VIDEO

Super-Resolution (SR) technique means to reconstruct High-Resolution (HR) images from a sequence of Low-Resolution (LR) observations,which has been a great focus for compressed video. Based on the theory of Projection Onto Convex Set (POCS),this paper constructs Quantization Constraint Set (QCS) using the quantization information extracted from the video bit stream. By combining the statistical properties of image and the Human Visual System (HVS),a novel Adaptive Quantization Constraint Set (AQCS) is proposed. Simulation results show that AQCS-based SR al-gorithm converges at a fast rate and obtains better performance in both objective and subjective quality,which is applicable for compressed video.

Super-Resolution Image Reconstruction Based on an Improved Maximum a Posteriori Algorithm

A maximum a posteriori（ MAP） algorithm is proposed to improve the accuracy of super resolution（ SR） reconstruction in traditional methods. The algorithm applies both joints image registration and SR reconstruction in the framework,but separates them in the process of iteratiion. Firstly,we estimate the shifting parameters through two lowresolution（ LR） images and use the parameters to reconstruct initial HR images. Then,we update the shifting parameters using HR images. The aforementioned steps are repeated until the ideal HR images are obtained. The metrics such as PSNR and SSIM are used to fully evaluate the quality of the reconstructed image. Experimental results indicate that the proposed method can enhance image resolution efficiently.

Channel attention based wavelet cascaded network for image super-resolution

Convolutional neural networks(CNNs) have shown great potential for image super-resolution(SR).However,most existing CNNs only reconstruct images in the spatial domain,resulting in insufficient high-frequency details of reconstructed images.To address this issue,a channel attention based wavelet cascaded network for image super-resolution(CWSR) is proposed.Specifically,a second-order channel attention(SOCA) mechanism is incorporated into the network,and the covariance matrix normalization is utilized to explore interdependencies between channel-wise features.Then,to boost the quality of residual features,the non-local module is adopted to further improve the global information integration ability of the network.Finally,taking the image loss in the spatial and wavelet domains into account,a dual-constrained loss function is proposed to optimize the network.Experimental results illustrate that CWSR outperforms several state-of-the-art methods in terms of both visual quality and quantitative metrics.

Single image super-resolution:a comprehensive review and recent insight

Super-resolution(SR)is a long-standing problem in image processing and computer vision and has attracted great attention from researchers over the decades.The main concept of SR is to reconstruct images from low-resolution(LR)to high-resolution(HR).It is an ongoing process in image technology,through up-sampling,de-blurring,and de-noising.Convolution neural network(CNN)has been widely used to enhance the resolution of images in recent years.Several alternative methods use deep learning to improve the progress of image super-resolution based on CNN.Here,we review the recent findings of single image super-resolution using deep learning with an emphasis on distillation knowledge used to enhance image super-resolution.,it is also to highlight the potential applications of image super-resolution in security monitoring,medical diagnosis,microscopy image processing,satellite remote sensing,communication transmission,the digital multimedia industry and video enhancement.Finally,we present the challenges and assess future trends in super-resolution based on deep learning.

Multi-example feature-constrained back-projection method for image super-resolution

Example-based super-resolution algorithms,which predict unknown high-resolution image information using a relationship model learnt from known high- and low-resolution image pairs, have attracted considerable interest in the field of image processing. In this paper, we propose a multi-example feature-constrained back-projection method for image super-resolution. Firstly, we take advantage of a feature-constrained polynomial interpolation method to enlarge the low-resolution image. Next, we consider low-frequency images of different resolutions to provide an example pair. Then, we use adaptive k NN search to find similar patches in the low-resolution image for every image patch in the high-resolution low-frequency image, leading to a regression model between similar patches to be learnt. The learnt model is applied to the low-resolution high-frequency image to produce high-resolution high-frequency information. An iterative back-projection algorithm is used as the final step to determine the final high-resolution image.Experimental results demonstrate that our method improves the visual quality of the high-resolution image.

Single image super-resolution via blind blurring estimation and anchored space mapping

It has been widely acknowledged that learning-based super-resolution(SR) methods are effective to recover a high resolution(HR) image from a single low resolution(LR) input image. However,there exist two main challenges in learning-based SR methods currently: the quality of training samples and the demand for computation. We proposed a novel framework for single image SR tasks aiming at these issues, which consists of blind blurring kernel estimation(BKE) and SR recovery with anchored space mapping(ASM). BKE is realized via minimizing the cross-scale dissimilarity of the image iteratively, and SR recovery with ASM is performed based on iterative least square dictionary learning algorithm(ILS-DLA). BKE is capable of improving the compatibility of training samples and testing samples effectively and ASM can reduce consumed time during SR recovery radically.Moreover, a selective patch processing(SPP) strategy measured by average gradient amplitude |grad| of a patch is adopted to accelerate the BKE process. The experimental results show that our method outruns several typical blind and non-blind algorithms on equal conditions.

Light field super-resolution using complementary-view feature attention

Light field(LF)cameras record multiple perspectives by a sparse sampling of real scenes,and these perspectives provide complementary information.This information is beneficial to LF super-resolution(LFSR).Compared with traditional single-image super-resolution,LF can exploit parallax structure and perspective correlation among different LF views.Furthermore,the performance of existing methods are limited as they fail to deeply explore the complementary information across LF views.In this paper,we propose a novel network,called the light field complementary-view feature attention network(LF-CFANet),to improve LFSR by dynamically learning the complementary information in LF views.Specifically,we design a residual complementary-view spatial and channel attention module(RCSCAM)to effectively interact with complementary information between complementary views.Moreover,RCSCAM captures the relationships between different channels,and it is able to generate informative features for reconstructing LF images while ignoring redundant information.Then,a maximum-difference information supplementary branch(MDISB)is used to supplement information from the maximum-difference angular positions based on the geometric structure of LF images.This branch also can guide the process of reconstruction.Experimental results on both synthetic and real-world datasets demonstrate the superiority of our method.The proposed LF-CFANet has a more advanced reconstruction performance that displays faithful details with higher SR accuracy than state-of-the-art methods.

RFCNet:Remote Sensing Image Super-Resolution Using Residual Feature Calibration Network

In the field of single remote sensing image Super-Resolution(SR),deep Convolutional Neural Networks(CNNs)have achieved top performance.To further enhance convolutional module performance in processing remote sensing images,we construct an efficient residual feature calibration block to generate expressive features.After harvesting residual features,we first divide them into two parts along the channel dimension.One part flows to the Self-Calibrated Convolution(SCC)to be further refined,and the other part is rescaled by the proposed Two-Path Channel Attention(TPCA)mechanism.SCC corrects local features according to their expressions under the deep receptive field,so that the features can be refined without increasing the number of calculations.The proposed TPCA uses the means and variances of feature maps to obtain accurate channel attention vectors.Moreover,a region-level nonlocal operation is introduced to capture long-distance spatial contextual information by exploring pixel dependencies at the region level.Extensive experiments demonstrate that the proposed residual feature calibration network is superior to other SR methods in terms of quantitative metrics and visual quality.

Edge preserving super-resolution infrared image reconstruction based on L1-and L2-norms

Super-resolution （SR） is a widely used tech- nology that increases image resolution using algorithmic methods. However, preserving the local edge structure and visual quality in infrared （IR） SR images is challenging because of their disadvantages, such as lack of detail, poor contrast, and blurry edges. Traditional and advanced methods maintain the quantitative measures, but they mostly fail to preserve edge and visual quality. This paper proposes an algorithm based on high frequency layer features. This algorithm focuses on the IR image edge texture in the reconstruction process. Experimental results show that the mean gradient of the IR image reconstructed by the proposed algorithm increased by 1.5, 1.4, and 1.2 times than that of the traditional algorithm based on L1- norm, L2-norm, and traditional mixed norm, respectively. The peak signal-to-noise ratio, structural similarity index, and visual effect of the reconstructed image also improved.

Exploring the usefulness of light field super-resolution for object detection

In order to solve the impact of image degradation on object detection, an object detection method based on light field super-resolution(LFSR) is proposed. This method takes LFSR as an image enhancement step to provide high-quality images for object detection without using expensive imaging equipment. To evaluate this method, three types of objects: person, bicycle, and car, are chosen and the results are compared from 5 parts: detected object quantity, mean confidence score, detection results in different scenes, error detection, and detection results from different images sizes and detection speed. Experimental results based on the common object in context(COCO) dataset show that the method incorporated LFSR improves performance of object detection models.

Super-resolution reconstruction of astronomical images using time-scale adaptive normalized convolution

In this work, we describe a new multiframe Super-Resolution（SR） framework based on time-scale adaptive Normalized Convolution（NC）, and apply it to astronomical images. The method mainly uses the conceptual basis of NC where each neighborhood of a signal is expressed in terms of the corresponding subspace expanded by the chosen polynomial basis function. Instead of the conventional NC, the introduced spatially adaptive filtering kernel is utilized as the applicability function of shape-adaptive NC, which fits the local image structure information including shape and orientation. This makes it possible to obtain image patches with the same modality,which are collected for polynomial expansion to maximize the signal-to-noise ratio and suppress aliasing artifacts across lines and edges. The robust signal certainty takes the confidence value at each point into account before a local polynomial expansion to minimize the influence of outliers.Finally, the temporal scale applicability is considered to omit accurate motion estimation since it is easy to result in annoying registration errors in real astronomical applications. Excellent SR reconstruction capability of the time-scale adaptive NC is demonstrated through fundamental experiments on both synthetic images and real astronomical images when compared with other SR reconstruction methods.