Research on single image super-resolution based on very deep super-resolution convolutional neural network

Single image super-resolution(SISR)is a fundamentally challenging problem because a low-resolution(LR)image can correspond to a set of high-resolution(HR)images,while most are not expected.Recently,SISR can be achieved by a deep learning-based method.By constructing a very deep super-resolution convolutional neural network(VDSRCNN),the LR images can be improved to HR images.This study mainly achieves two objectives:image super-resolution(ISR)and deblurring the image from VDSRCNN.Firstly,by analyzing ISR,we modify different training parameters to test the performance of VDSRCNN.Secondly,we add the motion blurred images to the training set to optimize the performance of VDSRCNN.Finally,we use image quality indexes to evaluate the difference between the images from classical methods and VDSRCNN.The results indicate that the VDSRCNN performs better in generating HR images from LR images using the optimized VDSRCNN in a proper method.

IRMIRS:Inception-ResNet-Based Network for MRI Image Super-Resolution

Medical image super-resolution is a fundamental challenge due to absorption and scattering in tissues.These challenges are increasing the interest in the quality of medical images.Recent research has proven that the rapid progress in convolutional neural networks(CNNs)has achieved superior performance in the area of medical image super-resolution.However,the traditional CNN approaches use interpolation techniques as a preprocessing stage to enlarge low-resolution magnetic resonance(MR)images,adding extra noise in the models and more memory consumption.Furthermore,conventional deep CNN approaches used layers in series-wise connection to create the deeper mode,because this later end layer cannot receive complete information and work as a dead layer.In this paper,we propose Inception-ResNet-based Network for MRI Image Super-Resolution known as IRMRIS.In our proposed approach,a bicubic interpolation is replaced with a deconvolution layer to learn the upsampling filters.Furthermore,a residual skip connection with the Inception block is used to reconstruct a high-resolution output image from a low-quality input image.Quantitative and qualitative evaluations of the proposed method are supported through extensive experiments in reconstructing sharper and clean texture details as compared to the state-of-the-art methods.

Fully 1×1 Convolutional Network for Lightweight Image Super-resolution

Deep convolutional neural networks,particularly large models with large kernels(3x3 or more),have achieved significant progress in single image super-resolution(SISR)tasks.However,the heavy computational footprint of such models prevents their de-ployment in real-time,resource-constrained environments.Conversely,1×1 convolutions have substantial computational efficiency,but struggle with aggregating local spatial representations,which is an essential capability for SISR models.In response to this dichotomy,we propose to harmonize the merits of both 3x3 and 1×1 kernels,and exploit their great potential for lightweight SISR tasks.Specific-ally,we propose a simple yet effective fully 1×1 convolutional network,named shift-Conv-based network(SCNet).By incorporating a parameter-free spatial-shift operation,the fully 1×1 convolutional network is equipped with a powerful representation capability and impressive computational efficiency.Extensive experiments demonstrate that SCNets,despite their fully 1×1 convolutional structure,consistently match or even surpass the performance of existing lightweight SR models that employ regular convolutions.The code and pretrained models can be found at .

Lightweight Image Super-Resolution via Weighted Multi-Scale Residual Network

The tradeoff between efficiency and model size of the convolutional neural network(CNN)is an essential issue for applications of CNN-based algorithms to diverse real-world tasks.Although deep learning-based methods have achieved significant improvements in image super-resolution(SR),current CNNbased techniques mainly contain massive parameters and a high computational complexity,limiting their practical applications.In this paper,we present a fast and lightweight framework,named weighted multi-scale residual network(WMRN),for a better tradeoff between SR performance and computational efficiency.With the modified residual structure,depthwise separable convolutions(DS Convs)are employed to improve convolutional operations’efficiency.Furthermore,several weighted multi-scale residual blocks(WMRBs)are stacked to enhance the multi-scale representation capability.In the reconstruction subnetwork,a group of Conv layers are introduced to filter feature maps to reconstruct the final high-quality image.Extensive experiments were conducted to evaluate the proposed model,and the comparative results with several state-of-the-art algorithms demonstrate the effectiveness of WMRN.

Deep-learning-based methods for super-resolution fluorescence microscopy

The algorithm used for reconstruction or resolution enhancement is one of the factors affectingthe quality of super-resolution images obtained by fluorescence microscopy.Deep-learning-basedalgorithms have achieved stateof-the-art performance in super-resolution fluorescence micros-copy and are becoming increasingly attractive.We firstly introduce commonly-used deep learningmodels,and then review the latest applications in terms of the net work architectures,the trainingdata and the loss functions.Additionally,we discuss the challenges and limits when using deeplearning to analyze the fluorescence microscopic data,and suggest ways to improve the reliability and robustness of deep learning applications.

Design of Network Cascade Structure for Image Super-Resolution

Image super resolution is an important field of computer research.The current mainstream image super-resolution technology is to use deep learning to mine the deeper features of the image,and then use it for image restoration.However,most of these models mentioned above only trained the images in a specific scale and do not consider the relationships between different scales of images.In order to utilize the information of images at different scales,we design a cascade network structure and cascaded super-resolution convolutional neural networks.This network contains three cascaded FSRCNNs.Due to each sub FSRCNN can process a specific scale image,our network can simultaneously exploit three scale images,and can also use the information of three different scales of images.Experiments on multiple datasets confirmed that the proposed network can achieve better performance for image SR.

Channel attention based wavelet cascaded network for image super-resolution

Convolutional neural networks(CNNs) have shown great potential for image super-resolution(SR).However,most existing CNNs only reconstruct images in the spatial domain,resulting in insufficient high-frequency details of reconstructed images.To address this issue,a channel attention based wavelet cascaded network for image super-resolution(CWSR) is proposed.Specifically,a second-order channel attention(SOCA) mechanism is incorporated into the network,and the covariance matrix normalization is utilized to explore interdependencies between channel-wise features.Then,to boost the quality of residual features,the non-local module is adopted to further improve the global information integration ability of the network.Finally,taking the image loss in the spatial and wavelet domains into account,a dual-constrained loss function is proposed to optimize the network.Experimental results illustrate that CWSR outperforms several state-of-the-art methods in terms of both visual quality and quantitative metrics.

Image Super-Resolution Reconstruction Based on Dual Residual Network

Research shows that deep learning algorithms can ffectivelyimprove a single image's super-resolution quality.However,if the algorithmis solely focused on increasing network depth and the desired result is not achieved,difficulties in the training process are more likely to arise.Simultaneously,the function space that can be transferred from a iow-resolution image to a high-resolution image is enormous,making finding a satisfactory solution difficult.In this paper,we propose a deep learning method for single image super-resolution.The MDRN network framework uses multi-scale residual blocks and dual learning to fully acquire features in low-resolution images.Finally,these features will be sent to the image reconstruction module torestore high-quality images.The function space is constrained by the closedloop formed by dual learning,which provides additional supervision forthe super-resolution reconstruction of the image.The up-sampling processincludes residual blocks with short-hop connections,so that the networkfocuses on learning high-frequency information,and strives to reconstructimages with richer feature details.The experimental results of ×4 and ×8 super-resolution reconstruction of the image show that the quality of thereconstructed image with this method is better than some existing experimental results of image super-resolution reconstruction in subjective visual ffectsand objective evaluation indicators.

Super-Resolution Stress Imaging for Terahertz-Elastic Based on SRCNN

Limited by diffraction limit, low spatial resolution is one of the shortcomings of terahertz imaging. Low spatial resolution is also one of the reasons limiting the development of stress measurement using terahertz imaging. In this paper, the full-field stress measurement using Terahertz Time Domain Spectroscopy (THz-TDS) is combined with Super-Resolution Convolutional Neural Network (SRCNN) algorithm to obtain stress fields with high spatial resolution. A modulation model from a plane stress state to a THz-TDS signal is constructed. A large number of simulated sets are obtained to train the SRCNN model. By applying the trained SRCNN model to imaging the numerical and physical stress fields, the improved spatial resolution of stress field calculated from the captured THz-TDS signal is obtained.

Super-resolution of PROBA-V images using convolutional neural networks

European Space Aqency(ESA)’s PROBA-V Earth observation(EO)satellite enables us to monitor our planet at a large scale to study the interaction between vegetation and climate,and provides guidance for important decisions on our common global future.However,the interval at which high-resolution images are recorded spans over several days,in contrast to the availability of lower-resolution images which is often daily.We collect an extensive dataset of both high-and low-resolution images taken by PROBA-V instruments during monthly periods to investigate Multi Image Super-resolution,a technique to merge several low-resolution images into one image of higher quality.We propose a convolutional neural network(CNN)that is able to cope with changes in illumination,cloud coverage,and landscape features which are introduced by the fact that the different images are taken over successive satellite passages at the same region.Given a bicubic upscaling of low resolution images taken under optimal conditions,we find the Peak Signal to Noise Ratio of the reconstructed image of the network to be higher for a large majority of different scenes.This shows that applied machine learning has the potential to enhance large amounts of previously collected EO data during multiple satellite passes.

A brief survey on deep learning based image super-resolution

Image super-resolution(SR)is an important technique for improving the resolution and quality of images.With the great progress of deep learning,image super-resolution achieves remarkable improvements recently.In this work,a brief survey on recent advances of deep learning based single image super-resolution methods is systematically described.The existing studies of SR techniques are roughly grouped into ten major categories.Besides,some other important issues are also introduced,such as publicly available benchmark datasets and performance evaluation metrics.Finally,this survey is concluded by highlighting four future trends.

Performance Evaluation of Super-Resolution Methods Using Deep-Learning and Sparse-Coding for Improving the Image Quality of Magnified Images in Chest Radiographs

Purpose: To detect small diagnostic signals such as lung nodules in chest radiographs, radiologists magnify a region-of-interest using linear interpolation methods. However, such methods tend to generate over-smoothed images with artifacts that can make interpretation difficult. The purpose of this study was to investigate the effectiveness of super-resolution methods for improving the image quality of magnified chest radiographs. Materials and Methods: A total of 247 chest X-rays were sampled from the JSRT database, then divided into 93 training cases with non-nodules and 154 test cases with lung nodules. We first trained two types of super-resolution methods, sparse-coding super-resolution (ScSR) and super-resolution convolutional neural network (SRCNN). With the trained super-resolution methods, the high-resolution image was then reconstructed using the super-resolution methods from a low-resolution image that was down-sampled from the original test image. We compared the image quality of the super-resolution methods and the linear interpolations (nearest neighbor and bilinear interpolations). For quantitative evaluation, we measured two image quality metrics: peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). For comparative evaluation of the super-resolution methods, we measured the computation time per image. Results: The PSNRs and SSIMs for the ScSR and the SRCNN schemes were significantly higher than those of the linear interpolation methods (p p p Conclusion: Super-resolution methods provide significantly better image quality than linear interpolation methods for magnified chest radiograph images. Of the two tested schemes, the SRCNN scheme processed the images fastest;thus, SRCNN could be clinically superior for processing radiographs in terms of both image quality and processing speed.

基于改进SRCNN模型的图像超分辨率重构

图像超分辨率重构是指将低分辨率图像生成对应的高分辨率图像,在许多领域有着重要作用。文章在SRCNN方法的基础上,提出了改进模型。首先,在SRCNN基础上使用小卷积代替大卷积。其次,加入残差结构。最后,在前两层网络后加入ReLU激活函数。结果表明,scale为3、4、6、8的PSNR分别提升了0.140 3 dB、0.084 5 dB、0.147 2 dB、0.113 5 dB,模型性能较改进前有所提升。

基于宽深超分辨率网络的信道估计方法

在正交频分复用(Orthogonal Frequency Division Multiplexing,OFDM)系统中由于快衰落导致信道特征不连续,常规的信道插值方法无法准确反应导频与整个信道之间的关联性。针对这一问题,提出了一种基于宽深超分辨率(Wide Deep Super-resolution,WDSR)网络的信道估计方法,把导频值通过最小二乘估计(Least Squares,LS)初步插值,再通过WDSR网络再次放大重构整个信道的响应。将信道估计插值上采样替换成初步插值和图像超分辨率上采样两步。仿真结果表明,与超分辨率卷积神经网络(Super-resolution Convolutional Neural Network,SRCNN)信道估计算法相比,在不同种类的信道以及导频数下WDSR信道估计方法均方误差性能提升约4.6 dB。

A survey for light field super-resolution

Compared to 2D imaging data,the 4D light field(LF)data retains richer scene’s structure information,which can significantly improve the computer’s perception capability,including depth estimation,semantic segmentation,and LF rendering.However,there is a contradiction between spatial and angular resolution during the LF image acquisition period.To overcome the above problem,researchers have gradually focused on the light field super-resolution(LFSR).In the traditional solutions,researchers achieved the LFSR based on various optimization frameworks,such as Bayesian and Gaussian models.Deep learning-based methods are more popular than conventional methods because they have better performance and more robust generalization capabilities.In this paper,the present approach can mainly divided into conventional methods and deep learning-based methods.We discuss these two branches in light field spatial super-resolution(LFSSR),light field angular super-resolution(LFASR),and light field spatial and angular super-resolution(LFSASR),respectively.Subsequently,this paper also introduces the primary public datasets and analyzes the performance of the prevalent approaches on these datasets.Finally,we discuss the potential innovations of the LFSR to propose the progress of our research field.

Information Purification Network for Remote Sensing Image Super-Resolution

Recently,several well-performing deep convolutional neural networks were proposed for remote sensing image super-resolution(SR).However,these methods rarely consider that remote sensing images are corruptible by additional noise,blurring,and other factors.Therefore,to eliminate the interference of these factors,especially the noise,we propose a novel information purification network(IPN)for remote sensing image SR.The proposed information purification block(IPB)can process channel-wise features differently by channel separation and rescale spatial-wise features adaptively through the proposed multi-scale spatial attention mechanism.We further design an information group to explore a more powerful expressive combination of IPBs.Moreover,long and short skip connections can transmit abundant low-frequency information,making IPBs pay more attention to high-frequency information.We mix the images under various degradation models as training data in the training phase.In this way,the network can directly reconstruct various degraded images.Experiments on AID and UC Merced Land-Use datasets under multiple degradation models demonstrate that the proposed IPN performs better than state-of-the-art methods.

RFCNet:Remote Sensing Image Super-Resolution Using Residual Feature Calibration Network

In the field of single remote sensing image Super-Resolution(SR),deep Convolutional Neural Networks(CNNs)have achieved top performance.To further enhance convolutional module performance in processing remote sensing images,we construct an efficient residual feature calibration block to generate expressive features.After harvesting residual features,we first divide them into two parts along the channel dimension.One part flows to the Self-Calibrated Convolution(SCC)to be further refined,and the other part is rescaled by the proposed Two-Path Channel Attention(TPCA)mechanism.SCC corrects local features according to their expressions under the deep receptive field,so that the features can be refined without increasing the number of calculations.The proposed TPCA uses the means and variances of feature maps to obtain accurate channel attention vectors.Moreover,a region-level nonlocal operation is introduced to capture long-distance spatial contextual information by exploring pixel dependencies at the region level.Extensive experiments demonstrate that the proposed residual feature calibration network is superior to other SR methods in terms of quantitative metrics and visual quality.

基于SRCNN的QR二维码-人脸重构算法

针对人脸识别技术存在的缺少生物信息的隐私保护、有很大的信息泄露风险问题,提出基于超分辨率卷积神经网络的QR二维码-人脸重构算法。该算法将获取到的人脸特征信息转化为QR二维码,并生成QR二维码图片,然后将存储的QR二维码图片与人脸特征信息对比,当比对结果达到一定阈值,实现人脸识别。该算法实现了QR二维码与人脸信息的重构,保证了人脸生物信息的准确、快速传递,也提高了人脸识别率,为生物信息的安全性和隐私保护提供了一种有效途径。

Deep Learning Based Single Image Super-resolution:A Survey

Single image super-resolution has attracted increasing attention and has a wide range of applications in satellite imaging, medical imaging, computer vision, security surveillance imaging, remote sensing, objection detection, and recognition. Recently, deep learning techniques have emerged and blossomed, producing " the state-of-the-art” in many domains. Due to their capability in feature extraction and mapping, it is very helpful to predict high-frequency details lost in low-resolution images. In this paper, we give an overview of recent advances in deep learning-based models and methods that have been applied to single image super-resolution tasks. We also summarize, compare and discuss various models from the past and present for comprehensive understanding and finally provide open problems and possible directions for future research.

A Super-resolution Perception-based Incremental Learning Approach for Power System Voltage Stability Assessment with Incomplete PMU Measurements

This paper develops a fully data-driven,missingdata tolerant method for post-fault short-term voltage stability(STVS)assessment of power systems against the incomplete PMU measurements.The super-resolution perception(SRP),based on a deep residual learning convolutional neural network,is employed to cope with the missing PMU measurements.The incremental broad learning(BL)is used to rapidly update the model to maintain and enhance the online application performance.Being different from the state-of-the-art methods,the proposed method is fully data-driven and can fill up missing data under any PMU placement information loss and network topology change scenario.Simulation results demonstrate that the proposed method has the best performance in terms of STVS assessment accuracy and missing-data tolerance among the existing methods on the benchmark testing system.