基于CNN与ELM的二次超分辨率重构方法研究

Two-Tie Image Super-Resolution Based on CNN and ELM

为了实现将低分辨率图像重构为高分辨率图像,弥补高、低分辨率图像间信息损失,文中提出了卷积神经网络与极限学习机结合的二次超分辨率重构方法.首先通过基于深度学习的超分辨率重构优化方法,快速训练端对端的卷积神经网络重构模型,学习结构化的图像信息;然后采用像素级的特征提取,并采用极限学习机模型对图像进行高频分量的补充,通过二次重构获得具有更好视觉效果的高分辨率图像.实验结果表明,文中的优化方法将原有卷积神经网络重构模型的训练效率提高了3个数量级,重构效果在主观和客观评估中均优于当前代表性的超分辨率重构方法.

With the rapid proliferation of information technology,there is a growing requirement for high quality images and videos.High-resolution images can offer more abundant details,which can not only satisfy people’s need for visual effect,also lay a solid foundation of implementing other visual analysis tasks.Image super-resolution is proven to be an effective method to provide high-resolution images.The key point of image super-resolution is to find the mapping relation and complementation information between low and high quality images and search the feasible solution space using this ill-posed problem.In order to reconstruct a high-resolution image from a low-resolution one,complementary information between low and high quality images,we propose a two-tie image super-resolution method combining CNN(Convolutional Neural Networks)and ELM(Extreme Learning Machines).At first,we establish an end-to-end CNN reconstruction model using an improved deep learning method,which can learn the structural image information.Then,we perform pixel-level feature extraction,where we use the high-frequency information learned by ELM to complement the lost component,thus fine-visual high-resolution images can be obtained after the second-time reconstruction.The main work and contributions of this paper are as follows:(1)An improved image super-resolution method based on deep learning.We make the following improvements on existing deep learning based high-resolution methods.First,the training data of CNN are processed according to their respective structural features.We utilize ISODATA algorithm to conduct clustering on the images after Sobel filtering in order to obtain two classes of training image sets,one of them is more complex and the other tends to be smooth.Then,we combine pre-training and fine-tuning strategies to train the network.In this work we use complicated images for pre-training and the whole training data set for fine-tuning.In the end,we make use of smaller scale parameters network to increase the training speed of model.Experimental results show that our improved model achieves the same super-resolution construction effect while only takes one thousandth iteration times compared to the original model[26],making the training phase more efficient.(2)A framework combining CNN and ELM to perform rapid two-tie reconstruction.To improve the image quality after CNN reconstruction,we perform pixel-wise feature extraction on those images.We train the ELM model with a smaller upscale factor than the global zoom factor and get the high-frequency components of low-resolution images.After that,we combine those components with the results from CNN based on their weights.Thus the two-tie image reconstruction can be implemented to get the ultimate high-resolution images.In addition,we also develop a demo which is capable of making visual improvements on original low-resolution text images based on the proposed method,and it can be deployed as a function of a remote immersive interaction system to break the limitation of low-resolution camera sensor,and perform the transmission of high-resolution text images.We perform sufficient experimental to demonstrate characteristics of proposed method and the results show that,comparing with the original work,our improved model make the training phase of CNN model more efficient,and the proposed method achieves better performance on majority of datasets compared to the state-of-the-arts.