Asymmetric Loss Based on Image Properties for Deep Learning-Based Image Restoration

Deep learning techniques have significantly improved image restoration tasks in recent years.As a crucial compo-nent of deep learning,the loss function plays a key role in network optimization and performance enhancement.However,the currently prevalent loss functions assign equal weight to each pixel point during loss calculation,which hampers the ability to reflect the roles of different pixel points and fails to exploit the image’s characteristics fully.To address this issue,this study proposes an asymmetric loss function based on the image and data characteristics of the image recovery task.This novel loss function can adjust the weight of the reconstruction loss based on the grey value of different pixel points,thereby effectively optimizing the network training by differentially utilizing the grey information from the original image.Specifically,we calculate a weight factor for each pixel point based on its grey value and combine it with the reconstruction loss to create a new loss function.This ensures that pixel points with smaller grey values receive greater attention,improving network recovery.In order to verify the effectiveness of the proposed asymmetric loss function,we conducted experimental tests in the image super-resolution task.The experimental results show that the model with the introduction of asymmetric loss weights improves all the indexes of the processing results without increasing the training time.In the typical super-resolution network SRCNN,by introducing asymmetric weights,it is possible to improve the peak signal-to-noise ratio(PSNR)by up to about 0.5%,the structural similarity index(SSIM)by up to about 0.3%,and reduce the root-mean-square error(RMSE)by up to about 1.7%with essentially no increase in training time.In addition,we also further tested the performance of the proposed method in the denoising task to verify the potential applicability of the method in the image restoration task.

Model-based deep learning for fiber bundle infrared image restoration

As the representative of flexibility in optical imaging media,in recent years,fiber bundles have emerged as a promising architecture in the development of compact visual systems.Dedicated to tackling the problems of universal honeycomb artifacts and low signal-to-noise ratio(SNR)imaging in fiber bundles,the iterative super-resolution reconstruction network based on a physical model is proposed.Under the constraint of solving the two subproblems of data fidelity and prior regularization term alternately,the network can efficiently“regenerate”the lost spatial resolution with deep learning.By building and calibrating a dual-path imaging system,the real-world dataset where paired low-resolution(LR)-high-resolution(HR)images on the same scene can be generated simultaneously.Numerical results on both the United States Air Force(USAF)resolution target and complex target objects demonstrate that the algorithm can restore high-contrast images without pixilated noise.On the basis of super-resolution reconstruction,compound eye image composition based on fiber bundle is also embedded in this paper for the actual imaging requirements.The proposed work is the first to apply a physical model-based deep learning network to fiber bundle imaging in the infrared band,effectively promoting the engineering application of thermal radiation detection.

Preconditioned Bminpert Algorithms for Matrix Equation AX = B and Their Applications in Color Image Restoration

The purpose of this paper is to show the preconditioned BMinPert algorithm and analyse the practical implementation. Then a posteriori backward error for BGMRES is given. Furthermore, we discuss their applications in color image restoration. The key differences between BMinPert and other methods such as BFGMRES-S(m, pf), GsGMRES and BGMRES are illustrated with numerical experiments which expound the advantages of BMinPert in the presence of sensitive data with ill-condition problems.

A METHOD TO APPROACH OPTIMAL RESTORATION IN IMAGE RESTORATION PROBLEMS WITHOUT NOISE ENERGY INFORMATION

This paper proposes a new image restoration technique, in which the resulting regularized image approximates the optimal solution steadily. The affect of the regular-ization operator and parameter on the lower band and upper band energy of the residue of the regularized image is theoretically analyzed by employing wavelet transform. This paper shows that regularization operator should generally be lowstop and highpass. So this paper chooses a lowstop and highpass operator as regularization operator, and construct an optimization model which minimizes the mean squares residue of regularized solution to determine regularization parameter. Although the model is random, on the condition of this paper, it can be solved and yields regularization parameter and regularized solution. Otherwise, the technique has a mechanism to predict noise energy. So, without noise information, it can also work and yield good restoration results.

Wavelet inverse scale space for image restoration

This paper proposes a model for image restoration by combining the wavelet shrinkage and inverse scale space （ISS） method. The ISS is applied to the wavelet representation to modify the retained wavelet coefficients, and the coefficients smaller than the threshold are set to zero. The curvature term of the ISS can remove the edge artifacts and preserve sharp edges. For the multiscale interpretation of the ISS and the multiscale property of the wavelet representation, small details are preserved. This paper illustrates that the wavelet ISS model can be deduced from the wavelet based on a total variation minimization problem. A stopping criterion is obtained from this minimization in the sense of the Bregman distance in the wavelet domain. Numerical examples show the improvement for the image denoising with the proposed method in the sense of the signal to noise ratio and with fewer details remained in the residue.

Dark channel prior based blurred image restoration method using total variation and morphology

The blurred image restoration method can dramatically highlight the image details and enhance the global contrast, which is of benefit to improvement of the visual effect during practical ap- plications. This paper is based on the dark channel prior principle and aims at the prior information absent blurred image degradation situation. A lot of improvements have been made to estimate the transmission map of blurred images. Since the dark channel prior principle can effectively restore the blurred image at the cost of a large amount of computation, the total variation （TV） and image morphology transform （specifically top-hat transform and bottom- hat transform） have been introduced into the improved method. Compared with original transmission map estimation methods, the proposed method features both simplicity and accuracy. The es- timated transmission map together with the element can restore the image. Simulation results show that this method could inhibit the ill-posed problem during image restoration, meanwhile it can greatly improve the image quality and definition.

Novel image restoration model coupling gradient fidelity term based on adaptive total variation

A novel image restoration model coupling with a gradient fidelity term based on adaptive total variation is proposed in this paper. In order to choose proper parameters, the selection criteria were analyzed theoretically, and a simple scheme to demonstrate its validity was adopted experimentally. To make fair comparisons of performances of three models, the same numerical algorithm was used to solve partial differential equations. Both the international standard test image on Lena and HR image of CBERS-02B of Dalian city were used to verify the performance of the model. Experimental results illustrate that the new model not only preserved the edge and important details but also alleviated the staircase effect effectively.

Robust Core Tensor Dictionary Learning with Modified Gaussian Mixture Model for Multispectral Image Restoration

The multispectral remote sensing image(MS-RSI)is degraded existing multi-spectral camera due to various hardware limitations.In this paper,we propose a novel core tensor dictionary learning approach with the robust modified Gaussian mixture model for MS-RSI restoration.First,the multispectral patch is modeled by three-order tensor and high-order singular value decomposition is applied to the tensor.Then the task of MS-RSI restoration is formulated as a minimum sparse core tensor estimation problem.To improve the accuracy of core tensor coding,the core tensor estimation based on the robust modified Gaussian mixture model is introduced into the proposed model by exploiting the sparse distribution prior in image.When applied to MS-RSI restoration,our experimental results have shown that the proposed algorithm can better reconstruct the sharpness of the image textures and can outperform several existing state-of-the-art multispectral image restoration methods in both subjective image quality and visual perception.

A Fast Filling Algorithm for Image Restoration Based on Contour Parity

Filling techniques are often used in the restoration of images.Yet the existing filling technique approaches either have high computational costs or present problems such as filling holes redundantly.This paper proposes a novel algorithm for filling holes and regions of the images.The proposed algorithm combines the advantages of both the parity-check filling approach and the region-growing inpainting technique.Pairing points of the region’s boundary are used to search and to fill the region.The scanning range of the filling method is within the target regions.The proposed method does not require additional working memory or assistant colors,and it can correctly fill any complex contours.Experimental results show that,compared to other approaches,the proposed algorithm fills regions faster and with lower computational cost.

Single foggy image restoration based on spatial correlation analysis of dark channel prior

Focusing on the degradation of foggy images, a restora- tion approach from a single image based on spatial correlation of dark channel prior is proposed. Firstly, the transmission of each pixel is estimated by the spatial correlation of dark channel prior. Secondly, a degradation model is utilized to restore the foggy image. Thirdly, the final recovered image, with enhanced contrast, is obtained by performing a post-processing technique based on just-noticeable difference. Experimental results demonstrate that the information of a foggy image can be recovered perfectly by the proposed method, even in the case of the abrupt depth changing scene.

Motion-Blurred Image Restoration Based on Joint Invertibility of PSFs

To implement restoration in a single motion blurred image,the PSF(Point Spread Function)is difficult to estimate and the image deconvolution is ill-posed as a result that a good recovery effect cannot be obtained.Considering that several different PSFs can get joint invertibility to make restoration wellposed,we proposed a motion-blurred image restoration method based on joint invertibility of PSFs by means of computational photography.Firstly,we designed a set of observation device which composed by multiple cameras with the same parameters to shoot the moving target in the same field of view continuously to obtain the target images with the same background.The target images have the same brightness,but different exposure time and different motion blur length.It is easy to estimate the blur PSFs of the target images make use of the sequence frames obtained by one camera.According to the motion blur superposition feature of the target and its background,the complete blurred target images can be extracted from the observed images respectively.Finally,for the same target images with different PSFs,the iterative restoration is solved by joint solution of multiple images in spatial domain.The experiments showed that the moving target observation device designed by this method had lower requirements on hardware conditions,and the observed images are more convenient to use joint-PSF solution for image restoration,and the restoration results maintained details well and had lower signal noise ratio(SNR).

Application of improved BPNN in image restoration-learning coefficient

A new method of artificial intelligence based on a new improved back propagation neural network （BPNN） algorithm is partially applied in the problem of image restoration. In order to over- come the inherited issues in conventional back propagation algorithm i.e. slow convergence rate, longer training time, hard to achieve global minima etc. , different methods have been used including the introduction of dynamic learning rate and dynamic momentum coefficient etc. With the passage of time different techniques has been used to improve the dynamicity of these coefficients. The meth- od applied in this paper improves the effect of learning coefficient η by using a new way to modify the value dynamically during learning process. The experimental results show that this helps in im- proving the efficiency overall both in visual effect and quality analysis.

Improved Multi-channel Blind Image Restoration Algorithm for UWB Radar

The imaging problem of low signal to noise ratio （SNR）echo is very important for ultra-wide band （UWB） through-wall radar. An improved multi-channel blind image restoration algorithm based on sub-space and constrained least square （CLS） is presented and applied to UWB radar system to deal with this issue. The high resolution of radar image is equivalent to multi-channel blind image restoration based on the improved model of the through-wall radar echo. And a new cost function is proposed to the multi-channel blind image restoration by considering the concept of sub-space as the limitation of blur identification. The proposed algorithm has all advantages of CLS and sub-space, and converts the image estimation to alternating-minimizing the two cost functions. Experimental results prove that the proposed algorithm is effective at improving the resolution of radar image even at low SNR.

Ultrasonic image restoration based on support vector machine for surfacing interface testing

In order to restore the degraded ultrasonic C-scan image for testing surfacing inteoface, a method based on support vector regression （SVR） network is proposed. By using the image of a simulating defect, the network is trained and a mapping relationship between the degraded and restored image is founded. The degraded C-scan image of Cu-Steel surfacing inteoface is processed by the trained network and improved image is obtained. The result shows that the method can effectively suppress the noise and deblur the defect edge in the image, and provide technique support for quality and reliability evaluation of the surfacing weld.

Image Restoration Using Hybrid Features Improvement on Morphological Component Analysis

Images are generally corrupted by impulse noise during acquisition and transmission.Noise deteriorates the quality of images.To remove corruption noise,we propose a hybrid approach to restoring a random noisecorrupted image,including a block matching 3D(BM3D)method,an adaptive non-local mean(ANLM)scheme,and the K-singular value decomposition(K-SVD)algorithm.In the proposed method,we employ the morphological component analysis(MCA)to decompose an image into the texture,structure,and edge parts.Then,the BM3D method,ANLM scheme,and K-SVD algorithm are utilized to eliminate noise in the texture,structure,and edge parts of the image,respectively.Experimental results show that the proposed approach can effectively remove interference random noise in different parts;meanwhile,the deteriorated image is able to be reconstructed well.

Greyscale based learning in BPNN for image restoration problem

A new method of back propagation learning with respect to the problem of image restora- tion which is named as greyscale based learning in back propagation neural networks （BPNN） is in- vestigated. It is observed that by using this method the value of mean square error （MSE） decreases significantly. In addition, this method also gives good visual results when it is applied in image resto- ration problem. This method is also useful to tackle the inherited drawback of falling into local mini- ma by reducing its effect on overall system by bifurcating the learning locally different for different grey scale values. The performance of this algorithm has been studied in detail with different combi- nations of weights. In short, this algorithm provides much better results especially when compared with the simple back propagation algorithm with any further enhancements and without going for hy- brid solutions.

A New Method of Image Restoration Technology Based on WGAN

With the development of image restoration technology based on deep learning,more complex problems are being solved,especially in image semantic inpainting based on context.Nowadays,image semantic inpainting techniques are becoming more mature.However,due to the limitations of memory,the instability of training,and the lack of sample diversity,the results of image restoration are still encountering difficult problems,such as repairing the content of glitches which cannot be well integrated with the original image.Therefore,we propose an image inpainting network based on Wasserstein generative adversarial network(WGAN)distance.With the corresponding technology having been adjusted and improved,we attempted to use the Adam algorithm to replace the traditional stochastic gradient descent,and another algorithm to optimize the training used in recent years.We evaluated our algorithm on the ImageNet dataset.We obtained high-quality restoration results,indicating that our algorithm improves the clarity and consistency of the image.

Weighted Nuclear Norm Minimization-Based Regularization Method for Image Restoration

Regularization methods have been substantially applied in image restoration due to the ill-posedness of the image restoration problem.Different assumptions or priors on images are applied in the construction of image regularization methods.In recent years,matrix low-rank approximation has been successfully introduced in the image denoising problem and significant denoising effects have been achieved.Low-rank matrix minimization is an NP-hard problem and it is often replaced with the matrix’s weighted nuclear norm minimization(WNNM).The assumption that an image contains an extensive amount of self-similarity is the basis for the construction of the matrix low-rank approximation-based image denoising method.In this paper,we develop a model for image restoration using the sum of block matching matrices’weighted nuclear norm to be the regularization term in the cost function.An alternating iterative algorithm is designed to solve the proposed model and the convergence analyses of the algorithm are also presented.Numerical experiments show that the proposed method can recover the images much better than the existing regularization methods in terms of both recovered quantities and visual qualities.

An overview of image restoration based on variational regularization

Image restoration is a complicated process in which the original information can be recovered from the degraded image model caused by lots of factors.Mathematically,image restoration problems are ill-posed inverse prob-lems.In this paper image restoration models and algorithms based on variational regularization are surveyed.First,we review and analyze the typical models for denoising,deblurring and inpainting.Second,we construct a unified restoration model based on variational regularization and summarize the typical numerical methods for the model.At last,we point out eight diffcult problems which remain open in this field.

Using restored two-dimensional X-ray images to reconstruct the three-dimensional magnetopause

Astronomical imaging technologies are basic tools for the exploration of the universe,providing basic data for the research of astronomy and space physics.The Soft X-ray Imager(SXI)carried by the Solar wind Magnetosphere Ionosphere Link Explorer(SMILE)aims to capture two-dimensional(2-D)images of the Earth’s magnetosheath by using soft X-ray imaging.However,the observed 2-D images are affected by many noise factors,destroying the contained information,which is not conducive to the subsequent reconstruction of the three-dimensional(3-D)structure of the magnetopause.The analysis of SXI-simulated observation images shows that such damage cannot be evaluated with traditional restoration models.This makes it difficult to establish the mapping relationship between SXIsimulated observation images and target images by using mathematical models.We propose an image restoration algorithm for SXIsimulated observation images that can recover large-scale structure information on the magnetosphere.The idea is to train a patch estimator by selecting noise–clean patch pairs with the same distribution through the Classification–Expectation Maximization algorithm to achieve the restoration estimation of the SXI-simulated observation image,whose mapping relationship with the target image is established by the patch estimator.The Classification–Expectation Maximization algorithm is used to select multiple patch clusters with the same distribution and then train different patch estimators so as to improve the accuracy of the estimator.Experimental results showed that our image restoration algorithm is superior to other classical image restoration algorithms in the SXI-simulated observation image restoration task,according to the peak signal-to-noise ratio and structural similarity.The restoration results of SXI-simulated observation images are used in the tangent fitting approach and the computed tomography approach toward magnetospheric reconstruction techniques,significantly improving the reconstruction results.Hence,the proposed technology may be feasible for processing SXI-simulated observation images.