Restoration of the JPEG Maximum Lossy Compressed Face Images with Hourglass Block-GAN

In the context of high compression rates applied to Joint Photographic Experts Group(JPEG)images through lossy compression techniques,image-blocking artifacts may manifest.This necessitates the restoration of the image to its original quality.The challenge lies in regenerating significantly compressed images into a state in which these become identifiable.Therefore,this study focuses on the restoration of JPEG images subjected to substantial degradation caused by maximum lossy compression using Generative Adversarial Networks(GAN).The generator in this network is based on theU-Net architecture.It features a newhourglass structure that preserves the characteristics of the deep layers.In addition,the network incorporates two loss functions to generate natural and high-quality images:Low Frequency(LF)loss and High Frequency(HF)loss.HF loss uses a pretrained VGG-16 network and is configured using a specific layer that best represents features.This can enhance the performance in the high-frequency region.In contrast,LF loss is used to handle the low-frequency region.The two loss functions facilitate the generation of images by the generator,which can mislead the discriminator while accurately generating high-and low-frequency regions.Consequently,by removing the blocking effects frommaximum lossy compressed images,images inwhich identities could be recognized are generated.This study represents a significant improvement over previous research in terms of the image resolution performance.

PSMFNet:Lightweight Partial Separation and Multiscale Fusion Network for Image Super-Resolution

The employment of deep convolutional neural networks has recently contributed to significant progress in single image super-resolution(SISR)research.However,the high computational demands of most SR techniques hinder their applicability to edge devices,despite their satisfactory reconstruction performance.These methods commonly use standard convolutions,which increase the convolutional operation cost of the model.In this paper,a lightweight Partial Separation and Multiscale Fusion Network(PSMFNet)is proposed to alleviate this problem.Specifically,this paper introduces partial convolution(PConv),which reduces the redundant convolution operations throughout the model by separating some of the features of an image while retaining features useful for image reconstruction.Additionally,it is worth noting that the existing methods have not fully utilized the rich feature information,leading to information loss,which reduces the ability to learn feature representations.Inspired by self-attention,this paper develops a multiscale feature fusion block(MFFB),which can better utilize the non-local features of an image.MFFB can learn long-range dependencies from the spatial dimension and extract features from the channel dimension,thereby obtaining more comprehensive and rich feature information.As the role of the MFFB is to capture rich global features,this paper further introduces an efficient inverted residual block(EIRB)to supplement the local feature extraction ability of PSMFNet.A comprehensive analysis of the experimental results shows that PSMFNet maintains a better performance with fewer parameters than the state-of-the-art models.

Pyramid Separable Channel Attention Network for Single Image Super-Resolution

Single Image Super-Resolution(SISR)technology aims to reconstruct a clear,high-resolution image with more information from an input low-resolution image that is blurry and contains less information.This technology has significant research value and is widely used in fields such as medical imaging,satellite image processing,and security surveillance.Despite significant progress in existing research,challenges remain in reconstructing clear and complex texture details,with issues such as edge blurring and artifacts still present.The visual perception effect still needs further enhancement.Therefore,this study proposes a Pyramid Separable Channel Attention Network(PSCAN)for the SISR task.Thismethod designs a convolutional backbone network composed of Pyramid Separable Channel Attention blocks to effectively extract and fuse multi-scale features.This expands the model’s receptive field,reduces resolution loss,and enhances the model’s ability to reconstruct texture details.Additionally,an innovative artifact loss function is designed to better distinguish between artifacts and real edge details,reducing artifacts in the reconstructed images.We conducted comprehensive ablation and comparative experiments on the Arabidopsis root image dataset and several public datasets.The experimental results show that the proposed PSCAN method achieves the best-known performance in both subjective visual effects and objective evaluation metrics,with improvements of 0.84 in Peak Signal-to-Noise Ratio(PSNR)and 0.017 in Structural Similarity Index(SSIM).This demonstrates that the method can effectively preserve high-frequency texture details,reduce artifacts,and have good generalization performance.

Multi-prior physics-enhanced neural network enables pixel super-resolution and twin-imagefree phase retrieval from single-shot hologram

Digital in-line holographic microscopy(DIHM)is a widely used interference technique for real-time reconstruction of living cells’morphological information with large space-bandwidth product and compact setup.However,the need for a larger pixel size of detector to improve imaging photosensitivity,field-of-view,and signal-to-noise ratio often leads to the loss of sub-pixel information and limited pixel resolution.Additionally,the twin-image appearing in the reconstruction severely degrades the quality of the reconstructed image.The deep learning(DL)approach has emerged as a powerful tool for phase retrieval in DIHM,effectively addressing these challenges.However,most DL-based strategies are datadriven or end-to-end net approaches,suffering from excessive data dependency and limited generalization ability.Herein,a novel multi-prior physics-enhanced neural network with pixel super-resolution(MPPN-PSR)for phase retrieval of DIHM is proposed.It encapsulates the physical model prior,sparsity prior and deep image prior in an untrained deep neural network.The effectiveness and feasibility of MPPN-PSR are demonstrated by comparing it with other traditional and learning-based phase retrieval methods.With the capabilities of pixel super-resolution,twin-image elimination and high-throughput jointly from a single-shot intensity measurement,the proposed DIHM approach is expected to be widely adopted in biomedical workflow and industrial measurement.

Using restored two-dimensional X-ray images to reconstruct the three-dimensional magnetopause

Astronomical imaging technologies are basic tools for the exploration of the universe,providing basic data for the research of astronomy and space physics.The Soft X-ray Imager(SXI)carried by the Solar wind Magnetosphere Ionosphere Link Explorer(SMILE)aims to capture two-dimensional(2-D)images of the Earth’s magnetosheath by using soft X-ray imaging.However,the observed 2-D images are affected by many noise factors,destroying the contained information,which is not conducive to the subsequent reconstruction of the three-dimensional(3-D)structure of the magnetopause.The analysis of SXI-simulated observation images shows that such damage cannot be evaluated with traditional restoration models.This makes it difficult to establish the mapping relationship between SXIsimulated observation images and target images by using mathematical models.We propose an image restoration algorithm for SXIsimulated observation images that can recover large-scale structure information on the magnetosphere.The idea is to train a patch estimator by selecting noise–clean patch pairs with the same distribution through the Classification–Expectation Maximization algorithm to achieve the restoration estimation of the SXI-simulated observation image,whose mapping relationship with the target image is established by the patch estimator.The Classification–Expectation Maximization algorithm is used to select multiple patch clusters with the same distribution and then train different patch estimators so as to improve the accuracy of the estimator.Experimental results showed that our image restoration algorithm is superior to other classical image restoration algorithms in the SXI-simulated observation image restoration task,according to the peak signal-to-noise ratio and structural similarity.The restoration results of SXI-simulated observation images are used in the tangent fitting approach and the computed tomography approach toward magnetospheric reconstruction techniques,significantly improving the reconstruction results.Hence,the proposed technology may be feasible for processing SXI-simulated observation images.

Hyperspectral Image Super-Resolution Meets Deep Learning:A Survey and Perspective

Hyperspectral image super-resolution,which refers to reconstructing the high-resolution hyperspectral image from the input low-resolution observation,aims to improve the spatial resolution of the hyperspectral image,which is beneficial for subsequent applications.The development of deep learning has promoted significant progress in hyperspectral image super-resolution,and the powerful expression capabilities of deep neural networks make the predicted results more reliable.Recently,several latest deep learning technologies have made the hyperspectral image super-resolution method explode.However,a comprehensive review and analysis of the latest deep learning methods from the hyperspectral image super-resolution perspective is absent.To this end,in this survey,we first introduce the concept of hyperspectral image super-resolution and classify the methods from the perspectives with or without auxiliary information.Then,we review the learning-based methods in three categories,including single hyperspectral image super-resolution,panchromatic-based hyperspectral image super-resolution,and multispectral-based hyperspectral image super-resolution.Subsequently,we summarize the commonly used hyperspectral dataset,and the evaluations for some representative methods in three categories are performed qualitatively and quantitatively.Moreover,we briefly introduce several typical applications of hyperspectral image super-resolution,including ground object classification,urban change detection,and ecosystem monitoring.Finally,we provide the conclusion and challenges in existing learning-based methods,looking forward to potential future research directions.

Super-resolution Restoration of Remote-sensing Images

A novel image restoration scheme, which is super-resolution image restoration algorithm Poisson-maximum-afterword-probability based on Markvo constraint (MPMAP) combined with evaluating image detail parameter D, has been proposed. The advantage of super-resolution algorithm MPMAP incorporated with parameter D lies in the fact that super-resolution algorithm MPMAP model is discrete, which is in accordance with remote-sensing imaging model, and the algorithm MPMAP is proved applicable to linear and non-linear imaging models with a unique solution when noise is not severe. According to simulation experiments for practical images, super-resolution algorithm MPMAP can retain image details better than most of traditional restoration methods; at the same time, the proposed parameter D can help to identify real point spread function (PSF) value of degradation process. Processing result of practical remote-sensing images by MPMAP combined with parameter D are given, it illustrates that MPMAP restoration scheme combined PSF estimation has a better restoration result than that of Photoshop processing, based on the same original images. It is proved that the proposed scheme is helpful to offset the lack of resolution of the original remote-sensing images and has its extensive application foreground.

Model-based deep learning for fiber bundle infrared image restoration

As the representative of flexibility in optical imaging media,in recent years,fiber bundles have emerged as a promising architecture in the development of compact visual systems.Dedicated to tackling the problems of universal honeycomb artifacts and low signal-to-noise ratio(SNR)imaging in fiber bundles,the iterative super-resolution reconstruction network based on a physical model is proposed.Under the constraint of solving the two subproblems of data fidelity and prior regularization term alternately,the network can efficiently“regenerate”the lost spatial resolution with deep learning.By building and calibrating a dual-path imaging system,the real-world dataset where paired low-resolution(LR)-high-resolution(HR)images on the same scene can be generated simultaneously.Numerical results on both the United States Air Force(USAF)resolution target and complex target objects demonstrate that the algorithm can restore high-contrast images without pixilated noise.On the basis of super-resolution reconstruction,compound eye image composition based on fiber bundle is also embedded in this paper for the actual imaging requirements.The proposed work is the first to apply a physical model-based deep learning network to fiber bundle imaging in the infrared band,effectively promoting the engineering application of thermal radiation detection.

IRMIRS:Inception-ResNet-Based Network for MRI Image Super-Resolution

Medical image super-resolution is a fundamental challenge due to absorption and scattering in tissues.These challenges are increasing the interest in the quality of medical images.Recent research has proven that the rapid progress in convolutional neural networks(CNNs)has achieved superior performance in the area of medical image super-resolution.However,the traditional CNN approaches use interpolation techniques as a preprocessing stage to enlarge low-resolution magnetic resonance(MR)images,adding extra noise in the models and more memory consumption.Furthermore,conventional deep CNN approaches used layers in series-wise connection to create the deeper mode,because this later end layer cannot receive complete information and work as a dead layer.In this paper,we propose Inception-ResNet-based Network for MRI Image Super-Resolution known as IRMRIS.In our proposed approach,a bicubic interpolation is replaced with a deconvolution layer to learn the upsampling filters.Furthermore,a residual skip connection with the Inception block is used to reconstruct a high-resolution output image from a low-quality input image.Quantitative and qualitative evaluations of the proposed method are supported through extensive experiments in reconstructing sharper and clean texture details as compared to the state-of-the-art methods.

Asymmetric Loss Based on Image Properties for Deep Learning-Based Image Restoration

Deep learning techniques have significantly improved image restoration tasks in recent years.As a crucial compo-nent of deep learning,the loss function plays a key role in network optimization and performance enhancement.However,the currently prevalent loss functions assign equal weight to each pixel point during loss calculation,which hampers the ability to reflect the roles of different pixel points and fails to exploit the image’s characteristics fully.To address this issue,this study proposes an asymmetric loss function based on the image and data characteristics of the image recovery task.This novel loss function can adjust the weight of the reconstruction loss based on the grey value of different pixel points,thereby effectively optimizing the network training by differentially utilizing the grey information from the original image.Specifically,we calculate a weight factor for each pixel point based on its grey value and combine it with the reconstruction loss to create a new loss function.This ensures that pixel points with smaller grey values receive greater attention,improving network recovery.In order to verify the effectiveness of the proposed asymmetric loss function,we conducted experimental tests in the image super-resolution task.The experimental results show that the model with the introduction of asymmetric loss weights improves all the indexes of the processing results without increasing the training time.In the typical super-resolution network SRCNN,by introducing asymmetric weights,it is possible to improve the peak signal-to-noise ratio(PSNR)by up to about 0.5%,the structural similarity index(SSIM)by up to about 0.3%,and reduce the root-mean-square error(RMSE)by up to about 1.7%with essentially no increase in training time.In addition,we also further tested the performance of the proposed method in the denoising task to verify the potential applicability of the method in the image restoration task.

Preconditioned Bminpert Algorithms for Matrix Equation AX = B and Their Applications in Color Image Restoration

The purpose of this paper is to show the preconditioned BMinPert algorithm and analyse the practical implementation. Then a posteriori backward error for BGMRES is given. Furthermore, we discuss their applications in color image restoration. The key differences between BMinPert and other methods such as BFGMRES-S(m, pf), GsGMRES and BGMRES are illustrated with numerical experiments which expound the advantages of BMinPert in the presence of sensitive data with ill-condition problems.

A generalized deep neural network approach for improving resolution of fluorescence microscopy images

Deep learning is capable of greatly promoting the progress of super-resolution imaging technology in terms of imaging and reconstruction speed,imaging resolution,and imagingflux.This paper proposes a deep neural network based on a generative adversarial network(GAN).The generator employs a U-Net-based network,which integrates Dense Net for the downsampling component.The proposed method has excellent properties,for example,the network model is trained with several different datasets of biological structures;the trained model can improve the imaging resolution of different microscopy imaging modalities such as confocal imaging and wide-field imaging;and the model demonstrates a generalized ability to improve the resolution of different biological structures even out of the datasets.In addition,experimental results showed that the method improved the resolution of caveolin-coated pits(CCPs)structures from 264 nm to 138 nm,a 1.91-fold increase,and nearly doubled the resolution of DNA molecules imaged while being transported through microfluidic channels.

Restoration of space-variant blurred image based on motion-blurred target segmentation

In imaging on moving target, it is easy to get space- variant blurred image. In order to recover the image and gain recognizable target, an approach to recover the space-variant blurred image is presented based on image segmentation. Be- cause of motion blur＇s convolution process, the pixels of observed image＇s target and background will be displaced and piled up to produce two superposition regions. As a result, the neighbor- ing pixels in the superposition regions will have similar grey level change. According to the pixel＇s motion-blur character, the target＇s blurred edge of superposition region could be detected. Canny operator can be recurred to detect the target edge which parallels the motion blur direction. Then in the segmentation process, the whole target image which has the character of integral convolution between motion blur and real target image can be obtained. At last, the target image is restored by deconvolution algorithms with adding zeros. The restoration result indicates that the approach can effectively solve the kind of problem of space-variant motion blurred image restoration.

A METHOD TO APPROACH OPTIMAL RESTORATION IN IMAGE RESTORATION PROBLEMS WITHOUT NOISE ENERGY INFORMATION

This paper proposes a new image restoration technique, in which the resulting regularized image approximates the optimal solution steadily. The affect of the regular-ization operator and parameter on the lower band and upper band energy of the residue of the regularized image is theoretically analyzed by employing wavelet transform. This paper shows that regularization operator should generally be lowstop and highpass. So this paper chooses a lowstop and highpass operator as regularization operator, and construct an optimization model which minimizes the mean squares residue of regularized solution to determine regularization parameter. Although the model is random, on the condition of this paper, it can be solved and yields regularization parameter and regularized solution. Otherwise, the technique has a mechanism to predict noise energy. So, without noise information, it can also work and yield good restoration results.

MAXIMUM ENTROPY RESTORATION METHOD OF LINEARLY DEGRADED BINARY IMAGE

This paper investigates the maximum entropy restoration of blurred binary image.In concerning with the binary property of image,a new maximum entropy restoration methodwith binary constraint is proposed.The properties of existence and uniqueness of solution arediscussed.The problem of maximum of entropy with two constraints is solved and the corre-sponding algorithm is given.In this paper,the maximum bounded entropy principle is employedconcerning the prior knowledge of binary image,and the maximum bounded entropy restora-tion method with binary constraint is put forward.The proposes methods,Wiener filter(WF)restoration method and maximum entropy restoration method are compared.The experimen-tal results show that the maximum entropy restoration method and maximum bounded entropyrestoration method with binary constraint can improve the quality of restored image.

Radial Basis Function Neural Network Based Super- Resolution Restoration for an Undersampled Image

To achieve restoration of high frequency information for an undersampled and degraded low-resolution image, a nonlinear and real-time processing method-the radial basis function (RBF) neural network based super-resolution method of restoration is proposed. The RBF network configuration and processing method is suitable for a high resolution restoration from an undersampled low-resolution image. The soft-competition learning scheme based on the k-means algorithm is used, and can achieve higher mapping approximation accuracy without increase in the network size. Experiments showed that the proposed algorithm can achieve a super-resolution restored image from an undersampled and degraded low-resolution image, and requires a shorter training time when compared with the multiplayer perception (MLP) network.

Blind-restoration-based blind separation method for permuted motion blurred images

A novel single-channel blind separation algorithm for permuted motion blurred images is proposed by using blind restoration in this paper. Both the motion direction and the length of the point spread function （PSF） are estimated by Radon transformation and extrema a detection. Using the estimated blur parameters, the permuted image is restored by performing the L-R blind restoration method. The permutation mixing matrices can be accurately estimated by classifying the ringing effect in the restored image, thereby the source images can be separated. Simulation results show a better separation efficiency for the permuted motion blurred image with various permutation operations. The proposed algorithm indicates a better performance on the robustness against Gaussian noise and lossy JPEG compression.

Super-resolution reconstruction of synthetic-aperture radar image using adaptive-threshold singular value decomposition technique

A super-resolution reconstruction approach of （SVD） technique was presented, and its performance was radar image using an adaptive-threshold singular value decomposition analyzed, compared and assessed detailedly. First, radar imaging model and super-resolution reconstruction mechanism were outlined. Then, the adaptive-threshold SVD super-resolution algorithm, and its two key aspects, namely the determination method of point spread function （PSF） matrix T and the selection scheme of singular value threshold, were presented. Finally, the super-resolution algorithm was demonstrated successfully using the measured synthetic-aperture radar （SAR） images, and a Monte Carlo assessment was carried out to evaluate the performance of the algorithm by using the input/output signal-to-noise ratio （SNR）. Five versions of SVD algorithms, namely 1 ） using all singular values, 2） using the top 80% singular values, 3） using the top 50% singular values, 4） using the top 20% singular values and 5） using singular values s such that S2≥/max（s2）/rinsNR were tested. The experimental results indicate that when the singular value threshold is set as Smax/（rinSNR）1/2, the super-resolution algorithm provides a good compromise between too much noise and too much bias and has good reconstruction results.

Image Super-Resolution Based on Generative Adversarial Networks: A Brief Review

Single image super resolution(SISR)is an important research content in the field of computer vision and image processing.With the rapid development of deep neural networks,different image super-resolution models have emerged.Compared to some traditional SISR methods,deep learning-based methods can complete the super-resolution tasks through a single image.In addition,compared with the SISR methods using traditional convolutional neural networks,SISR based on generative adversarial networks(GAN)has achieved the most advanced visual performance.In this review,we first explore the challenges faced by SISR and introduce some common datasets and evaluation metrics.Then,we review the improved network structures and loss functions of GAN-based perceptual SISR.Subsequently,the advantages and disadvantages of different networks are analyzed by multiple comparative experiments.Finally,we summarize the paper and look forward to the future development trends of GAN-based perceptual SISR.

Single color image super-resolution using sparse representation and color constraint

Color image super-resolution reconstruction based on the sparse representation model usually adopts the regularization norm(e.g.,L1 or L2).These methods have limited ability to keep image texture detail to some extent and are easy to cause the problem of blurring details and color artifacts in color reconstructed images.This paper presents a color super-resolution reconstruction method combining the L2/3 sparse regularization model with color channel constraints.The method converts the low-resolution color image from RGB to YCbCr.The L2/3 sparse regularization model is designed to reconstruct the brightness channel of the input low-resolution color image.Then the color channel-constraint method is adopted to remove artifacts of the reconstructed highresolution image.The method not only ensures the reconstruction quality of the color image details,but also improves the removal ability of color artifacts.The experimental results on natural images validate that our method has improved both subjective and objective evaluation.