Bridge the Gap Between Full-Reference and No-Reference:A Totally Full-Reference Induced Blind Image Quality Assessment via Deep Neural Networks

Blind image quality assessment(BIQA)is of fundamental importance in low-level computer vision community.Increasing interest has been drawn in exploiting deep neural networks for BIQA.Despite of the notable success achieved,there is a broad consensus that training deep convolutional neural networks(DCNN)heavily relies on massive annotated data.Unfortunately,BIQA is typically a small sample problem,resulting the generalization ability of BIQA severely restricted.In order to improve the accuracy and generalization ability of BIQA metrics,this work proposed a totally opinion-unaware BIQA in which no subjective annotations are involved in the training stage.Multiple full-reference image quality assessment(FR-IQA)metrics are employed to label the distorted image as a substitution of subjective quality annotation.A deep neural network(DNN)is trained to blindly predict the multiple FR-IQA score in absence of corresponding pristine image.In the end,a selfsupervised FR-IQA score aggregator implemented by adversarial auto-encoder pools the predictions of multiple FR-IQA scores into the final quality predicting score.Even though none of subjective scores are involved in the training stage,experimental results indicate that our proposed full reference induced BIQA framework is as competitive as state-of-the-art BIQA metrics.

Blind Image Quality Assessment by Pairwise Ranking Image Series

Image quality assessment(IQA)is constantly innovating,but there are still three types of stickers that have not been resolved:the“content sticker”-limitation of training set,the“annotation sticker”-subjective instability in opinion scores and the“distortion sticker”-disordered distortion settings.In this paper,a No-Reference Image Quality Assessment(NR IQA)approach is proposed to deal with the problems.For“content sticker”,we introduce the idea of pairwise comparison and generate a largescale ranking set to pre-train the network;For“annotation sticker”,the absolute noise-containing subjective scores are transformed into ranking comparison results,and we design an indirect unsupervised regression based on EigenValue Decomposition(EVD);For“distortion sticker”,we propose a perception-based distortion classification method,which makes the distortion types clear and refined.Experiments have proved that our NR IQA approach Experiments show that the algorithm performs well and has good generalization ability.Furthermore,the proposed perception based distortion classification method would be able to provide insights on how the visual related studies may be developed and to broaden our understanding of human visual system.

Image quality assessment based on perceptual grouping

To further explore the human visual system（ HVS）,the perceptual grouping（ PG）, which has been proven to play an important role in the HVS, is adopted to design an effective image quality assessment（ IQA） model. Compared with the existing fixed-window-based models, the proposed one is an adaptive window-like model that introduces the perceptual grouping strategy into the IQA model. It works as follows： first,it preprocesses the images by clustering similar pixels into a group to the greatest extent; then the structural similarity is used to compute the similarity of the superpixels between reference and distorted images; finally, it integrates all the similarity of superpixels of an image to yield a quality score. Experimental results on three databases（ LIVE, IVC and MICT） showthat the proposed method yields good performance in terms of correlation with human judgments of visual quality.

Color Image Quality Assessment Based on Structural Similarity

It is well-known that classical quality measures,such as Mean Squared Error(MSE),Weighted Mean Squared Error(WMSE)or Peak Signal-to-Noise Ratio(PSNR),are not always corresponding with visual observations.Structural similarity based image quality assessment was proposed under the assumption that the Human Visual System(HVS)is highly adapted for extracting structural information from an image.While the demand on high color quality increases in the media industry,color loss will make the visual quality different.In this paper,we proposed an improved quality assessment(QA)method by adding color comparison into the structural similarity(SSIM)measurement system for evaluating color image quality.Then we divided the task of similarity measurement into four comparisons:luminance,contrast,structure,and color.Experimental results show that the predicted quality scores of the proposed method are more effective and consistent with visual quality than the classical methods using five different distortion types of color image sets.

NEW VISUAL PERCEPTUAL POOLING STRATEGY FOR IMAGE QUALITY ASSESSMENT

Most of Image Quality Assessment (IQA) metrics consist of two processes. In the first process, quality map of image is measured locally. In the second process, the last quality score is converted from the quality map by using the pooling strategy. The first process had been made effective and significant progresses, while the second process was always done in simple ways. In the second process of the pooling strategy, the optimal perceptual pooling weights should be determined and computed according to Human Visual System (HVS). Thus, a reliable spatial pooling mathematical model based on HVS is an important issue worthy of study. In this paper, a new Visual Perceptual Pooling Strategy (VPPS) for IQA is presented based on contrast sensitivity and luminance sensitivity of HVS. Experimental results with the LIVE database show that the visual perceptual weights, obtained by the proposed pooling strategy, can effectively and significantly improve the performances of the IQA metrics with Mean Structural SIMilarity (MSSIM) or Phase Quantization Code (PQC). It is confirmed that the proposed VPPS demonstrates promising results for improving the performances of existing IQA metrics.

A METHOD OF IMAGE QUALITY ASSESSMENT FOR COMPRESSIVE SAMPLING VIDEO TRANSMISSION

Based on compressive sampling transmission model, we demonstrate here a method of quality evaluation for the reconstruction images, which is promising for the transmission of unstructured signal with reduced dimension. By this method, the auxiliary information of the recovery image quality is obtained as a feedback to control number of measurements from compressive sampling video stream. Therefore, the number of measurements can be easily derived at the condition of the absence of information sparsity, and the recovery image quality is effectively improved. Theoretical and experimental results show that this algorithm can estimate the quality of images effectively and is in well consistency with the traditional objective evaluation algorithm.

Blind Image Quality Assessment Based on Hybrid Fuzzy-Genetic Technique

A new method for no-reference image quality assessment based on hybrid fuzzy-genetic technique is proposed. Noise variance and edge sharpness level of the restored image are two basic metrics for assessing the performance of the restoration algorithm, then a fuzzy if-then inference system is developed to combine the two metrics to get a final quality score, and the parameters of the fuzzy membership function are trained with genetic algorithms. Experiments results show that the image quality score correlates well with mean opinion score and the proposed approach is robust and effective.

Blind Image Quality Assessment Based on Wavelet Power Spectrum in Perceptual Domain

Blind image quality assessment(BIQA) can assess the perceptual quality of a distorted image without a prior knowledge of its reference image or distortion type. In this paper, a novel BIQA model is developed in wavelet domain. Considering the multi-resolution and band-passing characteristics of discrete wavelet transform(DWT), an improvement over the power spectrum is put forward, i.e., dubbed wavelet power spectrum(WPS)estimation. Then, the concept of directional WPS is applied to simplify the calculation. Moreover, a rotationally symmetric modulation transfer function(MTF) of human visual system(HVS) is integrated as a filter, which makes the metric to be consistent with the human vision perception and more discriminative. Experiments are conducted on the LIVE databases and three other databases, and the results show that the proposed metric is highly correlated with subjective evaluations and it competes well with other state-of-the-art metrics in terms of effectiveness and robustness.

No-Reference Stereo Image Quality Assessment Based on Transfer Learning

In order to apply the deep learning to the stereo image quality evaluation,two problems need to be solved:The first one is that we have a bit of training samples,another is how to input the dimensional image’s left view or right view.In this paper,we transfer the 2D image quality evaluation model to the stereo image quality evaluation,and this method solves the first problem;use the method of principal component analysis is used to fuse the left and right views into an input image in order to solve the second problem.At the same time,the input image is preprocessed by phase congruency transformation,which further improves the performance of the algorithm.The structure of the deep convolution neural network consists of four convolution layers and three maximum pooling layers and two fully connected layers.The experimental results on LIVE3D image database show that the prediction quality score of the model is in good agreement with the subjective evaluation value.

A multimodal dense convolution network for blind image quality assessment

Technological advancements continue to expand the communications industry’s potential.Images,which are an important component in strengthening communication,are widely available.Therefore,image quality assessment(IQA)is critical in improving content delivered to end users.Convolutional neural networks(CNNs)used in IQA face two common challenges.One issue is that these methods fail to provide the best representation of the image.The other issue is that the models have a large number of parameters,which easily leads to overfitting.To address these issues,the dense convolution network(DSC-Net),a deep learning model with fewer parameters,is proposed for no-reference image quality assessment(NR-IQA).Moreover,it is obvious that the use of multimodal data for deep learning has improved the performance of applications.As a result,multimodal dense convolution network(MDSC-Net)fuses the texture features extracted using the gray-level co-occurrence matrix(GLCM)method and spatial features extracted using DSC-Net and predicts the image quality.The performance of the proposed framework on the benchmark synthetic datasets LIVE,TID2013,and KADID-10k demonstrates that the MDSC-Net approach achieves good performance over state-of-the-art methods for the NR-IQA task.

Image quality assessment metrics by using directional projection

Objective image quality measure, which is a fundamental and challenging job in image processing, evaluates the image quality consistently with human perception automatically. On the assumption that any image distortion could be modeled as the difference between the directional projection-based maps of reference and distortion images, we propose a new objective quality assessment method based on directional projection for full reference model. Experimental results show that the proposed metrics are well consistent with the subjective quality score.

A Novel Spatial Pooling Strategy for Image Quality Assessment

A variety of existing image quality assessment （IQA） metrics share a similar two-stage framework： at the first stage, a quality map is constructed by comparison between local regions of reference and distorted images; at the second stage, the spatial pooling is adopted to obtain overall quality score. In this work, we propose a novel spatial pooling strategy for image quality assessment through statistical analysis of the quality map. Our in-depth analysis indicates that the overall image quality is sensitive to the quality distribution. Based on the analysis, the quality histogram and statistical descriptors extracted from the quality map are used as input to the support vector regression to obtain the final objective quality score. Experimental results on three large public IQA databases have demonstrated that the proposed spatial pooling strategy can greatly improve the quality prediction performance of the original IQA metrics in terms of correlation with human subjective ratings.

Image quality assessment method based on nonlinear feature extraction in kernel space

To match human perception, extracting perceptual features effectively plays an important role in image quality assessment. In contrast to most existing methods that use linear transformations or models to represent images, we employ a complex mathematical expression of high dimensionality to reveal the statistical characteristics of the images. Furthermore, by introducing kernel methods to transform the linear problem into a nonlinear one, a full-reference image quality assessment method is proposed based on high-dimensional nonlinear feature extraction. Experiments on the LIVE, TID2008, and CSIQ databases demonstrate that nonlinear features offer competitive performance for image inherent quality representation and the proposed method achieves a promising performance that is consistent with human subjective evaluation.

SWVFS： a saliency weighted visual feature similarity metric for image quality assessment

In this paper, a saliency weighted visual feature similarity （SWVFS） metric is proposed for full reference im- age quality assessment （IQA）. Instead of traditional spatial pooling strategies, a visual saliency-based approach is em- ployed for better compliance with properties of the human visual system, where the saliency allocation is closely related to the activity of posterior parietal cortex and the pluvial nu- clei of the thalamus. Assuming that the saliency map actually represents the contribution of locally computed visual distor- tions to the overall image quality, the gradient similarity and the textural congruency are merged into the final image qual- ity indicator. The gradient and texture comparison play com- plementary roles in characterizing the local image distortion. Extensive experiments conducted on seven publicly available image databases show that the performance of SWVFS is competitive with the state-of-the-art IQA algorithms.

SIMPLE QUALITY ASSESSMENT FOR BINARY IMAGES

Usually image assessment methods could be classified into two categories: subjective as-sessments and objective ones. The latter are judged by the correlation coefficient with subjective quality measurement MOS (Mean Opinion Score). This paper presents an objective quality assessment algorithm special for binary images. In the algorithm, noise energy is measured by Euclidean distance between noises and signals and the structural effects caused by noise are described by Euler number change. The assessment on image quality is calculated quantitatively in terms of PSNR (Peak Signal to Noise Ratio). Our experiments show that the results of the algorithm are highly correlative with subjective MOS and the algorithm is more simple and computational saving than traditional objective assessment methods.

Target acquisition performance in the presence of JPEG image compression

This paper presents an investigation on the effect of JPEG compression on the similarity between the target image and the background,where the similarity is further used to determine the degree of clutter in the image.Four new clutter metrics based on image quality assessment are introduced,among which the Haar wavelet-based perceptual similarity index,known as HaarPSI,provides the best target acquisition prediction results.It is shown that the similarity between the target and the background at the boundary between visually lossless and visually lossy compression does not change significantly compared to the case when an uncompressed image is used.In future work,through subjective tests,it is necessary to check whether this presence of compression at the threshold of just noticeable differences will affect the human target acquisition performance.Similarity values are compared with the results of subjective tests of the well-known target Search_2 database,where the degree of agreement between objective and subjective scores,measured through linear correlation,reached a value of 90%.

No-reference noisy image quality assessment incorporating features of entropy, gradient, and kurtosis

Noise is the most common type of image distortion affecting human visual perception.In this paper,we propose a no-reference image quality assessment(IQA)method for noisy images incorporating the features of entropy,gradient,and kurtosis.Specifically,image noise estimation is conducted in the discrete cosine transform domain based on skewness invariance.In the principal component analysis domain,kurtosis feature is obtained by statistically counting the significant differences between images with and without noise.In addition,both the consistency between the entropy and kurtosis features and the subjective scores are improved by combining them with the gradient coefficient.Support vector regression is applied to map all extracted features into an integrated scoring system.The proposed method is evaluated in three mainstream databases(i.e.,LIVE,TID2013,and CSIQ),and the results demonstrate the superiority of the proposed method according to the Pearson linear correlation coefficient which is the most significant indicator in IQA.

Structured Computational Modeling of Human Visual System for No-reference Image Quality Assessment

Objective image quality assessment(IQA)plays an important role in various visual communication systems,which can automatically and efficiently predict the perceived quality of images.The human eye is the ultimate evaluator for visual experience,thus the modeling of human visual system(HVS)is a core issue for objective IQA and visual experience optimization.The traditional model based on black box fitting has low interpretability and it is difficult to guide the experience optimization effectively,while the model based on physiological simulation is hard to integrate into practical visual communication services due to its high computational complexity.For bridging the gap between signal distortion and visual experience,in this paper,we propose a novel perceptual no-reference(NR)IQA algorithm based on structural computational modeling of HVS.According to the mechanism of the human brain,we divide the visual signal processing into a low-level visual layer,a middle-level visual layer and a high-level visual layer,which conduct pixel information processing,primitive information processing and global image information processing,respectively.The natural scene statistics(NSS)based features,deep features and free-energy based features are extracted from these three layers.The support vector regression(SVR)is employed to aggregate features to the final quality prediction.Extensive experimental comparisons on three widely used benchmark IQA databases(LIVE,CSIQ and TID2013)demonstrate that our proposed metric is highly competitive with or outperforms the state-of-the-art NR IQA measures.

IMAGE QUALITY ASSESSMENT BASED ON CONTOUR AND REGION

Image Quality Assessment （IQA） is a fundamental problem in image processing. It is a common principle that human vision is hierarchical： we first perceive global structural information such as contours then focus on local regional details if necessary. Following this principle, we propose a novel framework for IQA by quantifying the degenerations of structural information and region content separately, and mapping both to obtain the objective score. The structural information can be obtained as contours by contour detec- tion techniques. Experiments are conducted to demonstrate its performance in comparison with multiple state-of-the-art methods on two large scale datasets.

Multi-Scale Blind Image Quality Predictor Based on Pyramidal Convolution

Traditional image quality assessment methods use the hand-crafted features to predict the image quality score,which cannot perform well in many scenes.Since deep learning promotes the development of many computer vision tasks,many IQA methods start to utilize the deep convolutional neural networks(CNN)for IQA task.In this paper,a CNN-based multi-scale blind image quality predictor is proposed to extract more effectivity multi-scale distortion features through the pyramidal convolution,which consists of two tasks:A distortion recognition task and a quality regression task.For the first task,image distortion type is obtained by the fully connected layer.For the second task,the image quality score is predicted during the distortion recognition progress.Experimental results on three famous IQA datasets show that the proposed method has better performance than the previous traditional algorithms for quality prediction and distortion recognition.