Perceptual Gradient Similarity Deviation for Full Reference Image Quality Assessment

Perceptual image quality assessment(IQA)is one of the most indispensable yet challenging problems in image processing and computer vision.It is quite necessary to develop automatic and efficient approaches that can accurately predict perceptual image quality consistently with human subjective evaluation.To further improve the prediction accuracy for the distortion of color images,in this paper,we propose a novel effective and efficient IQA model,called perceptual gradient similarity deviation(PGSD).Based on the gradient magnitude similarity,we proposed a gradient direction selection method to automatically determine the pixel-wise perceptual gradient.The luminance and chrominance channels are both took into account to characterize the quality degradation caused by intensity and color distortions.Finally,a multi-scale strategy is utilized and pooled with different weights to incorporate image details at different resolutions.Experimental results on LIVE,CSIQ and TID2013 databases demonstrate the superior performances of the proposed algorithm.

A Novel Objective Quality Assessment Method for Video Conferencing Coding

Most recently, due to the demand of immersive communication, region-of-interest-based(ROI) high efficiency video coding(HEVC) approaches in conferencing scenarios have become increasingly important. However, there exists no objective metric, specially developed for efficiently evaluating the perceived visual quality of video conferencing coding. Therefore, this paper proposes a novel objective quality assessment method, namely Gaussian mixture model based peak signal-tonoise ratio(GMM-PSNR), for the perceptual video conferencing coding. First, eye tracking experiments, together with a real-time technique of face and facial feature extraction, are introduced. In the experiments, importance of background, face, and facial feature regions is identified, and it is then quantified based on eye fixation points over test videos. Next, assuming that the distribution of the eye fixation points obeys Gaussian mixture model, we utilize expectation-maximization(EM) algorithm to generate an importance weight map for each frame of video conferencing coding, in light of a new term eye fixation points/pixel(efp/p). According to the generated weight map, GMM-PSNR is developed for quality assessment by assigning different weights to the distortion of each pixel in the video frame. Finally, we utilize some experiments to investigate the correlation of the proposed GMM-PSNR and other conventional objective metrics with subjective quality metrics. The experimental results show the effectiveness of GMM-PSNR.

Blind Image Quality Assessment by Pairwise Ranking Image Series

Image quality assessment(IQA)is constantly innovating,but there are still three types of stickers that have not been resolved:the“content sticker”-limitation of training set,the“annotation sticker”-subjective instability in opinion scores and the“distortion sticker”-disordered distortion settings.In this paper,a No-Reference Image Quality Assessment(NR IQA)approach is proposed to deal with the problems.For“content sticker”,we introduce the idea of pairwise comparison and generate a largescale ranking set to pre-train the network;For“annotation sticker”,the absolute noise-containing subjective scores are transformed into ranking comparison results,and we design an indirect unsupervised regression based on EigenValue Decomposition(EVD);For“distortion sticker”,we propose a perception-based distortion classification method,which makes the distortion types clear and refined.Experiments have proved that our NR IQA approach Experiments show that the algorithm performs well and has good generalization ability.Furthermore,the proposed perception based distortion classification method would be able to provide insights on how the visual related studies may be developed and to broaden our understanding of human visual system.

Beyond Video Quality:Evaluation of Spatial Presence in 360-Degree Videos

With the rapid development of immersive multimedia technologies,360-degree video services have quickly gained popularity and how to ensure sufficient spatial presence of end users when viewing 360-degree videos becomes a new challenge.In this regard,accurately acquiring users’sense of spatial presence is of fundamental importance for video service providers to improve their service quality.Unfortunately,there is no efficient evaluation model so far for measuring the sense of spatial presence for 360-degree videos.In this paper,we first design an assessment framework to clarify the influencing factors of spatial presence.Related parameters of 360-degree videos and headmounted display devices are both considered in this framework.Well-designed subjective experiments are then conducted to investigate the impact of various influencing factors on the sense of presence.Based on the subjective ratings,we propose a spatial presence assessment model that can be easily deployed in 360-degree video applications.To the best of our knowledge,this is the first attempt in literature to establish a quantitative spatial presence assessment model by using technical parameters that are easily extracted.Experimental results demonstrate that the proposed model can reliably predict the sense of spatial presence.

Estimating Reduced-Reference Video Quality for Quality-Based Streaming Video

Reduced-reference （RR） video-quality estimators send a small signature to the receiver. This signature comprises the original video content as well as the video stream. RR quality estimation provides reliability and involves a small data payload. While significant in theory, RR estimators have only recently been used in practice for quality monitoring and adaptive system con- trol in streaming-video frameworks. In this paper, we classify RR algorithms according to whether they are based on a） model- ing the signal distortion, b） modeling the human visual system, or c） analyzing the video signal source. We review proposed RR techniques for monitoring and controlling quality in streaming video systems.

No-Reference Image Quality Assessment Method Based on Visual Parameters

Recent studies on no-reference image quality assessment (NR-IQA) methods usually learn to evaluate the image quality by regressing from human subjective scores of the training samples. This study presented an NR-IQA method based on the basic image visual parameters without using human scored image databases in learning. We demonstrated that these features comprised the most basic characteristics for constructing an image and influencing the visual quality of an image. In this paper, the definitions, computational method, and relationships among these visual metrics were described. We subsequently proposed a no-reference assessment function, which was referred to as a visual parameter measurement index (VPMI), based on the integration of these visual metrics to assess image quality. It is established that the maximum of VPMI corresponds to the best quality of the color image. We verified this method using the popular assessment database—image quality assessment database (LIVE), and the results indicated that the proposed method matched better with the subjective assessment of human vision. Compared with other image quality assessment models, it is highly competitive. VPMI has low computational complexity, which makes it promising to implement in real-time image assessment systems.

Image quality assessment based on perceptual grouping

To further explore the human visual system（ HVS）,the perceptual grouping（ PG）, which has been proven to play an important role in the HVS, is adopted to design an effective image quality assessment（ IQA） model. Compared with the existing fixed-window-based models, the proposed one is an adaptive window-like model that introduces the perceptual grouping strategy into the IQA model. It works as follows： first,it preprocesses the images by clustering similar pixels into a group to the greatest extent; then the structural similarity is used to compute the similarity of the superpixels between reference and distorted images; finally, it integrates all the similarity of superpixels of an image to yield a quality score. Experimental results on three databases（ LIVE, IVC and MICT） showthat the proposed method yields good performance in terms of correlation with human judgments of visual quality.

Video classification for video quality prediction

In this paper we propose a novel method for video quality prediction using video classification. In essence, our ap- proach can serve two goals: (1) To measure the video quality of compressed video sequences without referencing to the original uncompressed videos, i.e., to realize No-Reference (NR) video quality evaluation; (2) To predict quality scores for uncompressed video sequences at various bitrates without actually encoding them. The use of our approach can help realize video streaming with ideal Quality of Service (QoS). Our approach is a low complexity solution, which is specially suitable for application to mobile video streaming where the resources at the handsets are scarce.

Recent Advances and Challenges of Visual Signal Quality Assessment

While quality assessment is essential for testing, optimizing, benchmarking, monitoring, and inspecting related systems and services, it also plays an essential role in the design of virtually all visual signal processing and communication algorithms, as well as various related decision-making processes. In this paper, we first provide an overview of recently derived quality assessment approaches for traditional visual signals (i.e., 2D images/videos), with highlights for new trends (such as machine learning approaches). On the other hand, with the ongoing development of devices and multimedia services, newly emerged visual signals (e.g., mobile/3D videos) are becoming more and more popular. This work focuses on recent progresses of quality metrics, which have been reviewed for the newly emerged forms of visual signals, which include scalable and mobile videos, High Dynamic Range (HDR) images, image segmentation results, 3D images/videos, and retargeted images.

A no-reference blocking artifact metric for B-DCT video

A new no-reference blocking artifact metric for B-DCT compression video is presented in this paper. We first present a new definition of blocking artifact and a new method for measuring perceptive blocking artifact based on HVS taking into account the luminance masking and activity masking characteristic. Then, we propose a new concept of blocking artifact cluster and the algorithm for clustering blocking artifacts. Considering eye movement and fixation, we select several clusters with most serious blocking artifacts and utilize the average of their blocking artifacts to assess the total blocking artifact of B-DCT reconstructed video. Experimental results illustrating the performance of the proposed method are presented and evaluated.

QI-BRiCE: Quality Index for Bleeding Regions in Capsule Endoscopy Videos

With the advent in services such as telemedicine and telesurgery,provision of continuous quality monitoring for these services has become a challenge for the network operators.Quality standards for provision of such services are application specic as medical imagery is quite different than general purpose images and videos.This paper presents a novel full reference objective video quality metric that focuses on estimating the quality of wireless capsule endoscopy(WCE)videos containing bleeding regions.Bleeding regions in gastrointestinal tract have been focused in this research,as bleeding is one of the major reasons behind several diseases within the tract.The method jointly estimates the diagnostic as well as perceptual quality of WCE videos,and accurately predicts the quality,which is in high correlation with the subjective differential mean opinion scores(DMOS).The proposed combines motion quality estimates,bleeding regions’quality estimates based on support vector machine(SVM)and perceptual quality estimates using the pristine and impaired WCE videos.Our method Quality Index for Bleeding Regions in Capsule Endoscopy(QI-BRiCE)videos is one of its kind and the results show high correlation in terms of Pearson’s linear correlation coefcient(PLCC)and Spearman’s rank order correlation coefcient(SROCC).An F-test is also provided in the results section to prove the statistical signicance of our proposed method.

Non-reference virtual view quality evaluation of MVD

The quality of virtual view based on multi-view video （MVD） plus depth format is often evaluated by PSNR or subjectively judged. However, due to synthesizing arbitrary view images, the virtual view images mostly have no reference images and are only assessed using non-reference. Virtual view images synthesized by depth estimation reference software （DERS） and view synthesis reference software （VSRS） often accompanied with blockiness and other distortions on the edge. In addition, matching level for the depth map and the corresponding texture maps of left and right views also affects the quality of the virtual view. This paper compares the edge similarity of the depth and the corresponding texture maps which generate the intermediate virtual view and combined with the virtual view＇s blockiness which causing blur to evaluate the quality of the virtual view. Experiment results show that the proposed method can reflect the quality of the virtual view better.

基于边缘辅助和多尺度Transformer的无参考屏幕内容图像质量评估

与从现实场景中拍摄的自然图像不同,屏幕内容图像是一种合成图像,通常由计算机生成的文本、图形和动画等各种多媒体形式组合而成.现有评估方法通常未能充分考虑图像边缘结构信息和全局上下文信息对屏幕内容图像质量感知的影响.为解决上述问题,本文提出一种基于边缘辅助和多尺度Transformer的无参考屏幕内容图像质量评估模型.首先,使用高斯拉普拉斯算子构造由失真屏幕内容图像高频信息组成的边缘结构图,然后通过卷积神经网络(Convolutional Neural Network,CNN)对输入的失真屏幕内容图像和相应的边缘结构图进行多尺度的特征提取与融合,以图像的边缘结构信息为模型训练提供额外的信息增益.此外,本文进一步构建了基于Transformer的多尺度特征编码模块,从而在CNN获得的局部特征基础上更好地建模不同尺度图像和边缘特征的全局上下文信息.实验结果表明,本文提出的方法在指标上优于其他现有的无参考和全参考屏幕内容图像质量评估方法,能够取得更高的主客观视觉感知一致性.

环境空气监测用氮中甲烷气体标准物质质量评估

为了解和掌握当前环境空气非甲烷总烃监测所使用的氮中甲烷气体标准物质的质量状况,通过对全国范围内开展环境空气非甲烷总烃监测的实验室中的99家进行使用情况调查,发现共涉及35家制造商。选择其中使用占比较高的10家,另外再随机选取6家,通过匿名采购等方式获取31瓶氮中甲烷气体标准物质。经两家测试单位分别测试后,采用归一化偏差(E_(n))法和相对偏差法两种方法评价所选标准物质的量值准确性和监测适用性,同时依照相关国家计量技术规范评估其证书完整性和规范性。结果显示,两种方法的评价结果一致。16家制造商中,14家的27瓶气体标准物质定值准确,2家的4瓶气体标准物质定值不准确;9家证书相对完整、规范,其余7家普遍存在缺少标准物质唯一标识等问题,部分制造商甚至存在涉嫌冒用证书、证书中的定值浓度超出认定范围等问题。鉴于氮中甲烷气体标准物质是保障非甲烷总烃监测结果准确性、可比性和可溯源性的重要基础,且制造商数量众多,建议使用者加强标准物质质量评估工作,并审查其所附证书的规范性。

环境监测用高锰酸盐指数和氨氮标准溶液量值比对研究

针对高锰酸盐指数(COD_(Mn))和氨氮(NH_(3)-N)标准溶液(以下简称液标)制造商、浓度水平和使用情况进行调研,根据调研结果选择常用的7个品牌COD_(Mn)液标和9个常用NH_(3)-N液标品牌开展量值比对和质量评价研究,匿名采购129支液标经盲样编码后采用《地表水环境质量标准》(GB 3838—2002)推荐标准进行分析测定,计算和评定测定结果的不确定度,以En法进行量值比对评价。研究表明,NH_(3)-N液标量值均满意,COD_(Mn)测试中的加盖操作会使测定结果偏高,按照证书要求操作时COD_(Mn)液标量值合格,未按照液标证书中要求进行液标样品前处理测定会造成测定结果的偏离,影响评价结果。建议监测人员合理选用和规范使用环境用液标,加强环境用液标的选购和管理,确保检测过程得到有效控制。

基于GMSL2的智能驾驶域控制器视频输出功能检测系统研究

智能驾驶域控制器承担着处理和分析来自各种传感器的数据。然而,随着车载摄像头数量的不断增加,域控制器内部的解串,加串,图像处理等各个环节都可能出现丢帧和像素异常问题,进而对其图像处理结果产生负面影响。为了精准评估智能驾驶域控制器的GMSL2视频输出功能,研究了一种双路GMSL2视频采集和视频质量对比系统。通过硬件板卡设计将GMSL2接口视频信号解串为MIPI CSI-2信号,由桥接芯片MC20901将MIPI CSI-2信号分离为FPGA可识别的LVDS信号和CMOS信号。通过XLINX XC7K325T-FFG900主控芯片进行FPGA逻辑设计,实现MIPI信号解析,YUV422转RGB888视频格式转码,DDR3缓存和PCIe 2.0×8总线上传。最后,以6位数码管百分秒表和24色标准色卡为拍摄对象,将拍摄到的视频帧结合图像特征提取、数码管穿线识别算法和RGB加权欧几里得色差公式,实现视频的丢帧和色差检测。实验结果表明:本系统能够实时采集双路YUV422 8 bit、4K、30 fps的GMSL2接口视频数据,并对智能驾驶域控制器输出的视频是否存在丢帧和色差问题进行定量分析,区分出被测件合格品与不合格品。提高了智能驾驶域控制视频输入输出功能检测结果的可靠性。

基于ResNet-TSM和BiGRU网络的移动视频感知质量评价模型

考虑到卡顿、质量切换、内容特征等因素对用户体验质量的影响都会直接体现在客户端的失真视频里,提出了一种客户端的移动视频感知质量评价模型。该模型无须对每种影响因素均进行表征和度量,而是基于深度特征提取+回归的思路,直接建立失真视频与平均意见分数之间的映射模型。首先,构建了ResNet-TSM网络结构,提取失真视频片段的深度时空特征;为了避免维度灾难,采用LargeVis算法对提取的深度特征进行降维,同时提升特征的表达与区分能力。然后,采用双向门控循环单元网络对视频的长时间依赖关系进行建模,得到各视频片段的打分,再利用时间平均池化方法将各片段分数进行聚合,得到整个视频的打分结果。在WaterlooSQoE-Ⅲ和LIVE-NFLX-Ⅱ数据集上的实验结果表明,提出的模型可以获得更高的预测精度。

结合视觉舒适度的无参考立体视频稳像效果评价

立体视频稳像效果评价是评价立体视频稳像算法性能的有效途径.针对当前缺乏立体视频稳像效果客观评价方法的问题,提出一种结合视觉舒适度的无参考立体视频稳像效果评价方法.将立体视频稳像前后的运动平滑度和视觉舒适度变化作为视频稳像前后的变化特征,结合主观评价训练得到立体视频稳像效果评价的支持向量回归模型;回归模型通过学习立体视频稳像前后的变化特征与主观评价结果之间的关系,最终获得直接评价任意立体视频稳像效果的能力.使用收集的55条仿真视频训练模型,并在10条真实视频上进行实验的结果表明,所提方法的稳定性较好,在视频量达到180条时,模型的评价结果趋于稳定,且模型评价结果与主观评价结果相关性达到93%,可用于立体视频稳像效果的客观评价.

降质中子照相图像清晰化及其质量度量方法

中子照相,也叫中子成像,是一种重要的射线无损检测技术。中子成像数据质量会受到准直比,样品与探测器间距离,散射中子,中子通量,γ射线、快中子等高能光子或粒子对CCD/CMOS相机芯片的干扰等因素影响。主要表现为高斯噪声、泊松噪声、几何不锐度(模糊)和白斑噪声等图像失真,尤其在中子通量受限的小型中子照相系统上愈发严重。为此,本文研究了中子照相图像的多种降质模型,构建了基于真实中子照相图像的多重失真数据集,依据深度学习生成对抗网络模型,设计了基于注意力机制的图像清晰化复原网络,通过端到端的训练,实现了对真实中子照相图像多重失真的有效抑制。同时,鉴于自然图像与中子照相图像的成像机理与失真类型的迥异,进一步设计了针对中子照相失真类型的图像无参考质量度量方案。由30组研究人员对不同复原方案效果的主观打分结果可知,本文所提出的图像清晰化方案在主观评价上取得了最好的表现力。同时,基于本文所提出的图像质量客观评价方法进行定量分析可知,本文复原方案较其他深度学习和传统图像复原方法在质量分数上均有超过10%的提升,这从侧面验证了本文复原方法的有效性。

联合全息平面与物平面的全息图质量评价

数字全息图可为用户提供更详细的3D信息;但其编码失真及特殊的性质为其质量评价带来了挑战。提出了一种联合全息平面与物平面的全参考全息图质量评价方法。首先,在全息平面上利用信息保真度准则度量了压缩全息图的退化程度。其次,在物平面上对重建的多视口参考全息图和压缩全息图提取显著性加权的梯度相似度,再利用Grünwald-Letnikov分数阶导数度量全局特征变化;结合二者计算重建的多视口参考全息图和压缩全息图的相似度。最后,将全息平面的计算结果和物平面的相似度作为联合特征,利用随机森林对以上特征进行训练,建立质量评价模型。实验结果表明,所提出方法的PLCC约为0.9179,SROCC约为0.9075,优于相关代表性全息图质量评价方法,与人眼主观感受契合度更高。