Perceptual Quality Assessment of Omnidirectional Images:Subjective Experiment and Objective Model Evaluation

Virtual reality(VR) environment can provide immersive experience to viewers.Under the VR environment, providing a good quality of experience is extremely important.Therefore, in this paper, we present an image quality assessment(IQA) study on omnidirectional images. We first build an omnidirectional IQA(OIQA) database, including 16 source images with their corresponding 320 distorted images. We add four commonly encountered distortions. These distortions are JPEG compression, JPEG2000 compression, Gaussian blur, and Gaussian noise. Then we conduct a subjective quality evaluation study in the VR environment based on the OIQA database. Considering that visual attention is more important in VR environment, head and eye movement data are also tracked and collected during the quality rating experiments. The 16 raw and their corresponding distorted images,subjective quality assessment scores, and the head-orientation data and eye-gaze data together constitute the OIQA database. Based on the OIQA database, we test some state-of-the-art full-reference IQA(FR-IQA) measures on equirectangular format or cubic formatomnidirectional images. The results show that applying FR-IQA metrics on cubic format omnidirectional images could improve their performance. The performance of some FR-IQA metrics combining the saliency weight of three different types are also tested based on our database. Some new phenomena different from traditional IQA are observed.

Blind Image Quality Assessment Based on Hybrid Fuzzy-Genetic Technique

A new method for no-reference image quality assessment based on hybrid fuzzy-genetic technique is proposed. Noise variance and edge sharpness level of the restored image are two basic metrics for assessing the performance of the restoration algorithm, then a fuzzy if-then inference system is developed to combine the two metrics to get a final quality score, and the parameters of the fuzzy membership function are trained with genetic algorithms. Experiments results show that the image quality score correlates well with mean opinion score and the proposed approach is robust and effective.

Human-Centric Composite-Quality Modeling and Assessment for Virtual Desktop Clouds

There are several motivations, such as mobility, cost, and secu- rity, that are behind the trend of traditional desktop users transi- tioning to thin-client-based virtual desktop clouds （VDCs）. Such a trend has led to the rising importance of human-centric performance modeling and assessment within user communities that are increasingly making use of desktop virtualization. In this paper, we present a novel reference architecture and its eas- ily deployable implementation for modeling and assessing objec- tive user quality of experience （QoE） in VDCs. This architec- ture eliminates the need for expensive, time-consuming subjec- tive testing and incorporates finite-state machine representa- tions for user workload generation. It also incorporates slow-mo- tion benchmarking with deep-packet inspection of application task performance affected by QoS variations. In this way, a ＂composite-quality＂ metric model of user QoE can be derived. We show how this metric can be customized to a particular user group profile with different application sets and can be used to a） identify dominant performance indicators and troubleshoot bottlenecks and b） obtain both absolute and relative objective user QoE measurements needed for pertinent selection of thin-client encoding configurations in VDCs. We validate our composite-quality modeling and assessment methodology by us- ing subjective and objective user QoE measurements in a re- al-world VDC called VDPilot, which uses RDP and PCoIP thin-client protocols. In our case study, actual users are pres- ent in virtual classrooms within a regional federated university system.

Recent Advances and Challenges of Visual Signal Quality Assessment

While quality assessment is essential for testing, optimizing, benchmarking, monitoring, and inspecting related systems and services, it also plays an essential role in the design of virtually all visual signal processing and communication algorithms, as well as various related decision-making processes. In this paper, we first provide an overview of recently derived quality assessment approaches for traditional visual signals (i.e., 2D images/videos), with highlights for new trends (such as machine learning approaches). On the other hand, with the ongoing development of devices and multimedia services, newly emerged visual signals (e.g., mobile/3D videos) are becoming more and more popular. This work focuses on recent progresses of quality metrics, which have been reviewed for the newly emerged forms of visual signals, which include scalable and mobile videos, High Dynamic Range (HDR) images, image segmentation results, 3D images/videos, and retargeted images.

Recent Advances in Image Dehazing

Images captured in hazy or foggy weather conditions can be seriously degraded by scattering of atmospheric particles,which reduces the contrast,changes the color,and makes the object features difficult to identify by human vision and by some outdoor computer vision systems.Therefore image dehazing is an important issue and has been widely researched in the field of computer vision.The role of image dehazing is to remove the influence of weather factors in order to improve the visual effects of the image and provide benefit to post-processing.This paper reviews the main techniques of image dehazing that have been developed over the past decade.Firstly,we innovatively divide a number of approaches into three categories:image enhancement based methods,image fusion based methods and image restoration based methods.All methods are analyzed and corresponding sub-categories are introduced according to principles and characteristics.Various quality evaluation methods are then described,sorted and discussed in detail.Finally,research progress is summarized and future research directions are suggested.

High Resolution SAR Image Algorithm with Sample Length Constraints for the Range Direction

The traditional Range Doppler(RD)algorithm is unable to meet practical needs owing to the limit of resolution.The order of fractional Fourier Transform(FrFT)and the length of sampling signals affect SAR imaging performance when FrFT is applied to RD algorithm.To overcome the above shortcomings,the purpose of this paper is to propose a high-resolution SAR image algorithm by using the optimal order of FrFT and the sample length constraints for the range direction.The expression of the optimal order of SAR range signals via FrFT is deduced in detail.The initial sample length and its constraints are proposed to obtain the best sample length of SAR range signals.Experimental results demonstrate that,when the range sampling-length changes in a certain interval,the best sampling-length will be obtained,which the best values of the range resolution,PSLR and ISLR,will be derived respectively.Compared with traditional RD algorithm,the main-lobe width of the peak-point target of the proposed algorithm is narrow in the range direction.While the peak amplitude of the first side-lobe is reduced significantly,those of other side-lobes also drop in various degrees.

Structured Computational Modeling of Human Visual System for No-reference Image Quality Assessment

Objective image quality assessment(IQA)plays an important role in various visual communication systems,which can automatically and efficiently predict the perceived quality of images.The human eye is the ultimate evaluator for visual experience,thus the modeling of human visual system(HVS)is a core issue for objective IQA and visual experience optimization.The traditional model based on black box fitting has low interpretability and it is difficult to guide the experience optimization effectively,while the model based on physiological simulation is hard to integrate into practical visual communication services due to its high computational complexity.For bridging the gap between signal distortion and visual experience,in this paper,we propose a novel perceptual no-reference(NR)IQA algorithm based on structural computational modeling of HVS.According to the mechanism of the human brain,we divide the visual signal processing into a low-level visual layer,a middle-level visual layer and a high-level visual layer,which conduct pixel information processing,primitive information processing and global image information processing,respectively.The natural scene statistics(NSS)based features,deep features and free-energy based features are extracted from these three layers.The support vector regression(SVR)is employed to aggregate features to the final quality prediction.Extensive experimental comparisons on three widely used benchmark IQA databases(LIVE,CSIQ and TID2013)demonstrate that our proposed metric is highly competitive with or outperforms the state-of-the-art NR IQA measures.

基于域随机化的绝缘子缺损数据自动生成与评价方法

机器代人巡检已逐步在电力巡检场景中得到广泛应用,绝缘子作为维护电力系统安全可靠稳定运行的重要设备,对其缺陷进行准确有效检测具有重要意义。绝缘子缺损识别是绝缘子缺陷检测中的重要任务,针对当前绝缘子缺损数据样本较少且样本不平衡,模型泛化能力差、数据标注不精确的问题,该文提出基于域随机化的缺损样本自动生成框架与样本图像质量评估方法,在虚拟域到真实域的绝缘子缺损检测域适应问题上取得了较好的效果,并基于该方法生成图像标注数据供开源使用。该文提出的域随机化数据生成方法首先基于程序化建模生成结构可调的绝缘子伞盘模型并基于纹理噪声模型建立了包含陶瓷绝缘子常见色彩、纹理信息的程序化纹理模型,进而基于网格噪声模型建立了绝缘子缺损切割模块,随后通过域随机化生成完整的绝缘子结构、纹理模型、缺损结构、背景信息与场景物体。在图像渲染和自动标注环节,首先基于相机对准与能见度自动生成与调整拍摄点及相机参数,进而提出了基于光线投射方法建立数据标注类别判定方法,设置实例对应的图像渲染通道进行图像渲染,完成批量数据生成。该文采用域随机化生成的3000张虚拟数据在不修改YOLO V5网络结构、模型参数的基础上训练模型,在300张真实绝缘子缺损图像上进行测试,正常绝缘子识别准确率达到97.8%,召回率92.1%,缺损绝缘子识别准确率79.0%,召回率75.9%,各检测类别的准确率和召回率均高于基于400张真实图像训练得到的检测模型的推理结果。该文提出的图像样本质量评估方法考虑了与真实域数据的相似度和样本在数据集中的独立性,将所得评价结果代入损失函数权重计算,进一步提升了推理结果,缺损绝缘子识别准确率85.3%,召回率77.8%。

基于对比性视觉-文本模型的光场图像质量评估

光场图像作为一种能够捕获场景每个位置光线信息的图像类型,在电子成像、医学影像和虚拟现实等领域具有广泛的应用前景.光场图像质量评估(Light Field Image Quality Assessment,LFIQA)旨在衡量此类图像的质量,但当前方法面临视觉效果与文本模态间异构性的重要挑战.为解决上述问题,本文提出了一种基于文本-视觉的多模态光场图像质量评估模型.具体来说,在视觉模态方面,我们设计了多任务模型,结合边缘自动阈值算法有效丰富了光场图像的关键表示特征.在文本模态方面,基于输入噪声特征与预测噪声特征的对比,准确识别光场图像的噪声类别,并验证了噪声预测对优化视觉表示的重要性.基于上述研究,进一步提出了一种优化的通用噪声文本配置方法,并结合边缘增强策略,显著提升了基线模型在光场图像质量评估中的准确性和泛化能力.此外,通过消融实验,评估了各组件对整体模型性能的贡献,验证了本文方法的有效性和稳健性.实验结果表明,该方法不仅在公开数据集Win5-LID和NBU-LF1.0的实验中表现出色,还在融合数据集中展示出优秀的实验结果,与现有最优算法相比,本文所提方法在两个数据库中的性能分别提升了2%和6%.本文提出的噪声验证策略和配置方法不仅为图像质量评估中的噪声预测任务提供了有价值的参考,也可用于其它噪声预测类型的辅助任务.

IMR算法与iDose^(4)算法对低剂量左房肺静脉CT图像质量影响的对比研究

目的比较全模型迭代重建(IMR)算法与混合迭代重建(iDose 4)算法对低剂量左房肺静脉CT图像质量的影响。方法招募2020年1月至12月因房颤于广西医科大学第二附属医院行射频消融术的患者30例,均接受低剂量左房肺静脉CT检查,管电压80 kV,管电流350 mA,扫描结束后采用IMR算法(IMR组)和iDose 4算法(iDose 4组)重建图像,对两组图像质量进行主观评分。于右上肺静脉(RSPV)、右下肺静脉(RIPV)、左上肺静脉(LSPV)、左下肺静脉(LIPV)开口处勾画感兴趣区(ROI),于左心房中心层面对左心房、降主动脉及胸壁脂肪勾画ROI。ROI选择密度均匀处勾画,记录各ROI内CT值和噪声值(SD值),计算左心房、RSPV、RIPV、LSPV、LIPV的对比噪声比(CNR)和信噪比(SNR),对比两组图像的客观评分。结果两名放射科医师对IMR组和iDose 4组图像的主观评价一致性高,组间相关系数(ICC)分别为0.895(P<0.001)、0.920(P<0.001)。IMR组图像质量主观评分显著高于iDose 4组[(4.50±0.57)分vs(3.86±0.68)分;t=3.898,P<0.001]。IMR组左心房、RSPV、RIPV、LSPV、LIPV的图像噪声值(SD值)低于iDose 4组,SNR和CNR高于iDose 4组,差异有统计学意义(P<0.05)。IMR组左心房、RSPV、RIPV、LSPV、LIPV的CT值低于iDose 4组,差异有统计学意义(P<0.05)。结论相较于iDose 4算法,IMR算法可显著降低低剂量左房肺静脉CT图像的噪声,提高SNR和CNR,从而提高图像质量,值得在临床中推广。

基于知识蒸馏的GAN生成图像质量评价方法

为了是提高GAN模型生成的图像质量的预测精度,以更好地符合人类对图像质量的主观评价,本研究介绍了一种基于知识蒸馏的半监督图像质量评价(IQA)方法。该方法利用CNN和ViT模型的结合,充分捕获全局和局部信息,学习高质量图像与失真图像之间特征的分布差异,并通过知识蒸馏传输高级特征信息。模型通过前向传播,得到图像质量评价分数。为了提高输入特征的多样性和模型的处理速度,本研究还采用了级联分组注意(CGA)机制对输入特征处理。通过在多个公共数据集上的实验表明,本研究方法的评价结果优于现有的评价方法,总体效果理想,相对鲁棒性相对优异,可以获得更加符合人类视觉效果的IQA结果。

基于实时图像处理算法的交通信号主动智能优化方法研究

为了提升城市交通的通勤效率,文中基于图像处理技术对交通信号灯的智能优化方案进行了研究。该方案引入了一种基于混合高斯模型的运动目标检测方法,可有效从道路背景中分离车辆的前景信息。并采用虚拟线圈法进行车辆统计,在识别出交叉路口的车道信息后按照不同车道分别统计车辆数量。同时还设计了一个以平均排队时长、排队长度和停车次数为优化目标的多目标约束规划模型,并通过粒子群算法完成了模型的求解。仿真结果表明,所提技术方案可适应不同交通流量交叉路口的车辆统计,且精度能够达到95.4%。相较于现有的Webster方案,文中信号灯优化方案的平均延误时间、平均停车次数与平均等待长度分别下降了2.76%、8.35%及12.44%。

Automated assessment of transthoracic echocardiogram image quality using deep neural networks

Background Standard views in two-dimensional echocardiography are well established but the qualities of acquired images are highly dependent on operator skills and are assessed subjectively.This study was aimed at providing an objective assessment pipeline for echocardiogram image quality by defining a new set of domain-specific quality indicators.Consequently,image quality assessment can thus be automated to enhance clinical measurements,interpretation,and real-time optimization.Methods We developed deep neural networks for the automated assessment of echocardiographic frames that were randomly sampled from 11,262 adult patients.The private echocardiography dataset consists of 33,784 frames,previously acquired between 2010 and 2020.Unlike non-medical images where full-reference metrics can be applied for image quality,echocardiogram's data are highly heterogeneous and requires blind-reference(IQA)metrics.Therefore,deep learning approaches were used to extract the spatiotemporal features and the image's quality indicators were evaluated against the mean absolute error.Our quality indicators encapsulate both anatomical and pathological elements to provide multivariate assessment scores for anatomical visibility,clarity,depth-gain and foreshortedness.Results The model performance accuracy yielded 94.4%,96.8%,96.2%,97.4%for anatomical visibility,clarity,depth-gain and foreshortedness,respectively.The mean model error of 0.375±0.0052 with computational speed of 2.52 ms per frame(real-time performance)was achieved.Conclusion The novel approach offers new insight to the objective assessment of transthoracic echocardiogram image quality and clinical quantification in A4C and PLAX views.It also lays stronger foundations for the operator's guidance system which can leverage the learning curve for the acquisition of optimum quality images during the transthoracic examination.

基于深度学习模型识别风电场避让物的方法研究

近年来,由于在新建风电场和需要进行技改的老旧风电场周边往往已建有风电场,因此风电场在新建或技改时除了需要避让生态保护红线、永久基本农田、自然保护区、风景名胜区等国家限建区域,以及需要考虑风电场内风电机组之间的尾流影响外,还需要重点关注周边风电场对新建或需要技改的老旧风电场的尾流影响。当针对尾流影响进行计算时,工程师采用当地遥感影像对周边已建风电场的风电机组逐个进行人工标记,但该方法容易造成风电机组机位统计缺漏,过失误差较大。提出一种基于深度学习模型识别风电场避让物的方法,利用深度学习模型对遥感影像中已建风电场的风电机组、民房等敏感因素进行识别标记,采用OpenWind软件对这些敏感因素进行避让,并快速进行风电机组布置。研究结果表明:利用计算机分析并标记避让物的结果准确、不易产生遗漏,可有效消除过失误差;同时,大幅降低了标记避让物的工作量,有效减少了大量的繁杂、低效、高重复性的工作,大幅提高了识别避让物的工作效率和结果精度。

Exploring the efficiency of image metric for assessing the visual quality of 3D mesh model

Recent developments in 3D graphics technology have led to extensive processes on 3D meshes(e.g.,compression,simplification,transmission and watermarking),these processes unavoidably cause the visual perceptual degradation of the 3D objects.The existing mesh visual quality evaluation metrics either require topology constrain or fail to reflect the perceived visual quality.Meanwhile,for the 3D objects that are observed on 2D screens by the users,it is reasonable to apply image metric to assess the distortion caused by mesh simplification.We attempt to explore the efficiency of image metric for assessing the visual fidelity of the simplified 3D model in this paper.For this purpose,several latest and most effective image metrics,2D snapshots,number and pooling algorithms are involved in our study,and finally tested on the IEETA simplification database.The statistical data allow the researcher to select the optimal parameter for this image-based mesh visual quality assessment and provide a new perspective for the design and performance assessment of mesh simplification algorithms.

基于视觉特性的无参考图像质量评价算法

针对传统无参考图像质量评价算法特征单一、需主观评价结果参与训练和对比度失真图像性能差等问题,提出一种结合颜色和纹理特征及视觉特性的改进算法。首先,为待评价图像构造一幅伪参考图像;其次,依据视觉特性将二者划分为主要与次要感兴趣区域,进一步将区域细化为更小的块并提取其中不同的颜色特征与纹理特征以建立多元高斯模型;最后,计算二者主要感兴趣区域之间、次要感兴趣区域之间的模型参数距离,作为质量分数评价图像的好坏。在公开数据集SPAQ和CSIQ上与传统算法QAC、NIQE、IL-NIQE等进行比较,计算线性皮尔森系数、斯皮尔曼秩相关系数、均方根误差等质量评价指标,实验结果表明所提算法优于传统算法。

基于PROSAIL模型及TM与实测数据的MODISLAI精度评价

利用PROSAIL模型和TM影像反演的叶面积指数LAI及地面实测数据,从空间和时间序列上分别对吉林省主要玉米种植带的MODISLAI产品精度进行验证分析,为准确评价和科学准确应用MODIS数据提供科学依据。研究结果表明,MODISLAI产品精度因玉米生长时期和植被类型而异,混合像元是造成MODISLAI产品低估玉米作物LAI值的主要原因之一;不同年份不同生长时期,MODISLAI和实测数据表现出不同的关系;在玉米生长灌浆期之后,MODISLAI产品与同期地面实测玉米LAI相比,约低估了33%～53%左右,MODISLAI产品值明显低于TM影像反演的LAI值,约低估玉米LAI值约30%～69%左右。因MODISLAI产品值与实际值有较大偏差,非常有必要对农作物MODISLAI产品值进行验证分析,研究结果可为准确利用MODISLAI产品及其算法改进后产品精度提供一种评价途径。

基于人眼视觉特性的图像视觉质量评价方法

为了更好地评价图像质量,在研究人眼视觉模型和人眼视觉特性的基础上,提出一种自顶向下的无参考图像客观质量评价方法。对待评价图像进行分块,并在每个色彩通道中对子块进行计算,输出平均视觉特性值,使评价模型既符合客观质量评价的优点,又兼顾人眼视觉感受,符合人眼主、客观一致性。选取美国TEXAS大学LIVE的图像数据库中图像的主观评价值对方法进行了实验证明,实验结果表明所提方法与主观评价取得了较好的一致性。

IP网络中视频流QoE评价模型

网络视频业务的兴起使网络运营商和服务提供商更加关注视频的用户体验(QoE),然而视频用户体验(QoE)值由于其主观性且评价过程复杂,难以在视频流传输中实时获取。通过实验分析了视频传输过程中服务质量(QoS)参数变化对视频QoE的影响,建立了客观、可测量的QoS参数与视频QoE之间映射模型,用可量化的QoS参数来评定视频QoE受网络性能的影响程度,以评估网络视频质量,该模型形式简单,能够实时监测视频质量。实验结果表明,该模型的评价结果能较好反映视频QoE。

基于自然图像统计的无参考图像质量评价

为了解决无参考图像情况下的质量评价问题,基于变换域的自然图像统计模型,提出一种适用于白噪声、高斯模糊、JPEG2000压缩等失真类型的无参考图像质量评价方法.利用自然图像Contourlet变换域子带均值间的线性关系,用失真条件下保持不变的低频子带均值预测未失真的高频子带均值,以高频子带预测均值与实际均值之间的差异计量图像的失真.结合人类视觉系统特性,对不同尺度、不同方向子带及子带内不同区域进行选取与加权,综合得到对失真图像的客观质量评价.实验表明,该质量评价测度与主观质量评价有较好的一致性,且在性能上优于峰值信噪比(PSNR).