基于边缘辅助和多尺度Transformer的无参考屏幕内容图像质量评估

No-Reference Screen Content Image Quality Assessment Based on Edge Assistance and Multi-Scale Transformer

与从现实场景中拍摄的自然图像不同,屏幕内容图像是一种合成图像,通常由计算机生成的文本、图形和动画等各种多媒体形式组合而成.现有评估方法通常未能充分考虑图像边缘结构信息和全局上下文信息对屏幕内容图像质量感知的影响.为解决上述问题,本文提出一种基于边缘辅助和多尺度Transformer的无参考屏幕内容图像质量评估模型.首先,使用高斯拉普拉斯算子构造由失真屏幕内容图像高频信息组成的边缘结构图,然后通过卷积神经网络(Convolutional Neural Network,CNN)对输入的失真屏幕内容图像和相应的边缘结构图进行多尺度的特征提取与融合,以图像的边缘结构信息为模型训练提供额外的信息增益.此外,本文进一步构建了基于Transformer的多尺度特征编码模块,从而在CNN获得的局部特征基础上更好地建模不同尺度图像和边缘特征的全局上下文信息.实验结果表明,本文提出的方法在指标上优于其他现有的无参考和全参考屏幕内容图像质量评估方法,能够取得更高的主客观视觉感知一致性.

Different from the natural images captured from real-world scenes,screen content images(SCI)are syn⁃thetic images typically composed of various multimedia contents,such as computer-generated text,graphics,and anima⁃tions.Existing SCI quality assessment methods usually fail to fully consider the impacts of image edge and global context on the perceived quality of screen content images.To address the above issues,this paper proposed a no-reference screen content image quality assessment model based on edge assistance and multi-scale Transformer.Firstly,an edge structure map consisting of the high-frequency information in a distorted SCI is constructed using Gaussian Laplace operators.Then a convolutional neural network(CNN)is used to extract and fuse the multi-scale features from the input distorted SCI and the corresponding edge structure map,thus providing additional edge information gain for model training.In addition,this paper further proposed a multi-scale feature encoding module based on Transformer to better model the global context infor⁃mation of different scale images and edge features on the basis of the local features obtained by CNN.The experimental re⁃sults show that the model proposed in this paper outperforms the state-of-the-art no-reference and full-reference SCI quality assessment methods,and achieves higher consistency with the subjective visual perception.