基于视觉误差与语义属性的零样本图像分类

Zero-shot image classification based on visual error and semantic attributes

在图像分类的实际应用过程中,部分类别可能完全没有带标签的训练数据。零样本学习(ZSL)的目的是将带标签类别的图像特征等知识迁移到无标签的类别上,实现无标签类别的正确分类。现有方法在测试时无法显式地区分输入图像属于已知类还是未知类,很大程度上导致未知类在传统设定下的ZSL和广义设定下的ZSL(GZSL)上的预测效果相差甚远。为此,提出一种融合视觉误差与属性语义信息的方法来缓解零样本图像分类中的预测偏置问题。首先,设计一种半监督学习方式的生成对抗网络架构来获取视觉误差信息,由此预测图像是否属于已知类;然后,提出融合属性语义信息的零样本图像分类网络来实现零样本图像分类;最后,测试融合视觉误差与属性语义的零样本图像分类方法在数据集AwA2和CUB上的效果。实验结果表明,与对比模型相比,所提方法有效缓解了预测偏置问题,其调和指标H在AwA2(Animal with Attributes)上提升了31.7个百分点,在CUB(Caltech-UCSD-Birds-200-2011)上提升了8.7个百分点。

In the practical applications of image classification,some categories may have no labeled training data at all.The purpose of Zero-Shot Learning(ZSL)is to transfer knowledge such as image features of labeled categories to unlabeled categories and to correctly classify the unlabeled categories.However,the existing state-of-the-art methods cannot explicitly distinguish the input image belonging to the known categories or unknown categories,which leads to a large performance gap for unlabeled categories between the traditional ZSL prediction and the Generalized ZSL(GZSL)prediction.Therefore,a method of fusing of visual error and semantic attributes was proposed to alleviate the prediction bias problem in zero-shot image classification.Firstly,a semi-supervised learning based generative adversarial network framework was designed to obtain visual error information,so as to predict whether the image belongs to the known categories.Then,a zero-shot image classification network combining semantic attributes was proposed to achieve zero-shot image classification.Finally,the performance of zero-shot image classification algorithm combining visual error and semantic attributes was tested on AwA2(Animal with Attributes)and CUB(Caltech-UCSD-Birds-200-2011)datasets.The experimental results show that,compared to the baseline models,the proposed method can effectively alleviate the prediction bias problem,and has the harmonic index H increased by 31.7 percentage points on AwA2 dataset and 8.7 percentage points on CUB dataset.