基于Swin Transformer的嵌入式零样本学习算法

Embedded Zero-shot Learning Algorithm Based on Swin Transformer

零样本学习旨在解决样本缺失情况下的分类问题.以往嵌入式零样本学习算法通常只利用可见类构建嵌入空间,在测试时不可避免会出现过拟合可见类的问题.基于此本文提出了一种基于类别语义相似度的多标签分类损失,该损失可在构建嵌入空间的过程中引导模型同时考虑与当前可见类语义上相似的未见类,进而将语义空间的相似性迁移到最终执行分类的嵌入空间.同时现有零样本学习算法大部分直接使用图像深度特征作为输入,特征提取过程没有考虑语义信息,基于此本文采用Swin Transformer作为骨干网络,输入原始图片利用自注意力机制得到基于语义信息的视觉特征.本文在3个零样本学习基准数据集上进行了大量实验,与目前最先进的算法相比取得了最佳的调和平均精度.

Zero-shot learning aims to solve the classification problem in the absence of samples.Previous embedding-based zero-shot learning algorithms usually construct an embedding space using only seen classes,which inevitably leads to overfitting problems for seen classes during testing phase.Based on this,this paper proposes a multi-label classification loss based on the semantic similarity of classes.This loss guides the model to simultaneously consider the unseen classes that are semantically similar to the current seen classes in the process of constructing the embedding space,and then migrates the similarity of the semantic space to the embedding space where classification is finally performed.Meanwhile,most of the existing zero-shot learning algorithms directly use the deep features of images as input,and the feature extraction process does not consider semantic information.Based on this,this paper adopts Swin Transformer as the backbone and inputs the original images to obtain the visual features based on semantic information using the self-attention mechanism.In this paper,extensive experiments are conducted on three zero-shot learning benchmark datasets,and the best harmonic mean is achieved compared with the current state-of-the-art methods,which validates the effectiveness of the method in this paper.