基于重构对比的广义零样本图像分类

Generalized Zero-Shot Image Classification Based on Reconstruction Contrast

广义零样本图像分类中常使用生成模型重构视觉信息或语义信息用于再进一步学习.然而,基于变分自编码器的方法对重构样本利用不够充分,表示性能欠缺.因此,文中提出基于重构对比的广义零样本图像分类模型.首先,使用两个变分自编码器将视觉信息和语义信息编码为同维度的低维隐向量,再将隐向量分别解码到两种模态.然后,使用投影模块投影视觉信息与语义模态的隐向量重构的视觉模态信息.最后,对投影后的特征进行重构对比学习.在保持变分自编码器重构性能的基础上增强编码器重构的判别性能,提高预训练特征在广义零样本图像分类任务上的应用能力.在4个标准数据集上的实验证实文中模型的有效性.

In generalized zero-shot image classification,generative models are often exploited to reconstruct visual or semantic information for further learning.However,the representation performance of the methods based on variational autoencoders is poor due to the underutilization of the reconstructed samples.Therefore,a generalized zero-shot image classification model based on reconstruction and contrastive learning is proposed.Firstly,two variational self-encoders are utilized to encode visual information and semantic information into low dimensional latent vectors of the same dimension,and then the latent vectors are decoded into two modes respectively.Next,the project modules are utilized to project both the original visual information and the visual information reconstructed from semantic modal latent vectors.Then,reconstruction contrastive learning is performed to learn the features after projection.The reconstruction performance of the encoder is maintained,the discriminative performance of the encoder is enhanced,and the application ability of pre-training features on the generalized zero-shot task is improved by the proposed method.The effectiveness of the proposed model is verified on four benchmark datasets.