基于异质图神经网络的服装兼容性预测

Heterogeneous Graph Neural Network for Fashion Outfit Compatibility Prediction

针对目前服装兼容性预测模型效果虚高,且对服装单品关系的建模没有考虑单品的类型,以及样本不均衡导致模型训练不充分的问题,提出一个基于异质图神经网络的服装兼容性预测模型.首先采用一种负样本采样策略,使用同类替换的原则重新构造负样本集,解决了现有模型效果虚高的问题;然后采用异质图神经网络,结合图全局池化技术实现了对同一套装内不同类型单品之间复杂关系的建模;最后通过难例学习,使得样本量较少的类型单品有更加均衡的训练机会.在Polyvore数据集上的实验结果表明,所提模型的AUC值达到0.838,综合兼容性预测性能优于对比方法.

To solve the problem of the inflated effectiveness of current clothing compatibility prediction models,as well as the lack of consideration for the type of clothing when modeling clothing relationships and insufficient model training because of imbalanced samples,we propose a clothing compatibility prediction model based on heterogeneous graph neural networks.The model first adopts a new negative sample sampling strategy,which reconstructs the negative sample set based on the principle of using similar items for replacement to cope with the problem of the inflated model’s effectiveness.Then,a heterogeneous graph neural network is used,together with graph global pooling techniques,to model the complex relationships between different types of items within the same outfit.Finally,through hard example learning,clothing items with fewer samples have more balanced train-ing opportunities.Experimental results on the Polyvore dataset show that the model achieves an AUC value of 0.838,with significant advantages in comprehensive performance compared to other methods.