加权多模态融合的服装搭配预测

Multimodal Fusion of Clothing Collocation Prediction

现有的服装搭配方法仅用图像模态或文本模态来进行服装搭配的预测。为解决多种模态间信息交互的问题,本文提出了基于带权重的多模态融合的服装搭配预测方法。该方法首先提取服装图像特征作为视觉特征,同时提取服装描述中的文本信息作为文本特征,然后将提取的特征加权融合,最终将融合的特征输入图神经网络模型中进行服装搭配预测。结果显示,融合后的特征将最重要的服装特征捕捉到服装表示中,能够有效提升服装搭配预测准确率。

The existing clothing matching methods only use image or text modalities for clothing matching prediction.To address the issue of information exchange between multiple modalities,this paper proposes a clothing matching prediction method based on weighted multimodal fusion.This method first extracts clothing image features as visual features,while extracting text information from clothing descriptions as text features.Then,the extracted features are weighted and fused,and finally,the fused features are input into the graph neural network model for clothing matching prediction.The results show that the fused features capture the most important clothing features in the clothing representation,which can effectively improve the accuracy of clothing matching prediction.