基于多模态融合的社交媒体文本地理位置预测方法

A social media geolocation prediction method based on multimodal fusion

挖掘社交媒体文本的地理位置信息能发现其空间关系,提出了基于多模态融合的社交媒体文本地理位置预测方法,利用文本获取的相关图片作为增强数据,构建图文数据集,以提高地理位置预测的准确性。多模态融合模型分别利用图片通道和文本通道提取两者的地理位置信息。同时,引入图文匹配模块对图文对进行降噪,解决图文不匹配问题。在Geotext数据集上进行的地理位置预测实验结果显示,与基线模型相比,中值误差距离降低了18.8%,平均误差距离降低了4.5%。

Geographical information extracted from social media text reveals underlying spatial correlations.A geo-graphical location prediction method for social media text based on multimodal fusion was proposed.By utilizing images associated with the text as augmented data,an integrated image-text dataset was constructed to enhance the accuracy of geographical location prediction.The multimodal fusion model employs separate channels for images and text to independently extract their respective geographical location information.Additionally,a text-image matching module was introduced to denoise the image-text pairs,effectively solving the issue of text-image misalignment.Experimental results on the Geotext dataset indicate that compared to the baseline model,the proposed method reduces the median error distance by 18.8%and the average error distance by 4.5%.