结合混合特征提取与深度学习的长文本语义相似度计算

Long text semantic similarity calculation combining hybrid feature extraction and deep learning

文本语义相似度计算是自然语言处理中一项非常重要的任务,但是目前对于文本语义相似度的研究多集中在短文本领域,而不是长文本。相较于短文本,长文本语义信息丰富,但同时语义信息容易分散。针对长文本语义信息分散的问题,提出一种特征提取模型,提取出长文本的主要语义信息;对提取的语义信息使用滑窗重叠的方法输入BERT预训练模型得到文本向量表示;然后,通过双向长短期记忆网络建模长文本的前后语义联系,将其映射到语义空间内;再通过线性层增加模型表示能力;最后,通过相似语义向量内积最大化和交叉熵损失函数进行微调。实验结果表明,该模型在CNSE和CNSS数据集上F1分数分别为0.84和0.91,性能优于基线模型。

Text semantic similarity calculation is a crucial task in natural language processing,but current research on similarity mostly focuses on short texts rather than long texts.Compared to short texts,long texts are semantically rich but their semantic information tends to be scattered.To address the issue of scattered semantic information in long texts,a feature extraction method is proposed to extract the main semantic information from long texts.The extracted semantic information is then fed into a BERT pre-training model using a sliding window overlap approach to obtain text vector representations.A bidirectional long short-term memory network is then utilized to model the contextual semantic relationships of long texts,mapping them into a semantic space.The model s representation ability is further enhanced through the addition of a linear layer.Finally,finetuning is performed by maximizing the inner product of similar semantic vectors and minimizing the cross-entropy loss function.Experiment results show that this method achieves F1 scores of 0.84 and 0.91 on the CNSE and CNSS datasets,outperforming the baseline models.