基于预训练模型与记忆卷积网络的立场检测研究

Study of Stance Detection Based on Pre-training Model and Memory Convolution Network

立场检测研究旨在研究特定文本针对特定话题所表达的支持、中立或反对立场,在以往的中文文本立场分析研究方法中,未关注文本结构间的依赖关系,且评论文本所隐含的立场往往是隐晦和不敏感的。该文提出了基于双向Transformer的大规模预训练语言模型BERT(Bidirectional Encoder Representation from Transformers)和长短时记忆(LSTM)以及卷积神经网络(CNN)相结合的立场分析方法来解决这个问题,同时,为解决BERT模型针对不同数据样本输入向量维度不一所导致的误差,提出了一种最优字个数维度判定算法对BERT模型输入进行分析。模型搭建上创新地采用并行输入输出的方法,充分利用了LSTM的全局特征提取和CNN的局部特征提取的优势,并且所用BERT模型更能对隐晦特征及不敏感特征进行提取,利用这一方法可以有效地判定不同目标对某一特定话题所表达的支持、中立或者反对立场。经过对比传统模型以及现有立场分析方法表明,所提模型拥有较好的性能,其F1值达到0.883。

The research of stance detection aims to study the support,neutrality or opposition expressed by specific texts on specific topics.In the previous research on the stance detection of Chinese texts,the dependence between text structures was not paid attention to,and the stance implied in the comment text was often obscure and insensitive.We propose a stance detection method combining Bidirectional Encoder Representation from Transformers,Long Short Time Memory(LSTM)and Convolutional Neural Network(CNN)to solve this problem.We propose an optimal word number dimension determination algorithm to analyze the input of BERT model.On the model construction,the parallel input and output method is used innovatively,making full use of the advantages of LSTM global feature extraction and CNN local feature extraction.The BERT model can extract more obscure features and insensitive features.This method can effectively determine the support,neutrality or opposition expressed by different targets for a specific topic.Compared with traditional models and existing stance detection methods,the proposed model has excellent performance,and its F1 value reaches 88.3%.