摘要
针对关系分类主流模型中存在的空间信息丢失和旋转不变性差的缺点,提出一个基于BERT和多头注意机制-胶囊网络(MA-CapsNet)的算法模型.该模型首先在句子的实体两端插入特殊符号,增强模型对实体信息的表示能力,再通过预训练的BERT语言模型获得包含上下文信息的语义向量表示,然后传入改进后的注重空间位置信息的胶囊网络中学习句子的语义特征并分类.同时引入多头注意力机制进一步提升模型的分类效果.在SemEval-2010 task 8关系分类数据集上,该算法模型取得了90.15%的宏F值.实验表明该模型架构能强化对句子语义特征的捕捉,改善关系分类任务的分类效果.
In response to the disadvantages of spatial information loss and poor rotation invariance in convolutional and recurrent neural networks in relational first classification,we proposed a MA-CapsNet model.The model inserted special symbols at the entity ends of a sentence firstly,then obtained a semantic vector representation containing contextual information through a pre-trained BERT language model,and passed it into the improved capsule network to learn the semantic features of the sentence and classify it.A Multi-Head Attention mechanism was also introduced to further enhance the classification effectiveness of the model.The model architecture enhanced the capture of semantic features of sentences and improved the learning of relational text features,and the method achieved an F-value of 90.15%on the SemEval-2010 task 8 relational classification dataset.
作者
钟志峰
侯瑞洁
ZHONG Zhifeng;HOU Ruijie(School of Computer Science and Information Engineering,HuBei University,Wuhan 430062,China)
出处
《湖北大学学报(自然科学版)》
CAS
2023年第3期396-403,共8页
Journal of Hubei University:Natural Science
基金
国家自然科学基金面上项目(61977021)资助。