期刊文献+

基于BERT和改进胶囊网络的关系分类方法

Relation classification based on BERT and improved capsule network
下载PDF
导出
摘要 针对关系分类主流模型中存在的空间信息丢失和旋转不变性差的缺点,提出一个基于BERT和多头注意机制-胶囊网络(MA-CapsNet)的算法模型.该模型首先在句子的实体两端插入特殊符号,增强模型对实体信息的表示能力,再通过预训练的BERT语言模型获得包含上下文信息的语义向量表示,然后传入改进后的注重空间位置信息的胶囊网络中学习句子的语义特征并分类.同时引入多头注意力机制进一步提升模型的分类效果.在SemEval-2010 task 8关系分类数据集上,该算法模型取得了90.15%的宏F值.实验表明该模型架构能强化对句子语义特征的捕捉,改善关系分类任务的分类效果. In response to the disadvantages of spatial information loss and poor rotation invariance in convolutional and recurrent neural networks in relational first classification,we proposed a MA-CapsNet model.The model inserted special symbols at the entity ends of a sentence firstly,then obtained a semantic vector representation containing contextual information through a pre-trained BERT language model,and passed it into the improved capsule network to learn the semantic features of the sentence and classify it.A Multi-Head Attention mechanism was also introduced to further enhance the classification effectiveness of the model.The model architecture enhanced the capture of semantic features of sentences and improved the learning of relational text features,and the method achieved an F-value of 90.15%on the SemEval-2010 task 8 relational classification dataset.
作者 钟志峰 侯瑞洁 ZHONG Zhifeng;HOU Ruijie(School of Computer Science and Information Engineering,HuBei University,Wuhan 430062,China)
出处 《湖北大学学报(自然科学版)》 CAS 2023年第3期396-403,共8页 Journal of Hubei University:Natural Science
基金 国家自然科学基金面上项目(61977021)资助。
关键词 关系分类 BERT模型 卷积神经网络 多头注意力机制 胶囊网络 relation classification BERT convolutional neural network multi-head attention capsule network
  • 相关文献

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部