基于标签注意力与多粒度网络的层级专利分类

Hierarchical patent classification based on label attention and multi-granularity network

针对中文专利的多层级自动分类任务中不同层级之间标签丰富语义信息及依赖关系和不同粒度大小的特征信息被忽略问题,提出一种RoBERTa-ALMG模型。通过RoBERTa预训练模型获取专利文本的高级语义表征,在标签注意力模块中借助双重多层感知机和注意力机制动态生成标签文本向量表示,通过前向传播过程实现不同层级之间的知识传递与信息共享,借助多粒度特征抽取模块捕捉层级之间的不同粒度特征和信息。在国家信息中心公布的数据集上的实验结果表明,该模型的表现优于其它模型。

In the multi-hierarchy automatic classification task of Chinese patents,the rich semantic information,dependencies of labels and the feature information of different granularities between hierarchies are ignored,and the RoBERTa-ALMG model was proposed.The advanced semantic representation of the patent text was obtained through RoBERTa pre-training model,and the label text vector representation was dynamically generated with the help of dual multilayer perceptron and attention mechanisms in the label attention module.Knowledge transfer and information sharing between different hierarchies were realized through the forward propagation process.Different granularity features and information between hierarchies were captured using the multi-granularity feature extraction module.Experimental results of the dataset published by the National Information Center show that the model outperforms other models.