基于BERT-Graphormer的中文专利文本分类模型研究

Research on Chinese Patent Text Classification Model Based on BERT-Graphorme

[目的/意义]随着专利申请量不断增长和专利分类类别的复杂化,针对提高专利分类工作效率、审查质量、节约人力资源的需求,构建中文专利文本分类模型。[方法/过程]通过微调Graphormer模型对专利标签的结构及信息进行建模,利用建模后的标签信息来增强BERT模型的文本表示。[结果/结论]相对于其他基线模型,该模型的Micro-F1与Macro-F1分数分别提升了1.6%与3.5%。实现了多标签专利的自动分类,通过对标签、文本的信息进行建模、融合,从而进一步提升模型的分类效果。

[Purpose/significance]With the increasing number of patent applications and the complexity of patent classification categories,the paper constructs a Chinese patent text classification model based on BERT-Graphormer in order to improve the efficiency of patent classification,review quality and save human resources.[Method/process]The paper models the structure and information of patent labels by fine-tuning the Graphormer model,and enhances the text representation of BERT model by using the modeled label information.[Result/conclusion]Compared with other baseline models,the Micro-F1 and Macro-F1 scores of the proposed model are increased by 1.6%and 3.5%.The study successfully achieved the automatic classification of multi-label patents.The model’s classification effectiveness is significantly improved by modeling and integrating information from labels and text.