生成式标签对抗的文本分类模型

Generative label adversarial text classification model

文本分类是自然语言处理(NLP)中的一项基础任务,目的是将文本数据分配至预先定义的类别。图卷积神经网络(GCN)与大规模的预训练模型BERT(Bidirectional Encoder Representations from Transformer)的结合在文本分类任务中取得了良好的效果。大规模异构图中GCN的无向的信息传递产生信息噪声影响模型的判断,造成模型分类能力下降,针对这一问题,提出一种生成式标签对抗模型,即类对抗图卷积网络(CAGCN)模型,以降低分类时无关信息的干扰,提升模型的分类性能。首先,采用TextGCN(Text Graph Convolutional Network)中的构图法构建邻接矩阵,结合GCN和BERT模型作为类生成器(CG);其次,在模型训练时采用伪标签特征训练法,并构建聚类器与类生成器联合训练;最后,在多个广泛使用的数据集上进行实验。实验结果表明,在泛用的分类数据集20NG、R8、R52、Ohsumed和MR上,CAGCN模型的分类准确率比RoBERTaGCN模型分别提高了1.2、0.1、0.5、1.7和0.5个百分点。

Text classification is a fundamental task in Natural Language Processing(NLP),aiming to assign text data to predefined categories.The combination of Graph Convolutional neural Network(GCN)and large-scale pre-trained model BERT(Bidirectional Encoder Representations from Transformer)has achieved excellent results in text classification tasks.Undirected information transmission of GCN in large-scale heterogeneous graphs produces information noise,which affects the judgment of the model and reduce the classification ability of the model.To solve this problem,a generative label adversarial model,the Class Adversarial Graph Convolutional Network(CAGCN)model,was proposed to reduce the interference of irrelevant information during classification and improve the classification performance of the model.Firstly,the composition method in TextGCN(Text Graph Convolutional Network)was used to construct the adjacency matrix,which was combined with GCN and BERT models as a Class Generator(CG).Secondly,the pseudo-label feature training method was used in the model training to construct a clueter.The cluster and the class generator were jointly trained.Finally,experiments were carried out on several widely used datasets.Experimental results show that the classification accuracy of CAGCN model is 1.2,0.1,0.5,1.7 and 0.5 percentage points higher than that of RoBERTaGCN model on the widely used classification datasets 20NG,R8,R52,Ohsumed and MR,respectively.