基于堆叠模型的司法短文本多标签分类

Multi-label Classification of Judicial Short Texts Based on Stacking Model

司法文书短文本的语义多样性和特征稀疏性等特点,对短文本多标签分类精度提出了很大的挑战,传统单一模型的分类算法已无法满足业务需求。为此,提出一种融合深度学习与堆叠模型的多标签分类方法。该方法将分类器划分成两个层次,第一层使用BERT、卷积神经网络、门限循环单元等深度学习方法作为基础分类器,每个基础分类器模型通过K折交叉验证得到所有数据的多标签分类概率值,将此概率值数据进行融合形成元数据;第二层使用自定义的深度神经网络作为混合器,以第一层的元数据为输入,通过训练多标签概率矩阵获取模型参数。该方法将强分类器关联在一起,获得比单个分类器更加强大的性能。实验结果表明,深度学习堆叠模型实现了87%左右的短文本分类F1分数,优于BERT、卷积神经网络、循环神经网络及其他单个模型的性能。

The semantic diversity and feature sparsity of short texts in judicial documents is a great challenge to the accuracy of multi-label classification,so the traditional single model classification algorithm can no longer meet the business needs.For this reason,we propose a multi-label classification method combining deep learning and stacking model.This method divides the classifiers into two layers.In the first layer,deep learning methods such as BERT,convolutional neural network and gated recurrent unit are used as the basic classifier.Each basic classifier model obtains the multi-label classification probability value of all data through K-fold cross-validation,which are merged to form metadata.In the second layer,the user-defined deep neural network is used as the mixer,and the metadata in the first layer is used as the input,and the model parameters are obtained by training the multi label probability matrix.This method associates the strong learners together and gains more powerful functions than a single classifier.The experiment shows that the proposed model stacking method achieves about 87%of the F1 score of short text classification,which is superior to BERT,convolutional neural network,cyclic neural network and other single models.