RHS-CNN:一种基于正则化层次Softmax的CNN文本分类模型

RHS-CNN: A CNN Text Classification Model Based on Regularized Hierarchical Softmax

传统的卷积神经网络分类模型(CNN)的输出层采用扁平式架构的标准Softmax,在数据量较大、类别较多的文本分类任务中计算复杂度高,训练耗时长;而基于霍夫曼树(Huffman tree)构建的改进算法--层次Softmax(hierarchical softmax,H-Softmax)能极大地提高训练速度,但由于加入了大量的节点参数,使得优化难度增加,优化需要更长的迭代步,且容易过拟合,继而影响模型的拟合速度和分类效果。为此,提出了改进算法模型RHS-CNN(regularization hierarchical softmax CNN),采用正则化的方法,对H-Softmax的节点参数进行约束,避免过拟合,增强模型的泛化能力。实验分析结果表明:所提出的方法在相应评价指标上相对Softmax、H-Softmax有着一定的提升。

The output layer of the traditional convolutional neural network classification model( CNN)adopts the standard Softmax of the flat architecture. In the classification task with large amount of text data and many categories,the computational complexity is high and the training takes a long time.The improved algorithm based on Huffman Tree,Hierarchical Softmax( H-Softmax),can greatly improve the training speed. However,due to the addition of a large number of node parameters,the optimization difficulty increases,and the optimization needs more steps,which in turn affects the model’s fitting speed and classification effect. To this end,this paper proposes an improved algorithm model RHS-CNN( Regularization Hierarchical Softmax CNN),using the regularization method to constrain the node parameters of H-Softmax,avoiding over-fitting and enhancing the generalization ability of the model. The experimental analysis shows that the proposed method has a certain improvement on Softmax and H-Softmax in the corresponding evaluation indicators.