基于XLNet的多层增强型注意力的电力文本分类

XLNet-based Power Text Classification of Multilayer Enhanced Attention

信息技术的发展使得电力企业由纸质化资源管理逐步过渡到数字资源管理,为提高数字资源管理效率,更迅速地反应客户需求,针对电网客户反应地电单文本,提出了基于XLNet的多层增强型注意力的电单文本分类。在此模型中,将中文字符通过XLNet预训练模型生成具有丰富上下文信息的词向量表征,残差网络结构获得词向量的丰富词义信息,双向长短时记忆网络实现上下文联系,多层注意力机制使得模型关注词义信息表征和上下文联系信息,并使用多层感知机进行多种类型的分类。为了验证模型的有效性,在电力客服文本数据集上进行验证,并取得了97.1%的文本分类准确度。

The development of information technology makes power enterprises gradually transition from paper-based resource management to digital resource management.In order to improve the efficiency of digital resource management and respond to customer needs more rapidly,a multilayer enhanced attention-based text classification of power bills based on XLNet is proposed for this paper for power grid customer response ground.In this model,Chinese characters are passed through XLNet pre-training model to generate word vector representations with rich contextual information,residual network structure to obtain rich lexical meaning information of word vectors,bidirectional long and short term memory network to achieve contextual connection,and multilayer attention mechanism makes the model focus on lexical meaning information representation and contextual connection information,and use multilayer perceptron to perform multiple types of classification.To verify the effectiveness of the model,it is validated on the electric power customer service text dataset and achieves 97.1%text classification accuracy.