基于BERT模型的中文新闻文本分类应用

Application of Chinese News Text Classification Based on BERT

AI赋能的互联网时代,带来了新闻数据量的爆发式增长,从这些亿万级的新闻文本数据中,探索出具有价值的有效新闻是非常重要的研究。本文在THUCNeWs数据集上使用CNN、RNN和BERT三种深度学习模型开展实验,其中BERT模型的准确率为0.9872,F1为0.97657。最后将训练好的BERT模型利用Streamlit工具部署在WEB端,实现了一个简明实用的中文新闻分类应用程序。实验证明了BERT模型在新闻文本分类任务中具有很高的准确性,不论在简明或是复杂、长文本或是短文本的分类条件下都具有较高应用前景。

In the AI-enabled Internet era,the amount of news data has been growing rapidly.It is very important to explore the valuable and effective news from these billions of news text data.In this paper,three deep learning models,CNN,RNN and BERT,are adopted on THUCNews dataset as experimental models.Accuracy of BERT model is 0.9872 and F1 is 0.97657.Finally,the trained BERT model is deployed on the WEB side with Streamlit tool,which achieves a concise and practical classification application.Experiments have proved that the BERT model has high accuracy in news text classification tasks,and has high application prospects under concise,complex,long text and short text classification conditions.