基于贝叶斯网的短文本特征扩展方法

Short Text Feature Extension Method Based on Bayesian Networks

针对短文本特征词稀疏、表示能力不足等问题,提出了一种基于贝叶斯网的短文本特征扩展方法。该方法根据短文本中特征词之间的依赖关系构建语义贝叶斯网,定义特征词与短文本之间的关联度。基于贝叶斯网的推理计算关联度,将与短文本关联密切的特征词扩展到短文本中,以达到降低短文本的噪声、改善特征稀疏的目的。在此基础上,以短文本分类作为基本的文本分析任务,分析所提方法的可行性和有效性。在Amazon评论数据集上进行实验,结果表明所提方法是可行和有效的。

Aiming at the problems of feature sparsity and insuffcient representation ability in short text,this paper proposed a feature extension method based on Bayesian networks.Firstly,the semantic Bayesian network is constructed by defining the dependencies between the feature words in the short texts.Then,the correlation degree is defined between the feature word and the short text,and the feature words closely related to the short text are selected.These words are further extended to the short text to reduce the noise and sparsity of short texts.Finally,this paper analyzed the feasibility and effectiveness of the proposed method with the short text classification as the basic task of text analysis.The experimental results on the Amazon product dataset show that the proposed method is feasible and effective.