基于遗传算法的文本特征选择

Text Feature Selection Based on Genetic Algorithm

文本特征选择是自然语言处理中的关键问题。针对文本特征的高维性和稀疏性问题,在过滤式特征选择算法文档-逆文档评率(term frequency-inverse document frequency,TF-IDF)的基础上,提出了用遗传算法对文本特征进行优化选择,使其最大程度地贴合后续的文本分类算法,在保证文本分类精确度的同时,降低特征维度以缩减预测时间。实验显示,该算法与单一的过滤式文本特征选择算法相比,能够有效减少所选文本特征数量(即降低特征维度),能有效提高文本的分类能力。

Text feature selection is a key issue in natural language processing.Due to the high-dimensional and sparsity of text features,based on the filter feature selection algorithm term frequency-inverse document frequency(TF-IDF),the genetic algorithm was used to optimize the text features.To maximize the fit of the subsequent text classification algorithm,while not effecting the accuracy of the text classification,reduce the feature dimension to reduce the prediction time.Experiments show that compared with a single filtered text feature selection algorithm,the algorithm can effectively reduce the number of selected text features(reduce the feature dimension)and effectively improve the text classification ability.