景区评论词频统计算法研究

Study on word frequency statistics algorithm of scenic sites

针对人们在出游前查看景区网络评价信息难以得到对该景区之整体评价的问题,提出了一种适用于海量数据的词频统计算法TF-CT.该算法采用余弦相似性算法对海量的文本数据进行词性分类,将具有相同表达态度的数据归为一类;采用TextRank算法对各类别中的一条数据进行关键词语提取;采用改进的TFIDF算法对提取的关键词进行词频统计,获取文本数据的表达态度.实验结果表明,与TFIDF算法相比,TF-CT算法在结果准确度和时间复杂度上具有更大的优势.

Aiming at the problem that it is difficult to get the overall evaluation of the scenic spot when people are viewing the network evaluation information before the trip,a word frequency statistics algorithm TF-CT that is suitable for massive data was proposed. The cosine similarity algorithm was used to classify these huge text data,so that the data with the same expression attitude was categorized into a class. The TextRank algorithm was used to extract the key words in one of the data in each category. The word frequency of the extracted keywords was used to obtain the attitude of text data using the TFIDF algorithm. Experimental results showed that compared with the TFIDF algorithm,the TF-CT algorithm had greater advantages in accuracy and time complexity.