基于频繁词义序列的检索结果聚类算法研究

Search result clustering algorithm based on frequent itemsets meaning sequence

目前大多搜索引擎结果聚类算法针对用户查询生成的网页摘要进行聚类,由于网页摘要较短且质量良莠不齐,聚类效果难以保证。提出了一种基于频繁词义序列的检索结果聚类算法,利用Word Net结合句法和语义特征对搜索结果构建聚类及标签。不像传统的基于向量空间模型的聚类算法,考虑了词语在文档中的序列模式。算法首先对文本进行预处理,生成压缩文档以降低文本数据维度,构建广义后缀树,挖掘出最大频繁项集,然后获取频繁词义序列。从文档中获取的有序频繁项集可以更好地反映文档的主题,把相同主题的搜索结果聚类在一起,与用户查询相关度高的优先排序。实验表明,该算法可以获得与查询相关的高质量聚类及基于语义的聚类标签,具有更高的聚类准确度和更高的运行效率,并且可扩展性良好。

Most of existing web page clustering algorithms are based on short and uneven snippets of web pages, which often cause bad clustering performance. This paper presents a clustering algorithm based on frequent itemsets meaning sequence, which combines the use of WordNet syntactic and semantic features to build the search results clustering and labeling. Most of existing text clustering algorithms use the vector space model, which treats documents as bags of words. A word（meaning）sequence is frequent if it occurs in more than certain percentage of the documents in the text database. Firstly, the text is pre-processed to generate compact document to reduce the dimension of the document, build generalized suffix tree, and dig out the maximum frequent itemsets, then the frequent word meaning sequences is generated. Document theme can be better reflected by frequent itemsets meaning sequence, the search results having same themes clustered together with the user＇s query prioritization highly relevant. Experimental results show that the clustering algorithm can obtain a high quality cluster that related to the query semantic tags, which has higher accuracy, efficiency and good scal- ability.