摘要
为了便于用户浏览搜索引擎返回结果,本文提出了一种基于TFIDF新的文本相似度计算方法,并提出使用具有近似线性时间复杂度的增量聚类算法对文本进行多层聚类的策略。同时,提出了一种从多文本中提取关键词的策略：提取簇中的名词或名词短语作为候选关键词,综合考虑每个候选关键词的词频、出现位置、长度和文本长度设置加权函数来计算其权重,不需要人工干预以及语料库的协助,自动提取权重最大的候选关键词作为类别关键词。在收集的百度、ODP语料以及公开测试的实验结果表明本文提出方法的有效性。
In order to facilitate the browse of the search results produced by search engines,this paper proposed a TFIDF-based new method to calculate the similarity of the documents and Web search results multi-level clustering by using one-pass clustering algorithm with linear time complexity.At the same time,we proposed a strategy to extract cluster keyword from multi-texts：selected noun or noun phrase as candidate cluster keywords,and took term frequency,the position of term occurring,the length of term and text into consideration to set a weighting function to compute every words weights of the search results,then automatically extracted the weightiest candidate keyword for each cluster generated by multi-level clustering without the intervene of human and the assistance of corpus.Experimental results on Baidu,ODP corpus and user investigation show the efficient and acceptance of our algorithm.
出处
《情报学报》
CSSCI
北大核心
2011年第5期464-470,共7页
Journal of the China Society for Scientific and Technical Information
基金
国家自然科学基金项目（60673191）
广东省自然科学基金项目（9151026005000002）
广东省高等学校自然科学研究重点项目（06Z012）
关键词
文本聚类
多层聚类
类别关键词提取
加权函数
text clustering
multi-level clustering
extracting keyword
weighting function
