基于MapReduce的Web标签SOINN聚类算法

MapReduce-based SOINN Clustering Algorithm for Web Tag

Web标签有助于用户根据自己特定的兴趣完成信息资源的分类、组织和检索。然而,正是由于协同标记系统特有的公开性、自由化的特点,采用其对信息资源进行描述、组织、分类和检索,存在着信息描述不精确、标签组织混乱和标签语意模糊等问题。在此背景下提出了3种基于特征向量表示法(FVR)的Web标签SOINN聚类算法:基于资源的特征向量表示法、基于其他共现标签的特征向量表示法和基于全集共现标签的特征向量表示法。同时应用MapReduce框架将SOINN算法进行并行化。实验表明,当类中心数量超过2000时,3种分布式聚类FVR算法的召回率和准确度优于原始算法,可获得很好的加速比。从而证明此分布式聚类算法具有很好的可扩展性,可以用于更为海量的Web日志聚类分析系统。

Web tag helps users to classify,organize and search internet resources according to their interests.Tag clustering can help to solve problems caused by openness and freedom of Web tag system,such as inaccurate information description,disorganized tags,ambiguity,and so on.Three tag feature vector representation （FVR） methods were presented which are resource-based FVR,other tag co-occurrence FVR and total tag co-occurrence FVR,can all apply to SOINN clustering algorithm.SOINN clustering can be parallelized by MapReduce model.Experiments show that accuracy and recall rate of three tag FVR are superior to original tag co-occurrence FVR and tag SOINN clustering by MapReduce owns optimum performance when the number of class center is more than 2000.The experimental results prove that distributed clustering algorithms proposed in this paper have good scalability which can be applied to more massive cluster Web tag analysis system.