Spark平台下基于聚类挖掘的影视资源智能推荐

Intelligent recommendation of video resources based on cluster mining under Spark platform

为了从海量的网络影视资源中快速准确地向用户推荐其感兴趣的视频,提出了一种Spark平台下基于邻近传播(AP)聚类的智能推荐方案。数据存储采用了分布式文件系统,并在弹性分布式数据集中采用AP聚类进行资源实时推荐,加快聚类速度。此外,将明可夫斯基(Minkowski)相似性度量引入到AP聚类中,替换原有的欧氏距离度量来构建相似度,以提高其聚类精度。在常用电影数据集MovieLens上进行实验验证。结果表明,Spark平台有效提升了聚类计算的效率。同时,相比于AP聚类和K-均值聚类算法,改进AP聚类的推荐准确率更高。

In order to quickly and accurately recommend videos appealing to users from massive network video resources,an intelligent recommendation scheme based on Affinity Propagation(AP)clustering in Spark platform is proposed.The distributed file system is used for data storage,and AP clustering is used in elastic distributed data set to realize real-time resource recommendation,which accelerates the clustering speed.In addition,Minkowski similarity measurement is introduced into AP clustering,replacing the original Euclidean distance measurement to construct similarity matrix to improve its clustering accuracy.Experiment verification is carried out on the commonly used movie data set MovieLens.The results show that spark platform improves the efficiency of clustering computing.At the same time,compared with AP clustering and K-means clustering algorithm,the recommended accuracy of improved AP clustering is higher.