基于Spark平台的推荐算法研究

Research on recommendation algorithm based on Spark platform

传统的推荐系统无论是在处理效率还是推荐准确度上都有一些不足之处,Spark作为新一代的数据处理的分布式框架,在数据处理效率上相较于hadoop有飞速的提升。主要针对Spark平台上的主流推荐算法ALS算法只考虑隐含信息而忽略了相似度信息的缺点展开研究,目的是进一步提高ALS算法的推荐精确度。通过将用户的相似度信息加权到ALS算法评分预测的公式中对ALS算法进行改进,并在Spark平台上使用MovieLens数据集验证改进前后的ALS算法的均方根误差(RMSE)值。实验证明,改进后的ALS算法的RMSE值更小,推荐的精度更高。

The traditional recommendation system has some shortcomings in both processing efficiency and recommendation accuracy.As a new generation of distributed framework for data processing,Spark has made rapid improvement in data processing efficiency compared with hadoop.This paper mainly studies ALS algorithm,the mainstream recommendation algorithm on Spark platform,which only considers implicit information but ignores similarity information,with the purpose of further improving the recommendation accuracy of ALS algorithm.In this paper,the ALS algorithm was improved by weighting the similarity information of users into the formula of ALS algorithm score prediction,and the RMSE value of ALS algorithm before and after the improvement was verified by using MovieLens dataset on Spark platform.Experiments show that the RMSE value of the improved ALS algorithm is smaller and the recommended accuracy is higher.