基于大数据分析与挖掘平台的个性化商品推荐研究及应用

Research and Application of Personalized Commodity Recommendation based on big data Analysis and Mining Platform

在以数据驱动为主导的大数据时代,信息资源量呈几何级增长,“信息超载”问题对数据分析与处理提出了更高的要求。从海量数据中提取有效信息并进行系统的分析与挖掘,从而满足用户的个性化需求将大大增强企业竞争力。本文结合Hadoop与Spark的优点,设计并搭建了包括HDFS、MongoDB、MLlib、Tableau等集群的大数据分析与挖掘平台,并实践了基于Amazon电商交易数据集的个性化商品推荐应用。利用SparkMLlib的ALS矩阵分解协同过滤推荐算法对用户购买行为进行模型训练和推荐,最后实验结果表明,此大数据分析与挖掘平台对于对个性化商品的推荐可实现不错的效果。

In the era of big data, which is dominated by data drive, the amount of information resources increases exponentially, and the problem of “information overload” puts forward higher requirements for data analysis and processing. Extracting effective information from massive data and carrying on systematic analysis and mining so as to meet the individual needs of users will greatly enhance the competitiveness of enterprises. Combining the advantages of Hadoop and Spark, this paper designs and builds an analysis and mining platform for big data, including HDFS,MongoDB,MLlib,Tableau and other clusters, and practices the personalized commodity recommendation based on Amazon e-commerce transaction data set. Application. The ALS matrix decomposition collaborative filtering recommendation algorithm of Spark MLlib is used to train and recommend the purchase behavior of users. Finally, the experimental results show that the big data analysis and mining platform can achieve a good effect on the recommendation of personalized goods.