期刊文献+

基于大数据分析与挖掘平台的个性化商品推荐研究及应用 被引量:8

Research and Application of Personalized Commodity Recommendation based on big data Analysis and Mining Platform
下载PDF
导出
摘要 在以数据驱动为主导的大数据时代,信息资源量呈几何级增长,“信息超载”问题对数据分析与处理提出了更高的要求。从海量数据中提取有效信息并进行系统的分析与挖掘,从而满足用户的个性化需求将大大增强企业竞争力。本文结合Hadoop与Spark的优点,设计并搭建了包括HDFS、MongoDB、MLlib、Tableau等集群的大数据分析与挖掘平台,并实践了基于Amazon电商交易数据集的个性化商品推荐应用。利用SparkMLlib的ALS矩阵分解协同过滤推荐算法对用户购买行为进行模型训练和推荐,最后实验结果表明,此大数据分析与挖掘平台对于对个性化商品的推荐可实现不错的效果。 In the era of big data, which is dominated by data drive, the amount of information resources increases exponentially, and the problem of “information overload” puts forward higher requirements for data analysis and processing. Extracting effective information from massive data and carrying on systematic analysis and mining so as to meet the individual needs of users will greatly enhance the competitiveness of enterprises. Combining the advantages of Hadoop and Spark, this paper designs and builds an analysis and mining platform for big data, including HDFS,MongoDB,MLlib,Tableau and other clusters, and practices the personalized commodity recommendation based on Amazon e-commerce transaction data set. Application. The ALS matrix decomposition collaborative filtering recommendation algorithm of Spark MLlib is used to train and recommend the purchase behavior of users. Finally, the experimental results show that the big data analysis and mining platform can achieve a good effect on the recommendation of personalized goods.
作者 李晓颖 赵安娜 周晓静 杨成伟 Li Xiaoying;Zhao Anna;Zhou Xiaojing;Yang Chengwei(school of Management Science and Engineering, Shandong University of Finance and Economics,Ji’nan Shandong,250014)
出处 《电子测试》 2019年第12期65-66,81,共3页 Electronic Test
基金 中国博士后科学基金第58批面上资助项目“面向媒体大数据分析任务的关联规则挖掘与并行处理系统(5M582104)” 山东省自然基金“基于云计算环境的大规模关联数据挖掘与并行优化方法研究(BS2015DX013)” 山东省自然基金(面上项目)“基于隐式反馈数据的情感分析与推荐方法研究(ZR2019MG037)” 山东省高等学校科技计划项目立项“分布式异构环境下动态资源管理策略与延迟调度方法研究(J14LN19)” 山东财经大学校级特色课程(A2017008)
关键词 HADOOP SPARK 大数据 推荐系统 数据挖掘 Hadoop Spark big data recommendation system data Mining
  • 相关文献

参考文献1

二级参考文献15

  • 1IDC. The Digital Universe of Opportunities:Rich Data and the Incdreasing Value of the Internet of Things [EB/OL]. [2014-04]. http://www.emc.com/leadership/digital-universe/2014iview/executive-summary.htm.
  • 2FERRERIA C R L , Traina J C, MACHADO T A J, et al. Clustering Very Large Multi-Dimensional Datasets with Mapreduce [C]. 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2011 ACM. San Diego: ACM Press, 2011: 690-698.
  • 3YU Y, HUANG C, LEE Y. An Intelligent Touring System Based on Mobile Social Network and Cloud Computing for Travel Recom- mendation[C]. 28th International Conference on Advanced Information Networking and Applications Workshops(AINA), 2014 IEEE. Victoria, Canada: IEEE Press, 2014:19-24.
  • 4WALUNJ S G, SADAFALE K. An Online Recommendation System for E-commerce Based on Apache Mahout Framework[C]. 2013 Annual Conference on Computers and People Research, 2013 ACM. Cincinnati: ACM Press,2013: 153-158.
  • 5ZAHARIA M, CHOWDHURY M, FRANKLIN M J, et al. Spark: Cluster Computing with Working Sets[C]. Proceedings of the 2nd USENIX Conference on Hot Topics in Cloud Computing , 2010:10-10.
  • 6ZAHARIA M, CHOWDHURY M, DAS T, et al. Resilient Distributed Datasets: A Fault-Tolerant Abstraction for in-Memory Cluster Computing[C]. Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation. USENIX Association, 2012:2-2.
  • 7X.LU,M.W.U. RAHMAN, N. ISLAM, D. SHANKAR. Accelerating Spark with RDMA for Big Data Processing: Early Experiences[C]. Proceedings of the 22nd Annual Symposium on High-Performance Interconnects.2010:9-16.
  • 8YANG J, HE SQ. The Optimization of Parallel DBN Based on Spark[C]. Proceedings of the 19th Asia Pacific Symposium on Intelligent and Evolutionary Systems,2016:157-169.
  • 9江小平,李成华,向文,张新访.云计算环境下朴素贝叶斯文本分类算法的实现[J].计算机应用,2011,31(9):2551-2554. 被引量:21
  • 10刘义,景宁,陈荦,熊伟.MapReduce框架下基于R-树的k-近邻连接算法[J].软件学报,2013,24(8):1836-1851. 被引量:60

共引文献21

同被引文献41

引证文献8

二级引证文献14

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部