摘要
搜索引擎用户行为分析是网络信息检索技术的研究热点。通过分析用户点击行为,利用Web数据挖掘技术获取有用信息,提高搜索引擎的检索算法和检索服务的效率,把用户从大量无序的搜索结果中解放出来。本文针对传统并行计算模型在易扩展和易编程方面遇到的瓶颈,给出一种基于Ha-doop的海量日志数据处理模型,通过基于Hadoop的分布式文件系统HDFS与MapReduce并行计算模型提高系统扩展性和易编程性,并应用该模型分析了Sogou搜索引擎一个月内约2 200万条查询日志,分析结果对于掌握用户搜索行为,评测及改进搜索引擎检索、排序算法等均有较好的指导意义。
Search engine user behaviors analysis is a focus of network information retrieval.It is a method of analyzing the user＇s behaviors through clicks to mine useful information to improve search engine＇s efficiency and retrieval services.In face of easy expansion and programming bottlenecks in traditional parallel computation models,a massive log data processing model based on Hadoop is given,which improves scalability and easy programming through Hadoop Distributed File System and MapReduce.Moreover,the experiment of analyzing 22 million query logs of the Sogou search engine in one month is carried out based on this model.The analysis result is instructive and meaningful to mastering the user＇s behaviors,evaluating and improving the searching and sorting algorithms.
出处
《计算机工程与科学》
CSCD
北大核心
2011年第4期115-120,共6页
Computer Engineering & Science
基金
广东省科技计划资助项目(2007B01020049)
