基于Map/Reduce的索引数据云存储模型研究

Study on Cloud Storage Model of Map/Reduce-based Index Data

针对目前搜索引擎引擎系统存在的数据量庞大、访问用户高并发性和搜索延迟性的特点,提出了基于云存储的文档索引分类存储模型,并在索引数据分类存储算法实现过程中,采用基于Map/Reduce编程模型的二次索引词权重计算,以降低分类过程中的模糊粒度.通过实验验证基于该存储模型的算法不仅可以提高海量数据索引库的数据处理效率,而且在一定程度上降低了检索系统查询延迟,提高了搜索效率.

The main problems of current search engine system applied on intelligent terminals are limited storage capacity with massive data,high-concurrency access of users and search delay of system.Aiming to tackling these problems,this paper proposes a cloud storage model of index classification and adopts a new index storage algorithm based on Map/Reduce programming model.The algorithm calculates the secondary weight of index term in the process of index classification in order to lower the fuzzy granularity of the classification.Based on the experimental results,the proposed storage model can not only improve the mass data processing efficiency,but also to some extent reduce query delay and ameliorate the search efficiency.