云计算分布式缓存技术在海量数据处理平台中的应用

Application of distributed cache techniques of Cloud Computing in massive data processing platform

随着云计算和大数据时代的到来,在满足用户对系统访问量、访问速度、访问安全的要求的同时,系统必须实时准确地处理迅猛增长的海量数据,而传统的缓存技术无法满足海量数据处理和用户高并发访问的需求。分布式缓存技术是最好的高性能缓存解决方案。本文研究如何利用云计算下分布式缓存技术在海量数据处理平台中解决该问题,分析研究了分布式缓存的关键技术、分布式缓存的一致性和分布式内存数据管理。在此基础上,分析并设计了分布式缓存系统的部署和整体架构。并将该分布式缓存系统的设计模式应用在某团购网上,进行了POC测试。测试结果证明分布式缓存技术可以缓解服务器的压力,解决海量数据和超高并发数据访问所带来的问题,提升了系统的性能、访问速度、可靠性以及降低响应延迟。

With the advent of the era of Cloud Computing and big data,massive data increased rapidly must be real-time accurately processed by the system,and requirements of visits to the system、access speed and access security to users are meeted at the same time,but traditional caching technology can＇t meet the needs of massive data processing and high concurrent access to the users. Distributed caching techniques is one of the best solution of high-performance cache. How to solve the problems in the mass data processing platform using distributed cache techniques of Cloud Computing is researched,and the key technology of distributed cache,consistency of distributed cache and distributed memory data management are also explored. Based on the aboved,deployment of distributed cache system and the overall architecture are analyzed and designed. And the design mode of the distributed cache system is used in the group buying site,and be tested by POC. Test results show that distributed cache techniques can alleviate the pressure of the server,solve the problems of massive data and high concurrent data access,and improve performance、access speed and reliability of the system,simultaneously reduce the response delay.