基于Hadoop/Hive的乳制品溯源数据计算及性能优化

Data calculation and performance optimization of dairy traceability based on Hadoop/Hive

为了提升传统乳制品溯源系统应对大规模企业生产数据的性能,本文分析了乳制品相关企业供应链业务流程、关键溯源单元和溯源信息,结合Hadoop/Hive大数据技术和分布式数据库技术,设计并构建了基于Hadoop/Hive的乳制品溯源框架.搭建模拟大数据环境并使用实际生产数据对系统性能进行测试,实验结果表明,引入Hadoop/Hive技术后,系统的平均数据存储速度、平均数据访问速度、平均数据交互速度分别提升了87.43%、27.10%、58.16%.改进后的乳制品溯源系统存储和处理大规模数据的能力明显优于传统的乳制品溯源系统.

In order to enhance the performance of traditional dairy traceability systems for the production data of large-scale enterprise,this paper analyzed the supply chain process of dairy enterprises,key traceability units and traceability information; combining Hadoop/Hive big data technology and distributed database technology,the paper designed and constructed a dairy products traceability framework based on Hadoop/Hive.We built a simulated large-scale data environment and used actual production data to test the system performance.The experimental results showed that after the introduction of the Hadoop/Hive technology system,the average data storage speed,the average data access speed,and the average data exchange rate increased by 87.43%,27.10% and 58.16%,respectively.The improved traceability system for dairy products is superior to the traditional dairy traceability system in storing and processing large-scale data.