湖北交通运输大数据存储应用平台设计与实现

Design and Implementation of Hubei Transportation Big Data Storage and Application Platform

针对当前湖北省交通运输数据的存储管理与应用瓶颈,结合目前大数据技术的发展背景,本文基于Hadoop生态体系设计与实现了针对湖北交通运输业务的大数据存储分析平台。文中从总体架构、存储结构、扩展架构、数据接入与治理、分布式任务调度的设计实现到平台的应用场景进行了论述,并使用10年的高速公路数据与具体业务场景对平台进行了存储与效率实验。实验结果表明,相比传统关系型数据库,该平台在数据存储、查询计算方面均具有非常明显的优越性。

Aiming at the current storage and application bottlenecks of transportation data in Hubei Province,combined with the development background of current big data technology,based on Hadoop ecosystem,a big data storage analysis platform for Hubei transportation business was designed and implemented.This article discusses the overall architecture,storage structure,extended architecture,data access and governance,design and implementation of distributed task scheduling,and application scenarios of the platform,and uses 10 years of highway data and specific business scenarios to store the platform.With efficiency experiments,the results prove that compared with traditional relational databases,this platform is not affected by the amount of data in terms of data storage,and has greatly improved query response speed.