基于流计算和大数据平台的实时交通流预测

Real-time traffic flow prediction based on big data platform and steam computing

目前交通流预测实时性差,很难满足在线分析和预测任务的需求,基于此提出一种Flink流计算框架和大数据平台结合的实时交通流预测方法。基于流计算框架实时捕捉和预处理数据,包括采用Flink的transform算子对数据进行校验和处理,将处理后的数据sink到大数据的HDFS文件系统,交由下一步的大数据并行框架进行分析建模与训练,实现基于流计算和大数据平台的实时交通流预测。实验结果表明,Flink能够实时捕捉和预处理交通流数据,把数据准时无误送入分布式文件系统中,在此基础上借助大数据框架下的并行分析和建模优势,在实时性数据分析与预测方面取得了较好的效果。

In real traffic scenarios,real-time data acquisition and real-time processing are extremely critical issues,and the current traffic flow prediction has poor real-time performance,which is difficult to meet the needs of online analysis tasks.Based on this,real-time traffic prediction method combined with the Flink stream computing framework and the big data platform was proposed,which was based on the stream computing framework to capture and preprocess data in real time,including the use of Flink’s transform operator to verify the data.The processed data sink to the BIG DATA HDFS file system,which was handed over to the next big data parallel framework for analyzing,modeling and training.The entire process of real-time traffic flow data inflow,pre-processing,and analysis modeling was simulated.Experimental results show that Flink can capture and preprocess traffic flow data in real time,and send the data into the distributed file system on time.On this basis,with the help of parallel analysis and modeling advantages under the framework of big data,it has a good effect on the real-time performance of data analysis and prediction,which is better than the offline processing mode of GPU.