地震大数据流式计算研究

Research on big data flow computing

在地震数据处理方面,对数据的实时性要求较高,而且随着地震检测台站的不断增多,地震数据的容量也在不断增加。针对这些特点结合大数据技术,提出一种基于Spark Streaming技术的地震数据流式处理算法。将原始地震数据经过清洗再由数据分发器Kafka传输至Spark大数据计算框架,使用Spark核心api-Spark Streaming进行流式计算,将计算结果实时存入数据库,最后在进行数据实时可视化。

In the aspect of seismic data processing,the requirement of real-time data is high,and with the increasing of seismic detection stations,the capacity of seismic data is also increasing.In view of these characteristics,combined with big data technology,a seismic data stream processing algorithm based on spark streaming technology is proposed.The original seismic data is cleaned,and then transmitted to the spark big data computing framework by the data distributor Kafka.The spark core API spark streaming is used for streaming calculation,and the calculation results are stored in the database in real time.Finally,the data is visualized in real time.