基于Flink on YARN平台的应用研究

对于海量实时数据而言,基于Flink on YARN平台可以对这些数据进行并行化处理。在接下来的工作机制和相关技术中,重点分析了基于此平台典型的流架构的数据传输模式,在此基础上,比对了流处理系统和批处理系统的差异点。对于不同数据处理系统,分析总结了Flink所面临的一些挑战,希望以此为Flink的进一步研究提供参考。

For massive real-time data,these data can be processed in parallel based on Flink on YARN platform.In the following working mechanism and related technologies,the data transmission mode based on the typical stream architecture of this platform is analyzed,and on this basis,the differences between the stream processing system and the batch processing system are compared.For different data processing systems,the analysis summarizes some of the challenges faced by Flink,hoping to provide a reference for Flink’s further research.