摘要
为了实现海量视频数据的高效并行处理,将视频数据集解耦合实现任务的高并行度,通过Spark读取数据流的同时获取关键帧的方式解决了解耦视频数据引起数据倍增问题,并对图片特征数据进行优化,进而在Spark上实现了具有高可扩展性并行处理海量视频数据的框架。在天河二号云平台上进行部署实验,实验结果表明,随着处理节点个数增加,本框架可以获得近线性的加速比。
In order to realize the effective processing of massive video data in parallel,this paper decoupled video files for getting high degree of tasks parallelism,resolved the problem of multiplied data after decoupling video files and optimized the feature of picture. It realized the better scalable framework of processing video data on Spark. By deploying and experimenting over Kylin Cloud on Tianhe-2,the experimental results show that the framework gets linear speed-up ratio with the increase of processing node.
出处
《计算机应用研究》
CSCD
北大核心
2017年第12期3811-3815,3819,共6页
Application Research of Computers
基金
国家"863"计划资助项目(2013AA01A212)
