基于云存储视频处理框架的研究与实现

Research and Implementation of Video Processing Framework Based on Cloud Storage

随着智慧城市的快速发展,视频技术作为基础数据采集手段已经被广泛使用。这会引发一个问题:短时间内生成的海量视频数据无法快速处理,从而严重影响数据时效性价值的问题愈来愈严重。文中提出一套基于HBase的分布式处理框架。该框架首先支持多客户端同时上传的视频,然后提取其中出现的人脸,最终建立一个可以保存在内存中的索引表进行查询加速。通过处理客户端上传的含有待查人脸的图像,该框架可以快速定位人脸在上传的视频中出现的位置。针对上述需要实现的功能,文中详细描述了实现该框架各部分中最重要的表的具体设计细节与设计目的,同时简述了人脸查询的具体流程,并从整个集群的角度优化集群的具体方法。最终通过在百万人脸中查询特定的一张来揭示集群性能。实验结果显示,该框架有较好的性能并完全能满足真实需求。

With the rapid development of smart city,video technology has been widely used as a basic data collection method which has caused a problem that the massive video data generated in a short time wouldn＇t process promptly,seriously affecting the timeliness of data value,becomes more and more serious. In this paper,a distributed processing framework based on HBase is proposed. It supports multi- client updated videos simultaneously,then extracts faces appeared in those videos and builds an index table stored in memory to increase query speed. Through processing a frame image which uploaded from client with special faces,the framework could locate those faces in those videos. In response to those functions which needs to be implemented,the details and design purpose of most important table in various parts of framework are described in this paper,meanwhile it outlines the specific processes of the face query,optimizing it from the perspective of entire cluster. Finally,an experiment that retrieves one special face in millions of magnitude of image data is used to reflect the effect of this framework. According to the experiment,this framework has good performance,and actual demand is satisfied completely.