云环境下海量GPS、视频监控数据存储检索研究

Research on Storage and Retrieval of Massive GPS and Video Surveillance Data in Cloud Environment

监控数据在不断增大,且80%以上为非结构化数据,利用传统关系型数据库存储管理时,占用机器内存较大,检索数据需对数据库全表扫描,管理性能低。为解决海量监控数据的存储与管理,采用HDFS分布式文件系统存储视频非结构化数据,研究设计Hbase属性信息表,利用Hbase分布式数据库存储属性信息,研究Hbase全局检索信息行键及MapReduce工作流程,构建检索,实现视频监控数据的检索。最终基于非结构化数据的检索技术以及分布式数据处理框架的研究,得到的海量监控数据存储与检索方案。

The monitoring data is increasing,and more than 80%of it is unstructured data.When using the traditional relational database for storage and management,it takes up a lot of machine memory,low management performance.In order to solve the storage and management of massive monitoring data,HDFS distributed file system is used to store video unstructured data,HBase attribute information table is researched and designed,HBase distributed database is used to store attribute information,HBase global retrieval information row key and MapReduce workflow are studied,retrieval is constructed,and video monitoring data retrieval is realized.Finally,the research of the Unstructured data retrieval technology and the distributed data processing framework provides the storage and retrieval scheme of the massive monitoring data.