摘要
针对无人船测试过程中数据量大、数据源多、数据记录分散等问题,提出一种面向无人船测试的多源数据HDFS存储优化方法。首先设计基于Hadoop的无人船测试云平台架构,分析无人船测试云平台的信息流;接着提出无人船测试多源数据容错机制及重复数据删除策略,优化无人船测试多源数据HDFS存储;最后对无人船测试多源数据进行HDFS存储实验。结果表明,Hadoop集群优化后比默认系统存储效率提升约20%,可满足无人船测试多源数据的存储需求。
Aiming at the problems of large amount of data,multiple data sources and scattered data records in the testing process of unmanned surface vessel(USV),a HDFS storage optimization method for multi-source data of USV testing was proposed.Firstly,the architecture of USV testing cloud platform based on Hadoop was designed,and the information flow of USV testing cloud platform was analyzed.Then,the strategy of multi-source fault tolerance and duplicate data deletion was proposed to optimize the HDFS storage of USV testing multi-source data.Finally,an experiment was designed to test the HDFS storage method of USV testing multi-source data.The results show that the efficiency of the optimized Hadoop cluster is 20% higher than that of default system,which meets the storage requirement of USV testing multi-source data.
作者
吴永旺
饶银辉
庄伟涛
子文江
杨捍东
余蓉
洪晓斌
WU Yongwang;RAO Yinhui;ZHUANG Weitao;ZI Wenjiang;YANG Handong;YU Rong;HONG Xiaobin(School of Mechanical and Automotive Engineering,South China University of Technology,Guangzhou 510640,China;Guangzhou Shipyard International Company Limited,Guangzhou 511462,China)
出处
《中国测试》
CAS
北大核心
2022年第5期123-127,共5页
China Measurement & Test
基金
广东省科技计划项目(2019B151502057)
广东省自然资源厅科技项目(GDoE[2019]A13)
广州市科技计划项目(201902010024)。