基于随机森林的高能物理数据放置策略

High Energy Physics Data Placement Strategy Based on Random Forest

随着LHAASO高海拔宇宙线等高能物理实验规模的不断扩大,每年需要存储PB级的海量物理数据。高能物理海量存储系统一般采用随机的数据放置策略,没有考虑数据访问场景和服务器节点、存储设备的差异性。针对以上问题,提出一种异构存储环境下基于随机森林算法的数据放置策略,根据存储设备性能差异划分快慢存储池,同时对后期文件的读写访问场景进行预测和识别,综合考虑当前设备负载为数据找到最佳的放置位置。使用真实物理实验数据验证了算法的有效性。

With the continuous developments of high energy physics experiments such as Large High Air Altitude Shower Observatory(LHAASO),a large amount of data at PB scale will be collected,stored and analyzed every year.At present,random data placement strategy which doesn’t fully consider the differences among data access scenarios,servers and storage devices is generally used.A data placement strategy based on random-forest algorithm is proposed.Storage devices are separated into storage pools(Fast pool,Normal pool)according to their performance.The algorithm will predict and identify a new file’s access pattern,and find one best place for it considering the load of target devices.This paper evaluates the performance of the algorithm with data samples collected from production storage system of LHAASO experiment.