基于深度强化学习的故障硬盘预测与处理方法

Deep Reinforcement Learning Based on Disk Failure Prediction and Operation Method

大数据技术发展产生的海量数据急需一种可靠的数据存储方法,现有的主动故障预测方法相比被动容错机制可取得更好的效果,但是故障硬盘预测领域仍有一些问题亟待解决。当前的故障硬盘预测方法大多是离线的,通过滑动窗口将硬盘数据切分为样本,使用欠采样或过采样解决样本不平衡问题。然而,硬盘数据的分布会随时间变化,下采样可能会丢失一些有用特征,过采样可能会导致训练过拟合,该样本使用方式也忽略了样本之间的时间相关性。为了解决这些问题,将存储系统中硬盘的整个运行维护过程视为一个顺序决策过程,使用深度强化学习DQN算法求解。在强化学习语境下,样本不平衡问题转化为稀疏奖励问题。通过奖励塑造及探索机制解决稀疏奖励问题,在模型部署后利用经验回放实现模型在线学习。在开源数据集BackBlaze上的实验验证了该方法的有效性。

The massive amount of data generated by the development of big data technology urgently requires a reliable data storage method.The existing proactive failure prediction methods have achieved better performance than traditional reactive methods,but some problems still exist to be solved in the disk failure prediction.Most current methods are offline and divide data into samples through a sliding window with subsampling or oversampling to solve the imbalance problem.However,the distribution of hard disk data changes over time,subsampling may lose some valuable features,and oversampling may lead to training overfitting.Meanwhile,these methods ignore the temporal dependency of disk data.To solve these problems,the proposed method casts a hard disk′s entire operation and maintenance process in the storage system as a markov decision process,which is solved using the reinforcement learning method DQN.Based on reinforcement learning,the imbalance problem turns into the sparse reward problem,which is solved by reward shaping and exploration.The replay buffer can update the model online by the stored tuples.Experiments on the open-source dataset BackBlaze verify the effectiveness of the proposed method.