融合AP聚类算法和宽度学习系统的分布外硬盘故障预测

Out-of-Distribution Hard Disk Failure Prediction with Affinity Propagation Clustering and Broad Learning Systems

硬盘是云数据中心最主要的存储设备,硬盘故障预测是保障数据安全的重要手段。但是,硬盘的故障与健康样本之间存在着极端的数量不平衡问题,这会导致模型偏差;此外,不同型号的硬盘数据分布存在一定的差异,在特定硬盘数据上训练的模型往往不适用于其他硬盘。对于这两个问题,文中提出了一种融合AP聚类算法和宽度学习系统的分布外硬盘故障预测方法。针对样本不平衡问题,文中使用AP聚类算法对硬盘故障出现前一阶段的样本集进行聚类,将与故障样本处于同一聚类簇的样本扩充为故障样本。针对不同型号硬盘分布存在差异的问题,文中结合流形正则化框架和宽度学习系统来学习硬盘数据的低维结构,提高模型对未知分布数据的泛化能力。实验结果表明,在AP聚类算法重采样的样本集上,相较于用于对比的重采样方法得到的样本集,多种故障预测方法的F1_Score取得了平均0.2的提升。此外,在分布外硬盘故障预测任务上,所提模型的F1_Score相比对比方法提升了0.1~0.2。

Hard disk is the primary storage device in cloud data centers,and hard disk failure prediction is crucial for ensuring data security.However,there is a significant imbalance between failure and healthy SMART samples,which can lead to model bias.Moreover,hard disk models exhibit varying data distributions.Prediction models trained on specific hard disk data may not be suitable for other hard disks.To address these issues,this paper proposes a method for out-of-distribution hard disk failure prediction by combining the AP clustering algorithm and the broad learning system.To tackle the sample imbalance problem,this paper uses the AP clustering algorithm to cluster samples close to failure and treats all samples in the cluster containing determined failure instances as additional failure samples.To address the distribution differences of hard disk models,this paper combines the manifold regularization framework and the broad learning system to learn the low-dimensional structure of hard disk data,thereby improving the model’s generalization ability to unknown data.Experimental results show that,on the dataset resampled by the AP clustering algorithm,the F1_Score of multiple methods increases by an average of 0.2 compared to the datasets resampled by comparative methods.Additionally,in the task of predicting out-of-distribution hard disk failures,the F1_Score of the proposed model increases by 0.1~0.2 compared to other methods.