基于深度强化学习的串联系统屏蔽故障维修策略研究

Maintenance Optimization for Series Systems Subjected to Masked Failures Based on Deep Reinforcement Learning

多部件串联系统维修决策优化问题得到越来越多的关注,目前绝大多数相关研究均假设系统失效时其失效部件是可直接观测的。实际的生产运营中,当系统发生失效时,其故障原因往往是屏蔽的,需要采用专业的工具进行诊断才能确定失效部件,继而进行维修。当诊断成本较高且部件接近更换役龄时,可以选择直接更换整个系统而不进行诊断。面向串联系统,考虑其发生屏蔽故障时,可以执行故障诊断确定失效部件继而更换,或者直接更换整个系统,建立半马尔科夫决策模型,以长周期成本率最低为目标函数,决策出每次屏蔽故障时的最佳动作。考虑到多部件导致的状态空间维数灾难问题,采用深度强化学习算法进行求解。最后将模型和算法应用于水电站液压自动抓梁系统,证明了其有效性。

More and more attentions has been paid to maintenance optimization of multi-component series systems in recent years.Most of related works assume that the failure parts of the system are self-announing upon each expected failure.In some real practise,the failure reason is masked,which can only be revealed with professional diagnosis equipment.Therefore,the whole system can be replaced instead of carrying out diagnosis in some cases.The maintennace optimization of series systems subjected to masked failures was investigated,upon which the diagnosis could be carried out to find the failed component and replace it,or the whole system could be replaced directly.The problem was formulated as a semi-Markov decision process to minimize the long-term average cost.Deep reinforcement learning method was adopted to deal with the curse of dimensionality.The effectiveness of the proposed model was validated by a numerical study on an automatic hydraulic grab system.