基于GA-Q-learning算法的虚拟维修作业规划模型

GA-Q-learning Algorithm-based Operation Planning Model for Virtual Maintenance Process

针对虚拟维修环境中任务执行过程存在的不确定性和随机性问题,提出了一种基于Q学习算法的作业策略规划模型,该方法将虚拟维修过程转化为选取不同动作参与状态转移的过程。在该过程中,采用试错机制和逆向求解的方法求解动作策略规划问题,并将任务特征匹配机制和顺序约束机制作为启发机制,保证策略学习过程中持续进化可行策略;在进化过程中,将动作因子赋予概率值,并采用遗传算法(GA)进化动作因子的概率分布,避免了策略学习过程中强化早期Q值较高的动作,为求解虚拟维修的最佳作业流程提供了一种行之有效的解决方法。将该方法应用于轮式挖掘机虚拟维修训练系统中,仿真结果表明,正确的动作在作业策略迭代过程中均能够获得较高的Q值,验证了方法的可行性和实用性。

To solve the uncertainty and randomization problems which happen in virtual maintenance process, a novel operation strategy planning model based on Q-learning algorithm is presented. The virtu- al maintenance process is transformed into a state transition process by using various actions. Correcting mechanism and inverse solution are used to solve task planning problem. To guarantee revolting continu- ously evolving feasible strategy, the characteristics matching mechanism and sequence constraint mecha- nism are proposed to aid in finding the optimal strategy. In the evolution process, a genetic algorithm is used to adjust the probability distribution of action value to avoid reinforcing early action with high Q-val- ue. Finally, an operation strategy optimal example for the virtual maintenance system is given to show that right action always can receive high Q-value in the evolution, which illustrates the feasibility and ap- plicability of the proposed methodology.