基于均值偏差奖赏函数的放煤口控制策略研究

Intelligent control strategy of drawing window in top-coal caving based on mean deviation reward function

根据液压支架的空间布局以及放煤口动作过程的特性,将放煤过程抽象为马尔科夫决策过程。同时,以强化学习为框架,在无需样本训练的情况下,利用Q-learning算法在线学习顶煤赋存状态与放煤口动作之间的映射关系,从而实现放煤口动作的最优决策。为保证放煤过程中煤岩分界面均匀下降,在Q-learning算法中设计了一种基于均值偏差的奖赏函数,并在Linux系统中建立了工作面连续进刀放煤三维仿真实验平台,对算法的有效性进行了验证。实验结果表明,基于均值偏差奖赏函数学习到的放煤口控制策略,能够保证在放顶煤过程中煤岩分界面更加均匀地下降。在工作面连续进刀放煤条件下,基于均值偏差奖赏函数Q-learning的智能放煤工艺,放煤平均奖励可达13467.8,比原Q-learning智能放煤工艺提高8.8%,比单轮顺序放煤等传统工艺提高约10%。

The actions of the top coal caving is abstracted to a Markov decision process by the spatial layout of the hydraulic supports and the characteristics of the windows action. Meanwhile, the reinforcement learning framework is employed to determine the optimal action of windows in top-coal caving, in which the Q-learning algorithm is adopted to learn the mapping between the state of top coal and the action of the windows online without preparing huge training samples. In the methodology, a new reward function based on mean deviation is designed for Q-learning to maintain the coal-rock boundary settlement uniform during top coal caving. Finally, a three-dimensional simulation experiment platform based on YADE discrete element analysis method is created in the Linux system, and the effectiveness of the proposed methodology is demonstrated by the experiment of cutting the coalface continuously. The results show that the coal-rock boundary driven by the proposed method is flatter during the coal falling, and the average reward of the agent for top coal caving can reach 13467.8. The reward 8.8% higher than the Q-learning method and 10% higher than the single-round sequential coal caving process.