摘要
根据液压支架的空间布局以及放煤口动作过程的特性,将放煤过程抽象为马尔科夫决策过程。同时,以强化学习为框架,在无需样本训练的情况下,利用Q-learning算法在线学习顶煤赋存状态与放煤口动作之间的映射关系,从而实现放煤口动作的最优决策。为保证放煤过程中煤岩分界面均匀下降,在Q-learning算法中设计了一种基于均值偏差的奖赏函数,并在Linux系统中建立了工作面连续进刀放煤三维仿真实验平台,对算法的有效性进行了验证。实验结果表明,基于均值偏差奖赏函数学习到的放煤口控制策略,能够保证在放顶煤过程中煤岩分界面更加均匀地下降。在工作面连续进刀放煤条件下,基于均值偏差奖赏函数Q-learning的智能放煤工艺,放煤平均奖励可达13467.8,比原Q-learning智能放煤工艺提高8.8%,比单轮顺序放煤等传统工艺提高约10%。
The actions of the top coal caving is abstracted to a Markov decision process by the spatial layout of the hydraulic supports and the characteristics of the windows action. Meanwhile, the reinforcement learning framework is employed to determine the optimal action of windows in top-coal caving, in which the Q-learning algorithm is adopted to learn the mapping between the state of top coal and the action of the windows online without preparing huge training samples. In the methodology, a new reward function based on mean deviation is designed for Q-learning to maintain the coal-rock boundary settlement uniform during top coal caving. Finally, a three-dimensional simulation experiment platform based on YADE discrete element analysis method is created in the Linux system, and the effectiveness of the proposed methodology is demonstrated by the experiment of cutting the coalface continuously. The results show that the coal-rock boundary driven by the proposed method is flatter during the coal falling, and the average reward of the agent for top coal caving can reach 13467.8. The reward 8.8% higher than the Q-learning method and 10% higher than the single-round sequential coal caving process.
作者
罗开成
高阳
杨艺
常亚军
袁瑞甫
LUO Kai-cheng;GAO Yang;YANG Yi;CHANG Ya-jun;YUAN Rui-fu(Zhengzhou Coal Mining Machinery Group Company Limited,Zhengzhou 450016,China;Zhengzhou Coal Machine Hydraulic Control Group Company Limited,Zhengzhou 450016,China;Collage of Electrical Engineering and Automation,Henan Polytechnic University,Jiaozuo 454000,China;State Collaborative Innovation Center of Coal Work Safety and Clean-efficient Utilization,Jiaozuo 454000,China)
出处
《煤炭工程》
北大核心
2022年第9期105-111,共7页
Coal Engineering
基金
国家重点研发计划项目(2018YFC0604502)
河南省煤矿智能开采技术创新中心支撑项目(2021YD01)
河南省科技攻关项目(212102210390)。
关键词
综合机械化开采
放顶煤
智能化
强化学习
fully mechanized mining
top-coal caving
intellectualization
reinforcement learning