基于个体-协同触发强化学习的多机器人行为决策方法

Multi-robot behavior decision making method based on individual-collaborative trigger reinforcement learning

为了提高多机器人行为最优决策控制中强化学习的效率和收敛速度,研究了多机器人的分布式马尔科夫建模与控制策略。根据机器人有限感知能力设计了个体-协同感知触发函数,机器人个体从环境观测结果计算个体-协同触发响应概率,定义一次触发过程后开始计算联合策略,减少机器人间通讯量和计算资源。引入双学习率改进Q学习算法,并将该算法应用于机器人行为决策。仿真实验结果表明,当机器人群组数量在20左右时,本文算法的协同效率较高,单位时步比为1.0850。同时距离调节参数η对机器人协同搜索效率有影响,当η=0.008时,所需的移动时步比和平均移动距离都能达到最小值。通过双学习率的引入,该算法较基于环境模型的强化学习算法具有更高的学习效率和适用性,平均性能提升35%,对于提高多机器人自主协同能力具有较高的理论意义及应用价值。

In order to improve the efficiency and convergence speed of reinforcement learning in multi-robot behavior optimal decision making control,the distributed Markov modeling and control strategy for multi-robots are studied in this paper.According to the limited perception ability of the robots,an individual-cooperative trigger perception function is designed.The individual robot calculates the individual-cooperative trigger response probability from the environment observation results,and defines that after a trigger process the joint strategy calculation starts,which reduces the communication amount and computing resources among robots.The Q-learning algorithm is improved through introducing the dual-learning rate strategy,which is applied to the behavior decision-making of robots.The simulation experiment results show that the algorithm proposed in this paper has quite high cooperative efficiency when the number of robots in the group is about 20.The unit time step ratio is 1.0850.At the same time,the distance adjustment parameterηhas an influence on the cooperative search efficiency of the robot.Whenηis 0.008,the required moving time step ratio and average moving distance can reach minimum.Through introducing the double learning rate,the proposed algorithm possesses higher learning efficiency and applicability compared with the reinforcement learning algorithm based on environment model,the average performance improvement reaches about 35%.The proposed algorithm has a high theoretical significance and application value for improving the autonomous cooperative ability of multi-robots.