基于图嵌入编码形态信息的非均匀多任务强化学习方法

Method for inhomogeneous multi-task reinforcement learning based on morphological information encoding by graph embedding

传统强化学习方法存在效率低下、泛化性能差、策略模型不可迁移的问题。针对此问题,提出了一种非均匀多任务强化学习方法,通过学习多个强化任务提升效率和泛化性能,将智能体形态构建为图,利用图神经网络能处理任意连接和大小的图来解决状态和动作空间维度不同的非均匀任务,突破模型不可迁移的局限,充分发挥图神经网络天然地利用图结构归纳偏差的优点,实现了模型高效训练和泛化性能提升,并可快速迁移到新任务。多任务学习实验结果表明,与以往方法相比,该方法在多任务学习和迁移学习实验中均表现出更好的性能,在迁移学习实验中展现出更准确的知识迁移。通过引入图结构偏差,使该方法具备更高的效率和更好的迁移泛化性能。

Traditional reinforcement learning methods have problems of low efficiency,poor generalization performance,and untransferable policy models.In response to this issue,this paper proposed an inhomogeneous multitask reinforcement learning method,which improved efficiency and generalization performance by learning multiple reinforcement tasks.It constructed the morphology of agent into a graph,and the graph neural network could handle graphs with any connection pattern and size graph,which was really suitable to solve inhomogeneous tasks with different dimensions of state and action space.This breaks through the limitations that model couldn’t be transferred and fully utilizes the advantages of graph neural network’s natural use of graph structure to induce bias.The model had achieved efficient training and improved generalization performance,and could be quickly migrated to new tasks.The results of multi task learning experiments show that compared with previous methods,this method exhibits better performance in both multi task learning and transfer learning experiments,and exhibits more accurate knowledge transfer in transfer learning experiments.By introducing bias in the structure of the agent graph,this method has achieved higher efficiency and better migration generalization performance.