一种新的基于隐喻地图的RPA路径规划算法

New RPA path planning algorithm based on metaphor map

智能化地制定机器人流程自动化(robotic process automation, RPA)执行路径有利于企业节约相关人力成本以及提高RPA的推广,提出基于改进深度双Q网络(double deep Q-learning algorithms, DDQN)算法进行RPA路径规划。首先针对存在RPA的作业环境即Web页面,不满足深度增强算法的探索条件的问题,借助隐喻地图的思想,通过构建虚拟环境来满足路径规划实验要求。同时为了提高DDQN算法探索效率,提出利用样本之间的位置信息的杰卡德系数,将其作为样本优先度结合基于排名的优先级(rank-based prioritization)构建新的采样方式。通过随机采用任务样本在虚拟环境上进行验证,证明其符合实验要求。进一步比较改进DDQN、深度Q网络(deep Q network, DQN)、DDQN、PPO以及SAC-Discrete算法的实验结果,结果显示改进算法的迭代次数更少、收敛速度更快以及回报值更高,验证了改进DDQN的有效性和可行性。

Intelligently formulating the RPA execution path is conducive to saving labor costs and improving the promotion of RPA for enterprises.For the first time,this paper proposed based on improving DDQN algorithm for RPA path planning.First of all,the problem that the working environment of RPA was a Web page,which didn’t meet the exploration conditions of the depth enhancement algorithm,with the help of the idea of metaphor map,it built the virtual environment to meet the requirements of the path planning experiment.At the same time,in order to improve the exploration efficiency of DDQN algorithm,this paper proposed to use the Jaccard coefficient of the location information between samples as a sample priority and combined it with rank-based prioritization to build new sampling methods.This paper randomly used task samples on the virtual environment to verify to demonstrate compliance with the experimental requirements.Further comparison of experimental results of the improved DDQN with DQN,DDQN,PPO and SAC-Discrete shows that the improved algorithm has fewer iterations,faster convergence speed,and higher return value,indicating the effectiveness and feasibility of the improving DDQN algorithm.