基于深度强化学习的大口径轴孔装配策略

Assembly strategy for large-diameter peg-in-hole based on deep reinforcement learning

针对大口径轴孔装配任务中存在的惯性冲击大、力控不稳定、装配精度差等问题,提出基于深度强化学习与模糊策略的大口径轴孔装配策略.该策略通过模糊动作生成器对强化学习算法输出的装配动作进行补偿,实现精确的状态跟踪.通过深度确定性决策梯度(DDPG)算法采集环境状态数据并计算输出动作,引导机器人改变装配状态.引入模糊动作生成器,与DDPG算法结合生成DDPGFA装配策略,利用模糊策略添加动作系数,提高装配动作准确性.在合理制定奖赏函数和模糊规则的基础上,实现训练过程的快速收敛.通过设定安全阈值保证在线学习过程中系统的受力安全.大口径轴孔装配仿真和实验结果表明,与未采用模糊动作的强化学习装配策略相比,DDPGFA策略能在更稳定的步数下完成装配,且离线训练速度提升约15%,装配接触力减小约30%.

A large-diameter peg-in-hole assembly strategy based on deep reinforcement learning and fuzzy strategy was proposed,in order to address the problems of large inertial impact,unstable force control,and poor assembly accuracy in the large-diameter peg-in-hole assembly task.In this strategy,the assembly actions output from the reinforcement learning algorithm were compensated by a fuzzy action generator to achieve accurate state tracking.The deep deterministic policy gradient(DDPG)algorithm was used to acquire the environmental state data and calculate the output actions,by which the robot’s assembly state was changed.A fuzzy action generator was introduced to combine with DDPG algorithm to generate DDPGFA assembly strategy,and the fuzzy strategy was utilized to add action coefficients to improve the accuracy of assembly actions.Based on the rational formulation of reward function and fuzzy rules,the rapid convergence of the training process was realized.System stresses were secured during e-learning by setting safety thresholds.Simulation and experimental results of large-diameter peg-inhole assembly showed that,the DDPGFA strategy can complete the assembly in a more stable number of steps,and the offline training speed was increased by about 15%and the assembly contact force was reduced by about 30%compared with the reinforcement learning assembly strategy without fuzzy actions.