基于强化学习的艾灸机器人温度控制策略研究

Study on temperature control strategy of moxibustion robot based on reinforcement learning

针对传统PID控制算法在艾灸机器人温度控制中存在参数辨识复杂、适应性差等问题,将强化学习引入到艾灸机器人温控领域中,提出了一种改进强化学习算法。首先,通过多物理场仿真软件和神经网络联合搭建智能体离线训练仿真环境,以解决智能体在线训练效率低下的问题;然后,提出一种结合奖励引导和余弦退火策略的改进强化学习算法,提高算法的收敛性和成功率;最后,将仿真环境训练后的模型迁移到真实环境进行实验验证。实验结果表明,温度超调量为0.2℃,稳态温度保持在43.1℃±0.4℃内,改进后的强化学习算法相比于传统PID控制算法的温度控制能力更好。

Aiming at the problems of complex parameter identification and poor adaptability of traditional PID control algorithm in temperature control of moxibustion robot,reinforcement learning is introduced into the field of temperature control of moxibustion robot,and an improved reinforcement learning algorithm is proposed.First,the offline training simulation environment of the agent is jointly built by multi-physics simulation software and neural network to solve the problem of low efficiency of online training of the agent;then,an improved reinforcement learning algorithm combining reward guidance and cosine annealing strategy is proposed to improve the convergence and success rate of the algorithm;finally,the model trained in the simulation environment is transferred to the real environment for experimental verification.The experimental results show that the temperature overshoot is 0.2℃,and the steady-state temperature is kept within 43.1±0.4℃.The improved reinforcement learning algorithm has better temperature control ability than the traditional PID control algorithm.