一种基于Dyna-Q学习的旋翼无人机视觉伺服智能控制方法

A visual servo intelligent control method for rotor UAV based on Dyna-Q learning

基于图像的视觉伺服机器人控制方法通过机器人的视觉获取图像信息,然后形成基于图像信息的闭环反馈来控制机器人的合理运动.经典视觉伺服的伺服增益的选取在大多数条件下是人工赋值的,故存在鲁棒性差、收敛速度慢等问题.针对该问题,提出一种基于Dyna-Q的旋翼无人机视觉伺服智能控制方法调节伺服增益以提高其自适应性.首先,使用基于费尔曼链码的图像特征提取算法提取目标特征点;然后,使用基于图像的视觉伺服形成特征误差的闭环控制;其次,针对旋翼无人机强耦合欠驱动的动力学特性提出一种解耦的视觉伺服控制模型;最后,建立使用Dyna-Q学习调节伺服增益的强化学习模型,通过训练可以使得旋翼无人机自主选择伺服增益.Dyna-Q学习在经典的Q学习的基础上通过建立环境模型来存储经验,环境模型产生的虚拟样本可以作为学习样本来进行值函数的迭代.实验结果表明,所提出的方法相比于传统控制方法PID控制以及经典的基于图像视觉伺服方法具有收敛速度快、稳定性高的优势.

The image-based visual servo control method of robots obtains the image information through the robot’s vision and then forms the closed-loop feedback based on the image information to control the robot’s reasonable movement.However, due to the problem of poor robustness and slow convergence, the selection of servo gain for classical visual servoing is artificial assignment under most conditions. Therefore, an intelligent servo control method based on Dyna-Q learning is proposed to adjust the servo gain to improve its adaptability. Firstly, this method uses the image feature extraction algorithm based on Felman chain code to extract the target feature point, then uses the image-based visual servoing to form the closed-loop control of the characteristic error. Then, this paper presents a decoupling visual servoing control model for the dynamic characteristics of rotor UAV’s strong coupling underactuated. Finally, a reinforcement learning model using Dyna-Q learning to adjust the servo gain is established, through which the rotor UAV can choose the servo gain independently. The Dyna-Q learning method learns to store experience on the basis of classical Q-Learning by setting up an environment model, and the virtual samples generated by the environment model can be used as learning samples to iterate the value function. The experimental results show that the proposed method is faster and more stable than the classical PID control and classical image based visual servo methods.