Dyna-QUF:Dyna-Q based univector field navigation for autonomous mobile robots in unknown environments 被引量：1

Dyna-QUF: Dyna-Q based univector field navigation for autonomous mobile robots in unknown environments

下载PDF

导出

摘要 A novel approach was presented to solve the navigation problem of autonomous mobile robots in unknown environments with dense obstacles based on a univector field method. In an obstacle-free environment, a robot is ensured to reach the goal position with the desired posture by following the univector field. Contrariwise, the univector field cannot guarantee that the robot will avoid obstacles in environments. In order to create an intelligent mobile robot being able to perform the obstacle avoidance task while following the univector field, Dyna-Q algorithm is developed to train the robot in learning moving directions to attain a collision-free path for its navigation. Simulations on the computer as well as experiments on the real world prove that the proposed algorithm is efficient for training the robot in reaching the goal position with the desired final orientation. A novel approach was presented to solve the navigation problem of autonomous mobile robots in unknown environments with dense obstacles based on a univector field method. In an obstacle-free environment, a robot is ensured to reach the goal position with the desired posture by following the univector field. Contrariwise, the univector field cannot guarantee that the robot will avoid obstacles in environments. In order to create an intelligent mobile robot being able to perform the obstacle avoidance task while following the univector field, Dyna-Q algorithm is developed to train the robot in learning moving directions to attain a collision-free path for its navigation. Simulations on the computer as well as experiments on the real world prove that the proposed algorithm is efficient for training the robot in reaching the goal position with the desired final orientation.

作者 VIET Hoang-huu CHOI Seung-yoon CHUNG Tae-choong

机构地区 Department of Computer Engineering

出处《Journal of Central South University》 SCIE EI CAS 2013年第5期1178-1188,共11页 中南大学学报（英文版）

基金 Project(2010-0012609) supported by the Basic Science Research Program,Korea

关键词自主移动机器人环境导航智能移动机器人目标位置移动方向现实世界 Q算法 Dyna-Q mobile robot reinforcement learning univector field

分类号 TP242 [自动化与计算机技术—检测技术与自动化装置] X24 [环境科学与工程—环境科学]

引文网络
相关文献

参考文献17

1DUDEK G, JENKIN M. Computational principles of mobile robotics [M]. Cambridge University Press, 2010: 80-105.
2KIM Y J, KIM J H, KWON D S. Evolutionary programming-based univector field navigation method for fast mobile robots [J]. IEEE Trans on Systems, Man, and Cybernetics B, 2001, 31(3): 450-458.
3PARK K H, KJM Y J, KIM J H. Modular Q-Iearning based multi-agent cooperation for robot soccer [J]. Robotics and Autonomous Systems, 2001, 35(2): 109-122.
4WATKJNS C. Learning from delayed rewards [D]. London: King's College, 1989.
5HONG Y, KIM J H. Footstep planning based on univector field method for humanoid robot [C]// Lecture Notes in Computer Science. Springer-Verlag Berlin, Heidelberg, 2009: 125-134.
6VIEN N A, VIET N H, PARK H J, LEE S G, CHUNG T C. Q-Learning based Univector Field Navigation Method for Mobile Robots [C]// Advances and Innovations in Systems, Computing Sciences and Software Engineering. Springer-Verlag New York Inc, 2007: 463-468.
7KAELBLING L P, LITTMAN M L, MOORE A W. Reinforcement learning: A survey [J]. Journal of Artificial Intelligence Research, 1996,4(1): 237-285.
8SUTTON R S, BARTO A G. Reinforcement learning: An introduction [M]. Cambridge: The MIT Press, 1998: 66-92.
9WATKINS C J C H, DAYAN P. Technical note Q-Learning [J]. Machine Learning, 1992,8(1): 279-292.
10KJM D H, KIM Y J, KJM K C, KfM J H, VADAKKEPAT P. Vector field based path planning and Petri-net based role selection mechanism with Q-Iearning for soccer robots [J]. Intelligent Automation and Soft Computing, 2000, 6(1): 75-87.

二级参考文献14

1邹小兵,蔡自兴,孙国荣.Non-smooth environment modeling and global path planning for mobile robots[J].Journal of Central South University of Technology,2003,10(3):248-254. 被引量：6
2文志强,蔡自兴.Global path planning approach based on ant colony optimization algorithm[J].Journal of Central South University of Technology,2006,13(6):707-712. 被引量：5
3祝晓才,董国华,蔡自兴,胡德文.Robust simultaneous tracking and stabilization of wheeled mobile robots not satisfying nonholonomic constraint[J].Journal of Central South University of Technology,2007,14(4):537-545. 被引量：5
4Andrew G. Barto,Sridhar Mahadevan.Recent Advances in Hierarchical Reinforcement Learning[J].Discrete Event Dynamic Systems (-).2003(1-2)
5KAELBLING L P,,LITTMAN M L,MOORE A W.Reinforcement learning:A survey[].Journal of Artificial Organs.1996
6SUTTON R S,BARTO A.Reinforcement learning:An introduction[]..1998
7BANERJEE B,STONE P.General game learning using knowledge transfer[].Proceedings of theth International Joint Conference on Artificial Intelligence.2007
8ASADI M,HUBER M.Effective control knowledge transfer through learning skill and representation hierarchies[].Proceedings of theth International Joint Conference on Artificial Intelligence.2007
9KONIDARIS G,BARTO A.Autonomous shaping:Knowledge transfer in reinforcement learning[].Proceedings of therd International Conference on Machine Learning.2006
10MEHTA N,NATARAJAN S,TADEPALLI P,FERN A.Transfer in variable-reward hierarchical reinforcement learning[].Workshop on Transfer Learning at Neural Information Processing Systems.2005