多Agent强化学习下的城市路网自适应交通信号协调配时决策研究综述

Review of Study on Adaptive Traffic Signal Coordinated Timing Decision of Urban Road Network under Multi-Agent Reinforcement Learning

相对于传统的交通信号配时决策方法,多Agent强化学习及其协调方法能更好地适应城市路网交通环境的变化。为探讨其在城市路网自适应交通信号配时决策中的应用,系统地总结了多Agent强化学习及协调机制的研究方法,详细地分析了国内外研究现状,并指出现有研究中存在的问题,在此基础上对未来研究进行了展望。研究结果表明,既有研究主要针对规模较小的路网,存在维数灾难问题,强化学习与协调机制结合研究还不够深入,相关学习参数分析不够细致,仿真环境和情景现实性不强。未来研究可以引入马尔科夫博弈提高决策协调性,嵌入混合交通流、公交优先等交通管理思想增强决策实用性,引入先验知识及其他学习技术加快学习速度,融入物联网、主动管理、大数据等先进理念和前沿技术增加决策的实时性,与交通诱导等集成提升决策的系统性。

Compared with the approaches of traditional traffic signal timing decision, multi-agent reinforcement learning and its coordination method can better adapt to the variation of traffic environment of urban road network. In order to explore its application in adaptive traffic signal coordinated timing decision of urban road network, the research methods of multi-agent reinforcement learning and its coordina- tion mechanism were systematically summarized, the research status at home and abroad were extensively analyzed, and the existing research problems were put forward. Finally, the directions of future re- search on this topic were discussed. The study results show that the existing research mainly aims at small scale road network and exists the problem of dimension disaster. The research of combination of re- inforcement learning and coordination mechanism isn＇t deep enough. The relevant learning parameter analysis isn＇t meticulous enough, and the reality of simulation environment and scene aren＇t strong enough. The future research can introduce the Markov game to improve coordination, embed traffic man- agement idea such as mixed traffic flow and bus priority to enhance practicability, add the priori knowl- edge to accelerate the learning speed, combine advanced concepts and cutting-edge technology such as the Internet of Things, active management, big data to increase the real-time performance of decision, and integrate traffic guidance to promote the systematieness of decision.