基于Q学习机场地面滑行路径动态规划

Q-Learning Applied to Airport Taxiway Dynamic Programming

塔台管制员目前对场面飞机滑行路线分配主要依据航班序列、进离港类型以及目标终点等为参考。本文通过对管制员管制行为使用Q学习建模,并引入未来航班先验概率滑行路线加入待检测冲突集合,使管制员Agent具备预估未来时刻冲突能力,对常见滑行冲突得到最优滑行路径。本文首先讨论了Q学习对于地面冲突离散状态数量的有限性,接着对状态的动作序列和回报函数进行设计。实验环境对地面滑行道优先级进行了人工标注,生成随机航班时刻样本集用于训练。仿真结果中管制员Agent能有效解决冲突,体现了该方案的可行性和优越性。

The path assignment of aircraft is mainly based on the schedule time, the type of flight-plan and the terminal point. In this paper, we use Q-Learning algorithm to study the behavior of controller, introduced the prior probability path to join the conflict path set to be detected. The Agent has the ability to predict the future conflict and to optimize taxing time for the situation of regular conflict while respecting aircraft separation and airport capacities. We begin by discussing the limited count of states in Q-Learning. Next, we design the actions and the reward function of the state. In the experiment, we have manually annotated the ground taxiway priority and generated random flightplan sample sets for training. The simulation results show that the controller Agent can solve the conflict effectively, which reflects the feasibility and superiority of this work.