基于深度强化学习的列车运行图冗余时间布局优化研究

Time supplements allocation model for railway timetables based on deep reinforcement learning

为精细化布局冗余时间,提升运行图抗干扰能力,本文考虑多种列车最小间隔时间约束,提出了基于深度强化学习的列车运行图停站-区间冗余时间联合优化模型。首先,基于武广高速铁路列车运行实绩,研究冗余时间作用特点及布局影响因素。其次,基于决策树回归算法构建环境模型,预测不同冗余时间下的列车晚点恢复效率;基于马尔可夫决策过程中的循环反馈结构,构建后验晚点时空分布学习闭环;基于Proximal Policy Optimization算法构建智能体。最后,进行实例验证,结果表明:(1)相比于线性优化结构,基于循环反馈优化结构的模型具有更准确的后验晚点时长估计精度,平均提升约11.8%;(2)本文模型收敛稳定,平均提升列车晚点恢复效率约14.87%,最高约45.17%。

To optimize the allocation of time supplements at stations and in sections precisely and enhance the robustness of railway timetables,this paper proposes a joint optimization model based on deep reinforcement learning that considers various constraints on train headways.First,the characteristics of the time supplement utilization and the influencing factors of the problem are studied according to the actual operation records of Wuhan-Guangzhou high-speed railway.Second,an environment model is constructed using decision-tree regression to predict the train delay recovery under different allocations.A learning loop of a posterior delay distribution based on the cyclic feedback structure of the Markov decision process is then constructed,and an agent is constructed on the basis of the Proximal Policy Optimization algorithm.Finally,examples are selected for model verification.The results indicate that(1)the model with a cyclic feedback optimization structure can more accurately estimate the temporal and spatial distributions of posterior delays than a model with a linear structure,with an average improvement of 11.8%,and(2)the proposed model converges stably,with an average improvement in the delay recovery efficiency of approximately 14.87%and a maximum improvement of approximately 45.17%.