基于深度强化学习的深圳市急救车调度算法

Ambulance Dispatching Algorithm in Shenzhen Based on Deep Reinforcement Learning

在院前急救领域中,急救反应时间是指患者拨打急救电话后,急救车到达现场的时间。传统急救车调度算法未全面考虑急救环境的动态性和复杂性因素,导致模型优化的急救反应时间与实际情况存在偏差。将急救车调度问题建模成马尔科夫决策过程,构建基于深度强化学习的急救车调度算法。以多层感知机作为评分网络结构,通过将急救站的动态信息映射为各个急救站的得分,确定急救车被调往各急救站的概率。同时,结合急救车调度的动态决策特点,利用强化学习中演员-评论家框架下的近端策略优化算法改进评分网络参数。在深圳市急救中心真实急救数据集上的实验结果表明,相比Fixed、DSM、MEXCLP等算法,该算法在每个急救事件中的急救反应时间平均缩短约80 s,并且在10 min内急救车的平均到达比例为36.5%,能够实时地将急救车调度到合适的急救站。

In prehospital emergency,the emergency response time refers to the time when the ambulance arrives at the scene after the patient dials the emergency phone number.The traditional ambulance dispatching algorithm does not fully consider the dynamics and complexity factors of the emergency environment,resulting in the deviation between the optimization emergenay response time of the model and the actual situation.The ambulance dispatching problem is modeled as a Markov Decision Process(MDP),and an ambulance dispatching algorithm based on deep reinforcement learning is constructed.The multilayer perceptron is used as the scoring network structure,and the dynamic information of the emergency station is mapped to the scores of each emergency station to determine the probability of the ambulance being transferred to each emergency station. Combined with the dynamic decision-making characteristics of ambulance dispatching,the Proximal Policy Optimization(PPO)algorithm under the Actor-Critic framework in reinforcement learning is used to improve the parameters of the scoring network.The experimental results for an actual emergency dataset of the Shenzhen center for prehospital care show that compared with the Fixed,DSM,MEXCLP,and other algorithms,the emergency response time of this algorithm in each emergency event is shortened by approximately 80 s on average,and the average arrival proportion of emergency vehicles within 10 min is 36.5%.The ambulances are dispatched to the appropriate emergency stations in real time.