基于深度强化学习的救护车动态重定位调度研究

Dynamic ambulance redeployment based on deep reinforcement learning

救护车是挽救患者生命的重要医疗资源,合理调配有限的救护车资源可以降低呼叫响应时间,提高医疗服务水平.本文面向救护车动态重定位调度问题,提出了一种基于强化学习的调度策略结构.为解决传统强化学习所面临的高维状态空间的挑战,本文基于深度Q值网络(DQN)方法,提出了一种考虑多种调度交互因子的算法RedCon-DQN,以在给定环境状态下得到最优的重定位调度策略.在此基础上,本文还提出了急救网络弹性概念,以评估各站点对全局救护优化目标的影响力.最后,基于南京市2016年~2017年的实际救护车呼叫及响应数据,构造了环境交互模拟器.在模拟器中通过大规模数据实验,验证了模型得到的调度策略相比已有方法的优越性,并分析了不同时段下调度策略的有效性及其特点.

Ambulance is one of the most crucial medical resources to save patients’lives.Appropriate allocations of limited ambulances to different emergency stations can effectively lower the response time and lift medical service quality.In view of this,we propose a reinforcement learning based scheduling structure to resolve the dynamic ambulance redeployment problems.In order to address the challenges aroused from high-dimensional state spaces,we propose RedCon-DQN by considering multiple scheduling interactive factors,which is based on Deep Q-value Network(DQN)and can output the optimized redeployment policy given specific environment.In addition,we propose a measurement,emergency-network resilience to evaluate the influences of each individual emergency station on the global optimization objectives.Finally,we construct a environment interactive simulator based on the emergency calls and response data of Nanjing from 2016 to 2017.We validate the advantages of the proposed redeployment policy over the state-of-the-art methods,and further analyze the effectiveness and characteristics in different time periods.