基于深度强化学习带时间窗的绿色车辆路径问题研究

Research on Green Vehicle Routing Problem with Time Window Based on Deep Reinforcement Learning

如何在客户规定的时间内合理安排车辆运输路线,一直是物流领域亟待解决的问题。基于此,文章提出使用基于软更新策略的决斗双重深度Q网络(Dueling Double Deep Q-network,D3QN),设计动作空间、状态空间与奖励函数,对带时间窗的绿色车辆路径问题进行建模与求解。选择了小、中、大规模的总计18个算例,将三种算法的实验结果在平均奖励、平均调度车辆数、平均里程和运算时间四个维度进行比较。实验结果表明:在大多数算例中,与Double DQN和Dueling DQN相比,D3QN能在可接受的增加时间范围内,获得更高的奖励函数,调度更少的车辆数,运输更短的里程,实现绿色调度的目标。

How to reasonably arrange vehicle transportation routes within the time specified by customers has always been an urgent problem in the field of logistics.Based on this,this paper proposes to use Dueling Double Deep Q-network(D3QN)based on soft update strategy to design action space,state space and reward function to model and solve the green vehicle routing problem with time window.A total of 18 small,medium and large scale examples are selected,and the experimental results of the three algorithms are compared in four dimensions:Average reward,average number of scheduled vehicles,average mileage and operation time.The experimental results show that,in most examples,compared with Double DQN and Dueling DQN,D3QN can obtain higher reward function,dispatch fewer vehicles,transport shorter mileage,and achieve the goal of green dispatch within the range of acceptable increase in time.