基于强化学习的改进NSGA-Ⅱ算法的城市快速路入口匝道控制

Urban expressway on-ramp control based on improved NSGA-Ⅱ algorithm of reinforcement learning

为了缓解城市快速路拥堵和尾气排放问题,提出了基于竞争结构和深度循环Q网络的改进非支配排序遗传算法(non-dominated sorting genetic algorithm Ⅱ based on dueling deep recurrent Q network, DRQN-NSGA-Ⅱ).该算法结合了基于竞争结构的深度Q网络(dueling deep Q network, Dueling DQN)、深度循环Q网络(deep recurrent Q network, DRQN)和NSGA-Ⅱ算法,将Dueling DRQN-NSGA-Ⅱ算法用于匝道控制问题.除了考虑匝道车辆汇入以提高快速路通行效率外,还考虑了环境和能源指标,将尾气排放和燃油消耗作为评价指标.除了与无控制情况及其他算法进行比较之外, Dueling DRQN-NSGA-Ⅱ还与NSGA-Ⅱ算法进行了比较.实验结果表明:与无控制情况相比,本算法能有效改善路网通行效率、缓解环境污染、减少能源损耗;相对于无控制情况,总花费时间(total time spent, TTS)减少了16.14%,总尾气排放(total emissions, TE)减少了9.56%,总燃油消耗(total fuel consumption, TF)得到了43.49%的改善.

To alleviate urban expressway congestion and exhaust emissions,an improved NSGA-Ⅱ algorithm based on dueling deep recurrent Q network(Dueling DRQN-NSGA-Ⅱ)was proposed.This method combined dueling deep Q network(Dueling DQN),deep recurrent Q network(DRQN),non-dominated sorting genetic algorithm II(NSGA-Ⅱ),and applied Dueling DRQN-NSGA-Ⅱ to ramp control.In addition to considering the merging of ramp vehicles to improve expressway traffic efficiency,the environmental and energy indicators were also considered,and the exhaust emissions and fuel consumption were used as evaluating indicators.Dueling DRQN-NSGA-Ⅱ algorithm was compared with NSGA-Ⅱ algorithm in addition to no control situation and other algorithm.The experimental results showed that compared to the no control situation, the proposed algorithm effectivelyimproved the road network traffic efficiency, alleviated environmental pollution and reducedenergy consumption. Compared with the no control situation, the total time spent (TTS)was reduced by 16.14%, the total emissions (TE) was reduced by 9.56%, while the totalfuel consumption (TF) was improved by 43.49%.