基于PER-DDPG算法的城市轨道交通越区切换研究

Research on Handover of Urban Rail Transit Based on PER-DDPG Algorithm

针对传统IEEE802.11越区切换方式存在较高的切换延时以及乒乓切换等问题,提出深度强化学习(Deep Q-Network,DQN)越区切换算法。通过对列车运行的特征状态信息进行提取输入,考虑列车运行速度及场强、切换阈值等动态信息构建越区切换模型。同时针对算法时间成本复杂度及稳定性,采用优先经验回放深度确定性策略梯度(Prioritized Experience Replay-Deep Deterministic Policy Gradient,PER-DDPG)算法,将列车状态空间信息传输至PER-DDPG网络中进行优化分析。结果表明基于PER-DDPG算法优化后的列车越区切换模型使用该算法时间计算成本降低,数据包传输延时约降低55%。

In order to solve the problems of the traditional IEEE802.11 off-zone switching mode,such as high switching delay and ping-pong switching,a deep Q-Network off-zone switching algorithm is proposed.A crossover algorithm of deep Q-Network is proposed.By extracting and inputting the characteristic state information of train operation,a crossover switching model is constructed considering the dynamic information of train running speed,field strength and switching threshold.Meanwhile,Prioritized Experience Replay-Deep Deterministic Policy Gradient is prioritized to the complexity and stability of the algorithm's time cost.The spatial information of train status is transmitted to PER-DDPG network for optimization analysis.The results show that the time calculation cost of the optimized train crossover model based on PER-DDPG algorithm is reduced.the packet transmission delay is reduced by about 55%.