通过优化地铁时刻表可有效降低地铁牵引能耗。为解决客流波动和车辆延误对实际节能率影响的问题,提出列车牵引和供电系统实时潮流计算分析模型和基于Dueling Deep Q Network(Dueling DQN)深度强化学习算法相结合的运行图节能优化方法,...通过优化地铁时刻表可有效降低地铁牵引能耗。为解决客流波动和车辆延误对实际节能率影响的问题,提出列车牵引和供电系统实时潮流计算分析模型和基于Dueling Deep Q Network(Dueling DQN)深度强化学习算法相结合的运行图节能优化方法,建立基于区间动态客流概率统计的时刻表迭代优化模型,降低动态客流变化对节能率的影响。对预测Q网络和目标Q网络分别选取自适应时刻估计和均方根反向传播方法,提高模型收敛快速性,同时以时刻表优化前、后总运行时间不变、乘客换乘时间和等待时间最小为优化目标,实现节能时刻表无感切换。以苏州轨道交通4号线为例验证方法的有效性,节能对比试验结果表明:在到达换乘站时刻偏差不超过2 s和列车全周转运行时间不变的前提下,列车牵引节能率达5.27%,车公里能耗下降4.99%。展开更多
异构VDES(VHF data exchange system)星座采用相同的通信频率和时分多址通信机制,使得异构星座重复覆盖区域内存在大量由时隙冲突造成的同频干扰,严重影响通信质量。针对此问题,提出一种基于深度Q网络(DQN)的星座间兼容策略。基于VDES...异构VDES(VHF data exchange system)星座采用相同的通信频率和时分多址通信机制,使得异构星座重复覆盖区域内存在大量由时隙冲突造成的同频干扰,严重影响通信质量。针对此问题,提出一种基于深度Q网络(DQN)的星座间兼容策略。基于VDES通信流程,设置船站作为资源信息中转节点,赋予卫星对通信环境的感知能力。在此基础上,将异构星座场景下的资源分配问题建模为强化学习问题,提出一种基于DQN的时隙资源分配算法。通过重构历史资源信息和当前资源信息,规划最优时隙资源分配方案,并根据结果对算法迭代优化。仿真结果表明,所提出的策略可以有效提高通信性能。展开更多
近年来深度强化学习作为一种高效可靠的机器学习方法被广泛应用在交通信号控制领域。目前,现有交通信号配时方法通常忽略了特殊车辆(例如救护车、消防车等)的优先通行;此外,基于传统深度强化学习的信号配时方法优化目标较为单一,导致其...近年来深度强化学习作为一种高效可靠的机器学习方法被广泛应用在交通信号控制领域。目前,现有交通信号配时方法通常忽略了特殊车辆(例如救护车、消防车等)的优先通行;此外,基于传统深度强化学习的信号配时方法优化目标较为单一,导致其在复杂交通场景中性能不佳。针对上述问题,基于Double DQN提出一种融合特殊车辆优先通行的双模式多目标信号配时方法(Dual-mode Multi-objective signal timing method based on Double DQN,DMDD),以提高不同交通场景下路口的通行效率。该方法首先基于路口的饱和状态选择信号控制模式,特殊车辆在紧急控制模式下被赋予更高的通行权重,有利于其更快通过路口;接着针对等待时长、队列长度和CO 2排放量3个指标分别设计神经网络进行奖励计算;最后利用Double DQN进行最优信号相位的选择,通过灵活切换信号相位以提升通行效率。基于SUMO的实验结果表明,DMDD与对比方法相比能有效缩短路口处特殊车辆的等待时长、队列长度和CO 2排放量,特殊车辆能够更快通过路口,有效地提高了通行效率。展开更多
In the face of the increasingly severe Botnet problem on the Internet,how to effectively detect Botnet traffic in realtime has become a critical problem.Although the existing deepQnetwork(DQN)algorithminDeep reinforce...In the face of the increasingly severe Botnet problem on the Internet,how to effectively detect Botnet traffic in realtime has become a critical problem.Although the existing deepQnetwork(DQN)algorithminDeep reinforcement learning can solve the problem of real-time updating,its prediction results are always higher than the actual results.In Botnet traffic detection,although it performs well in the training set,the accuracy rate of predicting traffic is as high as%;however,in the test set,its accuracy has declined,and it is impossible to adjust its prediction strategy on time based on new data samples.However,in the new dataset,its accuracy has declined significantly.Therefore,this paper proposes a Botnet traffic detection system based on double-layer DQN(DDQN).Two Q-values are designed to adjust the model in policy and action,respectively,to achieve real-time model updates and improve the universality and robustness of the model under different data sets.Experiments show that compared with the DQN model,when using DDQN,the Q-value is not too high,and the detectionmodel has improved the accuracy and precision of Botnet traffic.Moreover,when using Botnet data sets other than the test set,the accuracy and precision of theDDQNmodel are still higher than DQN.展开更多
文摘异构VDES(VHF data exchange system)星座采用相同的通信频率和时分多址通信机制,使得异构星座重复覆盖区域内存在大量由时隙冲突造成的同频干扰,严重影响通信质量。针对此问题,提出一种基于深度Q网络(DQN)的星座间兼容策略。基于VDES通信流程,设置船站作为资源信息中转节点,赋予卫星对通信环境的感知能力。在此基础上,将异构星座场景下的资源分配问题建模为强化学习问题,提出一种基于DQN的时隙资源分配算法。通过重构历史资源信息和当前资源信息,规划最优时隙资源分配方案,并根据结果对算法迭代优化。仿真结果表明,所提出的策略可以有效提高通信性能。
文摘近年来深度强化学习作为一种高效可靠的机器学习方法被广泛应用在交通信号控制领域。目前,现有交通信号配时方法通常忽略了特殊车辆(例如救护车、消防车等)的优先通行;此外,基于传统深度强化学习的信号配时方法优化目标较为单一,导致其在复杂交通场景中性能不佳。针对上述问题,基于Double DQN提出一种融合特殊车辆优先通行的双模式多目标信号配时方法(Dual-mode Multi-objective signal timing method based on Double DQN,DMDD),以提高不同交通场景下路口的通行效率。该方法首先基于路口的饱和状态选择信号控制模式,特殊车辆在紧急控制模式下被赋予更高的通行权重,有利于其更快通过路口;接着针对等待时长、队列长度和CO 2排放量3个指标分别设计神经网络进行奖励计算;最后利用Double DQN进行最优信号相位的选择,通过灵活切换信号相位以提升通行效率。基于SUMO的实验结果表明,DMDD与对比方法相比能有效缩短路口处特殊车辆的等待时长、队列长度和CO 2排放量,特殊车辆能够更快通过路口,有效地提高了通行效率。
基金the Liaoning Province Applied Basic Research Program,2023JH2/101600038.
文摘In the face of the increasingly severe Botnet problem on the Internet,how to effectively detect Botnet traffic in realtime has become a critical problem.Although the existing deepQnetwork(DQN)algorithminDeep reinforcement learning can solve the problem of real-time updating,its prediction results are always higher than the actual results.In Botnet traffic detection,although it performs well in the training set,the accuracy rate of predicting traffic is as high as%;however,in the test set,its accuracy has declined,and it is impossible to adjust its prediction strategy on time based on new data samples.However,in the new dataset,its accuracy has declined significantly.Therefore,this paper proposes a Botnet traffic detection system based on double-layer DQN(DDQN).Two Q-values are designed to adjust the model in policy and action,respectively,to achieve real-time model updates and improve the universality and robustness of the model under different data sets.Experiments show that compared with the DQN model,when using DDQN,the Q-value is not too high,and the detectionmodel has improved the accuracy and precision of Botnet traffic.Moreover,when using Botnet data sets other than the test set,the accuracy and precision of theDDQNmodel are still higher than DQN.
文摘针对传统深度Q学习网络(deep Q-learning network,DQN)在具有动态障碍物的路径规划下,移动机器人在探索时频繁碰撞难以移动至目标点的问题,通过在探索策略和经验回放机制上进行改进,提出一种改进的DQN算法。在探索策略上,利用快速搜索随机树(rapidly-exploring random tree,RRT)算法自动生成静态先验知识来指导动作选取,替代ε-贪婪策略的随机动作,提高智能体到达目标的成功率;在经验利用上,使用K-means算法设计一种聚类经验回放机制,根据动态障碍物的位置信息进行聚类分簇,着重采样与当前智能体状态相似的经验进行回放,使智能体更有效地避免碰撞动态障碍物。二维栅格化环境下的仿真实验表明,在动态环境下,该算法可以避开静态和动态障碍物,成功移动至目标点,验证了该算法在应对动态避障路径规划的可行性。
