通过优化地铁时刻表可有效降低地铁牵引能耗。为解决客流波动和车辆延误对实际节能率影响的问题,提出列车牵引和供电系统实时潮流计算分析模型和基于Dueling Deep Q Network(Dueling DQN)深度强化学习算法相结合的运行图节能优化方法,...通过优化地铁时刻表可有效降低地铁牵引能耗。为解决客流波动和车辆延误对实际节能率影响的问题,提出列车牵引和供电系统实时潮流计算分析模型和基于Dueling Deep Q Network(Dueling DQN)深度强化学习算法相结合的运行图节能优化方法,建立基于区间动态客流概率统计的时刻表迭代优化模型,降低动态客流变化对节能率的影响。对预测Q网络和目标Q网络分别选取自适应时刻估计和均方根反向传播方法,提高模型收敛快速性,同时以时刻表优化前、后总运行时间不变、乘客换乘时间和等待时间最小为优化目标,实现节能时刻表无感切换。以苏州轨道交通4号线为例验证方法的有效性,节能对比试验结果表明:在到达换乘站时刻偏差不超过2 s和列车全周转运行时间不变的前提下,列车牵引节能率达5.27%,车公里能耗下降4.99%。展开更多
异构VDES(VHF data exchange system)星座采用相同的通信频率和时分多址通信机制,使得异构星座重复覆盖区域内存在大量由时隙冲突造成的同频干扰,严重影响通信质量。针对此问题,提出一种基于深度Q网络(DQN)的星座间兼容策略。基于VDES...异构VDES(VHF data exchange system)星座采用相同的通信频率和时分多址通信机制,使得异构星座重复覆盖区域内存在大量由时隙冲突造成的同频干扰,严重影响通信质量。针对此问题,提出一种基于深度Q网络(DQN)的星座间兼容策略。基于VDES通信流程,设置船站作为资源信息中转节点,赋予卫星对通信环境的感知能力。在此基础上,将异构星座场景下的资源分配问题建模为强化学习问题,提出一种基于DQN的时隙资源分配算法。通过重构历史资源信息和当前资源信息,规划最优时隙资源分配方案,并根据结果对算法迭代优化。仿真结果表明,所提出的策略可以有效提高通信性能。展开更多
In the face of the increasingly severe Botnet problem on the Internet,how to effectively detect Botnet traffic in realtime has become a critical problem.Although the existing deepQnetwork(DQN)algorithminDeep reinforce...In the face of the increasingly severe Botnet problem on the Internet,how to effectively detect Botnet traffic in realtime has become a critical problem.Although the existing deepQnetwork(DQN)algorithminDeep reinforcement learning can solve the problem of real-time updating,its prediction results are always higher than the actual results.In Botnet traffic detection,although it performs well in the training set,the accuracy rate of predicting traffic is as high as%;however,in the test set,its accuracy has declined,and it is impossible to adjust its prediction strategy on time based on new data samples.However,in the new dataset,its accuracy has declined significantly.Therefore,this paper proposes a Botnet traffic detection system based on double-layer DQN(DDQN).Two Q-values are designed to adjust the model in policy and action,respectively,to achieve real-time model updates and improve the universality and robustness of the model under different data sets.Experiments show that compared with the DQN model,when using DDQN,the Q-value is not too high,and the detectionmodel has improved the accuracy and precision of Botnet traffic.Moreover,when using Botnet data sets other than the test set,the accuracy and precision of theDDQNmodel are still higher than DQN.展开更多
文摘异构VDES(VHF data exchange system)星座采用相同的通信频率和时分多址通信机制,使得异构星座重复覆盖区域内存在大量由时隙冲突造成的同频干扰,严重影响通信质量。针对此问题,提出一种基于深度Q网络(DQN)的星座间兼容策略。基于VDES通信流程,设置船站作为资源信息中转节点,赋予卫星对通信环境的感知能力。在此基础上,将异构星座场景下的资源分配问题建模为强化学习问题,提出一种基于DQN的时隙资源分配算法。通过重构历史资源信息和当前资源信息,规划最优时隙资源分配方案,并根据结果对算法迭代优化。仿真结果表明,所提出的策略可以有效提高通信性能。
基金the Liaoning Province Applied Basic Research Program,2023JH2/101600038.
文摘In the face of the increasingly severe Botnet problem on the Internet,how to effectively detect Botnet traffic in realtime has become a critical problem.Although the existing deepQnetwork(DQN)algorithminDeep reinforcement learning can solve the problem of real-time updating,its prediction results are always higher than the actual results.In Botnet traffic detection,although it performs well in the training set,the accuracy rate of predicting traffic is as high as%;however,in the test set,its accuracy has declined,and it is impossible to adjust its prediction strategy on time based on new data samples.However,in the new dataset,its accuracy has declined significantly.Therefore,this paper proposes a Botnet traffic detection system based on double-layer DQN(DDQN).Two Q-values are designed to adjust the model in policy and action,respectively,to achieve real-time model updates and improve the universality and robustness of the model under different data sets.Experiments show that compared with the DQN model,when using DDQN,the Q-value is not too high,and the detectionmodel has improved the accuracy and precision of Botnet traffic.Moreover,when using Botnet data sets other than the test set,the accuracy and precision of theDDQNmodel are still higher than DQN.
