摘要
在面对网络变化时,传统的TCP拥塞控制协议因其固有的规则机制只能做出固定的动作,既未充分利用链路带宽,也未从历史经验中学习,并且在发生拥塞时带宽恢复所用时间较长。近年来基于强化学习的拥塞控制协议(Reinforcement Learning Congestion Control,RL-CC)虽然可以有效地从历史经验中学习,但是它没有考虑历史经验在时序上存在的因果关系。对这种因果关系进行隐式提取,提出一种基于时序卷积网络和强化学习的拥塞窗口自适应智能化拥塞控制协议(Temporal convolutional network and Reinforcement Learning Congestion Control,TRL-CC)。TRL-CC通过NS-3仿真不同带宽的网络环境。大量的仿真实验表明,与NewReno和RL-CC做对比,TRL-CC在吞吐量方面提升32.8%和8.5%,时延降低41.3%和12%。
When the network changes,the protocol of traditional TCP congestion control can only make fixed actions due to its inherent rule mechanism,neither fully utilizes the link bandwidth,nor learns from historical experience,and when congestion occurs,it takes a long time to restore bandwidth.The congestion control protocol based on reinforcement learning(RL-CC)can effectively learn from historical experience,but it does not consider the causality of historical experience of time series.By implicitly extracting this causal relationship,a congestion window adaptive intelligent congestion control protocol based on temporal convolutional network and reinforcement learning(TRL-CC)is proposed.TRL-CC simulated network environments with different bandwidths through NS-3.A large number of simulation experiments show that compared with NewReno and RL-CC,TRL-CC has a 32.8%and 8.5%increase in throughput,and a delay reduction of 41.3%and 12%.
作者
卢光全
李建波
吕志强
Lu Guangquan;Li Jianbo;LüZhiqiang(College of Computer Science&Technology,Qingdao University,Qingdao 266071,Shandong,China)
出处
《计算机应用与软件》
北大核心
2023年第3期179-187,共9页
Computer Applications and Software
基金
国家重点研发计划重点专项项目(2018YFB2100303)
山东省高等学校青创科技计划创新团队项目(2020KJN011)
山东省博士后创新人才支持计划项目(40618030001)
国家自然科学基金项目(61802216)
中国博士后基金项目(2018M642613)。
关键词
TCP
AIMD
拥塞控制
强化学习
时序卷积网络
TCP
AIMD
Congestion control
Reinforcement learning
Temporal convolutional network