一种基于深度强化学习的TCP网络拥塞控制协议

A TCP NETWORK CONGESTION CONTROL PROTOCOL BASED ON DEEP REINFORCEMENT LEARNING

在面对网络变化时,传统的TCP拥塞控制协议因其固有的规则机制只能做出固定的动作,既未充分利用链路带宽,也未从历史经验中学习,并且在发生拥塞时带宽恢复所用时间较长。近年来基于强化学习的拥塞控制协议(Reinforcement Learning Congestion Control,RL-CC)虽然可以有效地从历史经验中学习,但是它没有考虑历史经验在时序上存在的因果关系。对这种因果关系进行隐式提取,提出一种基于时序卷积网络和强化学习的拥塞窗口自适应智能化拥塞控制协议(Temporal convolutional network and Reinforcement Learning Congestion Control,TRL-CC)。TRL-CC通过NS-3仿真不同带宽的网络环境。大量的仿真实验表明,与NewReno和RL-CC做对比,TRL-CC在吞吐量方面提升32.8%和8.5%,时延降低41.3%和12%。

When the network changes,the protocol of traditional TCP congestion control can only make fixed actions due to its inherent rule mechanism,neither fully utilizes the link bandwidth,nor learns from historical experience,and when congestion occurs,it takes a long time to restore bandwidth.The congestion control protocol based on reinforcement learning(RL-CC)can effectively learn from historical experience,but it does not consider the causality of historical experience of time series.By implicitly extracting this causal relationship,a congestion window adaptive intelligent congestion control protocol based on temporal convolutional network and reinforcement learning(TRL-CC)is proposed.TRL-CC simulated network environments with different bandwidths through NS-3.A large number of simulation experiments show that compared with NewReno and RL-CC,TRL-CC has a 32.8%and 8.5%increase in throughput,and a delay reduction of 41.3%and 12%.