基于深度强化学习的分布式资源管理

Distributed Resource Management Based on Deep Reinforcement Learning

超密集网络(Ultra-dense Network,UDN)可以有效提升网络的覆盖率和吞吐量,但密集部署的小基站会产生严重的干扰和能耗问题。为了减小网络干扰,进而提升网络的能源效率(Energy Efficiency,EE),提出了联合资源分配和功率控制的优化问题。对于这种复杂度高且难以求解的问题,提出了基于多智能体深度强化学习(Multi-agent Deep Reinforcement Learning,MADRL)框架的分布式优化算法。首先,各个智能体根据网络状态进行在线协作学习,得到相应的回报,产生训练数据;然后,利用深度神经网络(Deep Neural Network,DNN)对网络数据进行提取和训练,找到最佳的资源分配和功率控制策略。与其他算法相比,该算法不仅可以有效地提升网络能效,还具有很好的自适应能力。

Ultra-dense Network(UDN)can effectively improve network coverage and throughput.However,densely deployed small base stations will cause serious interference and energy consumption problems.In order to reduce network interference and improve the energy efficiency(EE)of the network,an optimization problem of joint resource allocation and power control is proposed in this paper.For such complex and difficult problems,a distributed optimization algorithm based on Multi-agent Deep Reinforcement Learning(MADRL)framework is proposed.First,each agent performs online collaborative learning according to the network status,obtains corresponding returns,and generates training data.Then,Deep Neural Network(DNN)is applied to find the best resource allocation and power control strategy by extracting and training network data.Compared with other algorithms,the algorithm in this paper not only can effectively improve the network energy efficiency,but also has a good adaptive ability.