基于强化学习的异构超密度网络资源分配算法

Resource Allocation Algorithm for Heterogeneous Ultra-dense Networks Based on Reinforcement Learning

为了保证下行链路用户服务质量(Quality of Service, QoS),提升异构超密度网络的频谱利用率(Spectrum Efficient, SE)和能源效率(Energy-Efficient, EE),提出了一种基于多智能体强化学习(Deep Reinforcement Learning, DRL)的频谱和功率联合分配算法。首先,以频谱利用率和能源效率为优化目标,用户服务质量为约束,得到资源分配优化函数。然后定义多智能体用户状态空间,奖励以及动作空间,通过较小的通信开销获得状态空间信息,得到一维状态空间数据,减少网络的输入数据量,用户利用自身的信道状态信息(Channel State Information, CSI)而不依赖全局信道状态信息,再根据状态空间信息得到频谱和功率分配策略。最后,通过训练深度神经网络找到最佳的资源分配策略。仿真结果表明,该算法可以实现较快的收敛速度,对比贪婪算法以及其他强化学习方法,能源效率均提升20%以上,频谱利用率分别提升27%和11%。

In order to ensure the quality of service(QoS) for downlink users and improve the spectrum efficiency(SE) and energy efficiency(EE) of heterogeneous ultra-dense networks, a multi-agent based joint spectrum and power allocation algorithm of deep reinforcement learning(DRL) is proposed. Firstly, we obtain the resource allocation optimization function with spectrum utilization and energy efficiency as the optimization goal. Secondly, the user state space, reward, and action space are defined, and state space information is obtained through relatively small communication overhead, which is one-dimensional data, and the amount of input data to the network is reduced. Users use their own channel state information(CSI) instead of relying on the global channel state information and then obtain spectrum and power allocation strategies based on the state information. Finally, the best resource allocation strategy is found by training a deep neural network. The simulation results show that the proposed algorithm can achieve a faster convergence speed. Compared with the greedy algorithm and other reinforcement learning methods, the energy efficiency is increased by more than 20%,and the spectrum utilization rate is increased by 27% and 11%,respectively.