智能电网中基于多智能体强化学习的频谱分配算法

Spectrum allocation algorithm based on multi-agent reinforcement learning in smart grid

针对智能电网中利用5G网络承载多样化电力终端的业务需求,提出了一种基于多智能体强化学习的频谱分配算法。首先,基于智能电网中部署的集成接入回程系统,考虑智能电网中轻量化和非轻量化终端业务的不同通信需求,将频谱分配问题建模为最大化系统总能效的非凸混合整数规划。其次,将前述问题构建为一个部分可观测的马尔可夫决策过程并转换为完全协作的多智能体问题,进而提出了一种集中训练分布执行框架下基于多智能体近端策略优化的频谱分配算法。最后,通过仿真验证了所提算法的性能。仿真结果表明,所提算法具有更快的收敛速度,通过有效减少层内与层间干扰、平衡接入与回程链路速率,可以将系统总速率提高25.2%。

In view of the fact that 5G networks are used to meet the service requirements of various power terminals in smart grid,a spectrum allocation algorithm based on multi-agent reinforcement learning was proposed.Firstly,for the integrated access backhaul system deployed in smart grid,considering the different communication requirements of services in lightweight and non-lightweight terminal,the spectrum allocation problem was formulated as a non-convex mixed-integer programming aiming to maximize the overall energy efficiency.Secondly,the above problem was modeled as a partially observable Markov decision process and transformed into a fully cooperative multi-agent problem,then a spectrum allocation algorithm was proposed which was based on multi-agent proximal policy optimization under the framework of centralized training and distributed execution.Finally,the performance of the proposed algorithm was verified by simulation.The results show that the proposed algorithm has a faster convergence speed and can increase the overall transmission rate by 25.2%through effectively reducing intra-layer and inter-layer interference and balancing the access and backhaul link rates.