基于替代迹的蜂窝网信道分配Actor-Critic算法

Actor-Critic algorithm for channel allocation in cellular network based on replace eligibility

针对D2D混合蜂窝网络在进行信道分配时难以在较高信噪比条件下进行自动信道分配的问题,提出了一种基于替代迹的蜂窝网络信道分配Actor-Critic算法。首先定义了信道分配问题的MDP模型,然后定义了基于替代迹的Actor-Critic算法。Actor采用模拟退火探索策略自适应控制状态空间的搜索,将critic部分求解的值函数的时间差分误差用于更新该策略的优先级,再依优先级对策略进行更新;critic部分采用基于替代迹的值函数更新方式,并计算值函数的时间差分误差,以指导actor改进策略。实验结果表明其具有系统吞吐量大和信噪比高的优点。

In allusion to the problem of allocating channel automatically under high channel noise ratio in cellular network with D2D,this paper proposed an Actor-Critic algorithm for channel allocation.Firstly,it established the MDP model for channel allocation problem and defined the algorithm based on replace eligibility.Actor took the exploring policy based on simulated annealing,and controled the exploring range adaptively via gradually adjusting the temperature.It renewed the policy by using the priority,which was computed from the TD(temporal difference)-error as from critic as the input.Critic part used the updating method based on replace eligibility to renew the temporal difference error to improve the actor’s policy.In order to verify this method,the channel allocation problem was simulated and verified,the result shows that the method has the advantages of large throughout capacity and high noise-signal ratio.