无线RF能量收集异构网络中基于Q-Learning的自适应功率控制

Q-Learning Based Adaptive Power Control for Wireless HetNets with RF Energy Harvesting

针对信息与能量同传Femtocell异构网络,建立了数学优化问题,目的是通过调节Femtocell发射功率来抑制干扰信号对信息传输的影响,同时有效利用干扰信号为设备充电,在满足用户信息通信服务质量以及传感器能量收集的条件下,最大化Femtocell基站群总容量。为求解此优化问题,设计了一种基于强化学习的自适应功率控制算法,给出了算法框架和求解方法,并提出一种基于距离影响和惩罚参数的分段式奖励函数。为了比较,还给出了一种基于网络容量和约束关系的奖励函数。仿真实验验证了所提算法的有效性,即在保障信息用户和能量用户通信服务质量的前提下,可提高Femtocell异构网络的总容量。实验还表明,采用所提分段式奖励函数能够取得更好的系统优化效果。

This paper investigates Femtocell HetNets with the information and energy transferring simultaneously. For such a network, we propose an optimization problem mathematically to explore the effect of the interference signal on the system by adjusting the transmit power, and charge the device by the interference signal. The goal of the problem is to maximize the sum capacity of the Femtocell base station group while satisfying the user’s quality of service(QoS) and sensors’ harvested energy. In order to solve this problem, we design an adaptive power control algorithm based on reinforcement learning. The algorithm framework and corresponding solution is presented. Meanwhile, we propose a piecewise reward function based on distance and penalty parameters.For comparison, we design another reward function based on network capacity and constraint relationship. Simulation results verify the effectiveness of the proposed algorithm. The total capacity of the system is increasing while satisfying the users’ QoS and sensors’ charging. And the numerical results show that the proposed piecewise reward function can achieve better system optimization results.