基于深度强化学习的卫星动态功率控制技术

Dynamic Power Allocation Technology for Satellites Based on Deep Reinforcement Learning

随着用户数量和数据业务的显著增长,卫星通信系统需要更高的吞吐量和更大的容量。在有限的无线资源条件下,高通量卫星如何通过智能化技术手段灵活高效地动态分配无线资源,成为当前卫星通信领域亟待解决的难题。在灵活载荷的框架下,传统的启发式方法存在计算复杂度高的问题,难以满足未来高实时的卫星通信业务需求。为解决以上难题满足未来灵活载荷的技术需求,建立了高通量卫星多波束通信数学模型,提出了一种基于深度强化学习的近端策略优化方法,可较低复杂度地动态控制高通量卫星各波束的功率分配,满足卫星请求容量和功率有效利用率的多优化目标需求。实验结果表明,基于近端策略优化的卫星动态功率控制技术能较好处理实际的业务需求,并能同时完成不同波束间功率的最优分配,极大地提升功率有效利用率。

With the significant increase in the number of users and data services,the satellite communication system requires higher throughput and larger capacity.With the limited wireless resources,it has become an urgent problem in the field of satellite communications for high-throughput satellites to dynamically allocate wireless resources with high flexibility and efficiency by intelligent technologies.Under the framework of flexible payload,the traditional heuristic method has the problem of high computational complexity,which is difficult to satisfy the requirements of future high real-time satellite communication services.To solve the above problem and meet the technical requirements of flexible payloads in the future,this paper established a high-throughput satellite multi-beam communication mathematical model and proposed a proximal strategy optimization method based on deep reinforcement learning,which can dynamically control the power distribution of each beam of a high-throughput satellite with low complexity to meet the requirements for multiple optimization objectives of satellite requested capacity and power effective utilization.The results show that the satellite dynamic power allocation technology based on the proximal strategy optimization can better handle actual business requirements,and can simultaneously complete the optimal power allocation between different beams,greatly improving the effective availability of power.