摘要
巨型低轨星座为载人飞船、空间站、遥感卫星等用户航天器提供低时延、大容量的通信通道存在波束资源分配优化的难题。针对采用离散时间的深度强化学习A2C(advanced actor-critic)的智能优化框架进行了研究,结合遗传算法中个体和基因概念、形成了可有效满足多用户、动态、并发接入需求的波束资源调度算法。基于仿真分析,提出的算法可在多种典型场景下具有适用性,支持在20 s内完成超过3000个任务的有效规划,任务成功率不低于91%。通过算法优化实现复杂度的降低,相对传统遗传算法可节约时间45%以上。同时对传统A2C算法框架中的收敛问题进行了优化,解决了传统全连接A2C算法无法收敛的难题,同时相比DQN(deep q-network)算法框架收敛速度提升38%以上。
The giant low-orbit constellation provided low-latency,large-capacity communication channels for user spacecraft such as manned spacecraft,space stations and remote sensing satellites,and there was a resource allocation optimizing problem of satellite beams.The intelligent optimization framework of A2C(advanced actor-critic)using discrete-time deep reinforcement learning was studied,and the beam resource scheduling algorithm that could effectively meet the needs of multi-users,dynamic and concurrent access was formed by combining the concepts of individuals and genes in genetic algorithms.Based on simulation and analysis,the proposed algorithm could be applicable in a variety of typical scenarios.The method could provide effective scheduling results for more than 3000 tasks in 20 s,and the task success rate was not less than 91%.The complexity was reduced by algorithm optimization,which could save more than 45%of the time compared with traditional genetic algorithms.At the same time,the convergence problem in the traditional A2C algorithm framework was optimized,which solved the non-convergence problem in the traditional fully connected A2C algorithm.Meanwhile,the convergence speed was increased by more than 38%compared with the DQN(deep q-network)algorithm.
作者
刘伟
郑润泽
张磊
高梓贺
陶滢
崔楷欣
LIU Wei;ZHENG Runze;ZHANG Lei;GAO Zihe;TAO Ying;CUI Kaixin(Innovation Center of Satellite Communication System,CNSA,Beijing 100094,China;Institute of Telecommunication and Navigation Satellites,China Academy of Space Technology,Beijing 100094,China;Northwestern Polytechnical University,Xi′an 710072,China;Beijing Institute of Technology,Beijing 100081,China)
出处
《中国空间科学技术》
CSCD
北大核心
2023年第3期123-133,共11页
Chinese Space Science and Technology
基金
国家重点研发计划(2021YFB2900603)
国家自然科学基金(61831008)。