多智能体强化学习驱动的无人机动态信道分配

Dynamic Channel Allocation for UAV Formation Driven by Multi-agent Reinforcement Learning

针对无人机编队在进行远距离实时视频传输时频谱资源不足且利用效率低、吞吐量要求较高、传输任务难以完成等问题,提出了多智能体强化学习驱动的动态信道分配算法,使得无人机编队可以根据传输任务和信道环境动态地选择使用的信道,实现了频谱资源的高效利用。该算法使用了集中式训练分布式执行的架构,通过联合探索和联合学习的方式保证了无人机间的探索和合作能力,使得每架无人机均可以依据局部观测信息同时独立分配自身使用信道,提高了算法的灵活性和可行性,并减少了频谱分配用时。仿真结果表明,该算法训练过程性能更好,执行时相比于现有算法可以提高编队整体的平均任务传输成功率。

For the problems of insufficient spectrum resources,low utilization efficiency,high throughput requirements,and difficulty in completing transmission tasks when unmanned aerial vehicle(UAV)formation performs long-distance real-time video transmission,a dynamic channel allocation algorithm driven by multi-agent reinforcement learning is proposed.The UAV formation can dynamically select channel to transmit according to the transmission task and the channel environment,which realizes the efficient use of spectrum resources.The algorithm uses a centralized training and distributed execution architecture to ensure exploration and cooperation capabilities between UAVs through joint exploration and joint learning,so that each UAV can independently allocate its own channels according to local observation.The flexibility and feasibility of the algorithm are improved,and the time for spectrum allocation is reduced.The simulation results show that this algorithm has better performance in the training process,and can improve the overall average task transmission success rate of the formation compared with the existing algorithms in execution process.