无人机集群协同主动搜索的强化学习策略研究

Collaborative altitude-adaptive reinforcement learning for active search with unmanned aerial vehicle swarms

在多变和复杂的灾害环境中,迅速定位幸存者是一项至关重要的任务,无人机(UAV,unmanned aerial vehicle)群的主动搜索能力在这一过程中发挥着关键作用。然而,无人机的传感器性能与其飞行高度紧密相关,覆盖范围和探测精度难以平衡。为了实现高效的搜索,无人机集群需要在高空飞行以覆盖更广的区域,同时在低空飞行以提高探测的准确性。此时,策略的制定对于无人机集群的协调和决策至关重要。为了应对这些挑战,提出了协同高度自适应强化学习(CARL,collaborative altitude-adaptive reinforcement learning)方法,该方法融合了可变高度传感器模型、基于信心的评估机制以及基于近端策略优化(PPO,proximal policy optimization)的高度自适应规划器。通过CARL方法,无人机能够根据实时情况动态地调整感知策略,并做出更加明智的决策。此外,引入了一种创新的奖励塑造策略,从而在广阔环境中最大化搜索效率。通过在多种条件下的模拟测试,CARL方法在提高完全搜索率方面表现出色,相较于基线方法提升了12%,充分证明了其在提升无人机集群在主动搜索任务中的有效性。

Active search with unmanned aerial vehicle(UAV)swarms in cluttered and unpredictable environments poses a critical challenge in search and rescue missions,where the rapid localizations of survivors are of paramount importance,as the majority of urban disaster victims are surface casualties.However,the altitude-dependent sensor performance of UAV introduces a crucial trade-off between coverage and accuracy,significantly influencing the coordination and decision-making of UAV swarms.The optimal strategy has to strike a balance between exploring larger areas at higher alti‐tudes and exploiting regions of high target probability at lower altitudes.To address these challenges,collaborative altitude-adaptive reinforcement learning(CARL)was proposed which incorporated an altitude-aware sensor model,a confidence-informed assessment module,and an altitude-adaptive planner based on proximal policy optimization(PPO)algorithms.CARL enabled UAV to dynamically adjust their sensing location and made informed decisions.Furthermore,a tailored reward shaping strategy was introduced,which maximized search efficiency in extensive environments.Com‐prehensive simulations under diverse conditions demonstrate that CARL surpasses baseline methods,achieves a 12%im‐provement in full recovery rate,and showcase its potential for enhancing the effectiveness of UAV swarms in active search missions.