无人机自组网中基于Q-learning算法的及时稳定路由策略

Timely and stable routing strategy based on Q-learning algorithm in UAV Ad hoc network

无人机自组网凭借其抗干扰能力强、适用于复杂地形、智能化程度高和成本较低的优点,近年来受到广泛关注,该网络中路由协议的设计与优化一直是核心研究问题。针对无人机自组网中因节点快速移动造成节点本地存储的路由未及时更新而失效的问题,提出一种基于Q-learning算法的动态感知优化链路状态路由协议(DSQ-OLSR)。该协议首先充分考虑了无人机自组网节点高速移动的特点,在选取多点中继(MPR)节点时添加了链路稳定性和链路存在时间这两个指标,使得选出的MPR节点集更稳定、合理;其次,结合Q-learning算法对TC消息的发送间隔进行自适应调整,使得在网络拓扑变动较小时增大TC发送间隔以减小控制开销,而在拓扑变动较大时减小TC发送间隔用于达到快速感知并构建网络拓扑的要求,进而实现数据的及时路由。仿真结果表明,与DT-OLSR协议相比,该协议在端到端时延、吞吐量、成功率和网络生存时间性能上分别提高了12.61%、9.28%、7.69%和5.86%,由此验证了其有效性。

The UAV Ad hoc network receives extensive attention in recent years due to its strong anti-interference ability,suitable for complex terrain,high degree of intelligence and low cost.The design and optimization of the routing protocol in this network are always the core research problem.Aiming at the problem that the node doesn’t update the local route in time due to it’s rapid movement in the UAV Ad hoc network,this paper proposed a dynamic sensor optimized link state routing protocol based on the Q-learning algorithm(dynamic sensor optimized link state routing based on Q-learning algorithm,DSQ-OLSR).Firstly,the proposed protocol fully considered the characteristics of the high-speed movement of nodes in the UAV Ad hoc network.When selected MPR(multi-point relay)nodes,this paper added two indicators of link stability and link existence time to make the selected MPR nodes set more stable and reasonable.In addition,this paper used the Q-learning algorithm to adjust the sending interval of TC messages adaptively.In order to reduce control overhead,the proposed algorithm increased the TC sen-ding interval when the network topology changed slightly.In order to meet the requirements of rapid perception and construction of the network topology and further realize the timely routing of data,the algorithm reduced the TC sending interval when the topology changed greatly.The simulation results show that compared with the DT-OLSR(dynamic topology link state routing based on Q-learning algorithm)protocol,the proposed protocol improves the end-to-end delay,throughput,success rate and network lifetime performance by 12.61%,9.28%,7.69%and 5.86%respectively,which verifies the effectiveness of this protocol.