基于增强学习的非协作认知无线网络路由算法研究

Research on the Non-Collaborative Cognitive Radio Networks Routing Algorithm Based on Reinforcement Learning

认知无线电网络(Cognitive Radio Networks,CRNs)的出现解决了由无线应用发展而引起的频谱稀缺问题.在CRNs中,次用户(Secondary Users,SUs)机会式地接入主用户(Primary Users,PUs)拥有的授权频谱.使用马尔科夫泊松过程(Markov Modulated Poisson Process,MMPP)对Pus的活动进行建模,提出基于增强学习的非协作认知无线网络路由算法.每个SU都想最小化自己流量的端到端时延,同时可以满足PU的服务质量(Qo S)需求.为了使SUs的路由决策能够适应环境变化和节点之间非协作交互的影响,我们将路由问题建模为非合作博弈的随机学习过程.然后,我们提出了一种解决路由问题的分布式增强学习算法,减少了SU之间由于信息交互带来的开销.仿真实验的结果表明了所提出的算法能够满足PU的Qo S需求,同时减少网络时延.

The Cognitive Radio Networks （CRNs） can solve the problem of spectrum scarcity caused by the development of wireless applications. In CRNs, Secondary Users （SUs） access the authorized spectrum owned by the Primary Users （PUs） opportunistically. In this paper, the Markov Modulated Poisson Process （MMPP） is used to model the activities of the PUs, and then a non-collaborative cognitive radio networks routing algorithm based on reinforcement learning is proposed. Each SU wants to minimize the end-to-end delay of its traffic while meet- ing the requirements of QoS of PUs. The routing decision should adapt to changing environment and the interactions between nodes are non-collaborative, we model the routing problem as a random learning process for non-cooperative games. Then, we propose a distributed reinforce-merit learning algorithm to solve the routing problem and reduce the cost of information interaction between SU. The simulation results show that the proposed algorithm can meet the QoS de- mand of PU and reduce network delay.