基于深度强化学习的无线异构网络中继决策研究

Study on Relay Decision in Wireless Heterogeneous Networks Based on Deep Reinforcement Learning

在物联网大规模多用户场景中,远端节点需通过中继接入网络。为解决中继在异构接入技术环境下的自适应接入控制问题,提出一种基于深度强化学习的智能中继接入控制策略,将中继对远端用户数据的收发过程视为一个部分可观察马尔可夫决策过程,通过动态决策中继工作状态,以实现最大化系统的总吞吐量和节点公平性目标。首先,建立具有中继的无线异构网的上行链路模型,以提高系统总吞吐量为优化目标,建立中继动态决策优化模型;其次,构建含有LSTM隐藏层的深度Q网络(DQN)作为行为状态值函数,以优化系统总吞吐量。测试结果表明深度强化学习无线异构网络中继决策方案(DRL-RAP)可在确保原有用户服务质量的前提下,为远端用户提供网络接入,系统总吞吐量在原有网络基础上显著提高,吞吐量最大可提高30%。

For large-scale multi-user scenarios of the Internet of Things,remote nodes need to access the network through relay.In order to solve the adaptive access control problem of relay in heterogeneous access technology environment,an intelligent relay access control strategy based on deep reinforcement learning is proposed,which regards the transmission and reception process of relay to remote user data as a partially observable Markov decision process,and dynamically decides the relay working state to maximize the total system throughput and node fairness.Firstly,the uplink model of wireless heterogeneous network with relay is established.With the goal of improving the total throughput of the system,the dynamic decision optimization model of relay is established.Secondly,a deep Q network(DQN)with LSTM hidden layer is constructed as a behavior state value function to optimize the total system throughput.Test results show that DRL-RAP can provide network access for remote users on the premise of ensuring the original user’s quality of service.The total throughput of the system is significantly improved on the basis of the original network,and the maximum throughput can be increased by 30%.