无线网络中基于深度Q学习的传输调度方案

Transmission scheduling scheme based on deep Q learning in wireless network

针对无线网络中的数据传输问题,提出一种基于深度Q学习(QL,Q learning)的传输调度方案。该方案通过建立马尔可夫决策过程(MDP,Markov decision process)系统模型来描述系统的状态转移情况;使用Q学习算法在系统状态转移概率未知的情况下学习和探索系统的状态转移信息,以获取调度节点的近似最优策略。另外,当系统状态的规模较大时,采用深度学习(DL,deep learning)的方法来建立状态和行为之间的映射关系,以避免策略求解中产生的较大计算量和存储空间。仿真结果表明,该方法在功耗、吞吐量、分组丢失率方面的性能逼近基于策略迭代的最优策略,且算法复杂度较低,解决了维灾问题。

To cope with the problem of data transmission in wireless networks,a deep Q learning based transmission scheduling scheme was proposed.The Markov decision process system model was formulated to describe the state transition of the system.The Q learning algorithm was adopted to learn and explore the system states transition information in the case of unknown system states transition probability to obtain the approximate optimal strategy of the schedule node.In addition,when the system state scale was big,the deep learning method was employed to map the relation between state and behavior to solve the problem of the large amount of computation and storage space in Q learning process.The simulation results show that the proposed scheme can approach the optimal strategy based on strategy iteration in terms of power consumption,throughput,packets loss rate.And the proposed scheme has a lower complexity,which can solve the problem of the curse of dimensionality.