Deep reinforcement learning-based resource allocation for D2D communications in heterogeneous cellular networks

Device-to-Device(D2D)communication-enabled Heterogeneous Cellular Networks(HCNs)have been a promising technology for satisfying the growing demands of smart mobile devices in fifth-generation mobile networks.The introduction of Millimeter Wave(mm-wave)communications into D2D-enabled HCNs allows higher system capacity and user data rates to be achieved.However,interference among cellular and D2D links remains severe due to spectrum sharing.In this paper,to guarantee user Quality of Service(QoS)requirements and effectively manage the interference among users,we focus on investigating the joint optimization problem of mode selection and channel allocation in D2D-enabled HCNs with mm-wave and cellular bands.The optimization problem is formulated as the maximization of the system sum-rate under QoS constraints of both cellular and D2D users in HCNs.To solve it,a distributed multiagent deep Q-network algorithm is proposed,where the reward function is redefined according to the optimization objective.In addition,to reduce signaling overhead,a partial information sharing strategy that does not observe global information is proposed for D2D agents to select the optimal mode and channel through learning.Simulation results illustrate that the proposed joint optimization algorithm possesses good convergence and achieves better system performance compared with other existing schemes.