基于深度强化学习的水声网络公平跨层MAC协议

Deep reinforcement learning based proportional-fair optimized cross-layer MAC protocol for underwater acoustic networks

针对水声通信异构网络中信道分配不公和节点能量受限的问题,基于深度强化学习方法,提出跨层联合优化公平信道接入和功率控制的媒介访问控制(POCL-MAC)协议。根据反馈ACK包获知时延状态下的信道冲突结果和接收机处信噪比,基于深度强化学习的状态、动作和奖励序列自主学习,调整认知用户的接入时隙和发射功率;采用公平函数实现异构网络中认知用户和主用户吞吐量性能的比例公平。设计了一个联合状态序列和独立式奖励函数,在不增加神经网络复杂度的前提下,提高跨层联合优化的子动作决策准确度。仿真结果表明,相比于传统DRL算法,所提算法实现了接近于最优公平性吞吐量性能,同时具有更好的能量利用效率。

In view of the unfair channel allocation and energy-constrained nodes in underwater acoustic communication heterogeneous networks,a proportional-fair optimized cross-layer medium access control(POCL-MAC)protocol is proposed based on deep reinforcement learning(DRL).It strives to optimize fair channel access and power control by cross-layer joint.By feedback ACK packets,the channel conflict outcomes and receiver-side signal-to-noise ratio(SNR)under delay status are obtained.Autonomous learning is carried out based on the state,action and reward sequence of DRL to adjust the access slot and transmission power of cognitive users.Fairness function is employed to achieve proportional fairness between cognitive users and primary users′throughput performance in heterogeneous networks.A joint state sequence and independent reward function are designed to enhance the accuracy of sub-action decisions in cross-layer joint optimization without increasing neural network complexity.Simulation results demonstrate that the proposed algorithm can achieve near-optimal fairness throughput performance while exhibiting better energy utilization efficiency in comparison with the traditional DRL-based algorithms.