Partially Distributed Channel and Power Management Based on Reinforcement Learning

This paper studies a dynamic multi-user wireless network,where users have no knowledge of the arrival rate and size of data block and suffer from a constraint on long-term average power consumption.Considering such a network,we address the problem of dynamically optimizing channel/power allocation,so as to minimize the long-term average data backlog.The design problem is shown to be a constrained Markov decision process.In order to solve the problem without knowledge on dynamics of the system,we introduce post-decision states and propose a resource allocation algorithm based on reinforcement learning.Since the channel/power allocation problem is coupled,the multiuser decision problem suffers from curses of dimensions(of state/action/outcome space).This makes centralized decision-making and optimization on channel/power allocation suffer from a long convergence time.As a countermeasure,a partially distributed resource allocation framework is proposed.The multiuser power allocation problem is decoupled into single-user decision problems,while channel allocation optimization is performed in a centralized manner.In order to further reduce computational complexity,we propose a low-complexity reinforcement learning method.Simulation results reveal that the proposed algorithm outperforms the state-of-the-art myopic optimizations in terms of energy efficiency and the backlog performance.