Resource Allocation in Multi-User Cellular Networks:A Transformer-Based Deep Reinforcement Learning Approach

To meet the communication services with diverse requirements,dynamic resource allocation has shown increasing importance.In this paper,we consider the multi-slot and multi-user resource allocation(MSMU-RA)in a downlink cellular scenario with the aim of maximizing system spectral efficiency while guaranteeing user fairness.We first model the MSMURA problem as a dual-sequence decision-making process,and then solve it by a novel Transformerbased deep reinforcement learning(TDRL)approach.Specifically,the proposed TDRL approach can be achieved based on two aspects:1)To adapt to the dynamic wireless environment,the proximal policy optimization(PPO)algorithm is used to optimize the multi-slot RA strategy.2)To avoid co-channel interference,the Transformer-based PPO algorithm is presented to obtain the optimal multi-user RA scheme by exploring the mapping between user sequence and resource sequence.Experimental results show that:i)the proposed approach outperforms both the traditional and DRL methods in spectral efficiency and user fairness,ii)the proposed algorithm is superior to DRL approaches in terms of convergence speed and generalization performance.

Resource Allocation for IRS Assistedmm Wave Wireless Powered Sensor Networks with User Cooperation

In this paper,we investigate IRS-aided user cooperation(UC)scheme in millimeter wave(mmWave)wirelesspowered sensor networks(WPSN),where two single-antenna users are wireless powered in the wireless energy transfer(WET)phase first and then cooperatively transmit information to a hybrid access point(AP)in the wireless information transmission(WIT)phase,following which the IRS is deployed to enhance the system performance of theWET andWIT.We maximized the weighted sum-rate problem by jointly optimizing the transmit time slots,power allocations,and the phase shifts of the IRS.Due to the non-convexity of the original problem,a semidefinite programming relaxation-based approach is proposed to convert the formulated problem to a convex optimization framework,which can obtain the optimal global solution.Simulation results demonstrate that the weighted sum throughput of the proposed UC scheme outperforms the non-UC scheme whether equipped with IRS or not.

Resource Allocation and User Association for HTTP Adaptive Streaming in Heterogeneous Cellular Networks with Small Cells

Video streaming,especially hypertext transfer protocol based(HTTP) adaptive streaming(HAS) of video,has been expected to be a dominant application over mobile networks in the near future,which brings huge challenge for the mobile networks.Although some works have been done for video streaming delivery in heterogeneous cellular networks,most of them focus on the video streaming scheduling or the caching strategy design.The problem of joint user association and rate allocation to maximize the system utility while satisfying the requirement of the quality of experience of users is largely ignored.In this paper,the problem of joint user association and rate allocation for HTTP adaptive streaming in heterogeneous cellular networks is studied,we model the optimization problem as a mixed integer programming problem.And to reduce the computational complexity,an optimal rate allocation using the Lagrangian dual method under the assumption of knowing user association for BSs is first solved.Then we use the many-to-one matching model to analyze the user association problem,and the joint user association and rate allocation based on the distributed greedy matching algorithm is proposed.Finally,extensive simulation results are illustrated to demonstrate the performance of the proposed scheme.

Context-Oriented Multi-RAT User Association and Resource Allocation with Triple Decision in 5G Heterogeneous Networks

In the upcoming 5 G heterogeneous networks, leveraging multiple radio access technologies(RATs) shows to be a crucial issue in achieving RAT multiplexing gain to meet the explosive traffic demand. For always best connection(ABC), users tend to activate parallel transmission across all available RATs. However from a system-wide perspective, this might not be optimal given the context of network load, interference and diverse service requirements. To intelligently determine how to use these multi-RAT access resources concurrently, this paper proposes a joint multi-RAT user association and resource allocation strategy with triple decision and integrated context awareness of users and networks. A dynamic game based ant colony algorithm(GACA) is designed to simultaneously maximize the system utility and the fairness of resource allocation. Simulation results show that it's more reasonable to make multi-RAT association decision from a system-wide viewpoint than from an individual one. Compared to max-SNR based and ABC based strategies, the proposed method alleviates network congestion and optimizes resource allocation. It obtains 39%~70% performance improvement.

Joint Resource Allocation Using Evolutionary Algorithms in Heterogeneous Mobile Cloud Computing Networks

The problem of joint radio and cloud resources allocation is studied for heterogeneous mobile cloud computing networks. The objective of the proposed joint resource allocation schemes is to maximize the total utility of users as well as satisfy the required quality of service(QoS) such as the end-to-end response latency experienced by each user. We formulate the problem of joint resource allocation as a combinatorial optimization problem. Three evolutionary approaches are considered to solve the problem: genetic algorithm(GA), ant colony optimization with genetic algorithm(ACO-GA), and quantum genetic algorithm(QGA). To decrease the time complexity, we propose a mapping process between the resource allocation matrix and the chromosome of GA, ACO-GA, and QGA, search the available radio and cloud resource pairs based on the resource availability matrixes for ACOGA, and encode the difference value between the allocated resources and the minimum resource requirement for QGA. Extensive simulation results show that our proposed methods greatly outperform the existing algorithms in terms of running time, the accuracy of final results, the total utility, resource utilization and the end-to-end response latency guaranteeing.

Energy Efficient Resource Allocation Approach for Renewable Energy Powered Heterogeneous Cellular Networks

In this paper,maximizing energy efficiency(EE)through radio resource allocation for renewable energy powered heterogeneous cellular networks(HetNet)with energy sharing,is investigated.Our goal is to maximize the network EE,conquer the instability of renewable energy sources and guarantee the fairness of users during allocating resources.We define the objective function as a sum weighted EE of all links in the HetNet.We formulate the resource allocation problem in terms of subcarrier assignment,power allocation and energy sharing,as a mixed combinatorial and non-convex optimization problem.We propose an energy efficient resource allocation scheme,including a centralized resource allocation algorithm for iterative subcarrier allocation and power allocation in which the power allocation problem is solved by analytically solving the Karush-Kuhn-Tucker(KKT)conditions of the problem and a water-filling problem thereafter and a low-complexity distributed resource allocation algorithm based on reinforcement learning(RL).Our numerical results show that both centralized and distributed algorithms converge with a few times of iterations.The numerical results also show that our proposed centralized and distributed resource allocation algorithms outperform the existing reference algorithms in terms of the network EE.

Deep reinforcement learning-based resource allocation for D2D communications in heterogeneous cellular networks

Device-to-Device(D2D)communication-enabled Heterogeneous Cellular Networks(HCNs)have been a promising technology for satisfying the growing demands of smart mobile devices in fifth-generation mobile networks.The introduction of Millimeter Wave(mm-wave)communications into D2D-enabled HCNs allows higher system capacity and user data rates to be achieved.However,interference among cellular and D2D links remains severe due to spectrum sharing.In this paper,to guarantee user Quality of Service(QoS)requirements and effectively manage the interference among users,we focus on investigating the joint optimization problem of mode selection and channel allocation in D2D-enabled HCNs with mm-wave and cellular bands.The optimization problem is formulated as the maximization of the system sum-rate under QoS constraints of both cellular and D2D users in HCNs.To solve it,a distributed multiagent deep Q-network algorithm is proposed,where the reward function is redefined according to the optimization objective.In addition,to reduce signaling overhead,a partial information sharing strategy that does not observe global information is proposed for D2D agents to select the optimal mode and channel through learning.Simulation results illustrate that the proposed joint optimization algorithm possesses good convergence and achieves better system performance compared with other existing schemes.

A Novel Resource Allocation Algorithm based on Downlink and Uplink Decouple Access Scheme in Heterogeneous Networks

Traditional cellular network requires that a user equipment(UE) should associate to the same base station(BS) in both the downlink(DL) and the uplink(UL). Based on dual connectivity(DC) introduced in LTE-Advanced R12, DL/UL decouple access scheme has been proposed, which is especially suitable for heterogeneous networks(Het Nets). This paper is the pioneer to take the DL/UL decouple access scheme into consideration and develop a novel resource allocation algorithm in a two-tier Het Net to improve the total system throughput in the UL and ease the load imbalance between macro base stations(MBSs) and pico base stations(PBSs). A model is formulated as a nonlinear integer programming, and the proposed algorithm is a sub-optimal algorithm based on the graph theory. First, an undirected and weighted interference graph is obtained. Next, the users are grouped to let users with large mutual interferences to be assigned to different clusters. Then, the users in different clusters are allocated to different resource blocks(RBs) by using the Hungarian algorithm. Simulation results show that the proposed algorithm can provide great promotions for both the total system throughput and the average cell edge user throughput and successfully ease the load imbalance between MBSs and PBSs.

Efficient Resource Allocation with Improved Interference Mitigation in FFR-Aided OFDMA Heterogeneous Networks

Fractional frequency reuse(FFR) has recently emerged as an efficient inter-cell interference coordination technique for orthogonal frequency division multiple access(OFDMA) based multi-tier cellular networks due to its low complexity, minimal signaling over-head, and coverage improvement. In this work, an intermediary region(IR) at the border of the center region(CR) and edge region(ER) is defined, which prevents severe cross-tier interference and is usually ignored by other schemes. Furthermore, a strategic resource allocation scheme is proposed, which allows macro users in this new region to be served more resources due to their good channel conditions close to the serving base station(BS), while femto users are assigned resource blocks from sub-bands that receive the least net interference from a set of usable sub-bands in any region. We find by analysis and simulation the optimal threshold for IR, which minimizes the cross-tier interference, and show that the femto throughput is also maximized for this threshold. Numerical results show the proposed scheme outperforms other notable schemes in terms of throughput and outage performances.

Robust resource allocation for NOMA-assisted heterogeneous networks

As a promising technology to improve spectrum efficiency and transmission coverage,Heterogeneous Network(HetNet)has attracted the attention of many scholars in recent years.Additionally,with the introduction of the Non-Orthogonal Multiple Access(NOMA)technology,the NOMA-assisted HetNet cannot only improve the system capacity but also allow more users to utilize the same frequency band resource,which makes the NOMA-assisted HetNet a hot topic.However,traditional resource allocation schemes assume that base stations can exactly estimate direct link gains and cross-tier link gains,which is impractical for practical HetNets due to the impact of channel delays and random perturbation.To further improve energy utilization and system robustness,in this paper,we investigate a robust resource allocation problem to maximize the total Energy Efficiency(EE)of Small-Cell Users(SCUs)in NOMA-assisted HetNets under imperfect channel state information.By considering bounded channel uncertainties,the robust resource optimization problem is formulated as a mixed-integer and nonlinear programming problem under the constraints of the cross-tier interference power of macrocell users,the maximum transmit power of small base station,the Resource Block(RB)assignment,and the quality of service requirement of each SCU.The original problem is converted into an equivalent convex optimization problem by using Dinkelbach's method and the successive convex approximation method.A robust Dinkelbach-based iteration algorithm is designed by jointly optimizing the transmit power and the RB allocation.Simulation results verify that the proposed algorithm has better EE and robustness than the existing algorithms.

Resource Allocation for Two-Tier RIS-Assisted Heterogeneous NOMA Networks

Reconfigurable intelligent surface(RIS)as a promising technology has been proposed to change weak communication environ-ments.However,most of the current resource allocation(RA)schemes have focused on RIS-assisted homogeneous networks,and there is still no open works about RA schemes of RIS-assisted heterogeneous networks(HetNets).In this paper,we design an RA scheme for a RIS-assisted HetNet with non-orthogonal multiple access to improve spectrum efficiency and transmission rates.In particular,we jointly optimize the transmit power of the small-cell base station and the phase-shift matrix of the RIS to maximize the sum rates of all small-cell users,subject to the unit modulus constraint,the minimum signal-to-interference-plus-noise ratio constraint,and the cross-tier interference constraint for protecting communication quality of microcell users.An efficient suboptimal RA scheme is proposed based on the alternating iteration ap-proach,and successive convex approximation and logarithmic transformation approach.Simulation results verify the effectiveness of the pro-posed scheme in terms of data rates.

User Scheduling and Slicing Resource Allocation in Industrial Internet of Things

Heterogeneous base station deployment enables to provide high capacity and wide area coverage.Network slicing makes it possible to allocate wireless resource for heterogeneous services on demand.These two promising technologies contribute to the unprecedented service in 5G.We establish a multiservice heterogeneous network model,which aims to raise the transmission rate under the delay constraints for active control terminals,and optimize the energy efficiency for passive network terminals.A policygradient-based deep reinforcement learning algorithm is proposed to make decisions on user association and power control in the continuous action space.Simulation results indicate the good convergence of the algorithm,and higher reward is obtained compared with other baselines.

Energy-Efficient Resource Allocation for Heterogeneous Network with Grouping D2D

In this paper,a new communication model is built named grouping D2D(GD2D).Different from the traditional D2D coordination,we proposed GD2D communication in licensed and unlicensed spectrum simultaneously.We formulate a resource allocation problem,which aims at maximizing the energy efficiency(EE)of the system while guaranteeing the quality-of-service(Qos)of users.To efficiently solve this problem,the non-convex optimization problem is first transformed into a convex optimization problem.By transforming the fractional-form problem into an equivalent subtractive-form problem,an iterative power allocation algorithm is proposed to maximize the system EE.Moreover,the optimal closedform power allocation expressions are derived by the Lagrangian approach.Simulation results show that our algorithm achieves higher EE performance than the traditional D2D communication scheme.

Resource Allocation for Physical Layer Security in Heterogeneous Network with Hidden Eavesdropper

The tremendous performance gain of heterogeneous networks(Het Nets) is at the cost of complicated resource allocation. Considering information security, the resource allocation for Het Nets becomes much more challenging and this is the focus of this paper. In this paper, the eavesdropper is hidden from the macro base stations. To relax the unpractical assumption on the channel state information on eavesdropper, a localization based algorithm is first given. Then a joint resource allocation algorithm is proposed in our work, which simultaneously considers physical layer security, cross-tier interference and joint optimization of power and subcarriers under fairness requirements. It is revealed in our work that the considered optimization problem can be efficiently solved relying on convex optimization theory and the Lagrangian dual decomposition method is exploited to solve the considered problem effectively. Moreover, in each iteration the closed-form optimal resource allocation solutions can be obtained based on the Karush-Kuhn-Tucker(KKT) conditions. Finally, the simulation results are given to show the performance advantages of the proposed algorithm.

A Distributed User Association and Resource Allocation Method in Cache-Enabled Small Cell Networks

Cache-enabled small cell networks have been regarded as a promising approach for network operators to cope with the explosive data traffic growth in future 5 G networks. However, the user association and resource allocation mechanism has not been thoroughly studied under given content placement situation. In this paper, we formulate the joint optimization problem of user association and resource allocation as a mixed integer nonlinear programming(MINLP) problem aiming at deriving a balance between the total utility of data rates and the total data rates retrieved from caches. To solve this problem, we propose a distributed relaxing-rounding method. Simulation results demonstrate that the distributed relaxing-rounding method outperforms traditional max-SINR method and range-expansion method in terms of both total utility of data rates and total data rates retrieved from caches in practical scenarios. In addition, effects of storage and backhaul capacities on the performance are also studied.

Optimal Energy Efficiency Resource Allocation Strategy for Cognitive Clustering Network under PUEA Attack

5G has pushed the use of radio spectrum to a new level,and cognitive clustering network can effectively improve the utilization of radio spectrum,which is a feasible way to solve the growing demand for wireless communications.However,cognitive clustering network is vulnerable to PUEA attack,which will lead to the degradation of system detection performance,thereby reducing the energy efficiency.Aiming at these problems,this paper investigates the optimal energy efficiency resource allocation scheme for cognitive clustering network under PUEA attack.A cooperative user selection algorithm based on selection factor is proposed to effectively resist PUEA user attack and improve detection performance.We construct the energy efficiency optimization problem under multi-constraint conditions and transform the nonlinear programming problem into parametric programming problem,which is solved by Lagrangian function and Karush-Kuhn-Tucker condition.Then the sub-gradient iterative algorithm based on optimal energy efficiency under PUEA attack is proposed and its complexity is analyzed.Simulation results indicate that proposed method is effective when subjected to PUEA attacks,and the impact of different parameters on energy efficiency is analyzed.

Modeling for heterogeneous multi-stage information propagation networks and maximizing information

In this paper, we propose a heterogeneous multi-stage model to study the effect of social reinforcement on information propagation. Both heterogeneity of network components and the heterogeneity of individual reinforcement thresholds are considered. An information outbreak condition is derived, according to which the outbreak scale and individual density of each state under specific propagation parameters can be deduced. Monte Carlo experiments are conducted in Facebook networks to demonstrate the outbreak condition, and we find that social reinforcement effects generally inhibit the propagation of information though it contributes to the emergence of certain hot spots simultaneously. Additionally, by applying Pontryagin's Maximum Principle, we derive the optimal control strategy in the case of limited control resources to maximize the information propagation. Then the forward–backward sweep method is utilized to verify its performance with numerical simulation.

Energy efficiency enhancement in heterogeneous networks:a joint resource allocation approach

To support the drastic growth of wireless multimedia services and the requirements of ubiquitous access, numerous wireless infrastructures which consume enormous energy, such as macrocell, small cell, distributed antenna systems and wireless sensor networks, have been deployed. Under the background of environmental protection, improving the energy efficiency（EE） in wireless networks is becoming more and more important. In this paper, an EE enhancement scheme in heterogeneous networks（Het Nets） by using a joint resource allocation approach is proposed. The Het Nets consists of a mix of macrocell and small cells. Firstly, we model this strategic coexistence as a multi-agent system in which decentralized resource management inspired from Reinforcement Learning are devised. Secondly, a Q-learning based joint resource allocation algorithm is designed. Meanwhile, with the consideration of the time-varying channel characteristics, we take the long-term learning reward into account. At last, simulation results show that the proposed decentralized algorithm can approximate to centralized algorithm with low-complexity and obtain high spectral efficiency（SE） in the meantime.

End-to-end utility-based resource allocation for multi-RAT nodes in heterogeneous cognitive wireless networks

This paper studies the problem of effective resource allocation for multi-radio access technologies （Multi-RAT） nodes in heterogeneous cognitive wireless networks （HCWNs）. End-to-end utility, which is defined as the delay of end-to-end communication, is taken into account in this paper. In the scenario of HCWNs, it is assumed that the cognitive radio nodes have the ability of Multi-RAT and can communicate with each other through different paths simultaneously by splitting the arrival packets. In this paper, the problem is formulated as the optimization of split ratio and power allocation of the source cognitive radio node to minimize the delay of end-to-end communication, and a low complexity step-by-step iterative algorithm is proposed. Numerical results show good performance of the proposed algorithm over two other conventional algorithms.