Resource Allocation in Multi-User Cellular Networks:A Transformer-Based Deep Reinforcement Learning Approach

To meet the communication services with diverse requirements,dynamic resource allocation has shown increasing importance.In this paper,we consider the multi-slot and multi-user resource allocation(MSMU-RA)in a downlink cellular scenario with the aim of maximizing system spectral efficiency while guaranteeing user fairness.We first model the MSMURA problem as a dual-sequence decision-making process,and then solve it by a novel Transformerbased deep reinforcement learning(TDRL)approach.Specifically,the proposed TDRL approach can be achieved based on two aspects:1)To adapt to the dynamic wireless environment,the proximal policy optimization(PPO)algorithm is used to optimize the multi-slot RA strategy.2)To avoid co-channel interference,the Transformer-based PPO algorithm is presented to obtain the optimal multi-user RA scheme by exploring the mapping between user sequence and resource sequence.Experimental results show that:i)the proposed approach outperforms both the traditional and DRL methods in spectral efficiency and user fairness,ii)the proposed algorithm is superior to DRL approaches in terms of convergence speed and generalization performance.

A dynamic and resource sharing virtual network mapping algorithm

Network virtualization can effectively establish dedicated virtual networks to implement various network functions.However,the existing research works have some shortcomings,for example,although computing resource properties of individual nodes are considered,node storage properties and the network topology properties are usually ignored in Virtual Network(VN)modelling,which leads to the inaccurate measurement of node availability and priority.In addition,most static virtual network mapping methods allocate fixed resources to users during the entire life cycle,and the users’actual resource requirements vary with the workload,which results in resource allocation redundancy.Based on the above analysis,in this paper,we propose a dynamic resource sharing virtual network mapping algorithm named NMA-PRS-VNE,first,we construct a new,more realistic network framework in which the properties of nodes include computing resources,storage resources and topology properties.In the node mapping process,three properties of the node are used to measure its mapping ability.Second,we consider the resources of adjacent nodes and links instead of the traditional method of measuring the availability and priority of nodes by considering only the resource properties,so as to more accurately select the physical mapping nodes that meet the constraints and conditions and improve the success rate of subsequent link mapping.Finally,we divide the resource requirements of Virtual Network Requests(VNRs)into basic subrequirements and variable sub-variable requirements to complete dynamic resource allocation.The former represents monopolizing resource requirements by the VNRs,while the latter represents shared resources by many VNRs with the probability of occupying resources,where we keep a balance between resource sharing and collision among users by calculating the collision probability.Simulation results show that the proposed NMAPRS-VNE can increase the average acceptance rate and network revenue by 15%and 38%,and reduce the network cost and link pressure by 25%and 17%.

Research on Dynamic Mathematical Resource Screening Methods Based on Machine Learning

The current digital educational resources are of many kinds and large quantities, to solve the problems existing in the existing dynamic resource selection methods, a dynamic resource selection method based on machine learning is proposed. Firstly, according to the knowledge structure and concepts of mathematical resources, combined with the basic components of dynamic mathematical resources, the knowledge structure graph of mathematical resources is constructed;according to the characteristics of mathematical resources, the interaction between users and resources is simulated, and the graph of the main body of the resources is identified, and the candidate collection of mathematical knowledge is selected;finally, according to the degree of matching between mathematical literature and the candidate collection, machine learning is utilized, and the mathematical resources are screened.

Dynamic resource allocation for high-speed railway downlink MIMO-OFDM system

The dynamic resource allocation problem in high-speed railway downlink orthogonal frequency-division multiplexing（OFDM） systems with multiple-input multiple-output（MIMO） antennas is investigated.Sub-carriers,antennas,time slots,and power are jointly considered.The problem of multi-dimensional resource allocation is formulated as a mixed-integer nonlinear programming problem.The effect of the moving speed on Doppler shift is analyzed to calculate the inter-carrier interference power.The optimization objective is to maximize the system throughput under the constraint of a total transmitted power that is no greater than a certain threshold.In order to reduce the computational complexity,a suboptimal solution to the optimization problem is obtained by a two-step method.First,sub-carriers,antennas,and time slots are assigned to users under the assumption of equal power allocation.Next,the power allocation problem is solved according to the result of the first-step resource allocation.Simulation results show that the proposed multi-dimensional resource allocation strategy has a significant performance improvement in terms of system throughput compared with the existing one.

Dynamic and Integrated Load-Balancing Scheduling Algorithm for Cloud Data Centers

One of the challenging scheduling problems in Cloud data centers is to take the allocation and migration of reconfigurable virtual machines as well as the integrated features of hosting physical machines into consideration. We introduce a Dynamic and Integrated Resource Scheduling algorithm (DAIRS) for Cloud data centers. Unlike traditional load-balance scheduling algorithms which often consider only one factor such as the CPU load in physical servers, DAIRS treats CPU, memory and network bandwidth integrated for both physical machines and virtual machines. We develop integrated measurement for the total imbalance level of a Cloud datacenter as well as the average imbalance level of each server. Simulation results show that DAIRS has good performance with regard to total imbalance level, average imbalance level of each server, as well as overall running time.

CHANNEL DE-ALLOCATION SCHEMES FOR GSM/GPRS NETWORKS

Several channel de-allocation schemes for GSM/GPRS(General Packet Radio Service) networks are proposed in this paper. For DRA (Dynamical Resource Allocation) with de-allocation mechanism, if a new voice call arrives and finds that all the channels are busy,then one of the GPRS packets which occupy more than one channel for data transmission may release a channel for the new voice call. This paper presents 5 de-allocation mechanisms, i.e.DA-RANDOM, DA-RICHEST, DA-POOREST, DA-OLDEST and DA-YOUNGEST, to select the GPRS packet for releasing the appropriate channel. Simulation results show that DAOLDEST achieves the best performance, especially in packets blocking probability, among all the de-allocation schemes. Although the performance of the proposed de-allocation schemes is not significantly different, they are all much better than that of the scheme without de-allocation.

QoS-Aware Dynamic Resource Management in Heterogeneous Mobile Cloud Computing Networks

In mobile cloud computing(MCC) systems,both the mobile access network and the cloud computing network are heterogeneous,implying the diverse configurations of hardware,software,architecture,resource,etc.In such heterogeneous mobile cloud(HMC) networks,both radio and cloud resources could become the system bottleneck,thus designing the schemes that separately and independently manage the resources may severely hinder the system performance.In this paper,we aim to design the network as the integration of the mobile access part and the cloud computing part,utilizing the inherent heterogeneity to meet the diverse quality of service(QoS)requirements of tenants.Furthermore,we propose a novel cross-network radio and cloud resource management scheme for HMC networks,which is QoS-aware,with the objective of maximizing the tenant revenue while satisfying the QoS requirements.The proposed scheme is formulated as a restless bandits problem,whose "indexability" feature guarantees the low complexity with scalable and distributed characteristics.Extensive simulation results are presented to demonstrate the significant performance improvement of the proposed scheme compared to the existing ones.

MODELING RADIO RESOURCE MANAGEMENT IN COGNITIVE RADIO SYSTEM BASED ON OFDMA USING COLORED PETRI NET

Cognitive Radio(CR) system based on Orthogonal Frequency Division Multiple Access(OFDMA),such as Wireless Regional Area Networks(WRAN) and Worldwide Interoperability for Microwave Access(WiMAX),often attempt to improve performance via dynamic radio resource management,which is characterized as concurrent processing of different traffic and nondeterministic system capacity.It is essential to design and evaluate such complex system using proper modeling and analysis tools.In the previous work,most of the communication systems were modeled as Markov Chain(MC) and Stochastic Petri Nets(SPN),which have the explicit limitation in evaluating adaptive OFDMA CR system with wide area traffic.In this paper,we develop an executable top-down hier-archical Colored Petri Net(CPN) model for adaptive OFDMA CR system,and analyze its performance using CPN tools.The results demonstrate that the CPN can model different radio resource manage-ment algorithms in CR Systems,and the CPN tools require less computational effort than Markov model using Matlab,with its flexibility and adaptability to the traffics which arrival interval and processing time are not exponentially distributed.

A dynamic model for exploring water-resource management scenarios in an inland arid area: Shanshan County, Northwestern China

Water scarcity is a challenge in many arid and semi-arid regions; this may lead to a series of environmental problems and could be stressed even further by the effects from climate change. This study focused on the water resource management in Shanshan County, an inland arid region located in northwestern China with a long history of groundwater overexploitation. A model of the supply and demand system in the study area from 2006 to2030, including effects from global climate change,was developed using a system dynamics(SD)modeling tool. This SD model was used to 1) explore the best water-resource management options by testing system responses under various scenarios and2) identify the principal factors affecting the responses, aiming for a balance of the groundwater system and sustainable socio-economic development.Three causes were identified as primarily responsible for water issues in Shanshan: low water-use efficiency low water reuse, and increase in industrial waterdemand. To address these causes, a combined scenario was designed and simulated, which was able to keep the water deficiency under 5% by 2030. The model provided some insights into the dynamic interrelations that generate system behavior and the key factors in the system that govern water demand and supply. The model as well as the study results may be useful in water resources management in Shanshan and may be applied, with appropriate modifications, to other regions facing similar water management challenges.

Efficient Network Slicing with Dynamic Resource Allocation

With the rapid development of wireless network technologies and the growing de⁃mand for a high quality of service(QoS),the effective management of network resources has attracted a lot of attention.For example,in a practical scenario,when a network shock oc⁃curs,a batch of affected flows needs to be rerouted to respond to the network shock to bring the entire network deployment back to the optimal state,and in the process of rerouting a batch of flows,the entire response time needs to be as short as possible.Specifically,we re⁃duce the time consumed for routing by slicing,but the routing success rate after slicing is re⁃duced compared with the unsliced case.In this context,we propose a two-stage dynamic net⁃work resource allocation framework that first makes decisions on the slices to which flows are assigned,and coordinates resources among slices to ensure a comparable routing suc⁃cess rate as in the unsliced case,while taking advantage of the time efficiency gains from slicing.

Project Management and Coordination of Distributed Cooperative Product Design in a Virtual Enterprise

The architecture of project management of distributed concurrent product design in a virtual enterprise is put forward. T he process of project management and its functions are presented. Product design process coordination is also discussed. First, based on the analysis of traditi onal project management, project management and coordination of distributed coop erative product design in the virtual enterprise is put forward. Then, aiming at the characteristics of a distributed concurrent product design process, the inh erent rules and complex interrelations in product development are studied. Accor dingly, the architecture of project management of distributed cooperative produc t design in a virtual enterprise is presented to adapt to distributed concurrent development of complex products. The main advantages of the architecture are al so discussed. Finally, the emphasis is placed on the project management process. Its main functions are set forth, such as project definition, task decompositio n and distribution, resource constraints and dynamic resource scheduling, proces s fusion, task scheduling and monitoring, project plan, cost and quality evaluat ion, etc.

Fault Aware Dynamic Resource Manager for Fault Recognition and Avoidance in Cloud

Fault tolerance(FT)schemes are intended to work on a minimized and static amount of physical resources.When a host failure occurs,the conventional FT frequently proceeds with the execution on the accessible working hosts.This methodology saves the execution state and applications to complete without disruption.However,the dynamicity of open cloud assets is not seen when taking scheduling choices.Existing optimization techniques are intended in dealing with resource scheduling.This method will be utilized for distributing the approaching tasks to the VMs.However,the dynamic scheduling for this procedure doesn’t accomplish the objective of adaptation of internal failure.The scheme prefers jobs in the activity list with the most elevated execution time on resources that can execute in a shorter timeframe,but it suffers with higher makespan;poor resource usage and unbalance load concerns.To overcome the above mentioned issue,Fault Aware Dynamic Resource Manager(FADRM)is proposed that enhances the mechanism to Multi-stage Resilience Manager at an application-level FT arrangement.Proposed FADRM method gives FT a Multi-stage Resilience Manager(MRM)in the client and application layers,and simultaneously decreases the over-head and degradations.It additionally provides safety to the application execution considering the clients,application and framework necessities.Based on experimental evaluations,Proposed Fault Aware Dynamic Resource Manager(FADRM)method 157.5 MakeSpan(MS)time,0.38 Fault Rate(FR),0.25 Failure Delay(FD)and improves 5.5 Performance Improvement Ratio(PIR)for 25,50,75 and 100 tasks and 475 MakeSpan(MS)time,0.40 Fault Rate(FR),1.30 Failure Delay(FD)and improves 6.75 improves Performance Improvement Ratio(PER)for 100,200,300 and 500 Tasks compare than existing methodologies.

Dynamic Resource Scheduling in Emergency Environment

Nowadays,emergency accidents could happen at any time.The accidents occur unpredictably and the accidents requirements are diversely.The accidents happen in a dynamic environment and the resource should be cooperative to solve the accidents.Most methods are focusing on minimizing the casualties and property losses in a static environment.However,they are lack in considering the dynamic and unpredictable event handling.In this paper,we propose a representative environmental model in representation of emergency and dynamic resource allocation model,and an adaptive mathematical model based on Genetic Algorithm(GA)to generate an optimal set of solution domain.The experimental results show that the proposed algorithm can get a set of better candidate solutions.

Extended Virtual Boundary and Its Application in Dynamic Spectrum Allocation

This paper presents an efficient dynamic spectrum allocation （DSA） scheme in a flexible spectrum licensing environment where multiple networks coexist and interfere with each other. In particular, an extension of virtual boundary concept in DSA is proposed, which is spectrally efficient than the previous virtual boundary concept applied to donor systems only. Here, the same technique is applied to both donor and rental systems so as to further reduce the occurrences where the insertion of guard bands is obligatory and as a result provides better spectral efficiency. The proposed extension improves the spectrum utilization without any compromise on interference and fairness issues.

Hierarchical Disturbance Propagation Mechanism and Improved Contract Net Protocol for Satellite TT&C Resource Dynamic Scheduling

The practical engineering of satellite tracking telemetry and command(TT&C)is often disturbed by unpredictable external factors,including the temporary rise in a significant quantity of satellite TT&C tasks,temporary failures and failures of some TT&C resources,and so on.To improve the adaptability and robustness of satellite TT&C systems when faced with uncertain dynamic disturbances,a hierarchical disturbance propagation mechanism and an improved contract network dynamic scheduling method for satellite TT&C resources were designed to address the dynamic scheduling problem of satellite TT&C resources.Firstly,the characteristics of the dynamic scheduling problem of satellite TT&C resources are analyzed,and a mathematical model is established with the weighted optimization objectives of maximizing the revenue from task completion and minimizing the degree of plan disturbance.Then,a bottom-up distributed dynamic collaborative scheduling framework for satellite TT&C resources is proposed,which includes a task layer,a resource layer,a central internal collaboration layer,and a central external collaboration layer.Dynamic disturbances are propagated layer by layer from the task layer to the central external collaboration layer in a bottom-up manner,using efficient heuristic strategies in the task layer and the resource layer,respectively.We use improved contract network algorithms in the center internal collaboration layer and the center external collaboration layer,the original scheduling plan is quickly adjusted to minimize the impact of disturbances while effectively completing dynamic task requirements.Finally,a large number of simulation experiments were carried out and compared with various comparative algorithms.The results show that the proposed algorithm can effectively improve the solution effect of satellite TT&C resource dynamic scheduling problems,and has good application prospects.

Sequential dynamic resource allocation in multi-beam satellite systems:A learning-based optimization method

Multi-beam antenna and beam hopping technologies are an effective solution for scarce satellite frequency resources.One of the primary challenges accompanying with Multi-Beam Satellites(MBS)is an efficient Dynamic Resource Allocation(DRA)strategy.This paper presents a learning-based Hybrid-Action Deep Q-Network(HADQN)algorithm to address the sequential decision-making optimization problem in DRA.By using a parameterized hybrid action space,HADQN makes it possible to schedule the beam pattern and allocate transmitter power more flexibly.To pursue multiple long-term QoS requirements,HADQN adopts a multi-objective optimization method to decrease system transmission delay,loss ratio of data packets and power consumption load simultaneously.Experimental results demonstrate that the proposed HADQN algorithm is feasible and greatly reduces in-orbit energy consumption without compromising QoS performance.

A many-objective evolutionary algorithm based on decomposition with dynamic resource allocation for irregular optimization

The multi-objective optimization problem has been encountered in numerous fields such as high-speed train head shape design,overlapping community detection,power dispatch,and unmanned aerial vehicle formation.To address such issues,current approaches focus mainly on problems with regular Pareto front rather than solving the irregular Pareto front.Considering this situation,we propose a many-objective evolutionary algorithm based on decomposition with dynamic resource allocation(Ma OEA/D-DRA)for irregular optimization.The proposed algorithm can dynamically allocate computing resources to different search areas according to different shapes of the problem’s Pareto front.An evolutionary population and an external archive are used in the search process,and information extracted from the external archive is used to guide the evolutionary population to different search regions.The evolutionary population evolves with the Tchebycheff approach to decompose a problem into several subproblems,and all the subproblems are optimized in a collaborative manner.The external archive is updated with the method of rithms using a variety of test problems with irregular Pareto front.Experimental results show that the proposed algorithèm out-p￡performs these five algorithms with respect to convergence speed and diversity of population members.By comparison with the weighted-sum approach and penalty-based boundary intersection approach,there is an improvement in performance after integration of the Tchebycheff approach into the proposed algorithm.

A Dynamic Resource Allocation Strategy with Reinforcement Learning for Multimodal Multi-objective Optimization

Many isolation approaches, such as zoning search, have been proposed to preserve the diversity in the decision space of multimodal multi-objective optimization(MMO). However, these approaches allocate the same computing resources for subspaces with different difficulties and evolution states. In order to solve this issue, this paper proposes a dynamic resource allocation strategy(DRAS)with reinforcement learning for multimodal multi-objective optimization problems(MMOPs). In DRAS, relative contribution and improvement are utilized to define the aptitude of subspaces, which can capture the potentials of subspaces accurately. Moreover, the reinforcement learning method is used to dynamically allocate computing resources for each subspace. In addition, the proposed DRAS is applied to zoning searches. Experimental results demonstrate that DRAS can effectively assist zoning search in finding more and better distributed equivalent Pareto optimal solutions in the decision space.

A novel model on dynamic resource allocation in optical networks

A novel model on dynamic resource allocation in the WDM optical networks is proposed, basing on the integrated considerations of the impacts of transmission impairments and service classification on dynamic resource allocation in the optical layer. In this model, the priorities of optical connection requests are mapped into different thresholds of transmission impairments, and a uniform method which is adopted to evaluate the virtual wavelength path (VWP) candidates is defined. The Advanced Preferred Wavelength Sets Algorithm (A-PWS) and the heuristic Dynamic Min-Cost & Optical Virtual Wavelength Path Algorithm (DMC-OVWP) are presented addressing the routing and wavelength assignment (RWA) problem based on dynamic traffic and multi priorities in wavelength-routed optical networks. For a received optical connection request, DMC-OVWP is employed to calculate a list of the VWP candidates, and an appropriate VWP which matches the request's priority is picked up to establish the lightpath by analyzing the transmission qualities of the VWP candidates.

Subcarrier and power allocation algorithm based on inter-cell interference mitigation for OFDMA system

This article proposes a dynamic subcarrier and power allocation algorithm for multicell orthogonal frequency division multiple access （OFDMA） downlink system, based on inter-cell interference （ICI） mitigation. Different from other ICI mitigation schemes, which pay little attention to power allocation in the system, the proposed algorithm assigns channels to each user, based on proportional-fair （PF） scheduling and ICI coordination, whereas allocating power is based on link gain distribution and the loading bit based on adaptive modulation and coding （AMC） in base transceiver station （BTS）. Simulation results show that the algorithm yields better performance for data services under fast fading.