MADDPG-D2: An Intelligent Dynamic Task Allocation Algorithm Based on Multi-Agent Architecture Driven by Prior Knowledge

Aiming at the problems of low solution accuracy and high decision pressure when facing large-scale dynamic task allocation(DTA)and high-dimensional decision space with single agent,this paper combines the deep reinforce-ment learning(DRL)theory and an improved Multi-Agent Deep Deterministic Policy Gradient(MADDPG-D2)algorithm with a dual experience replay pool and a dual noise based on multi-agent architecture is proposed to improve the efficiency of DTA.The algorithm is based on the traditional Multi-Agent Deep Deterministic Policy Gradient(MADDPG)algorithm,and considers the introduction of a double noise mechanism to increase the action exploration space in the early stage of the algorithm,and the introduction of a double experience pool to improve the data utilization rate;at the same time,in order to accelerate the training speed and efficiency of the agents,and to solve the cold-start problem of the training,the a priori knowledge technology is applied to the training of the algorithm.Finally,the MADDPG-D2 algorithm is compared and analyzed based on the digital battlefield of ground and air confrontation.The experimental results show that the agents trained by the MADDPG-D2 algorithm have higher win rates and average rewards,can utilize the resources more reasonably,and better solve the problem of the traditional single agent algorithms facing the difficulty of solving the problem in the high-dimensional decision space.The MADDPG-D2 algorithm based on multi-agent architecture proposed in this paper has certain superiority and rationality in DTA.

Dynamic optimal allocation of energy storage systems integrated within photovoltaic based on a dual timescale dynamics model

Energy storage systems(ESSs)operate as independent market participants and collaborate with photovoltaic(PV)generation units to enhance the flexible power supply capabilities of PV units.However,the dynamic variations in the profitability of ESSs in the electricity market are yet to be fully understood.This study introduces a dual-timescale dynamics model that integrates a spot market clearing(SMC)model into a system dynamics(SD)model to investigate the profit-aware capacity growth of ESSs and compares the profitability of independent energy storage systems(IESSs)with that of an ESS integrated within a PV(PV-ESS).Furthermore,this study aims to ascertain the optimal allocation of the PV-ESS.First,SD and SMC models were set up.Second,the SMC model simulated on an hourly timescale was incorporated into the SD model as a subsystem,a dual-timescale model was constructed.Finally,a development simulation and profitability analysis was conducted from 2022 to 2040 to reveal the dynamic optimal range of PV-ESS allocation.Additionally,negative electricity prices were considered during clearing processes.The simulation results revealed differences in profitability and capacity growth between IESS and PV-ESS,helping grid investors and policymakers to determine the boundaries of ESSs and dynamic optimal allocation of PV-ESSs.

An Advanced Dynamic Scheduling for Achieving Optimal Resource Allocation

Cloud computing distributes task-parallel among the various resources.Applications with self-service supported and on-demand service have rapid growth.For these applications,cloud computing allocates the resources dynami-cally via the internet according to user requirements.Proper resource allocation is vital for fulfilling user requirements.In contrast,improper resource allocations result to load imbalance,which leads to severe service issues.The cloud resources implement internet-connected devices using the protocols for storing,communi-cating,and computations.The extensive needs and lack of optimal resource allo-cating scheme make cloud computing more complex.This paper proposes an NMDS(Network Manager based Dynamic Scheduling)for achieving a prominent resource allocation scheme for the users.The proposed system mainly focuses on dimensionality problems,where the conventional methods fail to address them.The proposed system introduced three–threshold mode of task based on its size STT,MTT,LTT(small,medium,large task thresholding).Along with it,task mer-ging enables minimum energy consumption and response time.The proposed NMDS is compared with the existing Energy-efficient Dynamic Scheduling scheme(EDS)and Decentralized Virtual Machine Migration(DVM).With a Network Manager-based Dynamic Scheduling,the proposed model achieves excellence in resource allocation compared to the other existing models.The obtained results shows the proposed system effectively allocate the resources and achieves about 94%of energy efficient than the other models.The evaluation metrics taken for comparison are energy consumption,mean response time,percentage of resource utilization,and migration.

Dynamic Allocation of Manufacturing Tasks and Resources in Shared Manufacturing

Shared manufacturing is recognized as a new point-to-point manufac-turing mode in the digital era.Shared manufacturing is referred to as a new man-ufacturing mode to realize the dynamic allocation of manufacturing tasks and resources.Compared with the traditional mode,shared manufacturing offers more abundant manufacturing resources and flexible configuration options.This paper proposes a model based on the description of the dynamic allocation of tasks and resources in the shared manufacturing environment,and the characteristics of shared manufacturing resource allocation.The execution of manufacturing tasks,in which candidate manufacturing resources enter or exit at various time nodes,enables the dynamic allocation of manufacturing tasks and resources.Then non-dominated sorting genetic algorithm(NSGA-II)and multi-objective particle swarm optimization(MOPSO)algorithms are designed to solve the model.The optimal parameter settings for the NSGA-II and MOPSO algorithms have been obtained according to the experiments with various population sizes and iteration numbers.In addition,the proposed model’s efficiency,which considers the entries and exits of manufacturing resources in the shared manufacturing environment,is further demonstrated by the overlap between the outputs of the NSGA-II and MOPSO algorithms for optimal resource allocation.

Cooperative Relaying in a Three User Downlink NOMA System Using Dynamic Power Allocation

Non-orthogonal multiple access(NOMA)represents the latest addition to the array of multiple access techniques,enabling simultaneous servicing of multiple users within a singular resource block in terms of time,frequency,and code.A typical NOMA configuration comprises a base station along with proximate and distant users.The proximity users experience more favorable channel conditions in contrast to distant users,resulting in a compromised performance for the latter due to the less favorable channel conditions.When cooperative communication is integrated with NOMA,the overall system performance,including spectral efficiency and capacity,is further elevated.This study introduces a cooperative NOMA setup in the downlink,involving three users,and employs dynamic power allocation(DPA).Within this framework,User 2 acts as a relay,functioning under the decode-and-forward protocol,forwarding signals to both User 1 and User 3.This arrangement aims to bolster the performance of the user positioned farthest from the base station,who is adversely affected by weaker channel conditions.Theoretical and simulation outcomes reveal enhancements within the system’s performance.

Resource Allocation in Multi-User Cellular Networks:A Transformer-Based Deep Reinforcement Learning Approach

To meet the communication services with diverse requirements,dynamic resource allocation has shown increasing importance.In this paper,we consider the multi-slot and multi-user resource allocation(MSMU-RA)in a downlink cellular scenario with the aim of maximizing system spectral efficiency while guaranteeing user fairness.We first model the MSMURA problem as a dual-sequence decision-making process,and then solve it by a novel Transformerbased deep reinforcement learning(TDRL)approach.Specifically,the proposed TDRL approach can be achieved based on two aspects:1)To adapt to the dynamic wireless environment,the proximal policy optimization(PPO)algorithm is used to optimize the multi-slot RA strategy.2)To avoid co-channel interference,the Transformer-based PPO algorithm is presented to obtain the optimal multi-user RA scheme by exploring the mapping between user sequence and resource sequence.Experimental results show that:i)the proposed approach outperforms both the traditional and DRL methods in spectral efficiency and user fairness,ii)the proposed algorithm is superior to DRL approaches in terms of convergence speed and generalization performance.

Optimal allocation method of electric/air braking force of high-speed train considering axle load transfer

Reasonable distribution of braking force is a factor for a smooth,safe,and comfortable braking of trains.A dynamic optimal allocation strategy of electric-air braking force is proposed in this paper to solve the problem of the lack of consideration of adhesion difference of train wheelsets in the existing high-speed train electric-air braking force optimal allocation strategies.In this method,the braking strategy gives priority to the use of electric braking force.The force model of a single train in the braking process is analyzed to calculate the change of adhesion between the wheel and rail of each wheelset after axle load transfer,and then the adhesion of the train is estimated in real time.Next,with the goal of maximizing the total adhesion utilization ratio of trailer/motor vehicles,a linear programming distribution function is constructed.The proportional coefficient of adhesion utilization ratio of each train and the application upper limit of braking force in the function is updated according to the change time point of wheelset adhesion.Finally,the braking force is dynamically allocated.The simulation results of Matlab/Simulink show that the proposed algorithm not only uses the different adhesion limits of each trailer to reduce the total amount of braking force undertaken by the motor vehicle,but also considers the adhesion difference of each wheelset.The strategy can effectively reduce the risk and time of motor vehicles during the braking process and improve the stability of the train braking.

Centralized Dynamic Spectrum Allocation in Cognitive Radio Networks Based on Fuzzy Logic and Q-Learning

A novel centralized approach for Dynamic Spectrum Allocation (DSA) in the Cognitive Radio (CR) network is presented in this paper. Instead of giving the solution in terms of formulas modeling network environment such as linear programming or convex optimization, the new approach obtains the capability of iteratively on-line learning environment performance by using Reinforcement Learning (RL) algorithm after observing the variability and uncertainty of the heterogeneous wireless networks. Appropriate decision-making access actions can then be obtained by employing Fuzzy Inference System (FIS) which ensures the strategy being able to explore the possible status and exploit the experiences sufficiently. The new approach considers multi-objective such as spectrum efficiency and fairness between CR Access Points (AP) effectively. By interacting with the environment and accumulating comprehensive advantages, it can achieve the largest long-term reward expected on the desired objectives and implement the best action. Moreover, the present algorithm is relatively simple and does not require complex calculations. Simulation results show that the proposed approach can get better performance with respect to fixed frequency planning scheme or general dynamic spectrum allocation policy.

Dynamic resource allocation for high-speed railway downlink MIMO-OFDM system

The dynamic resource allocation problem in high-speed railway downlink orthogonal frequency-division multiplexing（OFDM） systems with multiple-input multiple-output（MIMO） antennas is investigated.Sub-carriers,antennas,time slots,and power are jointly considered.The problem of multi-dimensional resource allocation is formulated as a mixed-integer nonlinear programming problem.The effect of the moving speed on Doppler shift is analyzed to calculate the inter-carrier interference power.The optimization objective is to maximize the system throughput under the constraint of a total transmitted power that is no greater than a certain threshold.In order to reduce the computational complexity,a suboptimal solution to the optimization problem is obtained by a two-step method.First,sub-carriers,antennas,and time slots are assigned to users under the assumption of equal power allocation.Next,the power allocation problem is solved according to the result of the first-step resource allocation.Simulation results show that the proposed multi-dimensional resource allocation strategy has a significant performance improvement in terms of system throughput compared with the existing one.

Dynamic Bandwidth Allocation and Rate Coordination for DiffServ Environment

This paper presents a novel model for dynamic bandwidth allocation and rate coordination based on DiffServ and a bandwidth broker（BB）. In this model, assignment of bandwidth was made according to a periodic trace of network characteristics per application. And adjustment of transfer rate was accomplished through negotiation with applications by a bandwidth agent. This model was evaluated using network simulator 2 （NS-2）, and distinct improvements were found in respects of delay and packet loss of overall network and single flow. Finally, the model was suggested to be leveraged to multimedia applications with properties of lower delay and lower packet loss.

A Dynamic Distributed Spectrum Allocation Mechanism Based on Game Model in Fog Radio Access Networks

With the explosive growth of highspeed wireless data demand and the number of mobile devices, fog radio access networks(F-RAN) with multi-layer network structure becomes a hot topic in recent research. Meanwhile, due to the rapid growth of mobile communication traffic, high cost and the scarcity of wireless resources, it is especially important to develop an efficient radio resource management mechanism. In this paper, we focus on the shortcomings of resource waste, and we consider the actual situation of base station dynamic coverage and user requirements. We propose a spectrum pricing and allocation scheme based on Stackelberg game model under F-RAN framework, realizing the allocation of resource on demand. This scheme studies the double game between the users and the operators, as well as between the traditional operators and the virtual operators, maximizing the profits of the operators. At the same time, spectrum reuse technology is adopted to improve the utilization of network resource. By analyzing the simulation results, it is verified that our proposed scheme can not only avoid resource waste, but also effectively improve the operator's revenue efficiency and overall network resource utilization.

Approximate Dynamic Programming for Stochastic Resource Allocation Problems

A stochastic resource allocation model, based on the principles of Markov decision processes(MDPs), is proposed in this paper. In particular, a general-purpose framework is developed, which takes into account resource requests for both instant and future needs. The considered framework can handle two types of reservations(i.e., specified and unspecified time interval reservation requests), and implement an overbooking business strategy to further increase business revenues. The resulting dynamic pricing problems can be regarded as sequential decision-making problems under uncertainty, which is solved by means of stochastic dynamic programming(DP) based algorithms. In this regard, Bellman’s backward principle of optimality is exploited in order to provide all the implementation mechanisms for the proposed reservation pricing algorithm. The curse of dimensionality, as the inevitable issue of the DP both for instant resource requests and future resource reservations,occurs. In particular, an approximate dynamic programming(ADP) technique based on linear function approximations is applied to solve such scalability issues. Several examples are provided to show the effectiveness of the proposed approach.

Optimal dynamic spectrum allocation-assisted latency minimization for multiuser mobile edge computing

Mobile Edge Computing(MEC)has been envisioned as an efficient solution to provide computation-intensive yet latency-sensitive services for wireless devices.In this paper,we investigate the optimal dynamic spectrum allocation-assisted multiuser computation offloading in MEC for overall latency minimization.Specifically,we first focus on a static multiuser computation offloading scenario and jointly optimize users'offloading decisions,transmission durations,and Edge Servers'(ESs)resource allocations.Owing to the nonconvexity of our joint optimization problem,we identify its layered structure and decompose it into two problems:a subproblem and a top problem.For the subproblem,we propose a bisection search-based algorithm to efficiently find the optimal users'offloading decisions and ESs’resource allocations under a given transmission duration.Second,we use a linear search-based algorithm for solving the top problem to obtain the optimal transmission duration based on the result of the subproblem.Further,after solving the static scenario,we consider a dynamic scenario of multiuser computation offloading with time-varying channels and workload.To efficiently address this dynamic scenario,we propose a deep reinforcement learning-based online algorithm to determine the near-optimal transmission duration in a real-time manner.Numerical results are provided to validate our proposed algorithms for minimizing the overall latency in both static and dynamic offloading scenarios.We also demonstrate the advantages of our proposed algorithms compared to the conventional multiuser computation offloading schemes.

Dynamic bandwidth allocation algorithm for full-band utilization

To improve and optimize the bandwidth utilization for multi-service packet transporting system, a kind of Dynamic Full Bandwidth Utilized （DFBU） allocation algorithm allowing a single link to use far beyond its fair share bandwidth is presented. Three important parameters as the bound on max and minimum bandwidth, the maximum packet delay and the minimum bandwidth utilization are discussed and analyzed. Results of experiments show that the DFBU-algorithm is capable of making a single link in the system use all the spare bandwidth （up to full-bandwidth） while the performance of fairness and QoS requirement is still guaranteed.

Distributed Resource Allocation via Accelerated Saddle Point Dynamics

In this paper,accelerated saddle point dynamics is proposed for distributed resource allocation over a multi-agent network,which enables a hyper-exponential convergence rate.Specifically,an inertial fast-slow dynamical system with vanishing damping is introduced,based on which the distributed saddle point algorithm is designed.The dual variables are updated in two time scales,i.e.,the fast manifold and the slow manifold.In the fast manifold,the consensus of the Lagrangian multipliers and the tracking of the constraints are pursued by the consensus protocol.In the slow manifold,the updating of the Lagrangian multipliers is accelerated by inertial terms.Hyper-exponential stability is defined to characterize a faster convergence of our proposed algorithm in comparison with conventional primal-dual algorithms for distributed resource allocation.The simulation of the application in the energy dispatch problem verifies the result,which demonstrates the fast convergence of the proposed saddle point dynamics.

Dynamic Resource Allocation in TDD-Based Heterogeneous Cloud Radio Access Networks

To fulfill the explosive growth of network capacity, fifth generation(5G) standard has captured the attention and imagination of researchers and engineers around the world. In particular, heterogeneous cloud radio access network(H-CRAN), as a promising network paradigm in 5G system, is a hot research topic in recent years. However, the densely deployment of RRHs in H-CRAN leads to downlink/uplink traffic asymmetry and severe inter-cell interference which could seriously impair the network throughput and resource utilization. To simultaneously solve these two problems, we proposed a dynamic resource allocation(DRA) scheme for H-CRAN in TDD mode. Firstly, we design a clustering algorithm to group the RRHs into different sets. Secondly, we adopt coordinated multipoint technology to eliminate the interference in each set. Finally, we formulate the joint frame structure, power and subcarrier selection problem as a mixed strategy noncooperative game. The simulation results are presented to validate the effectiveness of our proposed algorithm by compared with the existing work.

Fuzzy Earthwork Dynamic Allocation and Optimization for Construction of High Concrete Face Rockfill Dam

Due to the complexity of earthwork allocation system for the construction of high concrete face rockfill dam,traditional allocation and planning are not able to function properly in the construction process with strong randomness.In this paper,the working mechanism of earthwork dynamic allocation system is analyzed comprehensively and a solution to fuzzy earthwork dynamic allocation is proposed on the basis of uncertain factors in the earthwork allocation of a hydropower project.Under the premise of actual situation and the experience of the construction site,an all-coefficient-fuzzy linear programming mathematical model with fuzzy parameters and constraints for earthwork allocation is established according to the structure unit weighted ranking criteria.In this way,the deficiency of certain allocation model can be overcome.The application results indicate that the proposed method is more rational compared with traditional earthwork allocation.

Spatio-Temporal Population Density and Spatial Dynamic Spectrum Allocation

A realistic population density distribution scenario in conjunction with the spatial dynamic spectrum allocation （DSA） is taken into account to mitigate the spectrum wastage in terms of extra guard bands. For the insertion of the extra guard bands, an efficient strategy based on self-assessment is applied to each victim cell individually and independently. Consequently, it is no more required to spread the extra guard band over the whole DSA region. Simulation results StlOW an improvement of 3% -4% in percentage of satisfied users for Universal Mobile Telecommunications System （UMTS） network and 4%-5% for Digital Video Broadcasting Terrestrial （DVB-T） network.

DYNAMIC RESOURCE ALLOCATION SCHEME UNDER TRAFFIC CONDITION IN SATELLITE SYSTEMS

As the traffic distribution in China mainland is far from uniform, a new traffic model in China mainland is presented on the basis of per-capita Gross Domestic Product (GDP) and density of population. Based on this characteristic traffic model, a new Traffic Dependent Dynamic Channel Allocation and Reservation (TDDCAR) technique is proposed, the simulation model is built, and the strategies' performance is evaluated through computer simulation. The simulation results show that, compared to the conventional Fixed Channel Allocation (FCA), TDDCAR estimates the traffic conditions in every spot beam and frequently adjusts the traffic according to current traffic conditions. It has achieved a significant improvement in new call blocking probability, handover blocking probability, and fair index, particularly, in heavy traffic conditions. The building of traffic model in China mainland and the analysis of the simulation results has been a key foundation for the study of resource allocation schemes in the future.

Dynamic Resource Pricing and Allocation in Multilayer Satellite Network

The goal of delivering high-quality service has spurred research of 6G satellite communication networks.The limited resource-allocation problem has been addressed by next-generation satellite communication networks,especially multilayer networks with multiple low-Earth-orbit(LEO)and nonlow-Earth-orbit(NLEO)satellites.In this study,the resource-allocation problem of a multilayer satellite network consisting of one NLEO and multiple LEO satellites is solved.The NLEO satellite is the authorized user of spectrum resources and the LEO satellites are unauthorized users.The resource allocation and dynamic pricing problems are combined,and a dynamic gamebased resource pricing and allocation model is proposed to maximize the market advantage of LEO satellites and reduce interference between LEO and NLEO satellites.In the proposed model,the resource price is formulated as the dynamic state of the LEO satellites,using the resource allocation strategy as the control variable.Based on the proposed dynamic game model,an openloop Nash equilibrium is analyzed,and an algorithm is proposed for the resource pricing and allocation problem.Numerical simulations validate the model and algorithm.