Channel assignment and power allocation for throughput improvement with PPO in B5G heterogeneous edge networks

In Beyond the Fifth Generation(B5G)heterogeneous edge networks,numerous users are multiplexed on a channel or served on the same frequency resource block,in which case the transmitter applies coding and the receiver uses interference cancellation.Unfortunately,uncoordinated radio resource allocation can reduce system throughput and lead to user inequity,for this reason,in this paper,channel allocation and power allocation problems are formulated to maximize the system sum rate and minimum user achievable rate.Since the construction model is non-convex and the response variables are high-dimensional,a distributed Deep Reinforcement Learning(DRL)framework called distributed Proximal Policy Optimization(PPO)is proposed to allocate or assign resources.Specifically,several simulated agents are trained in a heterogeneous environment to find robust behaviors that perform well in channel assignment and power allocation.Moreover,agents in the collection stage slow down,which hinders the learning of other agents.Therefore,a preemption strategy is further proposed in this paper to optimize the distributed PPO,form DP-PPO and successfully mitigate the straggler problem.The experimental results show that our mechanism named DP-PPO improves the performance over other DRL methods.

Power Allocation for SE Maximization in Uplink Massive MIMO System Under Minimum Rate Constraint

In this paper,we optimize the spectrum efficiency(SE)of uplink massive multiple-input multiple-output(MIMO)system with imperfect channel state information(CSI)over Rayleigh fading channel.The SE optimization problem is formulated under the constraints of maximum power and minimum rate of each user.Then,we develop a near-optimal power allocation(PA)scheme by using the successive convex approximation(SCA)method,Lagrange multiplier method,and block coordinate descent(BCD)method,and it can obtain almost the same SE as the benchmark scheme with lower complexity.Since this scheme needs three-layer iteration,a suboptimal PA scheme is developed to further reduce the complexity,where the characteristic of massive MIMO(i.e.,numerous receive antennas)is utilized for convex reformulation,and the rate constraint is converted to linear constraints.This suboptimal scheme only needs single-layer iteration,thus has lower complexity than the near-optimal scheme.Finally,we joint design the pilot power and data power to further improve the performance,and propose an two-stage algorithm to obtain joint PA.Simulation results verify the effectiveness of the proposed schemes,and superior SE performance is achieved.

Dynamic optimal allocation of energy storage systems integrated within photovoltaic based on a dual timescale dynamics model

Energy storage systems(ESSs)operate as independent market participants and collaborate with photovoltaic(PV)generation units to enhance the flexible power supply capabilities of PV units.However,the dynamic variations in the profitability of ESSs in the electricity market are yet to be fully understood.This study introduces a dual-timescale dynamics model that integrates a spot market clearing(SMC)model into a system dynamics(SD)model to investigate the profit-aware capacity growth of ESSs and compares the profitability of independent energy storage systems(IESSs)with that of an ESS integrated within a PV(PV-ESS).Furthermore,this study aims to ascertain the optimal allocation of the PV-ESS.First,SD and SMC models were set up.Second,the SMC model simulated on an hourly timescale was incorporated into the SD model as a subsystem,a dual-timescale model was constructed.Finally,a development simulation and profitability analysis was conducted from 2022 to 2040 to reveal the dynamic optimal range of PV-ESS allocation.Additionally,negative electricity prices were considered during clearing processes.The simulation results revealed differences in profitability and capacity growth between IESS and PV-ESS,helping grid investors and policymakers to determine the boundaries of ESSs and dynamic optimal allocation of PV-ESSs.

Mobile Crowdsourcing Task Allocation Based on Dynamic Self-Attention GANs

Crowdsourcing technology is widely recognized for its effectiveness in task scheduling and resource allocation.While traditional methods for task allocation can help reduce costs and improve efficiency,they may encounter challenges when dealing with abnormal data flow nodes,leading to decreased allocation accuracy and efficiency.To address these issues,this study proposes a novel two-part invalid detection task allocation framework.In the first step,an anomaly detection model is developed using a dynamic self-attentive GAN to identify anomalous data.Compared to the baseline method,the model achieves an approximately 4%increase in the F1 value on the public dataset.In the second step of the framework,task allocation modeling is performed using a twopart graph matching method.This phase introduces a P-queue KM algorithm that implements a more efficient optimization strategy.The allocation efficiency is improved by approximately 23.83%compared to the baseline method.Empirical results confirm the effectiveness of the proposed framework in detecting abnormal data nodes,enhancing allocation precision,and achieving efficient allocation.

Resource Allocation for Cognitive Network Slicing in PD-SCMA System Based on Two-Way Deep Reinforcement Learning

In this paper,we propose the Two-way Deep Reinforcement Learning(DRL)-Based resource allocation algorithm,which solves the problem of resource allocation in the cognitive downlink network based on the underlay mode.Secondary users(SUs)in the cognitive network are multiplexed by a new Power Domain Sparse Code Multiple Access(PD-SCMA)scheme,and the physical resources of the cognitive base station are virtualized into two types of slices:enhanced mobile broadband(eMBB)slice and ultrareliable low latency communication(URLLC)slice.We design the Double Deep Q Network(DDQN)network output the optimal codebook assignment scheme and simultaneously use the Deep Deterministic Policy Gradient(DDPG)network output the optimal power allocation scheme.The objective is to jointly optimize the spectral efficiency of the system and the Quality of Service(QoS)of SUs.Simulation results show that the proposed algorithm outperforms the CNDDQN algorithm and modified JEERA algorithm in terms of spectral efficiency and QoS satisfaction.Additionally,compared with the Power Domain Non-orthogonal Multiple Access(PD-NOMA)slices and the Sparse Code Multiple Access(SCMA)slices,the PD-SCMA slices can dramatically enhance spectral efficiency and increase the number of accessible users.

Optimization of resource allocation in FDD massive MIMO systems

The performance of massive MIMO systems relies heavily on the availability of Channel State Information at the Transmitter(CSIT).A large amount of work has been devoted to reducing the CSIT acquisition overhead at the pilot training and/or CsI feedback stage.In fact,the downlink communication generally includes three stages,i.e.,pilot training,CsI feedback,and data transmission.These three stages are mutually related and jointly determine the overall system performance.Unfortunately,there exist few studies on the reduction of csIT acquisition overhead from the global point of view.In this paper,we integrate the Minimum Mean Square Error(MMSE)channel estimation,Random Vector Quantization(RVQ)based limited feedback and Maximal Ratio Combining(MRC)precoding into a unified framework for investigating the resource allocation problem.In particular,we first approximate the covariance matrix of the quantization error with a simple expression and derive an analytical expression of the received Signal-to-Noise Ratio(SNR)based on the deterministic equivalence theory.Then the three performance metrics(the spectral efficiency,energy efficiency,and total energy consumption)oriented problems are formulated analytically.With practical system requirements,these three metrics can be collaboratively optimized.Finally,we propose an optimization solver to derive the optimal partition of channel coherence time.Experiment results verify the benefits of the proposed resource allocation schemes under three different scenarios and illustrate the tradeoff of resource allocation between three stages.

Learning-based user association and dynamic resource allocation in multi-connectivity enabled unmanned aerial vehicle networks

Unmanned Aerial Vehicles(UAvs)as aerial base stations to provide communication services for ground users is a flexible and cost-effective paradigm in B5G.Besides,dynamic resource allocation and multi-connectivity can be adopted to further harness the potentials of UAVs in improving communication capacity,in such situations such that the interference among users becomes a pivotal disincentive requiring effective solutions.To this end,we investigate the Joint UAV-User Association,Channel Allocation,and transmission Power Control(J-UACAPC)problem in a multi-connectivity-enabled UAV network with constrained backhaul links,where each UAV can determine the reusable channels and transmission power to serve the selected ground users.The goal was to mitigate co-channel interference while maximizing long-term system utility.The problem was modeled as a cooperative stochastic game with hybrid discrete-continuous action space.A Multi-Agent Hybrid Deep Reinforcement Learning(MAHDRL)algorithm was proposed to address this problem.Extensive simulation results demonstrated the effectiveness of the proposed algorithm and showed that it has a higher system utility than the baseline methods.

Joint Task Allocation and Resource Optimization for Blockchain Enabled Collaborative Edge Computing

Collaborative edge computing is a promising direction to handle the computation intensive tasks in B5G wireless networks.However,edge computing servers(ECSs)from different operators may not trust each other,and thus the incentives for collaboration cannot be guaranteed.In this paper,we propose a consortium blockchain enabled collaborative edge computing framework,where users can offload computing tasks to ECSs from different operators.To minimize the total delay of users,we formulate a joint task offloading and resource optimization problem,under the constraint of the computing capability of each ECS.We apply the Tammer decomposition method and heuristic optimization algorithms to obtain the optimal solution.Finally,we propose a reputation based node selection approach to facilitate the consensus process,and also consider a completion time based primary node selection to avoid monopolization of certain edge node and enhance the security of the blockchain.Simulation results validate the effectiveness of the proposed algorithm,and the total delay can be reduced by up to 40%compared with the non-cooperative case.

Adaptive Resource Allocation Algorithm for 5G Vehicular Cloud Communication

The current resource allocation in 5G vehicular networks for mobile cloud communication faces several challenges,such as low user utilization,unbalanced resource allocation,and extended adaptive allocation time.We propose an adaptive allocation algorithm for mobile cloud communication resources in 5G vehicular networks to address these issues.This study analyzes the components of the 5G vehicular network architecture to determine the performance of different components.It is ascertained that the communication modes in 5G vehicular networks for mobile cloud communication include in-band and out-of-band modes.Furthermore,this study analyzes the single-hop and multi-hop modes in mobile cloud communication and calculates the resource transmission rate and bandwidth in different communication modes.The study also determines the scenario of one-way and two-way vehicle lane cloud communication network connectivity,calculates the probability of vehicle network connectivity under different mobile cloud communication radii,and determines the amount of cloud communication resources required by vehicles in different lane scenarios.Based on the communication status of users in 5G vehicular networks,this study calculates the bandwidth and transmission rate of the allocated channels using Shannon’s formula.It determines the adaptive allocation of cloud communication resources,introduces an objective function to obtain the optimal solution after allocation,and completes the adaptive allocation process.The experimental results demonstrate that,with the application of the proposed method,the maximum utilization of user communication resources reaches approximately 99%.The balance coefficient curve approaches 1,and the allocation time remains under 2 s.This indicates that the proposed method has higher adaptive allocation efficiency.

A new quantum key distribution resource allocation and routing optimization scheme

Quantum key distribution(QKD)is a technology that can resist the threat of quantum computers to existing conventional cryptographic protocols.However,due to the stringent requirements of the quantum key generation environment,the generated quantum keys are considered valuable,and the slow key generation rate conflicts with the high-speed data transmission in traditional optical networks.In this paper,for the QKD network with a trusted relay,which is mainly based on point-to-point quantum keys and has complex changes in network resources,we aim to allocate resources reasonably for data packet distribution.Firstly,we formulate a linear programming constraint model for the key resource allocation(KRA)problem based on the time-slot scheduling.Secondly,we propose a new scheduling scheme based on the graded key security requirements(GKSR)and a new micro-log key storage algorithm for effective storage and management of key resources.Finally,we propose a key resource consumption(KRC)routing optimization algorithm to properly allocate time slots,routes,and key resources.Simulation results show that the proposed scheme significantly improves the key distribution success rate and key resource utilization rate,among others.

MADDPG-D2: An Intelligent Dynamic Task Allocation Algorithm Based on Multi-Agent Architecture Driven by Prior Knowledge

Aiming at the problems of low solution accuracy and high decision pressure when facing large-scale dynamic task allocation(DTA)and high-dimensional decision space with single agent,this paper combines the deep reinforce-ment learning(DRL)theory and an improved Multi-Agent Deep Deterministic Policy Gradient(MADDPG-D2)algorithm with a dual experience replay pool and a dual noise based on multi-agent architecture is proposed to improve the efficiency of DTA.The algorithm is based on the traditional Multi-Agent Deep Deterministic Policy Gradient(MADDPG)algorithm,and considers the introduction of a double noise mechanism to increase the action exploration space in the early stage of the algorithm,and the introduction of a double experience pool to improve the data utilization rate;at the same time,in order to accelerate the training speed and efficiency of the agents,and to solve the cold-start problem of the training,the a priori knowledge technology is applied to the training of the algorithm.Finally,the MADDPG-D2 algorithm is compared and analyzed based on the digital battlefield of ground and air confrontation.The experimental results show that the agents trained by the MADDPG-D2 algorithm have higher win rates and average rewards,can utilize the resources more reasonably,and better solve the problem of the traditional single agent algorithms facing the difficulty of solving the problem in the high-dimensional decision space.The MADDPG-D2 algorithm based on multi-agent architecture proposed in this paper has certain superiority and rationality in DTA.

A Comprehensive Survey on Joint Resource Allocation Strategies in Federated Edge Learning

Federated Edge Learning(FEL),an emerging distributed Machine Learning(ML)paradigm,enables model training in a distributed environment while ensuring user privacy by using physical separation for each user’s data.However,with the development of complex application scenarios such as the Internet of Things(IoT)and Smart Earth,the conventional resource allocation schemes can no longer effectively support these growing computational and communication demands.Therefore,joint resource optimization may be the key solution to the scaling problem.This paper simultaneously addresses the multifaceted challenges of computation and communication,with the growing multiple resource demands.We systematically review the joint allocation strategies for different resources(computation,data,communication,and network topology)in FEL,and summarize the advantages in improving system efficiency,reducing latency,enhancing resource utilization,and enhancing robustness.In addition,we present the potential ability of joint optimization to enhance privacy preservation by reducing communication requirements,indirectly.This work not only provides theoretical support for resource management in federated learning(FL)systems,but also provides ideas for potential optimal deployment in multiple real-world scenarios.By thoroughly discussing the current challenges and future research directions,it also provides some important insights into multi-resource optimization in complex application environments.

Joint Allocation of Computing and Connectivity Resources in Survivable Inter-Datacenter Elastic Optical Networks

Inter-datacenter elastic optical networks(EON)need to provide the service for the requests of cloud computing that require not only connectivity and computing resources but also network survivability.In this paper,to realize joint allocation of computing and connectivity resources in survivable inter-datacenter EONs,a survivable routing,modulation level,spectrum,and computing resource allocation algorithm(SRMLSCRA)algorithm and three datacenter selection strategies,i.e.Computing Resource First(CRF),Shortest Path First(SPF)and Random Destination(RD),are proposed for different scenarios.Unicast and manycast are applied to the communication of computing requests,and the routing strategies are calculated respectively.Simulation results show that SRMLCRA-CRF can serve the largest amount of protected computing tasks,and the requested calculation blocking probability is reduced by 29.2%,28.3%and 30.5%compared with SRMLSCRA-SPF,SRMLSCRA-RD and the benchmark EPS-RMSA algorithms respectively.Therefore,it is more applicable to the networks with huge calculations.Besides,SRMLSCRA-SPF consumes the least spectrum,thereby exhibiting its suitability for scenarios where the amount of calculation is small and communication resources are scarce.The results demonstrate that the proposed methods realize the joint allocation of computing and connectivity resources,and could provide efficient protection for services under single-link failure and occupy less spectrum.

Joint Optimization for on-Demand Deployment of UAVs and Spectrum Allocation in UAVs-Assisted Communication

To improve the efficiency and fairness of the spectrum allocation for ground communication assisted by unmanned aerial vehicles(UAVs),a joint optimization method for on-demand deployment and spectrum allocation of UAVs is proposed,which is modeled as a mixed-integer non-convex optimization problem(MINCOP).An algorithm to estimate the minimum number of required UAVs is firstly proposed based on the pre-estimation and simulated annealing.The MINCOP is then decomposed into three sub-problems based on the block coordinate descent method,including the spectrum allocation of UAVs,the association between UAVs and ground users,and the deployment of UAVs.Specifically,the optimal spectrum allocation is derived based on the interference mitigation and channel reuse.The association between UAVs and ground users is optimized based on local iterated optimization.A particle-based optimization algorithm is proposed to resolve the subproblem of the UAVs deployment.Simulation results show that the proposed method could effectively improve the minimum transmission rate of UAVs as well as user fairness of spectrum allocation.

Distributed Resource Allocation in Dispersed Computing Environment Based on UAV Track Inspection in Urban Rail Transit

With the rapid development of urban rail transit,the existing track detection has some problems such as low efficiency and insufficient detection coverage,so an intelligent and automatic track detectionmethod based onUAV is urgently needed to avoid major safety accidents.At the same time,the geographical distribution of IoT devices results in the inefficient use of the significant computing potential held by a large number of devices.As a result,the Dispersed Computing(DCOMP)architecture enables collaborative computing between devices in the Internet of Everything(IoE),promotes low-latency and efficient cross-wide applications,and meets users’growing needs for computing performance and service quality.This paper focuses on examining the resource allocation challenge within a dispersed computing environment that utilizes UAV inspection tracks.Furthermore,the system takes into account both resource constraints and computational constraints and transforms the optimization problem into an energy minimization problem with computational constraints.The Markov Decision Process(MDP)model is employed to capture the connection between the dispersed computing resource allocation strategy and the system environment.Subsequently,a method based on Double Deep Q-Network(DDQN)is introduced to derive the optimal policy.Simultaneously,an experience replay mechanism is implemented to tackle the issue of increasing dimensionality.The experimental simulations validate the efficacy of the method across various scenarios.

An Adaptive Hybrid Optimization Strategy for Resource Allocation in Network Function Virtualization

With the rapid development of Network Function Virtualization(NFV),the problem of low resource utilizationin traditional data centers is gradually being addressed.However,existing research does not optimize both localand global allocation of resources in data centers.Hence,we propose an adaptive hybrid optimization strategy thatcombines dynamic programming and neural networks to improve resource utilization and service quality in datacenters.Our approach encompasses a service function chain simulation generator,a parallel architecture servicesystem,a dynamic programming strategy formaximizing the utilization of local server resources,a neural networkfor predicting the global utilization rate of resources and a global resource optimization strategy for bottleneck andredundant resources.With the implementation of our local and global resource allocation strategies,the systemperformance is significantly optimized through simulation.

Resource Allocation for IRS Assistedmm Wave Wireless Powered Sensor Networks with User Cooperation

In this paper,we investigate IRS-aided user cooperation(UC)scheme in millimeter wave(mmWave)wirelesspowered sensor networks(WPSN),where two single-antenna users are wireless powered in the wireless energy transfer(WET)phase first and then cooperatively transmit information to a hybrid access point(AP)in the wireless information transmission(WIT)phase,following which the IRS is deployed to enhance the system performance of theWET andWIT.We maximized the weighted sum-rate problem by jointly optimizing the transmit time slots,power allocations,and the phase shifts of the IRS.Due to the non-convexity of the original problem,a semidefinite programming relaxation-based approach is proposed to convert the formulated problem to a convex optimization framework,which can obtain the optimal global solution.Simulation results demonstrate that the weighted sum throughput of the proposed UC scheme outperforms the non-UC scheme whether equipped with IRS or not.

Resource Allocation in Multi-User Cellular Networks:A Transformer-Based Deep Reinforcement Learning Approach

To meet the communication services with diverse requirements,dynamic resource allocation has shown increasing importance.In this paper,we consider the multi-slot and multi-user resource allocation(MSMU-RA)in a downlink cellular scenario with the aim of maximizing system spectral efficiency while guaranteeing user fairness.We first model the MSMURA problem as a dual-sequence decision-making process,and then solve it by a novel Transformerbased deep reinforcement learning(TDRL)approach.Specifically,the proposed TDRL approach can be achieved based on two aspects:1)To adapt to the dynamic wireless environment,the proximal policy optimization(PPO)algorithm is used to optimize the multi-slot RA strategy.2)To avoid co-channel interference,the Transformer-based PPO algorithm is presented to obtain the optimal multi-user RA scheme by exploring the mapping between user sequence and resource sequence.Experimental results show that:i)the proposed approach outperforms both the traditional and DRL methods in spectral efficiency and user fairness,ii)the proposed algorithm is superior to DRL approaches in terms of convergence speed and generalization performance.

Heterogeneous Task Allocation Model and Algorithm for Intelligent Connected Vehicles

With the development of vehicles towards intelligence and connectivity,vehicular data is diversifying and growing dramatically.A task allocation model and algorithm for heterogeneous Intelligent Connected Vehicle(ICV)applications are proposed for the dispersed computing network composed of heterogeneous task vehicles and Network Computing Points(NCPs).Considering the amount of task data and the idle resources of NCPs,a computing resource scheduling model for NCPs is established.Taking the heterogeneous task execution delay threshold as a constraint,the optimization problem is described as the problem of maximizing the utilization of computing resources by NCPs.The proposed problem is proven to be NP-hard by using the method of reduction to a 0-1 knapsack problem.A many-to-many matching algorithm based on resource preferences is proposed.The algorithm first establishes the mutual preference lists based on the adaptability of the task requirements and the resources provided by NCPs.This enables the filtering out of un-schedulable NCPs in the initial stage of matching,reducing the solution space dimension.To solve the matching problem between ICVs and NCPs,a new manyto-many matching algorithm is proposed to obtain a unique and stable optimal matching result.The simulation results demonstrate that the proposed scheme can improve the resource utilization of NCPs by an average of 9.6%compared to the reference scheme,and the total performance can be improved by up to 15.9%.

River width and depth as key factors of diurnal activity energy expenditure allocation for wintering Spot-billed Ducks in the Xin'an River Basin

Rivers are important habitats for wintering waterbirds.However,they are easily influenced by natural and human activities.An important approach for waterbirds to adapt to habitats is adjusting the activity time and energy expenditure allocation of diurnal behavior.The compensatory foraging hypothesis predicts that increased energy expenditure leads to longer foraging time,which in turn increases food intake and helps maintain a constant energy balance.However,it is unclear whether human-disturbed habitats result in increased energy expenditure related to safety or foraging.In this study,the scan sample method was used to observe the diurnal behavior of the wintering Spot-billed Duck(Anas poecilorhyncha) in two rivers in the Xin’an River Basin from October 2021 to March 2022.The allocation of time and energy expenditure for activity in both normal and disturbed environments was calculated.The results showed that foraging accounted for the highest percentage of time and energy expenditure.Additionally,foraging decreased in the disturbed environment than that in the normal environment.Resting behavior showed the opposite trend,while other behaviors were similar in both environments.The total diurnal energy expenditure of ducks in the disturbed environment was greater than that in the normal environment,with decreased foraging and resting time percentage and increased behaviors related to immediate safety(swimming and alert) and comfort.These results oppose the compensatory foraging hypothesis in favor of increased security.The optimal diurnal energy expenditure model included river width and water depth,which had a positive relationship;an increase in either of these two factors resulted in an increase in energy expenditure.This study provides a better understanding of energy allocation strategies underlying the superficial time allocation of wintering waterbirds according to environmental conditions.Exploring these changes can help understand the maximum fitness of wintering waterbirds in response to nature and human influences.