Role Dynamic Allocation of Human-Robot Cooperation Based on Reinforcement Learning in an Installation of Curtain Wall

A real-time adaptive roles allocation method based on reinforcement learning is proposed to improve humanrobot cooperation performance for a curtain wall installation task.This method breaks the traditional idea that the robot is regarded as the follower or only adjusts the leader and the follower in cooperation.In this paper,a self-learning method is proposed which can dynamically adapt and continuously adjust the initiative weight of the robot according to the change of the task.Firstly,the physical human-robot cooperation model,including the role factor is built.Then,a reinforcement learningmodel that can adjust the role factor in real time is established,and a reward and actionmodel is designed.The role factor can be adjusted continuously according to the comprehensive performance of the human-robot interaction force and the robot’s Jerk during the repeated installation.Finally,the roles adjustment rule established above continuously improves the comprehensive performance.Experiments of the dynamic roles allocation and the effect of the performance weighting coefficient on the result have been verified.The results show that the proposed method can realize the role adaptation and achieve the dual optimization goal of reducing the sum of the cooperator force and the robot’s Jerk.

Computing Resource Allocation for Blockchain-Based Mobile Edge Computing

Users and edge servers are not fullymutually trusted inmobile edge computing(MEC),and hence blockchain can be introduced to provide trustableMEC.In blockchain-basedMEC,each edge server functions as a node in bothMEC and blockchain,processing users’tasks and then uploading the task related information to the blockchain.That is,each edge server runs both users’offloaded tasks and blockchain tasks simultaneously.Note that there is a trade-off between the resource allocation for MEC and blockchain tasks.Therefore,the allocation of the resources of edge servers to the blockchain and theMEC is crucial for the processing delay of blockchain-based MEC.Most of the existing research tackles the problem of resource allocation in either blockchain or MEC,which leads to unfavorable performance of the blockchain-based MEC system.In this paper,we study how to allocate the computing resources of edge servers to the MEC and blockchain tasks with the aimtominimize the total systemprocessing delay.For the problem,we propose a computing resource Allocation algorithmfor Blockchain-based MEC(ABM)which utilizes the Slater’s condition,Karush-Kuhn-Tucker(KKT)conditions,partial derivatives of the Lagrangian function and subgradient projection method to obtain the solution.Simulation results show that ABM converges and effectively reduces the processing delay of blockchain-based MEC.

Mobile Crowdsourcing Task Allocation Based on Dynamic Self-Attention GANs

Crowdsourcing technology is widely recognized for its effectiveness in task scheduling and resource allocation.While traditional methods for task allocation can help reduce costs and improve efficiency,they may encounter challenges when dealing with abnormal data flow nodes,leading to decreased allocation accuracy and efficiency.To address these issues,this study proposes a novel two-part invalid detection task allocation framework.In the first step,an anomaly detection model is developed using a dynamic self-attentive GAN to identify anomalous data.Compared to the baseline method,the model achieves an approximately 4%increase in the F1 value on the public dataset.In the second step of the framework,task allocation modeling is performed using a twopart graph matching method.This phase introduces a P-queue KM algorithm that implements a more efficient optimization strategy.The allocation efficiency is improved by approximately 23.83%compared to the baseline method.Empirical results confirm the effectiveness of the proposed framework in detecting abnormal data nodes,enhancing allocation precision,and achieving efficient allocation.

Joint Allocation of Computing and Connectivity Resources in Survivable Inter-Datacenter Elastic Optical Networks

Inter-datacenter elastic optical networks(EON)need to provide the service for the requests of cloud computing that require not only connectivity and computing resources but also network survivability.In this paper,to realize joint allocation of computing and connectivity resources in survivable inter-datacenter EONs,a survivable routing,modulation level,spectrum,and computing resource allocation algorithm(SRMLSCRA)algorithm and three datacenter selection strategies,i.e.Computing Resource First(CRF),Shortest Path First(SPF)and Random Destination(RD),are proposed for different scenarios.Unicast and manycast are applied to the communication of computing requests,and the routing strategies are calculated respectively.Simulation results show that SRMLCRA-CRF can serve the largest amount of protected computing tasks,and the requested calculation blocking probability is reduced by 29.2%,28.3%and 30.5%compared with SRMLSCRA-SPF,SRMLSCRA-RD and the benchmark EPS-RMSA algorithms respectively.Therefore,it is more applicable to the networks with huge calculations.Besides,SRMLSCRA-SPF consumes the least spectrum,thereby exhibiting its suitability for scenarios where the amount of calculation is small and communication resources are scarce.The results demonstrate that the proposed methods realize the joint allocation of computing and connectivity resources,and could provide efficient protection for services under single-link failure and occupy less spectrum.

Joint Optimization for on-Demand Deployment of UAVs and Spectrum Allocation in UAVs-Assisted Communication

To improve the efficiency and fairness of the spectrum allocation for ground communication assisted by unmanned aerial vehicles(UAVs),a joint optimization method for on-demand deployment and spectrum allocation of UAVs is proposed,which is modeled as a mixed-integer non-convex optimization problem(MINCOP).An algorithm to estimate the minimum number of required UAVs is firstly proposed based on the pre-estimation and simulated annealing.The MINCOP is then decomposed into three sub-problems based on the block coordinate descent method,including the spectrum allocation of UAVs,the association between UAVs and ground users,and the deployment of UAVs.Specifically,the optimal spectrum allocation is derived based on the interference mitigation and channel reuse.The association between UAVs and ground users is optimized based on local iterated optimization.A particle-based optimization algorithm is proposed to resolve the subproblem of the UAVs deployment.Simulation results show that the proposed method could effectively improve the minimum transmission rate of UAVs as well as user fairness of spectrum allocation.

Resource Allocation in Multi-User Cellular Networks:A Transformer-Based Deep Reinforcement Learning Approach

To meet the communication services with diverse requirements,dynamic resource allocation has shown increasing importance.In this paper,we consider the multi-slot and multi-user resource allocation(MSMU-RA)in a downlink cellular scenario with the aim of maximizing system spectral efficiency while guaranteeing user fairness.We first model the MSMURA problem as a dual-sequence decision-making process,and then solve it by a novel Transformerbased deep reinforcement learning(TDRL)approach.Specifically,the proposed TDRL approach can be achieved based on two aspects:1)To adapt to the dynamic wireless environment,the proximal policy optimization(PPO)algorithm is used to optimize the multi-slot RA strategy.2)To avoid co-channel interference,the Transformer-based PPO algorithm is presented to obtain the optimal multi-user RA scheme by exploring the mapping between user sequence and resource sequence.Experimental results show that:i)the proposed approach outperforms both the traditional and DRL methods in spectral efficiency and user fairness,ii)the proposed algorithm is superior to DRL approaches in terms of convergence speed and generalization performance.

Learning-based user association and dynamic resource allocation in multi-connectivity enabled unmanned aerial vehicle networks

Unmanned Aerial Vehicles(UAvs)as aerial base stations to provide communication services for ground users is a flexible and cost-effective paradigm in B5G.Besides,dynamic resource allocation and multi-connectivity can be adopted to further harness the potentials of UAVs in improving communication capacity,in such situations such that the interference among users becomes a pivotal disincentive requiring effective solutions.To this end,we investigate the Joint UAV-User Association,Channel Allocation,and transmission Power Control(J-UACAPC)problem in a multi-connectivity-enabled UAV network with constrained backhaul links,where each UAV can determine the reusable channels and transmission power to serve the selected ground users.The goal was to mitigate co-channel interference while maximizing long-term system utility.The problem was modeled as a cooperative stochastic game with hybrid discrete-continuous action space.A Multi-Agent Hybrid Deep Reinforcement Learning(MAHDRL)algorithm was proposed to address this problem.Extensive simulation results demonstrated the effectiveness of the proposed algorithm and showed that it has a higher system utility than the baseline methods.

Starlet:Network defense resource allocation with multi-armed bandits for cloud-edge crowd sensing in IoT

The cloud platform has limited defense resources to fully protect the edge servers used to process crowd sensing data in Internet of Things.To guarantee the network's overall security,we present a network defense resource allocation with multi-armed bandits to maximize the network's overall benefit.Firstly,we propose the method for dynamic setting of node defense resource thresholds to obtain the defender(attacker)benefit function of edge servers(nodes)and distribution.Secondly,we design a defense resource sharing mechanism for neighboring nodes to obtain the defense capability of nodes.Subsequently,we use the decomposability and Lipschitz conti-nuity of the defender's total expected utility to reduce the difference between the utility's discrete and continuous arms and analyze the difference theoretically.Finally,experimental results show that the method maximizes the defender's total expected utility and reduces the difference between the discrete and continuous arms of the utility.

MADDPG-D2: An Intelligent Dynamic Task Allocation Algorithm Based on Multi-Agent Architecture Driven by Prior Knowledge

Aiming at the problems of low solution accuracy and high decision pressure when facing large-scale dynamic task allocation(DTA)and high-dimensional decision space with single agent,this paper combines the deep reinforce-ment learning(DRL)theory and an improved Multi-Agent Deep Deterministic Policy Gradient(MADDPG-D2)algorithm with a dual experience replay pool and a dual noise based on multi-agent architecture is proposed to improve the efficiency of DTA.The algorithm is based on the traditional Multi-Agent Deep Deterministic Policy Gradient(MADDPG)algorithm,and considers the introduction of a double noise mechanism to increase the action exploration space in the early stage of the algorithm,and the introduction of a double experience pool to improve the data utilization rate;at the same time,in order to accelerate the training speed and efficiency of the agents,and to solve the cold-start problem of the training,the a priori knowledge technology is applied to the training of the algorithm.Finally,the MADDPG-D2 algorithm is compared and analyzed based on the digital battlefield of ground and air confrontation.The experimental results show that the agents trained by the MADDPG-D2 algorithm have higher win rates and average rewards,can utilize the resources more reasonably,and better solve the problem of the traditional single agent algorithms facing the difficulty of solving the problem in the high-dimensional decision space.The MADDPG-D2 algorithm based on multi-agent architecture proposed in this paper has certain superiority and rationality in DTA.

Resource Allocation for Cognitive Network Slicing in PD-SCMA System Based on Two-Way Deep Reinforcement Learning

In this paper,we propose the Two-way Deep Reinforcement Learning(DRL)-Based resource allocation algorithm,which solves the problem of resource allocation in the cognitive downlink network based on the underlay mode.Secondary users(SUs)in the cognitive network are multiplexed by a new Power Domain Sparse Code Multiple Access(PD-SCMA)scheme,and the physical resources of the cognitive base station are virtualized into two types of slices:enhanced mobile broadband(eMBB)slice and ultrareliable low latency communication(URLLC)slice.We design the Double Deep Q Network(DDQN)network output the optimal codebook assignment scheme and simultaneously use the Deep Deterministic Policy Gradient(DDPG)network output the optimal power allocation scheme.The objective is to jointly optimize the spectral efficiency of the system and the Quality of Service(QoS)of SUs.Simulation results show that the proposed algorithm outperforms the CNDDQN algorithm and modified JEERA algorithm in terms of spectral efficiency and QoS satisfaction.Additionally,compared with the Power Domain Non-orthogonal Multiple Access(PD-NOMA)slices and the Sparse Code Multiple Access(SCMA)slices,the PD-SCMA slices can dramatically enhance spectral efficiency and increase the number of accessible users.

Online Learning-Based Offloading Decision and Resource Allocation in Mobile Edge Computing-Enabled Satellite-Terrestrial Networks

Mobile edge computing(MEC)-enabled satellite-terrestrial networks(STNs)can provide Internet of Things(IoT)devices with global computing services.Sometimes,the network state information is uncertain or unknown.To deal with this situation,we investigate online learning-based offloading decision and resource allocation in MEC-enabled STNs in this paper.The problem of minimizing the average sum task completion delay of all IoT devices over all time periods is formulated.We decompose this optimization problem into a task offloading decision problem and a computing resource allocation problem.A joint optimization scheme of offloading decision and resource allocation is then proposed,which consists of a task offloading decision algorithm based on the devices cooperation aided upper confidence bound(UCB)algorithm and a computing resource allocation algorithm based on the Lagrange multiplier method.Simulation results validate that the proposed scheme performs better than other baseline schemes.

Three-Level Optimal Scheduling and Power Allocation Strategy for Power System ContainingWind-Storage Combined Unit

To mitigate the impact of wind power volatility on power system scheduling,this paper adopts the wind-storage combined unit to improve the dispatchability of wind energy.And a three-level optimal scheduling and power allocation strategy is proposed for the system containing the wind-storage combined unit.The strategy takes smoothing power output as themain objectives.The first level is the wind-storage joint scheduling,and the second and third levels carry out the unit combination optimization of thermal power and the power allocation of wind power cluster(WPC),respectively,according to the scheduling power of WPC and ESS obtained from the first level.This can ensure the stability,economy and environmental friendliness of the whole power system.Based on the roles of peak shaving-valley filling and fluctuation smoothing of the energy storage system(ESS),this paper decides the charging and discharging intervals of ESS,so that the energy storage and wind power output can be further coordinated.Considering the prediction error and the output uncertainty of wind power,the planned scheduling output of wind farms(WFs)is first optimized on a long timescale,and then the rolling correction optimization of the scheduling output of WFs is carried out on a short timescale.Finally,the effectiveness of the proposed optimal scheduling and power allocation strategy is verified through case analysis.

River width and depth as key factors of diurnal activity energy expenditure allocation for wintering Spot-billed Ducks in the Xin'an River Basin

Rivers are important habitats for wintering waterbirds.However,they are easily influenced by natural and human activities.An important approach for waterbirds to adapt to habitats is adjusting the activity time and energy expenditure allocation of diurnal behavior.The compensatory foraging hypothesis predicts that increased energy expenditure leads to longer foraging time,which in turn increases food intake and helps maintain a constant energy balance.However,it is unclear whether human-disturbed habitats result in increased energy expenditure related to safety or foraging.In this study,the scan sample method was used to observe the diurnal behavior of the wintering Spot-billed Duck(Anas poecilorhyncha) in two rivers in the Xin’an River Basin from October 2021 to March 2022.The allocation of time and energy expenditure for activity in both normal and disturbed environments was calculated.The results showed that foraging accounted for the highest percentage of time and energy expenditure.Additionally,foraging decreased in the disturbed environment than that in the normal environment.Resting behavior showed the opposite trend,while other behaviors were similar in both environments.The total diurnal energy expenditure of ducks in the disturbed environment was greater than that in the normal environment,with decreased foraging and resting time percentage and increased behaviors related to immediate safety(swimming and alert) and comfort.These results oppose the compensatory foraging hypothesis in favor of increased security.The optimal diurnal energy expenditure model included river width and water depth,which had a positive relationship;an increase in either of these two factors resulted in an increase in energy expenditure.This study provides a better understanding of energy allocation strategies underlying the superficial time allocation of wintering waterbirds according to environmental conditions.Exploring these changes can help understand the maximum fitness of wintering waterbirds in response to nature and human influences.

Optimal allocation method of electric/air braking force of high-speed train considering axle load transfer

Reasonable distribution of braking force is a factor for a smooth,safe,and comfortable braking of trains.A dynamic optimal allocation strategy of electric-air braking force is proposed in this paper to solve the problem of the lack of consideration of adhesion difference of train wheelsets in the existing high-speed train electric-air braking force optimal allocation strategies.In this method,the braking strategy gives priority to the use of electric braking force.The force model of a single train in the braking process is analyzed to calculate the change of adhesion between the wheel and rail of each wheelset after axle load transfer,and then the adhesion of the train is estimated in real time.Next,with the goal of maximizing the total adhesion utilization ratio of trailer/motor vehicles,a linear programming distribution function is constructed.The proportional coefficient of adhesion utilization ratio of each train and the application upper limit of braking force in the function is updated according to the change time point of wheelset adhesion.Finally,the braking force is dynamically allocated.The simulation results of Matlab/Simulink show that the proposed algorithm not only uses the different adhesion limits of each trailer to reduce the total amount of braking force undertaken by the motor vehicle,but also considers the adhesion difference of each wheelset.The strategy can effectively reduce the risk and time of motor vehicles during the braking process and improve the stability of the train braking.

Multi-Agent Deep Deterministic Policy Gradien-Based Task Offloading Resource Allocation Joint Offloading

With the advancement of technology and the continuous innovation of applications, low-latency applications such as drones, online games and virtual reality are gradually becoming popular demands in modern society. However, these applications pose a great challenge to the traditional centralized mobile cloud computing paradigm, and it is obvious that the traditional cloud computing model is already struggling to meet such demands. To address the shortcomings of cloud computing, mobile edge computing has emerged. Mobile edge computing provides users with computing and storage resources by offloading computing tasks to servers at the edge of the network. However, most existing work only considers single-objective performance optimization in terms of latency or energy consumption, but not balanced optimization in terms of latency and energy consumption. To reduce task latency and device energy consumption, the problem of joint optimization of computation offloading and resource allocation in multi-cell, multi-user, multi-server MEC environments is investigated. In this paper, a dynamic computation offloading algorithm based on Multi-Agent Deep Deterministic Policy Gradient (MADDPG) is proposed to obtain the optimal policy. The experimental results show that the algorithm proposed in this paper reduces the delay by 5 ms compared to PPO, 1.5 ms compared to DDPG and 10.7 ms compared to DQN, and reduces the energy consumption by 300 compared to PPO, 760 compared to DDPG and 380 compared to DQN. This fully proves that the algorithm proposed in this paper has excellent performance.

A Deep Reinforcement Learning-Based Technique for Optimal Power Allocation in Multiple Access Communications

Formany years,researchers have explored power allocation(PA)algorithms driven bymodels in wireless networks where multiple-user communications with interference are present.Nowadays,data-driven machine learning methods have become quite popular in analyzing wireless communication systems,which among them deep reinforcement learning(DRL)has a significant role in solving optimization issues under certain constraints.To this purpose,in this paper,we investigate the PA problem in a k-user multiple access channels(MAC),where k transmitters(e.g.,mobile users)aim to send an independent message to a common receiver(e.g.,base station)through wireless channels.To this end,we first train the deep Q network(DQN)with a deep Q learning(DQL)algorithm over the simulation environment,utilizing offline learning.Then,the DQN will be used with the real data in the online training method for the PA issue by maximizing the sumrate subjected to the source power.Finally,the simulation results indicate that our proposedDQNmethod provides better performance in terms of the sumrate compared with the available DQL training approaches such as fractional programming(FP)and weighted minimum mean squared error(WMMSE).Additionally,by considering different user densities,we show that our proposed DQN outperforms benchmark algorithms,thereby,a good generalization ability is verified over wireless multi-user communication systems.

Bayesian network-based resilience assessment of interdependent infrastructure systems under optimal resource allocation strategies

Critical infrastructure systems(CISs)play a key role in the socio-economic activity of a society,but are exposed to an array of disruptive events that can greatly impact their function and performance.Therefore,understanding the underlying behaviors of CISs and their response to perturbations is needed to better prepare for,and mitigate the impact of,future disruptions.Resilience is one characteristic of CISs that influences the extent and severity of the impact induced by extreme events.Resilience is often dissected into four dimensions:robustness,redundancy,resourcefulness,and rapidity,known as the“4Rs”.This study proposes a framework to assess the resilience of an infrastructure network in terms of these four dimensions under optimal resource allocation strategies and incorporates interdependencies between different CISs,with resilience considered as a stochastic variable.The proposed framework combines an agent-based infrastructure interdependency model,advanced optimization algorithms,Bayesian network techniques,and Monte Carlo simulation to assess the resilience of an infrastructure network.The applicability and flexibility of the proposed framework is demonstrated with a case study using a network of CISs in Austin,Texas,where the resilience of the network is assessed and a“what-if”analysis is performed.

Effects of Sand Burial on the Survival, Growth, and Biomass Allocation in Semi-shrub Hedysarum laeve Seedlings

Hedysarum laeve Maxim. (Leguminosae) is one of the major species used frequently in revegetation of dune_field in the sandlands of the northern part of China by means of aerial sowing. Seedlings of the species after emergence above the sand surface may be buried in sand to various depths during its establishment in late spring and early summer. A study was made to examine the effects of sand burial at different levels of 0 (control), 33%, 67%, 100% and 133% of their shoot height, on the survivorship, growth, and biomass allocation pattern of H. laeve seedlings (one and two weeks old after emergence). When burial depth was up to 100% of their shoot height, about 70% seedlings died; and the burial at depth of 133% of their shoot height led to death of all seedlings. When seedlings was buried at depth of 33% and 67% of their shoot height, respectively, after six_week growth, their biomass of whole plant, blade, and root and relative growth rate were higher than the unburied counterparts. The seedlings in both 33% and 67% sand burial treatments did not significantly change their biomass allocation pattern comparing with the unburied ones. Furthermore, the number of leaves and shoot height of the seedlings in both 33% and 67% sand burial treatments were not significantly different from those of unburied individuals, respectively. The newly born leaves of the surviving seedlings, in 33%, 67%, and 100% burial treatments, during the period of experiment, were significantly more than those in control.

Resource allocation method for device-to-device communications in cellular networks

Based on the conflict graph model which is formulated as a binary integer optimization problem, a resource allocation method to support device-to-device （D2D） communications in ceUular networks is proposed. First, a frequency resource assignment algorithm is presented which assigns each D2D link one frequency resource block. For this algorithm, frequency resource blocks are assigned so that the frequency resource spatial reuse opportunities in the cellular networks can be fully exploited. Then a slot scheduling algorithm is presented which schedules time slots among D2D links assigned the same frequency resource block. For this algorithm, time slot resources are scheduled so that the proportional fairness among D2D links which are assigned the same frequency resource block can be achieved. The performance of the proposed method is evaluated via computer simulations. The simulation results show that the proposed method can well support D2D communications in cellular networks.

Resource allocation based on fairness and QoS provisioning for OFDMA-WLAN system

To satisfy different service requirements of multiple users in the orthogo nal frequency division multiple access wireless local area network OFDMA-WLAN system downlink transmission a resource allocation algorithm based on fairness and quality of service QoS provisioning is proposed. Different QoS requirements are converted into different rate requirements to calculate the QoSs atisfaction level.The optimization object is revised as a fairness-driven resource optimization function to provide fairness. The complex resource allocation problem is divided into channel allocation and power assignment sub-problems. The sub-problems are solved by the bipartite graph matching and water-filling based method.Compared with other algorithms the proposed algorithm sacrifices less data rate for higher fairnes and QoS satisfaction.The sim ulation results show that the proposed algorithm is capableo fp rovi ding QoS and fairness and performs better in a tradeoff among QoS fairness and data rate.