Role Dynamic Allocation of Human-Robot Cooperation Based on Reinforcement Learning in an Installation of Curtain Wall

A real-time adaptive roles allocation method based on reinforcement learning is proposed to improve humanrobot cooperation performance for a curtain wall installation task.This method breaks the traditional idea that the robot is regarded as the follower or only adjusts the leader and the follower in cooperation.In this paper,a self-learning method is proposed which can dynamically adapt and continuously adjust the initiative weight of the robot according to the change of the task.Firstly,the physical human-robot cooperation model,including the role factor is built.Then,a reinforcement learningmodel that can adjust the role factor in real time is established,and a reward and actionmodel is designed.The role factor can be adjusted continuously according to the comprehensive performance of the human-robot interaction force and the robot’s Jerk during the repeated installation.Finally,the roles adjustment rule established above continuously improves the comprehensive performance.Experiments of the dynamic roles allocation and the effect of the performance weighting coefficient on the result have been verified.The results show that the proposed method can realize the role adaptation and achieve the dual optimization goal of reducing the sum of the cooperator force and the robot’s Jerk.

Computing Resource Allocation for Blockchain-Based Mobile Edge Computing

Users and edge servers are not fullymutually trusted inmobile edge computing(MEC),and hence blockchain can be introduced to provide trustableMEC.In blockchain-basedMEC,each edge server functions as a node in bothMEC and blockchain,processing users’tasks and then uploading the task related information to the blockchain.That is,each edge server runs both users’offloaded tasks and blockchain tasks simultaneously.Note that there is a trade-off between the resource allocation for MEC and blockchain tasks.Therefore,the allocation of the resources of edge servers to the blockchain and theMEC is crucial for the processing delay of blockchain-based MEC.Most of the existing research tackles the problem of resource allocation in either blockchain or MEC,which leads to unfavorable performance of the blockchain-based MEC system.In this paper,we study how to allocate the computing resources of edge servers to the MEC and blockchain tasks with the aimtominimize the total systemprocessing delay.For the problem,we propose a computing resource Allocation algorithmfor Blockchain-based MEC(ABM)which utilizes the Slater’s condition,Karush-Kuhn-Tucker(KKT)conditions,partial derivatives of the Lagrangian function and subgradient projection method to obtain the solution.Simulation results show that ABM converges and effectively reduces the processing delay of blockchain-based MEC.

Resource Allocation in Multi-User Cellular Networks:A Transformer-Based Deep Reinforcement Learning Approach

To meet the communication services with diverse requirements,dynamic resource allocation has shown increasing importance.In this paper,we consider the multi-slot and multi-user resource allocation(MSMU-RA)in a downlink cellular scenario with the aim of maximizing system spectral efficiency while guaranteeing user fairness.We first model the MSMURA problem as a dual-sequence decision-making process,and then solve it by a novel Transformerbased deep reinforcement learning(TDRL)approach.Specifically,the proposed TDRL approach can be achieved based on two aspects:1)To adapt to the dynamic wireless environment,the proximal policy optimization(PPO)algorithm is used to optimize the multi-slot RA strategy.2)To avoid co-channel interference,the Transformer-based PPO algorithm is presented to obtain the optimal multi-user RA scheme by exploring the mapping between user sequence and resource sequence.Experimental results show that:i)the proposed approach outperforms both the traditional and DRL methods in spectral efficiency and user fairness,ii)the proposed algorithm is superior to DRL approaches in terms of convergence speed and generalization performance.

Online Learning-Based Offloading Decision and Resource Allocation in Mobile Edge Computing-Enabled Satellite-Terrestrial Networks

Mobile edge computing(MEC)-enabled satellite-terrestrial networks(STNs)can provide Internet of Things(IoT)devices with global computing services.Sometimes,the network state information is uncertain or unknown.To deal with this situation,we investigate online learning-based offloading decision and resource allocation in MEC-enabled STNs in this paper.The problem of minimizing the average sum task completion delay of all IoT devices over all time periods is formulated.We decompose this optimization problem into a task offloading decision problem and a computing resource allocation problem.A joint optimization scheme of offloading decision and resource allocation is then proposed,which consists of a task offloading decision algorithm based on the devices cooperation aided upper confidence bound(UCB)algorithm and a computing resource allocation algorithm based on the Lagrange multiplier method.Simulation results validate that the proposed scheme performs better than other baseline schemes.

Resource Allocation for Cognitive Network Slicing in PD-SCMA System Based on Two-Way Deep Reinforcement Learning

In this paper,we propose the Two-way Deep Reinforcement Learning(DRL)-Based resource allocation algorithm,which solves the problem of resource allocation in the cognitive downlink network based on the underlay mode.Secondary users(SUs)in the cognitive network are multiplexed by a new Power Domain Sparse Code Multiple Access(PD-SCMA)scheme,and the physical resources of the cognitive base station are virtualized into two types of slices:enhanced mobile broadband(eMBB)slice and ultrareliable low latency communication(URLLC)slice.We design the Double Deep Q Network(DDQN)network output the optimal codebook assignment scheme and simultaneously use the Deep Deterministic Policy Gradient(DDPG)network output the optimal power allocation scheme.The objective is to jointly optimize the spectral efficiency of the system and the Quality of Service(QoS)of SUs.Simulation results show that the proposed algorithm outperforms the CNDDQN algorithm and modified JEERA algorithm in terms of spectral efficiency and QoS satisfaction.Additionally,compared with the Power Domain Non-orthogonal Multiple Access(PD-NOMA)slices and the Sparse Code Multiple Access(SCMA)slices,the PD-SCMA slices can dramatically enhance spectral efficiency and increase the number of accessible users.

River width and depth as key factors of diurnal activity energy expenditure allocation for wintering Spot-billed Ducks in the Xin'an River Basin

Rivers are important habitats for wintering waterbirds.However,they are easily influenced by natural and human activities.An important approach for waterbirds to adapt to habitats is adjusting the activity time and energy expenditure allocation of diurnal behavior.The compensatory foraging hypothesis predicts that increased energy expenditure leads to longer foraging time,which in turn increases food intake and helps maintain a constant energy balance.However,it is unclear whether human-disturbed habitats result in increased energy expenditure related to safety or foraging.In this study,the scan sample method was used to observe the diurnal behavior of the wintering Spot-billed Duck(Anas poecilorhyncha) in two rivers in the Xin’an River Basin from October 2021 to March 2022.The allocation of time and energy expenditure for activity in both normal and disturbed environments was calculated.The results showed that foraging accounted for the highest percentage of time and energy expenditure.Additionally,foraging decreased in the disturbed environment than that in the normal environment.Resting behavior showed the opposite trend,while other behaviors were similar in both environments.The total diurnal energy expenditure of ducks in the disturbed environment was greater than that in the normal environment,with decreased foraging and resting time percentage and increased behaviors related to immediate safety(swimming and alert) and comfort.These results oppose the compensatory foraging hypothesis in favor of increased security.The optimal diurnal energy expenditure model included river width and water depth,which had a positive relationship;an increase in either of these two factors resulted in an increase in energy expenditure.This study provides a better understanding of energy allocation strategies underlying the superficial time allocation of wintering waterbirds according to environmental conditions.Exploring these changes can help understand the maximum fitness of wintering waterbirds in response to nature and human influences.

A Deep Reinforcement Learning-Based Technique for Optimal Power Allocation in Multiple Access Communications

Formany years,researchers have explored power allocation(PA)algorithms driven bymodels in wireless networks where multiple-user communications with interference are present.Nowadays,data-driven machine learning methods have become quite popular in analyzing wireless communication systems,which among them deep reinforcement learning(DRL)has a significant role in solving optimization issues under certain constraints.To this purpose,in this paper,we investigate the PA problem in a k-user multiple access channels(MAC),where k transmitters(e.g.,mobile users)aim to send an independent message to a common receiver(e.g.,base station)through wireless channels.To this end,we first train the deep Q network(DQN)with a deep Q learning(DQL)algorithm over the simulation environment,utilizing offline learning.Then,the DQN will be used with the real data in the online training method for the PA issue by maximizing the sumrate subjected to the source power.Finally,the simulation results indicate that our proposedDQNmethod provides better performance in terms of the sumrate compared with the available DQL training approaches such as fractional programming(FP)and weighted minimum mean squared error(WMMSE).Additionally,by considering different user densities,we show that our proposed DQN outperforms benchmark algorithms,thereby,a good generalization ability is verified over wireless multi-user communication systems.

An Adaptive Hybrid Optimization Strategy for Resource Allocation in Network Function Virtualization

With the rapid development of Network Function Virtualization(NFV),the problem of low resource utilizationin traditional data centers is gradually being addressed.However,existing research does not optimize both localand global allocation of resources in data centers.Hence,we propose an adaptive hybrid optimization strategy thatcombines dynamic programming and neural networks to improve resource utilization and service quality in datacenters.Our approach encompasses a service function chain simulation generator,a parallel architecture servicesystem,a dynamic programming strategy formaximizing the utilization of local server resources,a neural networkfor predicting the global utilization rate of resources and a global resource optimization strategy for bottleneck andredundant resources.With the implementation of our local and global resource allocation strategies,the systemperformance is significantly optimized through simulation.

Joint Task Allocation and Resource Optimization for Blockchain Enabled Collaborative Edge Computing

Collaborative edge computing is a promising direction to handle the computation intensive tasks in B5G wireless networks.However,edge computing servers(ECSs)from different operators may not trust each other,and thus the incentives for collaboration cannot be guaranteed.In this paper,we propose a consortium blockchain enabled collaborative edge computing framework,where users can offload computing tasks to ECSs from different operators.To minimize the total delay of users,we formulate a joint task offloading and resource optimization problem,under the constraint of the computing capability of each ECS.We apply the Tammer decomposition method and heuristic optimization algorithms to obtain the optimal solution.Finally,we propose a reputation based node selection approach to facilitate the consensus process,and also consider a completion time based primary node selection to avoid monopolization of certain edge node and enhance the security of the blockchain.Simulation results validate the effectiveness of the proposed algorithm,and the total delay can be reduced by up to 40%compared with the non-cooperative case.

Power Allocation for SE Maximization in Uplink Massive MIMO System Under Minimum Rate Constraint

In this paper,we optimize the spectrum efficiency(SE)of uplink massive multiple-input multiple-output(MIMO)system with imperfect channel state information(CSI)over Rayleigh fading channel.The SE optimization problem is formulated under the constraints of maximum power and minimum rate of each user.Then,we develop a near-optimal power allocation(PA)scheme by using the successive convex approximation(SCA)method,Lagrange multiplier method,and block coordinate descent(BCD)method,and it can obtain almost the same SE as the benchmark scheme with lower complexity.Since this scheme needs three-layer iteration,a suboptimal PA scheme is developed to further reduce the complexity,where the characteristic of massive MIMO(i.e.,numerous receive antennas)is utilized for convex reformulation,and the rate constraint is converted to linear constraints.This suboptimal scheme only needs single-layer iteration,thus has lower complexity than the near-optimal scheme.Finally,we joint design the pilot power and data power to further improve the performance,and propose an two-stage algorithm to obtain joint PA.Simulation results verify the effectiveness of the proposed schemes,and superior SE performance is achieved.

A new quantum key distribution resource allocation and routing optimization scheme

Quantum key distribution(QKD)is a technology that can resist the threat of quantum computers to existing conventional cryptographic protocols.However,due to the stringent requirements of the quantum key generation environment,the generated quantum keys are considered valuable,and the slow key generation rate conflicts with the high-speed data transmission in traditional optical networks.In this paper,for the QKD network with a trusted relay,which is mainly based on point-to-point quantum keys and has complex changes in network resources,we aim to allocate resources reasonably for data packet distribution.Firstly,we formulate a linear programming constraint model for the key resource allocation(KRA)problem based on the time-slot scheduling.Secondly,we propose a new scheduling scheme based on the graded key security requirements(GKSR)and a new micro-log key storage algorithm for effective storage and management of key resources.Finally,we propose a key resource consumption(KRC)routing optimization algorithm to properly allocate time slots,routes,and key resources.Simulation results show that the proposed scheme significantly improves the key distribution success rate and key resource utilization rate,among others.

Resource Allocation for IRS Assistedmm Wave Wireless Powered Sensor Networks with User Cooperation

In this paper,we investigate IRS-aided user cooperation(UC)scheme in millimeter wave(mmWave)wirelesspowered sensor networks(WPSN),where two single-antenna users are wireless powered in the wireless energy transfer(WET)phase first and then cooperatively transmit information to a hybrid access point(AP)in the wireless information transmission(WIT)phase,following which the IRS is deployed to enhance the system performance of theWET andWIT.We maximized the weighted sum-rate problem by jointly optimizing the transmit time slots,power allocations,and the phase shifts of the IRS.Due to the non-convexity of the original problem,a semidefinite programming relaxation-based approach is proposed to convert the formulated problem to a convex optimization framework,which can obtain the optimal global solution.Simulation results demonstrate that the weighted sum throughput of the proposed UC scheme outperforms the non-UC scheme whether equipped with IRS or not.

Stackelberg Game-Based Resource Allocation with Blockchain for Cold-Chain Logistics System

Cold-chain logistics system(CCLS)plays the role of collecting and managing the logistics data of frozen food.However,there always exist problems of information loss,data tampering,and privacy leakage in traditional centralized systems,which influence frozen food security and people’s health.The centralized management form impedes the development of the cold-chain logistics industry and weakens logistics data availability.This paper first introduces a distributed CCLS based on blockchain technology to solve the centralized management problem.This system aggregates the production base,storage,transport,detection,processing,and consumer to form a cold-chain logistics union.The blockchain ledger guarantees that the logistics data cannot be tampered with and establishes a traceability mechanism for food safety incidents.Meanwhile,to improve the value of logistics data,a Stackelberg game-based resource allocation model has been proposed between the logistics data resource provider and the consumer.The competition between resource price and volume balances the resource supplement and consumption.This model can help to achieve an optimal resource price when the Stackelberg game obtains Nash equilibrium.The two participants also can maximize their revenues with the optimal resource price and volume by utilizing the backward induction method.Then,the performance evaluations of transaction throughput and latency show that the proposed distributed CCLS is more secure and stable.The simulations about the variation trend of data price and amount,optimal benefits,and total benefits comparison of different forms show that the resource allocation model is more efficient and practical.Moreover,the blockchain-based CCLS and Stackelberg game-based resource allocation model also can promote the value of logistic data and improve social benefits.

Tasks-Oriented Joint Resource Allocation Scheme for the Internet of Vehicles with Sensing, Communication and Computing Integration

With the development of artificial intelligence(AI)and 5G technology,the integration of sensing,communication and computing in the Internet of Vehicles(Io V)is becoming a trend.However,the large amount of data transmission and the computing requirements of intelligent tasks lead to the complex resource management problems.In view of the above challenges,this paper proposes a tasks-oriented joint resource allocation scheme(TOJRAS)in the scenario of Io V.First,this paper proposes a system model with sensing,communication,and computing integration for multiple intelligent tasks with different requirements in the Io V.Secondly,joint resource allocation problems for real-time tasks and delay-tolerant tasks in the Io V are constructed respectively,including communication,computing and caching resources.Thirdly,a distributed deep Q-network(DDQN)based algorithm is proposed to solve the optimization problems,and the convergence and complexity of the algorithm are discussed.Finally,the experimental results based on real data sets verify the performance advantages of the proposed resource allocation scheme,compared to the existing ones.The exploration efficiency of our proposed DDQN-based algorithm is improved by at least about 5%,and our proposed resource allocation scheme improves the m AP performance by about 0.15 under resource constraints.

Vegetation C–N–P accumulation and allocation patterns at the community level in early restored plantations in the loess hilly-gully region

Accumulation of vegetation biomass is a crucial process for carbon fixation in the early stage of afforestation and a primary driving force for subsequent ecological functions.Accurately assessing the storage and allocation of elements in plantations is essential for their management and estimating carbon sink capacity.However,current knowledge of the storage and allocation patterns of elements within plant organs at the community level is limited.To clarify the distribution patterns of elements in plant organs at the community level,we measured the biomass within plant organs of five typical plantations in the early stage of afforestation in the loess hilly-gully region.We assessed the main drivers of element accumulation and distribution by employing redundancy analysis and random forest.Results revealed significant differences in biomass storages among plantations and a significant effect of plantation type on the storages of elements within plant organs.Furthermore,the dominant factors influencing C–N–P storage and allocation at the community level were found to be inconsistent.While the storage of elements was mainly influenced by stand openness,total soil nitrogen,and plant diversity,the allocation of elements in organs was mainly influenced by stand openness and soil water content.Overall,the spatial structure of the community had an important influence on both element storage and allocation,but soil conditions played a more important role in element allocation than in storage.Random forest results showed that at the community level,factors influencing element storage and allocation within plant organs often differed.The regulation of elemental storage could be regulated by the major growth demand resources,while the allocation was regulated by other limiting class factors,which often differed from those that had a significant effect on element storage.The differences in plant organ elemental storage and allocation drivers at the community level reflect community adaptation strategies and the regulation of resources by ecosystems in combination with plants.Our study provides valuable insights for enhancing plantation C sink estimates and serves as a reference for regulating element storage and allocation at the local scale.

NOMA-Based Collaborative Beam Hopping Frequency Allocation Mechanism for Future LEO Satellite Systems

Low Earth orbit(LEO) satellite systems provide terrestrial users with services that are not limited by geographical location. However, the conflict between existing allocation schemes and the business variability between beams is becoming increasingly prominent. Beam hopping technology allows for a more flexible and versatile approach to satellite resource allocation. This paper proposes a beam hopping pattern optimization scheme that jointly considers the interference threshold distance and beam service priority, reducing the inter-beam co-channel interference(CCI). In the cluster area, a non-orthogonal multiple access(NOMA)-based collaborative beam hopping(NCBH) scheme is proposed to minimize the cell-edge user(CEU) interference. Since there is a difference in channel gain between the CEU and cellcenter user(CCU), this scheme forms a NOMA cluster to perform power domain multiplexing and formulates a NOMA cluster pairing strategy according to the user location to reduce the CCI of the CEU. After NOMA cluster pairing, the optimal carrier frequency of the NOMA cluster is selected by a reinforcement learning algorithm. The simulation results verify the excellent performance of the proposed NCBH scheme regarding the user’s received power, transmission rate, and outage probability.

Efficient Resource Allocation Algorithm in Uplink OFDM-Based Cognitive Radio Networks

The computational complexity of resource allocation processes,in cognitive radio networks(CRNs),is a major issue to be managed.Furthermore,the complicated solution of the optimal algorithm for handling resource allocation in CRNs makes it unsuitable to adopt in real-world applications where both cognitive users,CRs,and primary users,PUs,exist in the identical geographical area.Hence,this work offers a primarily price-based power algorithm to reduce computational complexity in uplink scenarioswhile limiting interference to PUs to allowable threshold.Hence,this paper,compared to other frameworks proposed in the literature,proposes a two-step approach to reduce the complexity of the proposed mathematical model.In the first step,the subcarriers are assigned to the users of the CRN,while the cost function includes a pricing scheme to provide better power control algorithm with improved reliability proposed in the second stage.The main contribution of this paper is to lessen the complexity of the proposed algorithm and to offer flexibility in controlling the interference produced to the users of the primary networks,which has been achieved by including a pricing function in the proposed cost function.Finally,the performance of the proposed power and subcarrier algorithm is confirmed for orthogonal frequency-division multiplexing(OFDM).Simulation results prove that the performance of the proposed algorithm is better than other algorithms,albeit with a lesser complexity of O(NM)+O(Nlog(N)).

Intelligent Preamble Allocation for Coexistence of mMTC/URLLC Devices:A Hierarchical Q-Learning Based Approach

The emergence of various commercial and industrial Internet of Things(IoT)devices has brought great convenience to people’s life and production.Both low-power,massively connected mMTC devices(MDs)and highly reliable,low-latency URLLC devices(UDs)play an important role in different application scenarios.However,when dense MDs and UDs periodically initiate random access(RA)to connect the base station and send data,due to the limited preamble resources,preamble collisions are likely to occur,resulting in device access failure and data transmission delay.At the same time,due to the highreliability demands of UDs,which require smooth access and fast data transmission,it is necessary to reduce the failure rate of their RA process.To this end,we propose an intelligent preamble allocation scheme,which uses hierarchical reinforcement learning to partition the UD exclusive preamble resource pool at the base station side and perform preamble selection within each RA slot at the device side.In particular,considering the limited processing capacity and energy of IoT devices,we adopt the lightweight Qlearning algorithm on the device side and design simple states and actions for them.Experimental results show that the proposed intelligent scheme can significantly reduce the transmission failure rate of UDs and improve the overall access success rate of devices.

Optimizing Resource Allocation Framework for Multi-Cloud Environment

Cloud computingmakes dynamic resource provisioning more accessible.Monitoring a functioning service is crucial,and changes are made when particular criteria are surpassed.This research explores the decentralized multi-cloud environment for allocating resources and ensuring the Quality of Service(QoS),estimating the required resources,and modifying allotted resources depending on workload and parallelism due to resources.Resource allocation is a complex challenge due to the versatile service providers and resource providers.The engagement of different service and resource providers needs a cooperation strategy for a sustainable quality of service.The objective of a coherent and rational resource allocation is to attain the quality of service.It also includes identifying critical parameters to develop a resource allocation mechanism.A framework is proposed based on the specified parameters to formulate a resource allocation process in a decentralized multi-cloud environment.The three main parameters of the proposed framework are data accessibility,optimization,and collaboration.Using an optimization technique,these three segments are further divided into subsets for resource allocation and long-term service quality.The CloudSim simulator has been used to validate the suggested framework.Several experiments have been conducted to find the best configurations suited for enhancing collaboration and resource allocation to achieve sustained QoS.The results support the suggested structure for a decentralized multi-cloud environment and the parameters that have been determined.

Coalitional game based resource allocation in D2D-enabled V2V communication

The joint resource block(RB)allocation and power optimization problem is studied to maximize the sum-rate of the vehicle-to-vehicle(V2V)links in the device-to-device(D2D)-enabled V2V communication system,where one feasible cellular user(FCU)can share its RB with multiple V2V pairs.The problem is first formulated as a nonconvex mixed-integer nonlinear programming(MINLP)problem with constraint of the maximum interference power in the FCU links.Using the game theory,two coalition formation algorithms are proposed to accomplish V2V link partitioning and FCU selection,where the transferable utility functions are introduced to minimize the interference among the V2V links and the FCU links for the optimal RB allocation.The successive convex approximation(SCA)is used to transform the original problem into a convex one and the Lagrangian dual method is further applied to obtain the optimal transmit power of the V2V links.Finally,numerical results demonstrate the efficiency of the proposed resource allocation algorithm in terms of the system sum-rate.