Role Dynamic Allocation of Human-Robot Cooperation Based on Reinforcement Learning in an Installation of Curtain Wall

A real-time adaptive roles allocation method based on reinforcement learning is proposed to improve humanrobot cooperation performance for a curtain wall installation task.This method breaks the traditional idea that the robot is regarded as the follower or only adjusts the leader and the follower in cooperation.In this paper,a self-learning method is proposed which can dynamically adapt and continuously adjust the initiative weight of the robot according to the change of the task.Firstly,the physical human-robot cooperation model,including the role factor is built.Then,a reinforcement learningmodel that can adjust the role factor in real time is established,and a reward and actionmodel is designed.The role factor can be adjusted continuously according to the comprehensive performance of the human-robot interaction force and the robot’s Jerk during the repeated installation.Finally,the roles adjustment rule established above continuously improves the comprehensive performance.Experiments of the dynamic roles allocation and the effect of the performance weighting coefficient on the result have been verified.The results show that the proposed method can realize the role adaptation and achieve the dual optimization goal of reducing the sum of the cooperator force and the robot’s Jerk.

Computing Resource Allocation for Blockchain-Based Mobile Edge Computing

Users and edge servers are not fullymutually trusted inmobile edge computing(MEC),and hence blockchain can be introduced to provide trustableMEC.In blockchain-basedMEC,each edge server functions as a node in bothMEC and blockchain,processing users’tasks and then uploading the task related information to the blockchain.That is,each edge server runs both users’offloaded tasks and blockchain tasks simultaneously.Note that there is a trade-off between the resource allocation for MEC and blockchain tasks.Therefore,the allocation of the resources of edge servers to the blockchain and theMEC is crucial for the processing delay of blockchain-based MEC.Most of the existing research tackles the problem of resource allocation in either blockchain or MEC,which leads to unfavorable performance of the blockchain-based MEC system.In this paper,we study how to allocate the computing resources of edge servers to the MEC and blockchain tasks with the aimtominimize the total systemprocessing delay.For the problem,we propose a computing resource Allocation algorithmfor Blockchain-based MEC(ABM)which utilizes the Slater’s condition,Karush-Kuhn-Tucker(KKT)conditions,partial derivatives of the Lagrangian function and subgradient projection method to obtain the solution.Simulation results show that ABM converges and effectively reduces the processing delay of blockchain-based MEC.

An Adaptive Hybrid Optimization Strategy for Resource Allocation in Network Function Virtualization

With the rapid development of Network Function Virtualization(NFV),the problem of low resource utilizationin traditional data centers is gradually being addressed.However,existing research does not optimize both localand global allocation of resources in data centers.Hence,we propose an adaptive hybrid optimization strategy thatcombines dynamic programming and neural networks to improve resource utilization and service quality in datacenters.Our approach encompasses a service function chain simulation generator,a parallel architecture servicesystem,a dynamic programming strategy formaximizing the utilization of local server resources,a neural networkfor predicting the global utilization rate of resources and a global resource optimization strategy for bottleneck andredundant resources.With the implementation of our local and global resource allocation strategies,the systemperformance is significantly optimized through simulation.

Joint Task Allocation and Resource Optimization for Blockchain Enabled Collaborative Edge Computing

Collaborative edge computing is a promising direction to handle the computation intensive tasks in B5G wireless networks.However,edge computing servers(ECSs)from different operators may not trust each other,and thus the incentives for collaboration cannot be guaranteed.In this paper,we propose a consortium blockchain enabled collaborative edge computing framework,where users can offload computing tasks to ECSs from different operators.To minimize the total delay of users,we formulate a joint task offloading and resource optimization problem,under the constraint of the computing capability of each ECS.We apply the Tammer decomposition method and heuristic optimization algorithms to obtain the optimal solution.Finally,we propose a reputation based node selection approach to facilitate the consensus process,and also consider a completion time based primary node selection to avoid monopolization of certain edge node and enhance the security of the blockchain.Simulation results validate the effectiveness of the proposed algorithm,and the total delay can be reduced by up to 40%compared with the non-cooperative case.

Power Allocation for SE Maximization in Uplink Massive MIMO System Under Minimum Rate Constraint

In this paper,we optimize the spectrum efficiency(SE)of uplink massive multiple-input multiple-output(MIMO)system with imperfect channel state information(CSI)over Rayleigh fading channel.The SE optimization problem is formulated under the constraints of maximum power and minimum rate of each user.Then,we develop a near-optimal power allocation(PA)scheme by using the successive convex approximation(SCA)method,Lagrange multiplier method,and block coordinate descent(BCD)method,and it can obtain almost the same SE as the benchmark scheme with lower complexity.Since this scheme needs three-layer iteration,a suboptimal PA scheme is developed to further reduce the complexity,where the characteristic of massive MIMO(i.e.,numerous receive antennas)is utilized for convex reformulation,and the rate constraint is converted to linear constraints.This suboptimal scheme only needs single-layer iteration,thus has lower complexity than the near-optimal scheme.Finally,we joint design the pilot power and data power to further improve the performance,and propose an two-stage algorithm to obtain joint PA.Simulation results verify the effectiveness of the proposed schemes,and superior SE performance is achieved.

Resource Allocation in Multi-User Cellular Networks:A Transformer-Based Deep Reinforcement Learning Approach

To meet the communication services with diverse requirements,dynamic resource allocation has shown increasing importance.In this paper,we consider the multi-slot and multi-user resource allocation(MSMU-RA)in a downlink cellular scenario with the aim of maximizing system spectral efficiency while guaranteeing user fairness.We first model the MSMURA problem as a dual-sequence decision-making process,and then solve it by a novel Transformerbased deep reinforcement learning(TDRL)approach.Specifically,the proposed TDRL approach can be achieved based on two aspects:1)To adapt to the dynamic wireless environment,the proximal policy optimization(PPO)algorithm is used to optimize the multi-slot RA strategy.2)To avoid co-channel interference,the Transformer-based PPO algorithm is presented to obtain the optimal multi-user RA scheme by exploring the mapping between user sequence and resource sequence.Experimental results show that:i)the proposed approach outperforms both the traditional and DRL methods in spectral efficiency and user fairness,ii)the proposed algorithm is superior to DRL approaches in terms of convergence speed and generalization performance.

A new quantum key distribution resource allocation and routing optimization scheme

Quantum key distribution(QKD)is a technology that can resist the threat of quantum computers to existing conventional cryptographic protocols.However,due to the stringent requirements of the quantum key generation environment,the generated quantum keys are considered valuable,and the slow key generation rate conflicts with the high-speed data transmission in traditional optical networks.In this paper,for the QKD network with a trusted relay,which is mainly based on point-to-point quantum keys and has complex changes in network resources,we aim to allocate resources reasonably for data packet distribution.Firstly,we formulate a linear programming constraint model for the key resource allocation(KRA)problem based on the time-slot scheduling.Secondly,we propose a new scheduling scheme based on the graded key security requirements(GKSR)and a new micro-log key storage algorithm for effective storage and management of key resources.Finally,we propose a key resource consumption(KRC)routing optimization algorithm to properly allocate time slots,routes,and key resources.Simulation results show that the proposed scheme significantly improves the key distribution success rate and key resource utilization rate,among others.

Online Learning-Based Offloading Decision and Resource Allocation in Mobile Edge Computing-Enabled Satellite-Terrestrial Networks

Mobile edge computing(MEC)-enabled satellite-terrestrial networks(STNs)can provide Internet of Things(IoT)devices with global computing services.Sometimes,the network state information is uncertain or unknown.To deal with this situation,we investigate online learning-based offloading decision and resource allocation in MEC-enabled STNs in this paper.The problem of minimizing the average sum task completion delay of all IoT devices over all time periods is formulated.We decompose this optimization problem into a task offloading decision problem and a computing resource allocation problem.A joint optimization scheme of offloading decision and resource allocation is then proposed,which consists of a task offloading decision algorithm based on the devices cooperation aided upper confidence bound(UCB)algorithm and a computing resource allocation algorithm based on the Lagrange multiplier method.Simulation results validate that the proposed scheme performs better than other baseline schemes.

Resource Allocation for Cognitive Network Slicing in PD-SCMA System Based on Two-Way Deep Reinforcement Learning

In this paper,we propose the Two-way Deep Reinforcement Learning(DRL)-Based resource allocation algorithm,which solves the problem of resource allocation in the cognitive downlink network based on the underlay mode.Secondary users(SUs)in the cognitive network are multiplexed by a new Power Domain Sparse Code Multiple Access(PD-SCMA)scheme,and the physical resources of the cognitive base station are virtualized into two types of slices:enhanced mobile broadband(eMBB)slice and ultrareliable low latency communication(URLLC)slice.We design the Double Deep Q Network(DDQN)network output the optimal codebook assignment scheme and simultaneously use the Deep Deterministic Policy Gradient(DDPG)network output the optimal power allocation scheme.The objective is to jointly optimize the spectral efficiency of the system and the Quality of Service(QoS)of SUs.Simulation results show that the proposed algorithm outperforms the CNDDQN algorithm and modified JEERA algorithm in terms of spectral efficiency and QoS satisfaction.Additionally,compared with the Power Domain Non-orthogonal Multiple Access(PD-NOMA)slices and the Sparse Code Multiple Access(SCMA)slices,the PD-SCMA slices can dramatically enhance spectral efficiency and increase the number of accessible users.

Resource Allocation for IRS Assistedmm Wave Wireless Powered Sensor Networks with User Cooperation

In this paper,we investigate IRS-aided user cooperation(UC)scheme in millimeter wave(mmWave)wirelesspowered sensor networks(WPSN),where two single-antenna users are wireless powered in the wireless energy transfer(WET)phase first and then cooperatively transmit information to a hybrid access point(AP)in the wireless information transmission(WIT)phase,following which the IRS is deployed to enhance the system performance of theWET andWIT.We maximized the weighted sum-rate problem by jointly optimizing the transmit time slots,power allocations,and the phase shifts of the IRS.Due to the non-convexity of the original problem,a semidefinite programming relaxation-based approach is proposed to convert the formulated problem to a convex optimization framework,which can obtain the optimal global solution.Simulation results demonstrate that the weighted sum throughput of the proposed UC scheme outperforms the non-UC scheme whether equipped with IRS or not.

River width and depth as key factors of diurnal activity energy expenditure allocation for wintering Spot-billed Ducks in the Xin'an River Basin

Rivers are important habitats for wintering waterbirds.However,they are easily influenced by natural and human activities.An important approach for waterbirds to adapt to habitats is adjusting the activity time and energy expenditure allocation of diurnal behavior.The compensatory foraging hypothesis predicts that increased energy expenditure leads to longer foraging time,which in turn increases food intake and helps maintain a constant energy balance.However,it is unclear whether human-disturbed habitats result in increased energy expenditure related to safety or foraging.In this study,the scan sample method was used to observe the diurnal behavior of the wintering Spot-billed Duck(Anas poecilorhyncha) in two rivers in the Xin’an River Basin from October 2021 to March 2022.The allocation of time and energy expenditure for activity in both normal and disturbed environments was calculated.The results showed that foraging accounted for the highest percentage of time and energy expenditure.Additionally,foraging decreased in the disturbed environment than that in the normal environment.Resting behavior showed the opposite trend,while other behaviors were similar in both environments.The total diurnal energy expenditure of ducks in the disturbed environment was greater than that in the normal environment,with decreased foraging and resting time percentage and increased behaviors related to immediate safety(swimming and alert) and comfort.These results oppose the compensatory foraging hypothesis in favor of increased security.The optimal diurnal energy expenditure model included river width and water depth,which had a positive relationship;an increase in either of these two factors resulted in an increase in energy expenditure.This study provides a better understanding of energy allocation strategies underlying the superficial time allocation of wintering waterbirds according to environmental conditions.Exploring these changes can help understand the maximum fitness of wintering waterbirds in response to nature and human influences.

A Deep Reinforcement Learning-Based Technique for Optimal Power Allocation in Multiple Access Communications

Formany years,researchers have explored power allocation(PA)algorithms driven bymodels in wireless networks where multiple-user communications with interference are present.Nowadays,data-driven machine learning methods have become quite popular in analyzing wireless communication systems,which among them deep reinforcement learning(DRL)has a significant role in solving optimization issues under certain constraints.To this purpose,in this paper,we investigate the PA problem in a k-user multiple access channels(MAC),where k transmitters(e.g.,mobile users)aim to send an independent message to a common receiver(e.g.,base station)through wireless channels.To this end,we first train the deep Q network(DQN)with a deep Q learning(DQL)algorithm over the simulation environment,utilizing offline learning.Then,the DQN will be used with the real data in the online training method for the PA issue by maximizing the sumrate subjected to the source power.Finally,the simulation results indicate that our proposedDQNmethod provides better performance in terms of the sumrate compared with the available DQL training approaches such as fractional programming(FP)and weighted minimum mean squared error(WMMSE).Additionally,by considering different user densities,we show that our proposed DQN outperforms benchmark algorithms,thereby,a good generalization ability is verified over wireless multi-user communication systems.

Stackelberg Game-Based Resource Allocation with Blockchain for Cold-Chain Logistics System

Cold-chain logistics system(CCLS)plays the role of collecting and managing the logistics data of frozen food.However,there always exist problems of information loss,data tampering,and privacy leakage in traditional centralized systems,which influence frozen food security and people’s health.The centralized management form impedes the development of the cold-chain logistics industry and weakens logistics data availability.This paper first introduces a distributed CCLS based on blockchain technology to solve the centralized management problem.This system aggregates the production base,storage,transport,detection,processing,and consumer to form a cold-chain logistics union.The blockchain ledger guarantees that the logistics data cannot be tampered with and establishes a traceability mechanism for food safety incidents.Meanwhile,to improve the value of logistics data,a Stackelberg game-based resource allocation model has been proposed between the logistics data resource provider and the consumer.The competition between resource price and volume balances the resource supplement and consumption.This model can help to achieve an optimal resource price when the Stackelberg game obtains Nash equilibrium.The two participants also can maximize their revenues with the optimal resource price and volume by utilizing the backward induction method.Then,the performance evaluations of transaction throughput and latency show that the proposed distributed CCLS is more secure and stable.The simulations about the variation trend of data price and amount,optimal benefits,and total benefits comparison of different forms show that the resource allocation model is more efficient and practical.Moreover,the blockchain-based CCLS and Stackelberg game-based resource allocation model also can promote the value of logistic data and improve social benefits.

The relationships between maize(Zea mays L.)lodging resistance and yield formation depend on dry matter allocation to ear and stem

Lodging is a critical constraint to yield increase.There appear to be tradeoffs between yield formation and lodging resistance in maize.Hypothetically,it is feasible to reduce lodging risk as well as increase grain yield by optimizing dry-matter allocation to different organs under different environments.A three-year field experiment was conducted using four maize cultivars with differing lodging resistances and five growing environments in 2018–2020.Lodging-susceptible(LS)cultivars on average yielded more than lodging-resistant(LR)cultivars when lodging was not present.The yield components kernel number per ear(KN)and thousand-kernel weight(TKW)were both negatively correlated with lodging resistance traits(stalk bending strength,rind penetration strength,and dry matter weight per internode length).Before silking,the LR cultivar Lishou 1(LS1)transported more assimilates to the basal stem,resulting in a thicker basal stem,which reduced dry matter allocation to the ear and in turn KN.The lower KN of LS1 was also due partly to the lower plant height(PH),which increased lodging resistance but limited plant dry matter production.In contrast,the LS cultivars Xianyu 335(XY335)and Xundan 20(XD20)produced and allocated more photoassimilates to ears,but limited dry matter allocation to stems.After silking,LS cultivars showed higher TKW than LR cultivars as a function of high photoassimilate productivity and high assimilate allocation to the ear.The higher lodging resistance of LS1 was due mainly to the greater assimilate allocation to stem after silking and lower PH and ear height(EH).High-yielding and high-LR traits of Fumin(FM985)were related to optimized EH and stem anatomical structure,higher leaf productivity,low assimilate demand for kernel formation,and assimilate partitioning to ear.A high presilking temperature accelerated stem extension but reduced stem dry matter accumulation and basal stem strength.Post-silking temperature influences lodging resistance and yield more than other environmental factors.These results will be useful in understanding the tradeoffs between KN,KW,and LR in maize and environmental influences on these tradeoffs.

Channel Capacity and Power Allocation of MIMO Visible Light Communication System

In this paper,the channel capacity of the multiple-input multiple-output(MIMO)visible light communication(VLC)system is investigated under the peak,average optical and electrical power constraints.Finding the channel capacity of MIMO VLC is shown to be a mixed integer programming problem.To address this open problem,we propose an inexact gradient projection method to find the channel capacity-achieving discrete input distribution and the channel capacity of MIMO VLC.Also we derive both upper and lower bounds of the capacity of MIMO VLC with the closed-form expressions.Furthermore,by considering practical discrete constellation inputs,we develop the optimal power allocation scheme to maximize transmission rate of MIMO VLC system.Simulation results show that more discrete points are needed to achieve the channel capacity as SNR increases.Both the upper and lower bounds of channel capacity are tight at low SNR region.In addition,comparing the equal power allocation,the proposed power allocation scheme can significantly increase the rate for the low-order modulation inputs.

User Scheduling and Slicing Resource Allocation in Industrial Internet of Things

Heterogeneous base station deployment enables to provide high capacity and wide area coverage.Network slicing makes it possible to allocate wireless resource for heterogeneous services on demand.These two promising technologies contribute to the unprecedented service in 5G.We establish a multiservice heterogeneous network model,which aims to raise the transmission rate under the delay constraints for active control terminals,and optimize the energy efficiency for passive network terminals.A policygradient-based deep reinforcement learning algorithm is proposed to make decisions on user association and power control in the continuous action space.Simulation results indicate the good convergence of the algorithm,and higher reward is obtained compared with other baselines.

Tasks-Oriented Joint Resource Allocation Scheme for the Internet of Vehicles with Sensing, Communication and Computing Integration

With the development of artificial intelligence(AI)and 5G technology,the integration of sensing,communication and computing in the Internet of Vehicles(Io V)is becoming a trend.However,the large amount of data transmission and the computing requirements of intelligent tasks lead to the complex resource management problems.In view of the above challenges,this paper proposes a tasks-oriented joint resource allocation scheme(TOJRAS)in the scenario of Io V.First,this paper proposes a system model with sensing,communication,and computing integration for multiple intelligent tasks with different requirements in the Io V.Secondly,joint resource allocation problems for real-time tasks and delay-tolerant tasks in the Io V are constructed respectively,including communication,computing and caching resources.Thirdly,a distributed deep Q-network(DDQN)based algorithm is proposed to solve the optimization problems,and the convergence and complexity of the algorithm are discussed.Finally,the experimental results based on real data sets verify the performance advantages of the proposed resource allocation scheme,compared to the existing ones.The exploration efficiency of our proposed DDQN-based algorithm is improved by at least about 5%,and our proposed resource allocation scheme improves the m AP performance by about 0.15 under resource constraints.

Vegetation C–N–P accumulation and allocation patterns at the community level in early restored plantations in the loess hilly-gully region

Accumulation of vegetation biomass is a crucial process for carbon fixation in the early stage of afforestation and a primary driving force for subsequent ecological functions.Accurately assessing the storage and allocation of elements in plantations is essential for their management and estimating carbon sink capacity.However,current knowledge of the storage and allocation patterns of elements within plant organs at the community level is limited.To clarify the distribution patterns of elements in plant organs at the community level,we measured the biomass within plant organs of five typical plantations in the early stage of afforestation in the loess hilly-gully region.We assessed the main drivers of element accumulation and distribution by employing redundancy analysis and random forest.Results revealed significant differences in biomass storages among plantations and a significant effect of plantation type on the storages of elements within plant organs.Furthermore,the dominant factors influencing C–N–P storage and allocation at the community level were found to be inconsistent.While the storage of elements was mainly influenced by stand openness,total soil nitrogen,and plant diversity,the allocation of elements in organs was mainly influenced by stand openness and soil water content.Overall,the spatial structure of the community had an important influence on both element storage and allocation,but soil conditions played a more important role in element allocation than in storage.Random forest results showed that at the community level,factors influencing element storage and allocation within plant organs often differed.The regulation of elemental storage could be regulated by the major growth demand resources,while the allocation was regulated by other limiting class factors,which often differed from those that had a significant effect on element storage.The differences in plant organ elemental storage and allocation drivers at the community level reflect community adaptation strategies and the regulation of resources by ecosystems in combination with plants.Our study provides valuable insights for enhancing plantation C sink estimates and serves as a reference for regulating element storage and allocation at the local scale.

NOMA-Based Collaborative Beam Hopping Frequency Allocation Mechanism for Future LEO Satellite Systems

Low Earth orbit(LEO) satellite systems provide terrestrial users with services that are not limited by geographical location. However, the conflict between existing allocation schemes and the business variability between beams is becoming increasingly prominent. Beam hopping technology allows for a more flexible and versatile approach to satellite resource allocation. This paper proposes a beam hopping pattern optimization scheme that jointly considers the interference threshold distance and beam service priority, reducing the inter-beam co-channel interference(CCI). In the cluster area, a non-orthogonal multiple access(NOMA)-based collaborative beam hopping(NCBH) scheme is proposed to minimize the cell-edge user(CEU) interference. Since there is a difference in channel gain between the CEU and cellcenter user(CCU), this scheme forms a NOMA cluster to perform power domain multiplexing and formulates a NOMA cluster pairing strategy according to the user location to reduce the CCI of the CEU. After NOMA cluster pairing, the optimal carrier frequency of the NOMA cluster is selected by a reinforcement learning algorithm. The simulation results verify the excellent performance of the proposed NCBH scheme regarding the user’s received power, transmission rate, and outage probability.

Efficient Resource Allocation Algorithm in Uplink OFDM-Based Cognitive Radio Networks

The computational complexity of resource allocation processes,in cognitive radio networks(CRNs),is a major issue to be managed.Furthermore,the complicated solution of the optimal algorithm for handling resource allocation in CRNs makes it unsuitable to adopt in real-world applications where both cognitive users,CRs,and primary users,PUs,exist in the identical geographical area.Hence,this work offers a primarily price-based power algorithm to reduce computational complexity in uplink scenarioswhile limiting interference to PUs to allowable threshold.Hence,this paper,compared to other frameworks proposed in the literature,proposes a two-step approach to reduce the complexity of the proposed mathematical model.In the first step,the subcarriers are assigned to the users of the CRN,while the cost function includes a pricing scheme to provide better power control algorithm with improved reliability proposed in the second stage.The main contribution of this paper is to lessen the complexity of the proposed algorithm and to offer flexibility in controlling the interference produced to the users of the primary networks,which has been achieved by including a pricing function in the proposed cost function.Finally,the performance of the proposed power and subcarrier algorithm is confirmed for orthogonal frequency-division multiplexing(OFDM).Simulation results prove that the performance of the proposed algorithm is better than other algorithms,albeit with a lesser complexity of O(NM)+O(Nlog(N)).