Role Dynamic Allocation of Human-Robot Cooperation Based on Reinforcement Learning in an Installation of Curtain Wall

A real-time adaptive roles allocation method based on reinforcement learning is proposed to improve humanrobot cooperation performance for a curtain wall installation task.This method breaks the traditional idea that the robot is regarded as the follower or only adjusts the leader and the follower in cooperation.In this paper,a self-learning method is proposed which can dynamically adapt and continuously adjust the initiative weight of the robot according to the change of the task.Firstly,the physical human-robot cooperation model,including the role factor is built.Then,a reinforcement learningmodel that can adjust the role factor in real time is established,and a reward and actionmodel is designed.The role factor can be adjusted continuously according to the comprehensive performance of the human-robot interaction force and the robot’s Jerk during the repeated installation.Finally,the roles adjustment rule established above continuously improves the comprehensive performance.Experiments of the dynamic roles allocation and the effect of the performance weighting coefficient on the result have been verified.The results show that the proposed method can realize the role adaptation and achieve the dual optimization goal of reducing the sum of the cooperator force and the robot’s Jerk.

Computing Resource Allocation for Blockchain-Based Mobile Edge Computing

Users and edge servers are not fullymutually trusted inmobile edge computing(MEC),and hence blockchain can be introduced to provide trustableMEC.In blockchain-basedMEC,each edge server functions as a node in bothMEC and blockchain,processing users’tasks and then uploading the task related information to the blockchain.That is,each edge server runs both users’offloaded tasks and blockchain tasks simultaneously.Note that there is a trade-off between the resource allocation for MEC and blockchain tasks.Therefore,the allocation of the resources of edge servers to the blockchain and theMEC is crucial for the processing delay of blockchain-based MEC.Most of the existing research tackles the problem of resource allocation in either blockchain or MEC,which leads to unfavorable performance of the blockchain-based MEC system.In this paper,we study how to allocate the computing resources of edge servers to the MEC and blockchain tasks with the aimtominimize the total systemprocessing delay.For the problem,we propose a computing resource Allocation algorithmfor Blockchain-based MEC(ABM)which utilizes the Slater’s condition,Karush-Kuhn-Tucker(KKT)conditions,partial derivatives of the Lagrangian function and subgradient projection method to obtain the solution.Simulation results show that ABM converges and effectively reduces the processing delay of blockchain-based MEC.

An Adaptive Hybrid Optimization Strategy for Resource Allocation in Network Function Virtualization

With the rapid development of Network Function Virtualization(NFV),the problem of low resource utilizationin traditional data centers is gradually being addressed.However,existing research does not optimize both localand global allocation of resources in data centers.Hence,we propose an adaptive hybrid optimization strategy thatcombines dynamic programming and neural networks to improve resource utilization and service quality in datacenters.Our approach encompasses a service function chain simulation generator,a parallel architecture servicesystem,a dynamic programming strategy formaximizing the utilization of local server resources,a neural networkfor predicting the global utilization rate of resources and a global resource optimization strategy for bottleneck andredundant resources.With the implementation of our local and global resource allocation strategies,the systemperformance is significantly optimized through simulation.

Joint Task Allocation and Resource Optimization for Blockchain Enabled Collaborative Edge Computing

Collaborative edge computing is a promising direction to handle the computation intensive tasks in B5G wireless networks.However,edge computing servers(ECSs)from different operators may not trust each other,and thus the incentives for collaboration cannot be guaranteed.In this paper,we propose a consortium blockchain enabled collaborative edge computing framework,where users can offload computing tasks to ECSs from different operators.To minimize the total delay of users,we formulate a joint task offloading and resource optimization problem,under the constraint of the computing capability of each ECS.We apply the Tammer decomposition method and heuristic optimization algorithms to obtain the optimal solution.Finally,we propose a reputation based node selection approach to facilitate the consensus process,and also consider a completion time based primary node selection to avoid monopolization of certain edge node and enhance the security of the blockchain.Simulation results validate the effectiveness of the proposed algorithm,and the total delay can be reduced by up to 40%compared with the non-cooperative case.

Power Allocation for SE Maximization in Uplink Massive MIMO System Under Minimum Rate Constraint

In this paper,we optimize the spectrum efficiency(SE)of uplink massive multiple-input multiple-output(MIMO)system with imperfect channel state information(CSI)over Rayleigh fading channel.The SE optimization problem is formulated under the constraints of maximum power and minimum rate of each user.Then,we develop a near-optimal power allocation(PA)scheme by using the successive convex approximation(SCA)method,Lagrange multiplier method,and block coordinate descent(BCD)method,and it can obtain almost the same SE as the benchmark scheme with lower complexity.Since this scheme needs three-layer iteration,a suboptimal PA scheme is developed to further reduce the complexity,where the characteristic of massive MIMO(i.e.,numerous receive antennas)is utilized for convex reformulation,and the rate constraint is converted to linear constraints.This suboptimal scheme only needs single-layer iteration,thus has lower complexity than the near-optimal scheme.Finally,we joint design the pilot power and data power to further improve the performance,and propose an two-stage algorithm to obtain joint PA.Simulation results verify the effectiveness of the proposed schemes,and superior SE performance is achieved.

Resource Allocation in Multi-User Cellular Networks:A Transformer-Based Deep Reinforcement Learning Approach

To meet the communication services with diverse requirements,dynamic resource allocation has shown increasing importance.In this paper,we consider the multi-slot and multi-user resource allocation(MSMU-RA)in a downlink cellular scenario with the aim of maximizing system spectral efficiency while guaranteeing user fairness.We first model the MSMURA problem as a dual-sequence decision-making process,and then solve it by a novel Transformerbased deep reinforcement learning(TDRL)approach.Specifically,the proposed TDRL approach can be achieved based on two aspects:1)To adapt to the dynamic wireless environment,the proximal policy optimization(PPO)algorithm is used to optimize the multi-slot RA strategy.2)To avoid co-channel interference,the Transformer-based PPO algorithm is presented to obtain the optimal multi-user RA scheme by exploring the mapping between user sequence and resource sequence.Experimental results show that:i)the proposed approach outperforms both the traditional and DRL methods in spectral efficiency and user fairness,ii)the proposed algorithm is superior to DRL approaches in terms of convergence speed and generalization performance.

A new quantum key distribution resource allocation and routing optimization scheme

Quantum key distribution(QKD)is a technology that can resist the threat of quantum computers to existing conventional cryptographic protocols.However,due to the stringent requirements of the quantum key generation environment,the generated quantum keys are considered valuable,and the slow key generation rate conflicts with the high-speed data transmission in traditional optical networks.In this paper,for the QKD network with a trusted relay,which is mainly based on point-to-point quantum keys and has complex changes in network resources,we aim to allocate resources reasonably for data packet distribution.Firstly,we formulate a linear programming constraint model for the key resource allocation(KRA)problem based on the time-slot scheduling.Secondly,we propose a new scheduling scheme based on the graded key security requirements(GKSR)and a new micro-log key storage algorithm for effective storage and management of key resources.Finally,we propose a key resource consumption(KRC)routing optimization algorithm to properly allocate time slots,routes,and key resources.Simulation results show that the proposed scheme significantly improves the key distribution success rate and key resource utilization rate,among others.

Online Learning-Based Offloading Decision and Resource Allocation in Mobile Edge Computing-Enabled Satellite-Terrestrial Networks

Mobile edge computing(MEC)-enabled satellite-terrestrial networks(STNs)can provide Internet of Things(IoT)devices with global computing services.Sometimes,the network state information is uncertain or unknown.To deal with this situation,we investigate online learning-based offloading decision and resource allocation in MEC-enabled STNs in this paper.The problem of minimizing the average sum task completion delay of all IoT devices over all time periods is formulated.We decompose this optimization problem into a task offloading decision problem and a computing resource allocation problem.A joint optimization scheme of offloading decision and resource allocation is then proposed,which consists of a task offloading decision algorithm based on the devices cooperation aided upper confidence bound(UCB)algorithm and a computing resource allocation algorithm based on the Lagrange multiplier method.Simulation results validate that the proposed scheme performs better than other baseline schemes.

Resource Allocation for Cognitive Network Slicing in PD-SCMA System Based on Two-Way Deep Reinforcement Learning

In this paper,we propose the Two-way Deep Reinforcement Learning(DRL)-Based resource allocation algorithm,which solves the problem of resource allocation in the cognitive downlink network based on the underlay mode.Secondary users(SUs)in the cognitive network are multiplexed by a new Power Domain Sparse Code Multiple Access(PD-SCMA)scheme,and the physical resources of the cognitive base station are virtualized into two types of slices:enhanced mobile broadband(eMBB)slice and ultrareliable low latency communication(URLLC)slice.We design the Double Deep Q Network(DDQN)network output the optimal codebook assignment scheme and simultaneously use the Deep Deterministic Policy Gradient(DDPG)network output the optimal power allocation scheme.The objective is to jointly optimize the spectral efficiency of the system and the Quality of Service(QoS)of SUs.Simulation results show that the proposed algorithm outperforms the CNDDQN algorithm and modified JEERA algorithm in terms of spectral efficiency and QoS satisfaction.Additionally,compared with the Power Domain Non-orthogonal Multiple Access(PD-NOMA)slices and the Sparse Code Multiple Access(SCMA)slices,the PD-SCMA slices can dramatically enhance spectral efficiency and increase the number of accessible users.

Resource Allocation for IRS Assistedmm Wave Wireless Powered Sensor Networks with User Cooperation

In this paper,we investigate IRS-aided user cooperation(UC)scheme in millimeter wave(mmWave)wirelesspowered sensor networks(WPSN),where two single-antenna users are wireless powered in the wireless energy transfer(WET)phase first and then cooperatively transmit information to a hybrid access point(AP)in the wireless information transmission(WIT)phase,following which the IRS is deployed to enhance the system performance of theWET andWIT.We maximized the weighted sum-rate problem by jointly optimizing the transmit time slots,power allocations,and the phase shifts of the IRS.Due to the non-convexity of the original problem,a semidefinite programming relaxation-based approach is proposed to convert the formulated problem to a convex optimization framework,which can obtain the optimal global solution.Simulation results demonstrate that the weighted sum throughput of the proposed UC scheme outperforms the non-UC scheme whether equipped with IRS or not.

River width and depth as key factors of diurnal activity energy expenditure allocation for wintering Spot-billed Ducks in the Xin'an River Basin

Rivers are important habitats for wintering waterbirds.However,they are easily influenced by natural and human activities.An important approach for waterbirds to adapt to habitats is adjusting the activity time and energy expenditure allocation of diurnal behavior.The compensatory foraging hypothesis predicts that increased energy expenditure leads to longer foraging time,which in turn increases food intake and helps maintain a constant energy balance.However,it is unclear whether human-disturbed habitats result in increased energy expenditure related to safety or foraging.In this study,the scan sample method was used to observe the diurnal behavior of the wintering Spot-billed Duck(Anas poecilorhyncha) in two rivers in the Xin’an River Basin from October 2021 to March 2022.The allocation of time and energy expenditure for activity in both normal and disturbed environments was calculated.The results showed that foraging accounted for the highest percentage of time and energy expenditure.Additionally,foraging decreased in the disturbed environment than that in the normal environment.Resting behavior showed the opposite trend,while other behaviors were similar in both environments.The total diurnal energy expenditure of ducks in the disturbed environment was greater than that in the normal environment,with decreased foraging and resting time percentage and increased behaviors related to immediate safety(swimming and alert) and comfort.These results oppose the compensatory foraging hypothesis in favor of increased security.The optimal diurnal energy expenditure model included river width and water depth,which had a positive relationship;an increase in either of these two factors resulted in an increase in energy expenditure.This study provides a better understanding of energy allocation strategies underlying the superficial time allocation of wintering waterbirds according to environmental conditions.Exploring these changes can help understand the maximum fitness of wintering waterbirds in response to nature and human influences.

A Deep Reinforcement Learning-Based Technique for Optimal Power Allocation in Multiple Access Communications

Formany years,researchers have explored power allocation(PA)algorithms driven bymodels in wireless networks where multiple-user communications with interference are present.Nowadays,data-driven machine learning methods have become quite popular in analyzing wireless communication systems,which among them deep reinforcement learning(DRL)has a significant role in solving optimization issues under certain constraints.To this purpose,in this paper,we investigate the PA problem in a k-user multiple access channels(MAC),where k transmitters(e.g.,mobile users)aim to send an independent message to a common receiver(e.g.,base station)through wireless channels.To this end,we first train the deep Q network(DQN)with a deep Q learning(DQL)algorithm over the simulation environment,utilizing offline learning.Then,the DQN will be used with the real data in the online training method for the PA issue by maximizing the sumrate subjected to the source power.Finally,the simulation results indicate that our proposedDQNmethod provides better performance in terms of the sumrate compared with the available DQL training approaches such as fractional programming(FP)and weighted minimum mean squared error(WMMSE).Additionally,by considering different user densities,we show that our proposed DQN outperforms benchmark algorithms,thereby,a good generalization ability is verified over wireless multi-user communication systems.

Stackelberg Game-Based Resource Allocation with Blockchain for Cold-Chain Logistics System

Cold-chain logistics system(CCLS)plays the role of collecting and managing the logistics data of frozen food.However,there always exist problems of information loss,data tampering,and privacy leakage in traditional centralized systems,which influence frozen food security and people’s health.The centralized management form impedes the development of the cold-chain logistics industry and weakens logistics data availability.This paper first introduces a distributed CCLS based on blockchain technology to solve the centralized management problem.This system aggregates the production base,storage,transport,detection,processing,and consumer to form a cold-chain logistics union.The blockchain ledger guarantees that the logistics data cannot be tampered with and establishes a traceability mechanism for food safety incidents.Meanwhile,to improve the value of logistics data,a Stackelberg game-based resource allocation model has been proposed between the logistics data resource provider and the consumer.The competition between resource price and volume balances the resource supplement and consumption.This model can help to achieve an optimal resource price when the Stackelberg game obtains Nash equilibrium.The two participants also can maximize their revenues with the optimal resource price and volume by utilizing the backward induction method.Then,the performance evaluations of transaction throughput and latency show that the proposed distributed CCLS is more secure and stable.The simulations about the variation trend of data price and amount,optimal benefits,and total benefits comparison of different forms show that the resource allocation model is more efficient and practical.Moreover,the blockchain-based CCLS and Stackelberg game-based resource allocation model also can promote the value of logistic data and improve social benefits.

The relationships between maize(Zea mays L.)lodging resistance and yield formation depend on dry matter allocation to ear and stem

Lodging is a critical constraint to yield increase.There appear to be tradeoffs between yield formation and lodging resistance in maize.Hypothetically,it is feasible to reduce lodging risk as well as increase grain yield by optimizing dry-matter allocation to different organs under different environments.A three-year field experiment was conducted using four maize cultivars with differing lodging resistances and five growing environments in 2018–2020.Lodging-susceptible(LS)cultivars on average yielded more than lodging-resistant(LR)cultivars when lodging was not present.The yield components kernel number per ear(KN)and thousand-kernel weight(TKW)were both negatively correlated with lodging resistance traits(stalk bending strength,rind penetration strength,and dry matter weight per internode length).Before silking,the LR cultivar Lishou 1(LS1)transported more assimilates to the basal stem,resulting in a thicker basal stem,which reduced dry matter allocation to the ear and in turn KN.The lower KN of LS1 was also due partly to the lower plant height(PH),which increased lodging resistance but limited plant dry matter production.In contrast,the LS cultivars Xianyu 335(XY335)and Xundan 20(XD20)produced and allocated more photoassimilates to ears,but limited dry matter allocation to stems.After silking,LS cultivars showed higher TKW than LR cultivars as a function of high photoassimilate productivity and high assimilate allocation to the ear.The higher lodging resistance of LS1 was due mainly to the greater assimilate allocation to stem after silking and lower PH and ear height(EH).High-yielding and high-LR traits of Fumin(FM985)were related to optimized EH and stem anatomical structure,higher leaf productivity,low assimilate demand for kernel formation,and assimilate partitioning to ear.A high presilking temperature accelerated stem extension but reduced stem dry matter accumulation and basal stem strength.Post-silking temperature influences lodging resistance and yield more than other environmental factors.These results will be useful in understanding the tradeoffs between KN,KW,and LR in maize and environmental influences on these tradeoffs.

Channel Capacity and Power Allocation of MIMO Visible Light Communication System

In this paper,the channel capacity of the multiple-input multiple-output(MIMO)visible light communication(VLC)system is investigated under the peak,average optical and electrical power constraints.Finding the channel capacity of MIMO VLC is shown to be a mixed integer programming problem.To address this open problem,we propose an inexact gradient projection method to find the channel capacity-achieving discrete input distribution and the channel capacity of MIMO VLC.Also we derive both upper and lower bounds of the capacity of MIMO VLC with the closed-form expressions.Furthermore,by considering practical discrete constellation inputs,we develop the optimal power allocation scheme to maximize transmission rate of MIMO VLC system.Simulation results show that more discrete points are needed to achieve the channel capacity as SNR increases.Both the upper and lower bounds of channel capacity are tight at low SNR region.In addition,comparing the equal power allocation,the proposed power allocation scheme can significantly increase the rate for the low-order modulation inputs.

User Scheduling and Slicing Resource Allocation in Industrial Internet of Things

Heterogeneous base station deployment enables to provide high capacity and wide area coverage.Network slicing makes it possible to allocate wireless resource for heterogeneous services on demand.These two promising technologies contribute to the unprecedented service in 5G.We establish a multiservice heterogeneous network model,which aims to raise the transmission rate under the delay constraints for active control terminals,and optimize the energy efficiency for passive network terminals.A policygradient-based deep reinforcement learning algorithm is proposed to make decisions on user association and power control in the continuous action space.Simulation results indicate the good convergence of the algorithm,and higher reward is obtained compared with other baselines.

Tasks-Oriented Joint Resource Allocation Scheme for the Internet of Vehicles with Sensing, Communication and Computing Integration

With the development of artificial intelligence(AI)and 5G technology,the integration of sensing,communication and computing in the Internet of Vehicles(Io V)is becoming a trend.However,the large amount of data transmission and the computing requirements of intelligent tasks lead to the complex resource management problems.In view of the above challenges,this paper proposes a tasks-oriented joint resource allocation scheme(TOJRAS)in the scenario of Io V.First,this paper proposes a system model with sensing,communication,and computing integration for multiple intelligent tasks with different requirements in the Io V.Secondly,joint resource allocation problems for real-time tasks and delay-tolerant tasks in the Io V are constructed respectively,including communication,computing and caching resources.Thirdly,a distributed deep Q-network(DDQN)based algorithm is proposed to solve the optimization problems,and the convergence and complexity of the algorithm are discussed.Finally,the experimental results based on real data sets verify the performance advantages of the proposed resource allocation scheme,compared to the existing ones.The exploration efficiency of our proposed DDQN-based algorithm is improved by at least about 5%,and our proposed resource allocation scheme improves the m AP performance by about 0.15 under resource constraints.

An Online Malicious Spam Email Detection System Using Resource Allocating Network with Locality Sensitive Hashing

In this paper, we propose a new online system that can quickly detect malicious spam emails and adapt to the changes in the email contents and the Uniform Resource Locator (URL) links leading to malicious websites by updating the system daily. We introduce an autonomous function for a server to generate training examples, in which double-bounce emails are automatically collected and their class labels are given by a crawler-type software to analyze the website maliciousness called SPIKE. In general, since spammers use botnets to spread numerous malicious emails within a short time, such distributed spam emails often have the same or similar contents. Therefore, it is not necessary for all spam emails to be learned. To adapt to new malicious campaigns quickly, only new types of spam emails should be selected for learning and this can be realized by introducing an active learning scheme into a classifier model. For this purpose, we adopt Resource Allocating Network with Locality Sensitive Hashing (RAN-LSH) as a classifier model with a data selection function. In RAN-LSH, the same or similar spam emails that have already been learned are quickly searched for a hash table in Locally Sensitive Hashing (LSH), in which the matched similar emails located in “well-learned” are discarded without being used as training data. To analyze email contents, we adopt the Bag of Words (BoW) approach and generate feature vectors whose attributes are transformed based on the normalized term frequency-inverse document frequency (TF-IDF). We use a data set of double-bounce spam emails collected at National Institute of Information and Communications Technology (NICT) in Japan from March 1st, 2013 until May 10th, 2013 to evaluate the performance of the proposed system. The results confirm that the proposed spam email detection system has capability of detecting with high detection rate.

Zoning Search With Adaptive Resource Allocating Method for Balanced and Imbalanced Multimodal Multi-Objective Optimization

Maintaining population diversity is an important task in the multimodal multi-objective optimization.Although the zoning search(ZS)can improve the diversity in the decision space,assigning the same computational costs to each search subspace may be wasteful when computational resources are limited,especially on imbalanced problems.To alleviate the above-mentioned issue,a zoning search with adaptive resource allocating(ZS-ARA)method is proposed in the current study.In the proposed ZS-ARA,the entire search space is divided into many subspaces to preserve the diversity in the decision space and to reduce the problem complexity.Moreover,the computational resources can be automatically allocated among all the subspaces.The ZS-ARA is compared with seven algorithms on two different types of multimodal multi-objective problems(MMOPs),namely,balanced and imbalanced MMOPs.The results indicate that,similarly to the ZS,the ZS-ARA achieves high performance with the balanced MMOPs.Also,it can greatly assist a“regular”algorithm in improving its performance on the imbalanced MMOPs,and is capable of allocating the limited computational resources dynamically.

A Mathematical Model of Global Equity for Allocating Asteroid Resources Based on Analytic Hierarchy Process

As the Earth’s resources are utilized, we are increasingly seeking access to space-based resources. One of the most promising solutions to the problem of resource scarcity is asteroid mining. However, it brings with it the problem of resource allocation. Because of the different levels of development of countries, the way of equal distribution is not reasonable. In this paper, we firstly elaborate and define the abstract global equity in a concrete way, linking global equity with comprehensive national strength, and build a new global equity model on this basis. In this way, we can apply the established model to the distribution of space-based resources or other resources. This paper first establishes the corresponding relation between global equity and comprehensive national power, and then determines several important indexes affecting comprehensive national power according to Klein equation. Further, this paper selected four representative large countries and determined the data of each country in each important index by consulting relevant materials. On this basis, the analytic hierarchy process was used to determine the weight of each country in resource allocation. By comparing the weight index obtained by the model in this paper with the actual resource allocation ratio, we can find that they are in good agreement [1]. Therefore, it can be concluded that when allocating space-based resources or other resources, we can use the global equity model established in this paper to calculate the weight index and allocate resources on this basis to ensure the realization of global equity [2].