Channel assignment and power allocation for throughput improvement with PPO in B5G heterogeneous edge networks

In Beyond the Fifth Generation(B5G)heterogeneous edge networks,numerous users are multiplexed on a channel or served on the same frequency resource block,in which case the transmitter applies coding and the receiver uses interference cancellation.Unfortunately,uncoordinated radio resource allocation can reduce system throughput and lead to user inequity,for this reason,in this paper,channel allocation and power allocation problems are formulated to maximize the system sum rate and minimum user achievable rate.Since the construction model is non-convex and the response variables are high-dimensional,a distributed Deep Reinforcement Learning(DRL)framework called distributed Proximal Policy Optimization(PPO)is proposed to allocate or assign resources.Specifically,several simulated agents are trained in a heterogeneous environment to find robust behaviors that perform well in channel assignment and power allocation.Moreover,agents in the collection stage slow down,which hinders the learning of other agents.Therefore,a preemption strategy is further proposed in this paper to optimize the distributed PPO,form DP-PPO and successfully mitigate the straggler problem.The experimental results show that our mechanism named DP-PPO improves the performance over other DRL methods.

A self-organization formation configuration based assignment probability and collision detection

The formation control of multiple unmanned aerial vehicles(multi-UAVs)has always been a research hotspot.Based on the straight line trajectory,a multi-UAVs target point assignment algorithm based on the assignment probability is proposed to achieve the shortest overall formation path of multi-UAVs with low complexity and reduce the energy consumption.In order to avoid the collision between UAVs in the formation process,the concept of safety ball is introduced,and the collision detection based on continuous motion of two time slots and the lane occupation detection after motion is proposed to avoid collision between UAVs.Based on the idea of game theory,a method of UAV motion form setting based on the maximization of interests is proposed,including the maximization of self-interest and the maximization of formation interest is proposed,so that multi-UAVs can complete the formation task quickly and reasonably with the linear trajectory assigned in advance.Finally,through simulation verification,the multi-UAVs target assignment algorithm based on the assignment probability proposed in this paper can effectively reduce the total path length,and the UAV motion selection method based on the maximization interests can effectively complete the task formation.

Development of SNP parentage assignment techniques in the yellowfin seabream Acanthopagrus latus

Acanthopagrus latus is an essential aquaculture species on the south coast of China.However,there is a lack of systematic breeding of A.latus,which considerably limits the sustainable development of A.latus.As a result,genetic improvements are urgently needed to breed new strains of A.latus with rapid growth and strong resistance to disease.During selective breeding,it is necessary to estimate the genetic parameters of the target trait,which in turn depends on an accurate disentangled pedigree for the selective population.Therefore,it is necessary to establish the parentage assignment technique for A.latus.In this study,95 individuals selected from their parents and their 14 families were used as experimental material.SNPs were developed by genome resequencing,and highly polymorphic SNPs were screened on the basis of optimized filtering parameters.A total of 14392738 SNPs were discovered and 205 SNPs were selected for parentage assignment using the CERVUS software.In the model where the gender of the parents is known,the assignment success rate is 98.61%for the male parent,97.22%for the female parent,and 95.83%for the parent pair.In the model where the gender of the parents is unknown,the assignment success rate is 100%for a single parent and 90.28%for the parent pair.The results of this study were expected to serve as a reference for the breeding of new varieties of A.latus.

Multi-Objective Teaching-Learning-Based Optimizer for a Multi-Weeding Robot Task Assignment Problem

With the emergence of the artificial intelligence era,all kinds of robots are traditionally used in agricultural production.However,studies concerning the robot task assignment problem in the agriculture field,which is closely related to the cost and efficiency of a smart farm,are limited.Therefore,a Multi-Weeding Robot Task Assignment(MWRTA)problem is addressed in this paper to minimize the maximum completion time and residual herbicide.A mathematical model is set up,and a Multi-Objective Teaching-Learning-Based Optimization(MOTLBO)algorithm is presented to solve the problem.In the MOTLBO algorithm,a heuristicbased initialization comprising an improved Nawaz Enscore,and Ham(NEH)heuristic and maximum loadbased heuristic is used to generate an initial population with a high level of quality and diversity.An effective teaching-learning-based optimization process is designed with a dynamic grouping mechanism and a redefined individual updating rule.A multi-neighborhood-based local search strategy is provided to balance the exploitation and exploration of the algorithm.Finally,a comprehensive experiment is conducted to compare the proposed algorithm with several state-of-the-art algorithms in the literature.Experimental results demonstrate the significant superiority of the proposed algorithm for solving the problem under consideration.

Reinforcement Learning Based Quantization Strategy Optimal Assignment Algorithm for Mixed Precision

The quantization algorithm compresses the original network by reducing the numerical bit width of the model,which improves the computation speed. Because different layers have different redundancy and sensitivity to databit width. Reducing the data bit width will result in a loss of accuracy. Therefore, it is difficult to determinethe optimal bit width for different parts of the network with guaranteed accuracy. Mixed precision quantizationcan effectively reduce the amount of computation while keeping the model accuracy basically unchanged. In thispaper, a hardware-aware mixed precision quantization strategy optimal assignment algorithm adapted to low bitwidth is proposed, and reinforcement learning is used to automatically predict the mixed precision that meets theconstraints of hardware resources. In the state-space design, the standard deviation of weights is used to measurethe distribution difference of data, the execution speed feedback of simulated neural network accelerator inferenceis used as the environment to limit the action space of the agent, and the accuracy of the quantization model afterretraining is used as the reward function to guide the agent to carry out deep reinforcement learning training. Theexperimental results show that the proposed method obtains a suitable model layer-by-layer quantization strategyunder the condition that the computational resources are satisfied, and themodel accuracy is effectively improved.The proposed method has strong intelligence and certain universality and has strong application potential in thefield of mixed precision quantization and embedded neural network model deployment.

Task assignment in ground-to-air confrontation based on multiagent deep reinforcement learning

The scale of ground-to-air confrontation task assignments is large and needs to deal with many concurrent task assignments and random events.Aiming at the problems where existing task assignment methods are applied to ground-to-air confrontation,there is low efficiency in dealing with complex tasks,and there are interactive conflicts in multiagent systems.This study proposes a multiagent architecture based on a one-general agent with multiple narrow agents(OGMN)to reduce task assignment conflicts.Considering the slow speed of traditional dynamic task assignment algorithms,this paper proposes the proximal policy optimization for task assignment of general and narrow agents(PPOTAGNA)algorithm.The algorithm based on the idea of the optimal assignment strategy algorithm and combined with the training framework of deep reinforcement learning(DRL)adds a multihead attention mechanism and a stage reward mechanism to the bilateral band clipping PPO algorithm to solve the problem of low training efficiency.Finally,simulation experiments are carried out in the digital battlefield.The multiagent architecture based on OGMN combined with the PPO-TAGNA algorithm can obtain higher rewards faster and has a higher win ratio.By analyzing agent behavior,the efficiency,superiority and rationality of resource utilization of this method are verified.

UAVs cooperative task assignment and trajectory optimization with safety and time constraints

This paper proposes new methods and strategies for Multi-UAVs cooperative attacks with safety and time constraints in a complex environment.Delaunay triangle is designed to construct a map of the complex flight environment for aerial vehicles.Delaunay-Map,Safe Flight Corridor(SFC),and Relative Safe Flight Corridor(RSFC)are applied to ensure each UAV flight trajectory's safety.By using such techniques,it is possible to avoid the collision with obstacles and collision between UAVs.Bezier-curve is further developed to ensure that multi-UAVs can simultaneously reach the target at the specified time,and the trajectory is within the flight corridor.The trajectory tracking controller is also designed based on model predictive control to track the planned trajectory accurately.The simulation and experiment results are presented to verifying developed strategies of Multi-UAV cooperative attacks.

An Air Defense Weapon Target Assignment Method Based on Multi-Objective Artificial Bee Colony Algorithm

With the advancement of combat equipment technology and combat concepts,new requirements have been put forward for air defense operations during a group target attack.To achieve high-efficiency and lowloss defensive operations,a reasonable air defense weapon assignment strategy is a key step.In this paper,a multi-objective and multi-constraints weapon target assignment(WTA)model is established that aims to minimize the defensive resource loss,minimize total weapon consumption,and minimize the target residual effectiveness.An optimization framework of air defense weapon mission scheduling based on the multiobjective artificial bee colony(MOABC)algorithm is proposed.The solution for point-to-point saturated attack targets at different operational scales is achieved by encoding the nectar with real numbers.Simulations are performed for an imagined air defense scenario,where air defense weapons are saturated.The non-dominated solution sets are obtained by the MOABC algorithm to meet the operational demand.In the case where there are more weapons than targets,more diverse assignment schemes can be selected.According to the inverse generation distance(IGD)index,the convergence and diversity for the solutions of the non-dominated sorting genetic algorithm III(NSGA-III)algorithm and the MOABC algorithm are compared and analyzed.The results prove that the MOABC algorithm has better convergence and the solutions are more evenly distributed among the solution space.

Shuffled frog leaping algorithm with non-dominated sorting for dynamic weapon-target assignment

The dynamic weapon target assignment(DWTA)problem is of great significance in modern air combat.However,DWTA is a highly complex constrained multi-objective combinatorial optimization problem.An improved elitist non-dominated sorting genetic algorithm-II(NSGA-II)called the non-dominated shuffled frog leaping algorithm(NSFLA)is proposed to maximize damage to enemy targets and minimize the self-threat in air combat constraints.In NSFLA,the shuffled frog leaping algorithm(SFLA)is introduced to NSGA-II to replace the inside evolutionary scheme of the genetic algorithm(GA),displaying low optimization speed and heterogeneous space search defects.Two improvements have also been raised to promote the internal optimization performance of SFLA.Firstly,the local evolution scheme,a novel crossover mechanism,ensures that each individual participates in updating instead of only the worst ones,which can expand the diversity of the population.Secondly,a discrete adaptive mutation algorithm based on the function change rate is applied to balance the global and local search.Finally,the scheme is verified in various air combat scenarios.The results show that the proposed NSFLA has apparent advantages in solution quality and efficiency,especially in many aircraft and the dynamic air combat environment.

Joint target assignment and power allocation in the netted C-MIMO radar when tracking multi-targets in the presence of self-defense blanket jamming

The netted radar system(NRS)has been proved to possess unique advantages in anti-jamming and improving target tracking performance.Effective resource management can greatly ensure the combat capability of the NRS.In this paper,based on the netted collocated multiple input multiple output(CMIMO)radar,an effective joint target assignment and power allocation(JTAPA)strategy for tracking multi-targets under self-defense blanket jamming is proposed.An architecture based on the distributed fusion is used in the radar network to estimate target state parameters.By deriving the predicted conditional Cramer-Rao lower bound(PC-CRLB)based on the obtained state estimation information,the objective function is formulated.To maximize the worst case tracking accuracy,the proposed JTAPA strategy implements an online target assignment and power allocation of all active nodes,subject to some resource constraints.Since the formulated JTAPA is non-convex,we propose an efficient two-step solution strategy.In terms of the simulation results,the proposed algorithm can effectively improve tracking performance in the worst case.

An Intelligent Algorithm for Solving Weapon-Target Assignment Problem:DDPG-DNPE Algorithm

Aiming at the problems of traditional dynamic weapon-target assignment algorithms in command decisionmaking,such as large computational amount,slow solution speed,and low calculation accuracy,combined with deep reinforcement learning theory,an improved Deep Deterministic Policy Gradient algorithm with dual noise and prioritized experience replay is proposed,which uses a double noise mechanism to expand the search range of the action,and introduces a priority experience playback mechanism to effectively achieve data utilization.Finally,the algorithm is simulated and validated on the ground-to-air countermeasures digital battlefield.The results of the experiment show that,under the framework of the deep neural network for intelligent weapon-target assignment proposed in this paper,compared to the traditional RELU algorithm,the agent trained with reinforcement learning algorithms,such asDeepDeterministic Policy Gradient algorithm,Asynchronous Advantage Actor-Critic algorithm,Deep Q Network algorithm performs better.It shows that the use of deep reinforcement learning algorithms to solve the weapon-target assignment problem in the field of air defense operations is scientific.In contrast to other reinforcement learning algorithms,the agent trained by the improved Deep Deterministic Policy Gradient algorithm has a higher win rate and reward in confrontation,and the use of weapon resources is more efficient.It shows that the model and algorithm have certain superiority and rationality.The results of this paper provide new ideas for solving the problemof weapon-target assignment in air defense combat command decisions.

SiamDLA: Dynamic Label Assignment for Siamese Visual Tracking

Label assignment refers to determining positive/negative labels foreach sample to supervise the training process. Existing Siamese-based trackersprimarily use fixed label assignment strategies according to human priorknowledge;thus, they can be sensitive to predefined hyperparameters and failto fit the spatial and scale variations of samples. In this study, we first developa novel dynamic label assignment (DLA) module to handle the diverse datadistributions and adaptively distinguish the foreground from the backgroundbased on the statistical characteristics of the target in visual object tracking.The core of DLA module is a two-step selection mechanism. The first stepselects candidate samples according to the Euclidean distance between trainingsamples and ground truth, and the second step selects positive/negativesamples based on the mean and standard deviation of candidate samples.The proposed approach is general-purpose and can be easily integrated intoanchor-based and anchor-free trackers for optimal sample-label matching.According to extensive experimental findings, Siamese-based trackers withDLA modules can refine target locations and outperformbaseline trackers onOTB100, VOT2019, UAV123 and LaSOT. Particularly, DLA-SiamRPN++improves SiamRPN++ by 1% AUC and DLA-SiamCAR improves Siam-CAR by 2.5% AUC on OTB100. Furthermore, hyper-parameters analysisexperiments show that DLA module hardly increases spatio-temporal complexity,the proposed approach maintains the same speed as the originaltracker without additional overhead.

Bee Colony Optimization Algorithm for Routing and Wavelength Assignment Based on Directional Guidance in Satellite Optical Networks

With the development of satellite communication,in order to solve the problems of shortage of on-board resources and refinement of delay requirements to improve the communication performance of satellite optical networks,this paper proposes a bee colony optimization algorithm for routing and wavelength assignment based on directional guidance(DBCO-RWA)in satellite optical networks.In D-BCORWA,directional guidance based on relative position and link load is defined,and then the link cost function in the path search stage is established based on the directional guidance factor.Finally,feasible solutions are expanded in the global optimization stage.The wavelength utilization,communication success probability,blocking rate,communication hops and convergence characteristic are simulated.The results show that the performance of the proposed algorithm is improved compared with existing algorithms.

Automatic Team Assignment and Jersey Number Recognition in Football Videos

Football is one of the most-watched sports,but analyzing players’per-formance is currently difficult and labor intensive.Performance analysis is done manually,which means that someone must watch video recordings and then log each player’s performance.This includes the number of passes and shots taken by each player,the location of the action,and whether or not the play had a successful outcome.Due to the time-consuming nature of manual analyses,interest in automatic analysis tools is high despite the many interdependent phases involved,such as pitch segmentation,player and ball detection,assigning players to their teams,identifying individual players,activity recognition,etc.This paper proposes a system for developing an automatic video analysis tool for sports.The proposed system is the first to integrate multiple phases,such as segmenting the field,detecting the players and the ball,assigning players to their teams,and iden-tifying players’jersey numbers.In team assignment,this research employed unsu-pervised learning based on convolutional autoencoders(CAEs)to learn discriminative latent representations and minimize the latent embedding distance between the players on the same team while simultaneously maximizing the dis-tance between those on opposing teams.This paper also created a highly accurate approach for the real-time detection of the ball.Furthermore,it also addressed the lack of jersey number datasets by creating a new dataset with more than 6,500 images for numbers ranging from 0 to 99.Since achieving a high perfor-mance in deep learning requires a large training set,and the collected dataset was not enough,this research utilized transfer learning(TL)to first pretrain the jersey number detection model on another large dataset and then fine-tune it on the target dataset to increase the accuracy.To test the proposed system,this paper presents a comprehensive evaluation of its individual stages as well as of the sys-tem as a whole.

^(1)H and ^(13)C NMR spectral assignments for low-concentration bile acids in biological samples

Bile acids are the main body of enterohepatic circulation in vivo.They have essential functions such as emulsifying fat,bacteriostasis and regulating multiple metabolic pathways as signal molecules.However,the assignments of NMR signals for some lowconcentration bile acids are still needed.This study combined 1D nuclear magnetic resonance(NMR)and 2D NMR techniques including 1He1H correlation spectroscopy(COSY),1He1H total correlation spectroscopy(TOCSY),1H J-resolved spectroscopy(J-Res),1He13C heteronuclear single quantum coherence spectroscopy(HSQC),and 1He13C heteronuclear multiple bond correlation spectroscopy(HMBC)to assign the 1H and 13C signals of six bile acids in aqueous solution at physiological pH(~7.4)and nine bile acids in methanol.These data are of importance to the NMR-based studies on lipid digestion,absorption,and metabolism.

Bus frequency optimization in a large-scale multi-modal transportation system:integrating 3D-MFD and dynamic traffic assignment

A properly designed public transport system is expected to improve traffic efficiency.A high-frequency bus service would decrease the waiting time for passengers,but the interaction between buses and cars might result in more serious congestion.On the other hand,a low-frequency bus service would increase the waiting time for passengers and would not reduce the use of private cars.It is important to strike a balance between high and low frequencies in order to minimize the total delays for all road users.It is critical to formulate the impacts of bus frequency on congestion dynamics and mode choices.However,as far as the authors know,most proposed bus frequency optimization formulations are based on static demand and the Bureau of Public Roads function,and do not properly consider the congestion dynamics and their impacts on mode choices.To fill this gap,this paper proposes a bi-level optimization model.A three-dimensional Macroscopic Fundamental Diagram based modeling approach is developed to capture the bi-modal congestion dynamics.A variational inequality model for the user equilibrium in mode choices is presented and solved using a double projection algorithm.A surrogate model-based algorithm is used to solve the bi-level programming problem.

Hybrid Policy of Routing and Spectrum Assignment in Elastic Optical Networks

The major challenge in elastic optical networks is to determine the path of a connection and to allocate spectral resources on the links of this path. This problem consists of two sub-problems, routing and spectrum allocation. In the literature, these sub-problems are solved with a predefined order for all topology node pairs. Recent work proposes hybrid resolution algorithms based on connection demand and network state to provide a solution to these problems. However, the blocking rate of new connection requests has become problematic. In this work, we propose a hybrid routing and spectrum assignment policy to improve blocking rate of new connection requests. The proposed solution consists to change the routing policy of a pair node if the connection request is blocked. This algorithm improves the blocking rate of new connection requests.

ACS-based resource assignment and task scheduling in grid

To solve the deadlock problem of tasks that the interdependence between tasks fails to consider during the course of resource assignment and task scheduling based on the heuristics algorithm, an improved ant colony system （ACS） based algorithm is proposed. First, how to map the resource assignment and task scheduling （RATS） problem into the optimization selection problem of task resource assignment graph （TRAG） and to add the semaphore mechanism in the optimal TRAG to solve deadlocks are explained. Secondly, how to utilize the grid pheromone system model to realize the algorithm based on ACS is explicated. This refers to the construction of TRAG by the random selection of appropriate resources for each task by the user agent and the optimization of TRAG through the positive feedback and distributed parallel computing mechanism of the ACS. Simulation results show that the proposed algorithm is effective and efficient in solving the deadlock problem.

A Discrete-Time Stochastic Traffic Assignment Model

A discrete time stochastic traffic assignment model is proposed. The model provides a discrete time description of the variations of flows on a road network during a day or a peak period. The congestion effect at links and link junctions are taken into account. The first in first out principle is enforced on all links at all periods of the day. A stochastic user equilibrium assignment is achieved when the tripmaker is unable to find better travel alternatives. A computational procedure is also presented.

Survey of the research on dynamic weapon-target assignment problem

The basic concepts and models of weapon-target assignment （WTA） are introduced and the mathematical nature of the WTA models is also analyzed. A systematic survey of research on WTA problem is provided. The present research on WTA is focused on models and algorithms. In the research on models of WTA, the static WTA models are mainly studied and the dynamic WTA models are not fully studied in deed. In the research on algorithms of WTA, the intelligent algorithms are often used to solve the WTA problem. The small scale of static WTA problems has been solved very well, however, the large scale of dynamic WTA problems has not been solved effectively so far. Finally, the characteristics of dynamic WTA are analyzed and directions for the future research on dynamic WTA are discussed.