Target Tracking and Obstacle Avoidance for Multi-agent Networks with Input Constraints

In this paper, the problems of target tracking and obstacle avoidance for multi-agent networks with input constraints are investigated. When there is a moving obstacle, the control objectives are to make the agents track a moving target and to avoid collisions among agents. First, without considering the input constraints, a novel distributed controller can be obtained based on the potential function. Second, at each sampling time, the control algorithm is optimized. Furthermore, to solve the problem that agents cannot effectively avoid the obstacles in dynamic environment where the obstacles are moving, a new velocity repulsive potential is designed. One advantage of the designed control algorithm is that each agent only requires local knowledge of its neighboring agents. Finally, simulation results are provided to verify the effectiveness of the proposed approach.

Leader-following formation control of multi-agent networks based on distributed observers

To investigate the leader-following formation control, in this paper we present the design problem of control protocols and distributed observers under which the agents can achieve and maintain the desired formation from any initial states, while the velocity converges to that of the virtual leader whose velocity cannot be measured by agents in real time. The two cases of switching topologies without communication delay and fixed topology with time-varying communication delay are both considered for multi-agent networks. By using the Lyapunov stability theory, the issue of stability is analysed for multi-agent systems with switching topologies. Then, by considering the time-varying communication delay, the sufficient condition is proposed for the multi-agent systems with fixed topology. Finally, two numerical examples are given to illustrate the effectiveness of the proposed leader-following formation control protocols.

Pinning consensus analysis of multi-agent networks with arbitrary topology

In this paper the pinning consensus of multi-agent networks with arbitrary topology is investigated. Based on the properties of M-matrix, some criteria of pinning consensus are established for the continuous multi-agent network and the results show that the pinning consensus of the dynamical system depends on the smallest real part of the eigenvalue of the matrix which is composed of the Laplacian matrix of the multi-agent network and the pinning control gains. Meanwhile, the relevant work for the discrete-time system is studied and the corresponding criterion is also obtained. Particularly, the fundamental problem of pinning consensus, that is, what kind of node should be pinned, is investigated and the positive answers to this question are presented. Finally, the correctness of our theoretical findings is demonstrated by some numerical simulated examples.

Collective surrounding control in multi-agent networks

We study the collective encirclement control problem of a multi-agent system in this paper, where the agents move collectively to encircle the multiple targets. An algorithm is proposed and a sufficient condition is derived to realize the collective encirclement motion. Some simulation results are provided to show the effectiveness of the obtained theoretical results.

Rigidity based formation tracking for multi-agent networks

This paper considers the formation tracking problem under a rigidity framework, where the target formation is specified as a minimally and infinitesimally rigid formation and the desired velocity of the group is available to only a subset of the agents. The following two cases are considered： the desired velocity is constant, and the desired velocity is timevarying. In the first case, a distributed linear estimator is constructed for each agent to estimate the desired velocity. The velocity estimation and a formation acquisition term are employed to design the control inputs for the agents, where the rigidity matrix plays a central role. In the second case, a distributed non-smooth estimator is constructed to estimate the time-varying velocity, which is shown to converge in a finite time. Theoretical analysis shows that the formation tracking problem can be solved under the proposed control algorithms and estimators. Simulation results are also provided to show the validity of the derived results.

Safety-Constrained Multi-Agent Reinforcement Learning for Power Quality Control in Distributed Renewable Energy Networks

This paper examines the difficulties of managing distributed power systems,notably due to the increasing use of renewable energy sources,and focuses on voltage control challenges exacerbated by their variable nature in modern power grids.To tackle the unique challenges of voltage control in distributed renewable energy networks,researchers are increasingly turning towards multi-agent reinforcement learning(MARL).However,MARL raises safety concerns due to the unpredictability in agent actions during their exploration phase.This unpredictability can lead to unsafe control measures.To mitigate these safety concerns in MARL-based voltage control,our study introduces a novel approach:Safety-ConstrainedMulti-Agent Reinforcement Learning(SC-MARL).This approach incorporates a specialized safety constraint module specifically designed for voltage control within the MARL framework.This module ensures that the MARL agents carry out voltage control actions safely.The experiments demonstrate that,in the 33-buses,141-buses,and 322-buses power systems,employing SC-MARL for voltage control resulted in a reduction of the Voltage Out of Control Rate(%V.out)from0.43,0.24,and 2.95 to 0,0.01,and 0.03,respectively.Additionally,the Reactive Power Loss(Q loss)decreased from 0.095,0.547,and 0.017 to 0.062,0.452,and 0.016 in the corresponding systems.

Finite-time Prescribed Performance Time-Varying Formation Control for Second-Order Multi-Agent Systems With Non-Strict Feedback Based on a Neural Network Observer

This paper studies the problem of time-varying formation control with finite-time prescribed performance for nonstrict feedback second-order multi-agent systems with unmeasured states and unknown nonlinearities.To eliminate nonlinearities,neural networks are applied to approximate the inherent dynamics of the system.In addition,due to the limitations of the actual working conditions,each follower agent can only obtain the locally measurable partial state information of the leader agent.To address this problem,a neural network state observer based on the leader state information is designed.Then,a finite-time prescribed performance adaptive output feedback control strategy is proposed by restricting the sliding mode surface to a prescribed region,which ensures that the closed-loop system has practical finite-time stability and that formation errors of the multi-agent systems converge to the prescribed performance bound in finite time.Finally,a numerical simulation is provided to demonstrate the practicality and effectiveness of the developed algorithm.

A Transmission Design in Dynamic Heterogeneous V2V Networks Through Multi-Agent Deep Reinforcement Learning

In highly dynamic and heterogeneous vehicular communication networks,it is challenging to efficiently utilize network resources and ensure demanding performance requirements of safetyrelated applications.This paper investigates machinelearning-assisted transmission design in a typical multi-user vehicle-to-vehicle(V2V)communication scenario.The transmission process proceeds sequentially along the discrete time steps,where several source nodes intend to deliver multiple different types of messages to their respective destinations within the same spectrum.Due to rapid movement of vehicles,real-time acquirement of channel knowledge and central coordination of all transmission actions are in general hard to realize.We consider applying multi-agent deep reinforcement learning(MADRL)to handle this issue.By transforming the transmission design problem into a stochastic game,a multi-agent proximal policy optimization(MAPPO)algorithm under a centralized training and decentralized execution framework is proposed such that each source decides its own transmission message type,power level,and data rate,based on local observations of the environment and feedback,to maximize its energy efficiency.Via simulations we show that our method achieves better performance over conventional methods.

Multi-Agent Deep Reinforcement Learning for Cross-Layer Scheduling in Mobile Ad-Hoc Networks

Due to the fading characteristics of wireless channels and the burstiness of data traffic,how to deal with congestion in Ad-hoc networks with effective algorithms is still open and challenging.In this paper,we focus on enabling congestion control to minimize network transmission delays through flexible power control.To effectively solve the congestion problem,we propose a distributed cross-layer scheduling algorithm,which is empowered by graph-based multi-agent deep reinforcement learning.The transmit power is adaptively adjusted in real-time by our algorithm based only on local information(i.e.,channel state information and queue length)and local communication(i.e.,information exchanged with neighbors).Moreover,the training complexity of the algorithm is low due to the regional cooperation based on the graph attention network.In the evaluation,we show that our algorithm can reduce the transmission delay of data flow under severe signal interference and drastically changing channel states,and demonstrate the adaptability and stability in different topologies.The method is general and can be extended to various types of topologies.

基于Multi-Agent的无人机集群体系自主作战系统设计

针对无人集群自主作战体系设计中的关键问题,提出基于Multi-Agent的无人集群自主作战系统设计方法。建立无人集群各节点的Agent模型及其推演规则;对于仿真系统模块化和通用化的需求,设计系统互操作式接口和无人集群自主作战的交互关系;开展无人集群系统仿真推演验证。仿真结果表明,所提设计方案不仅能够有效开展并完成自主作战网络生成-集群演化-效能评估的全过程动态演示验证,而且能够通过重复随机试验进一步评估无人集群的协同作战效能,最后总结了集群协同作战的策略和经验。

基于Multi-Agent的水电站变压器故障诊断系统

为了精准、快速完成水电站变压器的故障诊断,设计基于Multi-Agent的水电站变压器故障诊断系统。变压器状态监控agent将检测到的变压器故障信息发送给系统管理agent,系统管理agent通过通信agent将变压器故障信息发送给变压器故障诊断agent,变压器故障诊断agent利用小波变换方法提取变压器故障特征,并将其作为IFOA-SVM模型输入,完成变压器故障分类后,获取变压器故障诊断结果,该结果通过通信agent显示给用户。实验表明,该系统可有效诊断变压器故障诊断,诊断成功率受系统故障信息丢失率的影响较小,诊断耗时、耗能小,并具有较高故障诊断成功率。

UAV-Assisted Dynamic Avatar Task Migration for Vehicular Metaverse Services: A Multi-Agent Deep Reinforcement Learning Approach

Avatars, as promising digital representations and service assistants of users in Metaverses, can enable drivers and passengers to immerse themselves in 3D virtual services and spaces of UAV-assisted vehicular Metaverses. However, avatar tasks include a multitude of human-to-avatar and avatar-to-avatar interactive applications, e.g., augmented reality navigation,which consumes intensive computing resources. It is inefficient and impractical for vehicles to process avatar tasks locally. Fortunately, migrating avatar tasks to the nearest roadside units(RSU)or unmanned aerial vehicles(UAV) for execution is a promising solution to decrease computation overhead and reduce task processing latency, while the high mobility of vehicles brings challenges for vehicles to independently perform avatar migration decisions depending on current and future vehicle status. To address these challenges, in this paper, we propose a novel avatar task migration system based on multi-agent deep reinforcement learning(MADRL) to execute immersive vehicular avatar tasks dynamically. Specifically, we first formulate the problem of avatar task migration from vehicles to RSUs/UAVs as a partially observable Markov decision process that can be solved by MADRL algorithms. We then design the multi-agent proximal policy optimization(MAPPO) approach as the MADRL algorithm for the avatar task migration problem. To overcome slow convergence resulting from the curse of dimensionality and non-stationary issues caused by shared parameters in MAPPO, we further propose a transformer-based MAPPO approach via sequential decision-making models for the efficient representation of relationships among agents. Finally, to motivate terrestrial or non-terrestrial edge servers(e.g., RSUs or UAVs) to share computation resources and ensure traceability of the sharing records, we apply smart contracts and blockchain technologies to achieve secure sharing management. Numerical results demonstrate that the proposed approach outperforms the MAPPO approach by around 2% and effectively reduces approximately 20% of the latency of avatar task execution in UAV-assisted vehicular Metaverses.

Pluggable multitask diffractive neural networks based on cascaded metasurfaces

Optical neural networks have significant advantages in terms of power consumption,parallelism,and high computing speed,which has intrigued extensive attention in both academic and engineering communities.It has been considered as one of the powerful tools in promoting the fields of imaging processing and object recognition.However,the existing optical system architecture cannot be reconstructed to the realization of multi-functional artificial intelligence systems simultaneously.To push the development of this issue,we propose the pluggable diffractive neural networks(P-DNN),a general paradigm resorting to the cascaded metasurfaces,which can be applied to recognize various tasks by switching internal plug-ins.As the proof-of-principle,the recognition functions of six types of handwritten digits and six types of fashions are numerical simulated and experimental demonstrated at near-infrared regimes.Encouragingly,the proposed paradigm not only improves the flexibility of the optical neural networks but paves the new route for achieving high-speed,low-power and versatile artificial intelligence systems.

Activation Redistribution Based Hybrid Asymmetric Quantization Method of Neural Networks

The demand for adopting neural networks in resource-constrained embedded devices is continuously increasing.Quantization is one of the most promising solutions to reduce computational cost and memory storage on embedded devices.In order to reduce the complexity and overhead of deploying neural networks on Integeronly hardware,most current quantization methods use a symmetric quantization mapping strategy to quantize a floating-point neural network into an integer network.However,although symmetric quantization has the advantage of easier implementation,it is sub-optimal for cases where the range could be skewed and not symmetric.This often comes at the cost of lower accuracy.This paper proposed an activation redistribution-based hybrid asymmetric quantizationmethod for neural networks.The proposedmethod takes data distribution into consideration and can resolve the contradiction between the quantization accuracy and the ease of implementation,balance the trade-off between clipping range and quantization resolution,and thus improve the accuracy of the quantized neural network.The experimental results indicate that the accuracy of the proposed method is 2.02%and 5.52%higher than the traditional symmetric quantization method for classification and detection tasks,respectively.The proposed method paves the way for computationally intensive neural network models to be deployed on devices with limited computing resources.Codes will be available on https://github.com/ycjcy/Hybrid-Asymmetric-Quantization.

Biodiversity metrics on ecological networks: Demonstrated with animal gastrointestinal microbiomes

Biodiversity has become a terminology familiar to virtually every citizen in modern societies.It is said that ecology studies the economy of nature,and economy studies the ecology of humans;then measuring biodiversity should be similar with measuring national wealth.Indeed,there have been many parallels between ecology and economics,actually beyond analogies.For example,arguably the second most widely used biodiversity metric,Simpson(1949)’s diversity index,is a function of familiar Gini-index in economics.One of the biggest challenges has been the high“diversity”of diversity indexes due to their excessive“speciation”-there are so many indexes,similar to each country’s sovereign currency-leaving confused diversity practitioners in dilemma.In 1973,Hill introduced the concept of“numbers equivalent”,which is based on Renyi entropy and originated in economics,but possibly due to his abstruse interpretation of the concept,his message was not widely received by ecologists until nearly four decades later.What Hill suggested was similar to link the US dollar to gold at the rate of$35 per ounce under the Bretton Woods system.The Hill numbers now are considered most appropriate biodiversity metrics system,unifying Shannon,Simpson and other diversity indexes.Here,we approach to another paradigmatic shift-measuring biodiversity on ecological networks-demonstrated with animal gastrointestinal microbiomes representing four major invertebrate classes and all six vertebrate classes.The network diversity can reveal the diversity of species interactions,which is a necessary step for understanding the spatial and temporal structures and dynamics of biodiversity across environmental gradients.

A data-driven model of drop size prediction based on artificial neural networks using small-scale data sets

An artificial neural network(ANN)method is introduced to predict drop size in two kinds of pulsed columns with small-scale data sets.After training,the deviation between calculate and experimental results are 3.8%and 9.3%,respectively.Through ANN model,the influence of interfacial tension and pulsation intensity on the droplet diameter has been developed.Droplet size gradually increases with the increase of interfacial tension,and decreases with the increase of pulse intensity.It can be seen that the accuracy of ANN model in predicting droplet size outside the training set range is reach the same level as the accuracy of correlation obtained based on experiments within this range.For two kinds of columns,the drop size prediction deviations of ANN model are 9.6%and 18.5%and the deviations in correlations are 11%and 15%.

Discovering Latent Variables for the Tasks With Confounders in Multi-Agent Reinforcement Learning

Efficient exploration in complex coordination tasks has been considered a challenging problem in multi-agent reinforcement learning(MARL). It is significantly more difficult for those tasks with latent variables that agents cannot directly observe. However, most of the existing latent variable discovery methods lack a clear representation of latent variables and an effective evaluation of the influence of latent variables on the agent. In this paper, we propose a new MARL algorithm based on the soft actor-critic method for complex continuous control tasks with confounders. It is called the multi-agent soft actor-critic with latent variable(MASAC-LV) algorithm, which uses variational inference theory to infer the compact latent variables representation space from a large amount of offline experience.Besides, we derive the counterfactual policy whose input has no latent variables and quantify the difference between the actual policy and the counterfactual policy via a distance function. This quantified difference is considered an intrinsic motivation that gives additional rewards based on how much the latent variable affects each agent. The proposed algorithm is evaluated on two collaboration tasks with confounders, and the experimental results demonstrate the effectiveness of MASAC-LV compared to other baseline algorithms.

A survey on multi-agent reinforcement learning and its application

Multi-agent reinforcement learning(MARL)has been a rapidly evolving field.This paper presents a comprehensive survey of MARL and its applications.We trace the historical evolution of MARL,highlight its progress,and discuss related survey works.Then,we review the existing works addressing inherent challenges and those focusing on diverse applications.Some representative stochastic games,MARL means,spatial forms of MARL,and task classification are revisited.We then conduct an in-depth exploration of a variety of challenges encountered in MARL applications.We also address critical operational aspects,such as hyperparameter tuning and computational complexity,which are pivotal in practical implementations of MARL.Afterward,we make a thorough overview of the applications of MARL to intelligent machines and devices,chemical engineering,biotechnology,healthcare,and societal issues,which highlights the extensive potential and relevance of MARL within both current and future technological contexts.Our survey also encompasses a detailed examination of benchmark environments used in MARL research,which are instrumental in evaluating MARL algorithms and demonstrate the adaptability of MARL to diverse application scenarios.In the end,we give our prospect for MARL and discuss their related techniques and potential future applications.

Influencer identification of dynamical networks based on an information entropy dimension reduction method

Identifying critical nodes or sets in large-scale networks is a fundamental scientific problem and one of the key research directions in the fields of data mining and network science when implementing network attacks, defense, repair and control.Traditional methods usually begin from the centrality, node location or the impact on the largest connected component after node destruction, mainly based on the network structure.However, these algorithms do not consider network state changes.We applied a model that combines a random connectivity matrix and minimal low-dimensional structures to represent network connectivity.By using mean field theory and information entropy to calculate node activity,we calculated the overlap between the random parts and fixed low-dimensional parts to quantify the influence of node impact on network state changes and ranked them by importance.We applied this algorithm and the proposed importance algorithm to the overall analysis and stratified analysis of the C.elegans neural network.We observed a change in the critical entropy of the network state and by utilizing the proposed method we can calculate the nodes that indirectly affect muscle cells through neural layers.

Verifying hierarchical network nonlocality in general quantum networks

Recently, a class of innovative notions on quantum network nonlocality(QNN), called full quantum network nonlocality(FQNN), have been proposed in Phys. Rev. Lett. 128 010403(2022). As the generalization of full network nonlocality(FNN), l-level quantum network nonlocality(l-QNN) was defined in arxiv. 2306.15717 quant-ph(2024). FQNN is a NN that can be generated only from a network with all sources being non-classical. This is beyond the existing standard network nonlocality, which may be generated from a network with only a non-classical source. One of the challenging tasks is to establish corresponding Bell-like inequalities to demonstrate the FQNN or l-QNN. Up to now, the inequality criteria for FQNN and l-QNN have only been established for star and chain networks. In this paper, we devote ourselves to establishing Bell-like inequalities for networks with more complex structures. Note that star and chain networks are special kinds of tree-shaped networks. We first establish the Bell-like inequalities for verifying l-QNN in k-forked tree-shaped networks. Such results generalize the existing inequalities for star and chain networks. Furthermore, we find the Bell-like inequality criteria for l-QNN for general acyclic and cyclic networks. Finally, we discuss the demonstration of l-QNN in the well-known butterfly networks.