Resilience Against Replay Attacks:A Distributed Model Predictive Control Scheme for Networked Multi-Agent Systems

In this paper,a resilient distributed control scheme against replay attacks for multi-agent networked systems subject to input and state constraints is proposed.The methodological starting point relies on a smart use of predictive arguments with a twofold aim:1)Promptly detect malicious agent behaviors affecting normal system operations;2)Apply specific control actions,based on predictive ideas,for mitigating as much as possible undesirable domino effects resulting from adversary operations.Specifically,the multi-agent system is topologically described by a leader-follower digraph characterized by a unique leader and set-theoretic receding horizon control ideas are exploited to develop a distributed algorithm capable to instantaneously recognize the attacked agent.Finally,numerical simulations are carried out to show benefits and effectiveness of the proposed approach.

Sufficient Condition for Average Consensus of Directed Networked Multi-agent Systems with Time Delays

The average consensus problem in a directed network of multi-agent systems with communication time delays was investigated. The directed networks were balanced and weakly connected with fixed or switching topology digraph. Based on frequency domain analysis method, a sufficient condition of asymptotic stability of multi-agent systems with time delays was obtained,where the analytic formula between the maximum time delay and the directed network structure was provided. The maximum time delay can be derived directly and easily by the eigenvalue of Laplacian L. Numerical examples confirm the effectiveness of the proposed technique.

Finite-time Prescribed Performance Time-Varying Formation Control for Second-Order Multi-Agent Systems With Non-Strict Feedback Based on a Neural Network Observer

This paper studies the problem of time-varying formation control with finite-time prescribed performance for nonstrict feedback second-order multi-agent systems with unmeasured states and unknown nonlinearities.To eliminate nonlinearities,neural networks are applied to approximate the inherent dynamics of the system.In addition,due to the limitations of the actual working conditions,each follower agent can only obtain the locally measurable partial state information of the leader agent.To address this problem,a neural network state observer based on the leader state information is designed.Then,a finite-time prescribed performance adaptive output feedback control strategy is proposed by restricting the sliding mode surface to a prescribed region,which ensures that the closed-loop system has practical finite-time stability and that formation errors of the multi-agent systems converge to the prescribed performance bound in finite time.Finally,a numerical simulation is provided to demonstrate the practicality and effectiveness of the developed algorithm.

CoopAI-Route: DRL Empowered Multi-Agent Cooperative System for Efficient QoS-Aware Routing for Network Slicing in Multi-Domain SDN

The emergence of beyond 5G networks has the potential for seamless and intelligent connectivity on a global scale.Network slicing is crucial in delivering services for different,demanding vertical applications in this context.Next-generation applications have time-sensitive requirements and depend on the most efficient routing path to ensure packets reach their intended destinations.However,the existing IP(Internet Protocol)over a multi-domain network faces challenges in enforcing network slicing due to minimal collaboration and information sharing among network operators.Conventional inter-domain routing methods,like Border Gateway Protocol(BGP),cannot make routing decisions based on performance,which frequently results in traffic flowing across congested paths that are never optimal.To address these issues,we propose CoopAI-Route,a multi-agent cooperative deep reinforcement learning(DRL)system utilizing hierarchical software-defined networks(SDN).This framework enforces network slicing in multi-domain networks and cooperative communication with various administrators to find performance-based routes in intra-and inter-domain.CoopAI-Route employs the Distributed Global Topology(DGT)algorithm to define inter-domain Quality of Service(QoS)paths.CoopAI-Route uses a DRL agent with a message-passing multi-agent Twin-Delayed Deep Deterministic Policy Gradient method to ensure optimal end-to-end routes adapted to the specific requirements of network slicing applications.Our evaluation demonstrates CoopAI-Route’s commendable performance in scalability,link failure handling,and adaptability to evolving topologies compared to state-of-the-art methods.

Safety-Constrained Multi-Agent Reinforcement Learning for Power Quality Control in Distributed Renewable Energy Networks

This paper examines the difficulties of managing distributed power systems,notably due to the increasing use of renewable energy sources,and focuses on voltage control challenges exacerbated by their variable nature in modern power grids.To tackle the unique challenges of voltage control in distributed renewable energy networks,researchers are increasingly turning towards multi-agent reinforcement learning(MARL).However,MARL raises safety concerns due to the unpredictability in agent actions during their exploration phase.This unpredictability can lead to unsafe control measures.To mitigate these safety concerns in MARL-based voltage control,our study introduces a novel approach:Safety-ConstrainedMulti-Agent Reinforcement Learning(SC-MARL).This approach incorporates a specialized safety constraint module specifically designed for voltage control within the MARL framework.This module ensures that the MARL agents carry out voltage control actions safely.The experiments demonstrate that,in the 33-buses,141-buses,and 322-buses power systems,employing SC-MARL for voltage control resulted in a reduction of the Voltage Out of Control Rate(%V.out)from0.43,0.24,and 2.95 to 0,0.01,and 0.03,respectively.Additionally,the Reactive Power Loss(Q loss)decreased from 0.095,0.547,and 0.017 to 0.062,0.452,and 0.016 in the corresponding systems.

Multilayer Satellite Network Collaborative Mobile Edge Caching:A GCN-Based Multi-Agent Approach

With the explosive growth of highdefinition video streaming data,a substantial increase in network traffic has ensued.The emergency of mobile edge caching(MEC)can not only alleviate the burden on core network,but also significantly improve user experience.Integrating with the MEC and satellite networks,the network is empowered popular content ubiquitously and seamlessly.Addressing the research gap between multilayer satellite networks and MEC,we study the caching placement problem in this paper.Initially,we introduce a three-layer distributed network caching management architecture designed for efficient and flexible handling of large-scale networks.Considering the constraint on satellite capacity and content propagation delay,the cache placement problem is then formulated and transformed into a markov decision process(MDP),where the content coded caching mechanism is utilized to promote the efficiency of content delivery.Furthermore,a new generic metric,content delivery cost,is proposed to elaborate the performance of caching decision in large-scale networks.Then,we introduce a graph convolutional network(GCN)-based multi-agent advantage actor-critic(A2C)algorithm to optimize the caching decision.Finally,extensive simulations are conducted to evaluate the proposed algorithm in terms of content delivery cost and transferability.

Consensus of networked control multi-agent systems using a double-layer encryption scheme

This paper addresses the decentralized consensus problem for a system of multiple dynamic agents with remote controllers via networking,known as a networked control multi-agent system(NCMAS).It presents a challenging scenario where partial dynamic entities or remote control units are vulnerable to disclosure attacks,making them potentially malicious.To tackle this issue,we propose a secure decentralized control design approach employing a double-layer cryptographic strategy.This approach not only ensures that the input and output information of the benign entities remains protected from the malicious entities but also practically achieves consensus performance.The paper provides an explicit design,supported by theoretical proof and numerical verification,covering stability,steady-state error,and the prevention of computation overflow or underflow.

Tracking Control of Multi-Agent Systems Using a Networked Predictive PID Tracking Scheme

With the rapid development of network technology and control technology,a networked multi-agent control system is a key direction of modern industrial control systems,such as industrial Internet systems.This paper studies the tracking control problem of networked multi-agent systems with communication constraints,where each agent has no information on the dynamics of other agents except their outputs.A networked predictive proportional integral derivative(PPID)tracking scheme is proposed to achieve the desired tracking performance,compensate actively for communication delays,and simplify implementation in a distributed manner.This scheme combines the past,present and predictive information of neighbour agents to form a tracking error signal for each agent,and applies the proportional,integral,and derivative of the agent tracking error signal to control each individual agent.The criteria of the stability and output tracking consensus of multi-agent systems with the networked PPID tracking scheme are derived through detailed analysis on the closed-loop systems.The effectiveness of the networked PPID tracking scheme is illustrated via an example.

Pinning synchronization of networked multi-agent systems: spectral analysis

Pinning synchronization of a networked multi-agent system with a directed communication topology is investigated from a spectral analysis approach. Some new types of synchronized regions for networked systems with different nonlinear agent dynamics and inner coupling structures are discovered. The eigenvalue distributions of the coupling and control matrices for different types of directed networks are obtained. The effects of the network topology, pinning density and pinning strength on the network synchronizability are examined through extensive numerical simulations. It is shown that the synchronizability of the pinned network can be effectively improved by increasing pinning density and pinning strength for some types of synchronized regions, whereas too large the pinning density and pinning strength will lead to desynchronization for other types. It is found that directed random networks are not always easier to synchronize than directed small-world networks, and a denser eigenvalue distribution may not always imply better synchronizability.

基于Multi-Agent的无人机集群体系自主作战系统设计

针对无人集群自主作战体系设计中的关键问题,提出基于Multi-Agent的无人集群自主作战系统设计方法。建立无人集群各节点的Agent模型及其推演规则;对于仿真系统模块化和通用化的需求,设计系统互操作式接口和无人集群自主作战的交互关系;开展无人集群系统仿真推演验证。仿真结果表明,所提设计方案不仅能够有效开展并完成自主作战网络生成-集群演化-效能评估的全过程动态演示验证,而且能够通过重复随机试验进一步评估无人集群的协同作战效能,最后总结了集群协同作战的策略和经验。

基于Multi-Agent的水电站变压器故障诊断系统

为了精准、快速完成水电站变压器的故障诊断,设计基于Multi-Agent的水电站变压器故障诊断系统。变压器状态监控agent将检测到的变压器故障信息发送给系统管理agent,系统管理agent通过通信agent将变压器故障信息发送给变压器故障诊断agent,变压器故障诊断agent利用小波变换方法提取变压器故障特征,并将其作为IFOA-SVM模型输入,完成变压器故障分类后,获取变压器故障诊断结果,该结果通过通信agent显示给用户。实验表明,该系统可有效诊断变压器故障诊断,诊断成功率受系统故障信息丢失率的影响较小,诊断耗时、耗能小,并具有较高故障诊断成功率。

Event-Triggered Differentially Private Average Consensus for Multi-agent Network

This paper investigates the differentially private problem of the average consensus for a class of discrete-time multi-agent network systems(MANSs). Based on the MANSs,a new distributed differentially private consensus algorithm(DPCA) is developed. To avoid continuous communication between neighboring agents, a kind of intermittent communication strategy depending on an event-triggered function is established in our DPCA. Based on our algorithm, we carry out the detailed analysis including its convergence, its accuracy, its privacy and the trade-off between the accuracy and the privacy level, respectively. It is found that our algorithm preserves the privacy of initial states of all agents in the whole process of consensus computation. The trade-off motivates us to find the best achievable accuracy of our algorithm under the free parameters and the fixed privacy level. Finally, numerical experiment results testify the validity of our theoretical analysis.

UAV-Assisted Dynamic Avatar Task Migration for Vehicular Metaverse Services: A Multi-Agent Deep Reinforcement Learning Approach

Avatars, as promising digital representations and service assistants of users in Metaverses, can enable drivers and passengers to immerse themselves in 3D virtual services and spaces of UAV-assisted vehicular Metaverses. However, avatar tasks include a multitude of human-to-avatar and avatar-to-avatar interactive applications, e.g., augmented reality navigation,which consumes intensive computing resources. It is inefficient and impractical for vehicles to process avatar tasks locally. Fortunately, migrating avatar tasks to the nearest roadside units(RSU)or unmanned aerial vehicles(UAV) for execution is a promising solution to decrease computation overhead and reduce task processing latency, while the high mobility of vehicles brings challenges for vehicles to independently perform avatar migration decisions depending on current and future vehicle status. To address these challenges, in this paper, we propose a novel avatar task migration system based on multi-agent deep reinforcement learning(MADRL) to execute immersive vehicular avatar tasks dynamically. Specifically, we first formulate the problem of avatar task migration from vehicles to RSUs/UAVs as a partially observable Markov decision process that can be solved by MADRL algorithms. We then design the multi-agent proximal policy optimization(MAPPO) approach as the MADRL algorithm for the avatar task migration problem. To overcome slow convergence resulting from the curse of dimensionality and non-stationary issues caused by shared parameters in MAPPO, we further propose a transformer-based MAPPO approach via sequential decision-making models for the efficient representation of relationships among agents. Finally, to motivate terrestrial or non-terrestrial edge servers(e.g., RSUs or UAVs) to share computation resources and ensure traceability of the sharing records, we apply smart contracts and blockchain technologies to achieve secure sharing management. Numerical results demonstrate that the proposed approach outperforms the MAPPO approach by around 2% and effectively reduces approximately 20% of the latency of avatar task execution in UAV-assisted vehicular Metaverses.

Target Tracking and Obstacle Avoidance for Multi-agent Networks with Input Constraints

In this paper, the problems of target tracking and obstacle avoidance for multi-agent networks with input constraints are investigated. When there is a moving obstacle, the control objectives are to make the agents track a moving target and to avoid collisions among agents. First, without considering the input constraints, a novel distributed controller can be obtained based on the potential function. Second, at each sampling time, the control algorithm is optimized. Furthermore, to solve the problem that agents cannot effectively avoid the obstacles in dynamic environment where the obstacles are moving, a new velocity repulsive potential is designed. One advantage of the designed control algorithm is that each agent only requires local knowledge of its neighboring agents. Finally, simulation results are provided to verify the effectiveness of the proposed approach.

Multi-agent system application in accordance with game theory in bi-directional coordination network model

The multi-agent system is the optimal solution to complex intelligent problems. In accordance with the game theory, the concept of loyalty is introduced to analyze the relationship between agents' individual income and global benefits and build the logical architecture of the multi-agent system. Besides, to verify the feasibility of the method, the cyclic neural network is optimized, the bi-directional coordination network is built as the training network for deep learning, and specific training scenes are simulated as the training background. After a certain number of training iterations, the model can learn simple strategies autonomously. Also,as the training time increases, the complexity of learning strategies rises gradually. Strategies such as obstacle avoidance, firepower distribution and collaborative cover are adopted to demonstrate the achievability of the model. The model is verified to be realizable by the examples of obstacle avoidance, fire distribution and cooperative cover. Under the same resource background, the model exhibits better convergence than other deep learning training networks, and it is not easy to fall into the local endless loop.Furthermore, the ability of the learning strategy is stronger than that of the training model based on rules, which is of great practical values.

Multi-Agent Network Intrusion Active Defense Model Based on Immune Theory

Inspired by the immune theory and multi-agent systems, an immune multi-agent active defense model for network intrusion is established. The concept of immune agent is introduced, and its running mechanism is established. The method, which uses antibody concentration to quantitatively describe the degree of intrusion danger, is presented. This model implements the multi-layer and distributed active defense mechanism for network intrusion. The experiment results show that this model is a good solution to the network security defense.

Multi-agent cooperative intrusion response in mobile adhoc networks

The nature of adhoc networks makes them vulnerable to security attacks. Many security technologies such as intrusion prevention and intrusion detection are passive in response to intrusions in that their countermea- sures are only to protect the networks, and there is no automated network-wide counteraction against detected intrusions, the architecture of cooperation intrusion response based multi-agent is propose. The architecture is composed of mobile agents. Monitor agent resides on every node and monitors its neighbor nodes. Decision agent collects information from monitor nodes and detects an intrusion by security policies. When an intruder is found in the architecture, the block agents will get to the neighbor nodes of the intruder and form the mobile firewall to isolate the intruder. In the end, we evaluate it by simulation.

Service Function Chain Deployment Algorithm Based on Multi-Agent Deep Reinforcement Learning

Aiming at the rapid growth of network services,which leads to the problems of long service request processing time and high deployment cost in the deployment of network function virtualization service function chain(SFC)under 5G networks,this paper proposes a multi-agent deep deterministic policy gradient optimization algorithm for SFC deployment(MADDPG-SD).Initially,an optimization model is devised to enhance the request acceptance rate,minimizing the latency and deploying the cost SFC is constructed for the network resource-constrained case.Subsequently,we model the dynamic problem as a Markov decision process(MDP),facilitating adaptation to the evolving states of network resources.Finally,by allocating SFCs to different agents and adopting a collaborative deployment strategy,each agent aims to maximize the request acceptance rate or minimize latency and costs.These agents learn strategies from historical data of virtual network functions in SFCs to guide server node selection,and achieve approximately optimal SFC deployment strategies through a cooperative framework of centralized training and distributed execution.Experimental simulation results indicate that the proposed method,while simultaneously meeting performance requirements and resource capacity constraints,has effectively increased the acceptance rate of requests compared to the comparative algorithms,reducing the end-to-end latency by 4.942%and the deployment cost by 8.045%.

Distributed Fault-Tolerant Containment Control for Nonlinear Multi-Agent Systems Under Directed Network Topology via Hierarchical Approach

This paper investigates the distributed fault-tolerant containment control(FTCC)problem of nonlinear multi-agent systems(MASs)under a directed network topology.The proposed control framework which is independent on the global information about the communication topology consists of two layers.Different from most existing distributed fault-tolerant control(FTC)protocols where the fault in one agent may propagate over network,the developed control method can eliminate the phenomenon of fault propagation.Based on the hierarchical control strategy,the FTCC problem with a directed graph can be simplified to the distributed containment control of the upper layer and the fault-tolerant tracking control of the lower layer.Finally,simulation results are given to demonstrate the effectiveness of the proposed control protocol.

Multi-agent reinforcement learning for edge information sharing in vehicular networks

To guarantee the heterogeneous delay requirements of the diverse vehicular services,it is necessary to design a full cooperative policy for both Vehicle to Infrastructure(V2I)and Vehicle to Vehicle(V2V)links.This paper investigates the reduction of the delay in edge information sharing for V2V links while satisfying the delay requirements of the V2I links.Specifically,a mean delay minimization problem and a maximum individual delay minimization problem are formulated to improve the global network performance and ensure the fairness of a single user,respectively.A multi-agent reinforcement learning framework is designed to solve these two problems,where a new reward function is proposed to evaluate the utilities of the two optimization objectives in a unified framework.Thereafter,a proximal policy optimization approach is proposed to enable each V2V user to learn its policy using the shared global network reward.The effectiveness of the proposed approach is finally validated by comparing the obtained results with those of the other baseline approaches through extensive simulation experiments.