A Cognitive Model for Multi-Agent Collaboration

In multi-agent system, agents work together for solving complex tasks and reaching common goals. In this paper, we propose a cognitive model for multi-agent collaboration. Based on the cognitive model, an agent architecture will also be presented. This agent has BDI, awareness and policy driven mechanism concurrently. These approaches are integrated in one agent that will make multi-agent collaboration more practical in the real world.

A Weighted Mean Field Reinforcement Learning Algorithm for Large-Scale Multi-Agent Collaboration

Reinforcement learning has been proven to be an effective approach for solving multi-agent coordination problems in a dynamic open environment.For dealing with multi-agent cooper-ation issues,the mean field multi-agent reinforcement leaming method can better overcome the problems of slow learning speed,unstable convergent performance,and poor learning effect.However,the original mean field algorithm cannot extract features well when agents cooperate.In order to solve the large-scale multi-agent coordination problem,in this paper,the mean field multi-agent reinforcement learning algorithm is improved and optimized by combining the multi-head attention mechanism,and the attention-based mean field(MFA)structure is designed.The employment of a multi-head attention mechanism can optimize the interaction among agents,extract more effective cluster features and enable agents to learn more efficient strategies.This paper first introduces the framework structure of MFA and then expounds on the relevant theoretical basis based on the Q-Learning and Actor-Critic algorithms,and finally conducts large-scale multi-agent cooperative experiments on the MAgent platform.The ex-perimental results show that compared with the baseline algorithm,the attention-based mean field Q-learning(MFQA)and attention-based Actor-Critic(MFACA)algorithms can make large-scale multi-agent clusters converge to higher rewards,and perform better than the original mean field multi-agent algorithm.

基于Multi-Agent的无人机集群体系自主作战系统设计

针对无人集群自主作战体系设计中的关键问题,提出基于Multi-Agent的无人集群自主作战系统设计方法。建立无人集群各节点的Agent模型及其推演规则;对于仿真系统模块化和通用化的需求,设计系统互操作式接口和无人集群自主作战的交互关系;开展无人集群系统仿真推演验证。仿真结果表明,所提设计方案不仅能够有效开展并完成自主作战网络生成-集群演化-效能评估的全过程动态演示验证,而且能够通过重复随机试验进一步评估无人集群的协同作战效能,最后总结了集群协同作战的策略和经验。

UAV-Assisted Dynamic Avatar Task Migration for Vehicular Metaverse Services: A Multi-Agent Deep Reinforcement Learning Approach

Avatars, as promising digital representations and service assistants of users in Metaverses, can enable drivers and passengers to immerse themselves in 3D virtual services and spaces of UAV-assisted vehicular Metaverses. However, avatar tasks include a multitude of human-to-avatar and avatar-to-avatar interactive applications, e.g., augmented reality navigation,which consumes intensive computing resources. It is inefficient and impractical for vehicles to process avatar tasks locally. Fortunately, migrating avatar tasks to the nearest roadside units(RSU)or unmanned aerial vehicles(UAV) for execution is a promising solution to decrease computation overhead and reduce task processing latency, while the high mobility of vehicles brings challenges for vehicles to independently perform avatar migration decisions depending on current and future vehicle status. To address these challenges, in this paper, we propose a novel avatar task migration system based on multi-agent deep reinforcement learning(MADRL) to execute immersive vehicular avatar tasks dynamically. Specifically, we first formulate the problem of avatar task migration from vehicles to RSUs/UAVs as a partially observable Markov decision process that can be solved by MADRL algorithms. We then design the multi-agent proximal policy optimization(MAPPO) approach as the MADRL algorithm for the avatar task migration problem. To overcome slow convergence resulting from the curse of dimensionality and non-stationary issues caused by shared parameters in MAPPO, we further propose a transformer-based MAPPO approach via sequential decision-making models for the efficient representation of relationships among agents. Finally, to motivate terrestrial or non-terrestrial edge servers(e.g., RSUs or UAVs) to share computation resources and ensure traceability of the sharing records, we apply smart contracts and blockchain technologies to achieve secure sharing management. Numerical results demonstrate that the proposed approach outperforms the MAPPO approach by around 2% and effectively reduces approximately 20% of the latency of avatar task execution in UAV-assisted vehicular Metaverses.

Designing Proportional-Integral Consensus Protocols for Second-Order Multi-Agent Systems Using Delayed and Memorized State Information

This paper is concerned with consensus of a secondorder linear time-invariant multi-agent system in the situation that there exists a communication delay among the agents in the network.A proportional-integral consensus protocol is designed by using delayed and memorized state information.Under the proportional-integral consensus protocol,the consensus problem of the multi-agent system is transformed into the problem of asymptotic stability of the corresponding linear time-invariant time-delay system.Note that the location of the eigenvalues of the corresponding characteristic function of the linear time-invariant time-delay system not only determines the stability of the system,but also plays a critical role in the dynamic performance of the system.In this paper,based on recent results on the distribution of roots of quasi-polynomials,several necessary conditions for Hurwitz stability for a class of quasi-polynomials are first derived.Then allowable regions of consensus protocol parameters are estimated.Some necessary and sufficient conditions for determining effective protocol parameters are provided.The designed protocol can achieve consensus and improve the dynamic performance of the second-order multi-agent system.Moreover,the effects of delays on consensus of systems of harmonic oscillators/double integrators under proportional-integral consensus protocols are investigated.Furthermore,some results on proportional-integral consensus are derived for a class of high-order linear time-invariant multi-agent systems.

Finite-time Prescribed Performance Time-Varying Formation Control for Second-Order Multi-Agent Systems With Non-Strict Feedback Based on a Neural Network Observer

This paper studies the problem of time-varying formation control with finite-time prescribed performance for nonstrict feedback second-order multi-agent systems with unmeasured states and unknown nonlinearities.To eliminate nonlinearities,neural networks are applied to approximate the inherent dynamics of the system.In addition,due to the limitations of the actual working conditions,each follower agent can only obtain the locally measurable partial state information of the leader agent.To address this problem,a neural network state observer based on the leader state information is designed.Then,a finite-time prescribed performance adaptive output feedback control strategy is proposed by restricting the sliding mode surface to a prescribed region,which ensures that the closed-loop system has practical finite-time stability and that formation errors of the multi-agent systems converge to the prescribed performance bound in finite time.Finally,a numerical simulation is provided to demonstrate the practicality and effectiveness of the developed algorithm.

Safety-Constrained Multi-Agent Reinforcement Learning for Power Quality Control in Distributed Renewable Energy Networks

This paper examines the difficulties of managing distributed power systems,notably due to the increasing use of renewable energy sources,and focuses on voltage control challenges exacerbated by their variable nature in modern power grids.To tackle the unique challenges of voltage control in distributed renewable energy networks,researchers are increasingly turning towards multi-agent reinforcement learning(MARL).However,MARL raises safety concerns due to the unpredictability in agent actions during their exploration phase.This unpredictability can lead to unsafe control measures.To mitigate these safety concerns in MARL-based voltage control,our study introduces a novel approach:Safety-ConstrainedMulti-Agent Reinforcement Learning(SC-MARL).This approach incorporates a specialized safety constraint module specifically designed for voltage control within the MARL framework.This module ensures that the MARL agents carry out voltage control actions safely.The experiments demonstrate that,in the 33-buses,141-buses,and 322-buses power systems,employing SC-MARL for voltage control resulted in a reduction of the Voltage Out of Control Rate(%V.out)from0.43,0.24,and 2.95 to 0,0.01,and 0.03,respectively.Additionally,the Reactive Power Loss(Q loss)decreased from 0.095,0.547,and 0.017 to 0.062,0.452,and 0.016 in the corresponding systems.

Targeted multi-agent communication algorithm based on state control

As an important mechanism in multi-agent interaction,communication can make agents form complex team relationships rather than constitute a simple set of multiple independent agents.However,the existing communication schemes can bring much timing redundancy and irrelevant messages,which seriously affects their practical application.To solve this problem,this paper proposes a targeted multiagent communication algorithm based on state control(SCTC).The SCTC uses a gating mechanism based on state control to reduce the timing redundancy of communication between agents and determines the interaction relationship between agents and the importance weight of a communication message through a series connection of hard-and self-attention mechanisms,realizing targeted communication message processing.In addition,by minimizing the difference between the fusion message generated from a real communication message of each agent and a fusion message generated from the buffered message,the correctness of the final action choice of the agent is ensured.Our evaluation using a challenging set of Star Craft II benchmarks indicates that the SCTC can significantly improve the learning performance and reduce the communication overhead between agents,thus ensuring better cooperation between agents.

Robust and Trustworthy Data Sharing Framework Leveraging On-Chain and Off-Chain Collaboration

The proliferation of Internet of Things(IoT)systems has resulted in the generation of substantial data,presenting new challenges in reliable storage and trustworthy sharing.Conventional distributed storage systems are hindered by centralized management and lack traceability,while blockchain systems are limited by low capacity and high latency.To address these challenges,the present study investigates the reliable storage and trustworthy sharing of IoT data,and presents a novel system architecture that integrates on-chain and off-chain data manage systems.This architecture,integrating blockchain and distributed storage technologies,provides high-capacity,high-performance,traceable,and verifiable data storage and access.The on-chain system,built on Hyperledger Fabric,manages metadata,verification data,and permission information of the raw data.The off-chain system,implemented using IPFS Cluster,ensures the reliable storage and efficient access to massive files.A collaborative storage server is designed to integrate on-chain and off-chain operation interfaces,facilitating comprehensive data operations.We provide a unified access interface for user-friendly system interaction.Extensive testing validates the system’s reliability and stable performance.The proposed approach significantly enhances storage capacity compared to standalone blockchain systems.Rigorous reliability tests consistently yield positive outcomes.With average upload and download throughputs of roughly 20 and 30 MB/s,respectively,the system’s throughput surpasses the blockchain system by a factor of 4 to 18.

Computation Tree Logic Model Checking of Multi-Agent Systems Based on Fuzzy Epistemic Interpreted Systems

Model checking is an automated formal verification method to verify whether epistemic multi-agent systems adhere to property specifications.Although there is an extensive literature on qualitative properties such as safety and liveness,there is still a lack of quantitative and uncertain property verifications for these systems.In uncertain environments,agents must make judicious decisions based on subjective epistemic.To verify epistemic and measurable properties in multi-agent systems,this paper extends fuzzy computation tree logic by introducing epistemic modalities and proposing a new Fuzzy Computation Tree Logic of Knowledge(FCTLK).We represent fuzzy multi-agent systems as distributed knowledge bases with fuzzy epistemic interpreted systems.In addition,we provide a transformation algorithm from fuzzy epistemic interpreted systems to fuzzy Kripke structures,as well as transformation rules from FCTLK formulas to Fuzzy Computation Tree Logic(FCTL)formulas.Accordingly,we transform the FCTLK model checking problem into the FCTL model checking.This enables the verification of FCTLK formulas by using the fuzzy model checking algorithm of FCTL without additional computational overheads.Finally,we present correctness proofs and complexity analyses of the proposed algorithms.Additionally,we further illustrate the practical application of our approach through an example of a train control system.

Runout prediction of potential landslides based on the multi-source data collaboration analysis on historical cases

Long runout landslides involve a massive amount of energy and can be extremely hazardous owing to their long movement distance,high mobility and strong destructive power.Numerical methods have been widely used to predict the landslide runout but a fundamental problem remained is how to determine the reliable numerical parameters.This study proposes a framework to predict the runout of potential landslides through multi-source data collaboration and numerical analysis of historical landslide events.Specifically,for the historical landslide cases,the landslide-induced seismic signal,geophysical surveys,and possible in-situ drone/phone videos(multi-source data collaboration)can validate the numerical results in terms of landslide dynamics and deposit features and help calibrate the numerical(rheological)parameters.Subsequently,the calibrated numerical parameters can be used to numerically predict the runout of potential landslides in the region with a similar geological setting to the recorded events.Application of the runout prediction approach to the 2020 Jiashanying landslide in Guizhou,China gives reasonable results in comparison to the field observations.The numerical parameters are determined from the multi-source data collaboration analysis of a historical case in the region(2019 Shuicheng landslide).The proposed framework for landslide runout prediction can be of great utility for landslide risk assessment and disaster reduction in mountainous regions worldwide.

Collaboration strategy for software dynamic evolution of multi-agent system

As the ability of a single agent is limited while information and resources in multi-agent systems are distributed, cooperation is necessary for agents to accomplish a complex task. In the open and changeable environment on the Internet, it is of great significance to research a system flexible and capable in dynamic evolution that can find a collaboration method for agents which can be used in dynamic evolution process. With such a method, agents accomplish tasks for an overall target and at the same time, the collaborative relationship of agents can be adjusted with the change of environment. A method of task decomposition and collaboration of agents by improved contract net protocol is introduced. Finally, analysis on the result of the experiments is performed to verify the improved contract net protocol can greatly increase the efficiency of communication and collaboration in multi-agent system.

基于Multi-Agent在炉渣厂生产中的应用研究

随着社会的发展和科技的进步,炉渣已经成为现今较为重要的生产原料。为了进一步提高炉渣的利用率,对炉渣生产车间的调度问题的研究成为当前主要的研究方向,同时也存在着重要的理论和价值。基于上述背景,在文章中使用了Multi-Agent系统,深入地研究了炉渣生产中的作业车间调度问题,并首次对Multi-Agent系统及其在生产车间调度中的使用状况做出了简要介绍,还首次提出了一种采用Multi-Agent系统黑板模型的炉渣生产车间调度方法,并定义了生产车间的管理Agent、工作单元管理Agent、任务管理Agent、事件Agent等的生产车间调度系统模式,并给出了通过黑板模型进行的作业车间调度管理框架,为提高炉渣厂的竞争力提供理论依据。

An Optimal Control-Based Distributed Reinforcement Learning Framework for A Class of Non-Convex Objective Functionals of the Multi-Agent Network

This paper studies a novel distributed optimization problem that aims to minimize the sum of the non-convex objective functionals of the multi-agent network under privacy protection, which means that the local objective of each agent is unknown to others. The above problem involves complexity simultaneously in the time and space aspects. Yet existing works about distributed optimization mainly consider privacy protection in the space aspect where the decision variable is a vector with finite dimensions. In contrast, when the time aspect is considered in this paper, the decision variable is a continuous function concerning time. Hence, the minimization of the overall functional belongs to the calculus of variations. Traditional works usually aim to seek the optimal decision function. Due to privacy protection and non-convexity, the Euler-Lagrange equation of the proposed problem is a complicated partial differential equation.Hence, we seek the optimal decision derivative function rather than the decision function. This manner can be regarded as seeking the control input for an optimal control problem, for which we propose a centralized reinforcement learning(RL) framework. In the space aspect, we further present a distributed reinforcement learning framework to deal with the impact of privacy protection. Finally, rigorous theoretical analysis and simulation validate the effectiveness of our framework.

Distributed Adaptive Output Consensus of Unknown Heterogeneous Non-Minimum Phase Multi-Agent Systems

This article addresses the leader-following output consensus problem of heterogeneous linear multi-agent systems with unknown agent parameters under directed graphs.The dynamics of followers are allowed to be non-minimum phase with unknown arbitrary individual relative degrees.This is contrary to many existing works on distributed adaptive control schemes where agent dynamics are required to be minimum phase and often of the same relative degree.A distributed adaptive pole placement control scheme is developed,which consists of a distributed observer and an adaptive pole placement control law.It is shown that under the proposed distributed adaptive control scheme,all signals in the closed-loop system are bounded and the outputs of all the followers track the output of the leader asymptotically.The effectiveness of the proposed scheme is demonstrated by one practical example and one numerical example.

Distributed fault diagnosis observer for multi-agent system against actuator and sensor faults

Component failures can cause multi-agent system(MAS)performance degradation and even disasters,which provokes the demand of the fault diagnosis method.A distributed sliding mode observer-based fault diagnosis method for MAS is developed in presence of actuator and sensor faults.Firstly,the actuator and sensor faults are extended to the system state,and the system is transformed into a descriptor system form.Then,a sliding mode-based distributed unknown input observer is proposed to estimate the extended state.Furthermore,adaptive laws are introduced to adjust the observer parameters.Finally,the effectiveness of the proposed method is demonstrated with numerical simulations.

Deep reinforcement learning based multi-level dynamic reconfiguration for urban distribution network:a cloud-edge collaboration architecture

With the construction of the power Internet of Things(IoT),communication between smart devices in urban distribution networks has been gradually moving towards high speed,high compatibility,and low latency,which provides reliable support for reconfiguration optimization in urban distribution networks.Thus,this study proposed a deep reinforcement learning based multi-level dynamic reconfiguration method for urban distribution networks in a cloud-edge collaboration architecture to obtain a real-time optimal multi-level dynamic reconfiguration solution.First,the multi-level dynamic reconfiguration method was discussed,which included feeder-,transformer-,and substation-levels.Subsequently,the multi-agent system was combined with the cloud-edge collaboration architecture to build a deep reinforcement learning model for multi-level dynamic reconfiguration in an urban distribution network.The cloud-edge collaboration architecture can effectively support the multi-agent system to conduct“centralized training and decentralized execution”operation modes and improve the learning efficiency of the model.Thereafter,for a multi-agent system,this study adopted a combination of offline and online learning to endow the model with the ability to realize automatic optimization and updation of the strategy.In the offline learning phase,a Q-learning-based multi-agent conservative Q-learning(MACQL)algorithm was proposed to stabilize the learning results and reduce the risk of the next online learning phase.In the online learning phase,a multi-agent deep deterministic policy gradient(MADDPG)algorithm based on policy gradients was proposed to explore the action space and update the experience pool.Finally,the effectiveness of the proposed method was verified through a simulation analysis of a real-world 445-node system.

Global trends in international research collaboration, 1980-2021

Purpose:The aim of this study is to analyze the evolution of international research collaboration from 1980 to 2021.The study examines the main global patterns as well as those specific to individual countries,country groups,and different areas of research.Design/methodology/approach:The study is based on the Web of Science Core collection database.More than 50 million publications are analyzed using co-authorship data.International collaboration is defined as publications having authors affiliated with institutions located in more than one country.Findings:At the global level,the share of publications representing international collaboration has gradually increased from 4.7%in 1980 to 25.7%in 2021.The proportion of such publications within each country is higher and,in 2021,varied from less than 30%to more than 90%.There are notable disparities in the temporal trends,indicating that the process of internationalization has impacted countries in different ways.Several factors such as country size,income level,and geopolitics may explain the variance.Research limitations:Not all international research collaboration results in joint co-authored scientific publications.International co-authorship is a partial indicator of such collaboration.Another limitation is that the applied full counting method does not take into account the number of authors representing in each country in the publication.Practical implications:The study provides global averages,indicators,and concepts that can provide a useful framework of reference for further comparative studies of international research collaboration.Originality/value:Long-term macro-level studies of international collaboration are rare,and as a novelty,this study includes an analysis by the World Bank’s division of countries into four income groups.

MULTI-AGENT COLLABORATIVE DESIGN SYSTEM MODEL BASED ON THE INTERNET AND EXPERIMENTS

Development of complicated products is a project of system engineering It involves extensive and complicated knowledge,design methods and auxiliary technology Various factors affect each other So,modern product development is a typical group problem with distributed and dynamic features It is apparent superiority to solve this problem with a multi agent system representing various knowledge domains Distributed artificial intelligence knowledge being used,the multi agent collaborative design system concept and model based on Internet environment are put forward The realizing method of product developing agents,interactive process among multi agents,and organization and implementing of the design project of the multi agent collaborative design system are discussed in detail Application examples are also presented.

Monitoring Climate Change and Air Quality Project Initiative: A Case Study of the Collaboration between ECSTAR and TeroSpace in Thailand

In Thailand, the alteration of weather patterns has resulted in an increase in instances of irregular rainfall, contributing to the occurrence of droughts. The decline of water levels in dams, due to the combined effects of climate change and prolonged droughts, has had a significant impact on agricultural productivity. Drought has a profound effect on the terrestrial biosphere and the atmospheric water cycle and can also contribute to air pollution. Researchers have found a strong correlation between air pollution and drought severity. In response to this pressing issue, the Excellence Center of Space Technology and Research (ECSTAR) at King Mongkut’s Institute of Technology Ladkrabang has joined forces with TeroSpace company to launch an initiative aimed at promoting sustainable growth in Chiang Rai, a province in Thailand known for its rich biodiversity. ECSTAR and TeroSpace’s partnership on the sustainable growth initiative in Thailand’s Chiang Rai province focuses on expanding their collaboration to include international organizations such as the Centre National d’Etudes Spatiales (CNES), which will provide access to satellite imagery and climate and weather information to improve decision-making in various areas of development. CNES is a French organization in charge of space-related activities in France. The collaboration between France and Thailand for this project, in the context of the France-Thailand Year of Innovation 2023, will be crucial for the successful initiation and execution of this research project. The project aims to explore the relationship between air pollution and climate change through the deployment of air quality monitoring devices in designated locations, connected to a global data-sharing network. The results of this research will be valuable to policymakers as they consider the interplay between air pollution and climate change and make efforts to address these challenges.