基于Multi-Agent的无人机集群体系自主作战系统设计

针对无人集群自主作战体系设计中的关键问题,提出基于Multi-Agent的无人集群自主作战系统设计方法。建立无人集群各节点的Agent模型及其推演规则;对于仿真系统模块化和通用化的需求,设计系统互操作式接口和无人集群自主作战的交互关系;开展无人集群系统仿真推演验证。仿真结果表明,所提设计方案不仅能够有效开展并完成自主作战网络生成-集群演化-效能评估的全过程动态演示验证,而且能够通过重复随机试验进一步评估无人集群的协同作战效能,最后总结了集群协同作战的策略和经验。

基于Multi-Agent的水电站变压器故障诊断系统

为了精准、快速完成水电站变压器的故障诊断,设计基于Multi-Agent的水电站变压器故障诊断系统。变压器状态监控agent将检测到的变压器故障信息发送给系统管理agent,系统管理agent通过通信agent将变压器故障信息发送给变压器故障诊断agent,变压器故障诊断agent利用小波变换方法提取变压器故障特征,并将其作为IFOA-SVM模型输入,完成变压器故障分类后,获取变压器故障诊断结果,该结果通过通信agent显示给用户。实验表明,该系统可有效诊断变压器故障诊断,诊断成功率受系统故障信息丢失率的影响较小,诊断耗时、耗能小,并具有较高故障诊断成功率。

Cascaded ELM-Based Joint Frame Synchronization and Channel Estimation over Rician Fading Channel with Hardware Imperfections

Due to the interdependency of frame synchronization(FS)and channel estimation(CE),joint FS and CE(JFSCE)schemes are proposed to enhance their functionalities and therefore boost the overall performance of wireless communication systems.Although traditional JFSCE schemes alleviate the influence between FS and CE,they show deficiencies in dealing with hardware imperfection(HI)and deterministic line-of-sight(LOS)path.To tackle this challenge,we proposed a cascaded ELM-based JFSCE to alleviate the influence of HI in the scenario of the Rician fading channel.Specifically,the conventional JFSCE method is first employed to extract the initial features,and thus forms the non-Neural Network(NN)solutions for FS and CE,respectively.Then,the ELMbased networks,named FS-NET and CE-NET,are cascaded to capture the NN solutions of FS and CE.Simulation and analysis results show that,compared with the conventional JFSCE methods,the proposed cascaded ELM-based JFSCE significantly reduces the error probability of FS and the normalized mean square error(NMSE)of CE,even against the impacts of parameter variations.

UAV-Assisted Dynamic Avatar Task Migration for Vehicular Metaverse Services: A Multi-Agent Deep Reinforcement Learning Approach

Avatars, as promising digital representations and service assistants of users in Metaverses, can enable drivers and passengers to immerse themselves in 3D virtual services and spaces of UAV-assisted vehicular Metaverses. However, avatar tasks include a multitude of human-to-avatar and avatar-to-avatar interactive applications, e.g., augmented reality navigation,which consumes intensive computing resources. It is inefficient and impractical for vehicles to process avatar tasks locally. Fortunately, migrating avatar tasks to the nearest roadside units(RSU)or unmanned aerial vehicles(UAV) for execution is a promising solution to decrease computation overhead and reduce task processing latency, while the high mobility of vehicles brings challenges for vehicles to independently perform avatar migration decisions depending on current and future vehicle status. To address these challenges, in this paper, we propose a novel avatar task migration system based on multi-agent deep reinforcement learning(MADRL) to execute immersive vehicular avatar tasks dynamically. Specifically, we first formulate the problem of avatar task migration from vehicles to RSUs/UAVs as a partially observable Markov decision process that can be solved by MADRL algorithms. We then design the multi-agent proximal policy optimization(MAPPO) approach as the MADRL algorithm for the avatar task migration problem. To overcome slow convergence resulting from the curse of dimensionality and non-stationary issues caused by shared parameters in MAPPO, we further propose a transformer-based MAPPO approach via sequential decision-making models for the efficient representation of relationships among agents. Finally, to motivate terrestrial or non-terrestrial edge servers(e.g., RSUs or UAVs) to share computation resources and ensure traceability of the sharing records, we apply smart contracts and blockchain technologies to achieve secure sharing management. Numerical results demonstrate that the proposed approach outperforms the MAPPO approach by around 2% and effectively reduces approximately 20% of the latency of avatar task execution in UAV-assisted vehicular Metaverses.

Finite-time Prescribed Performance Time-Varying Formation Control for Second-Order Multi-Agent Systems With Non-Strict Feedback Based on a Neural Network Observer

This paper studies the problem of time-varying formation control with finite-time prescribed performance for nonstrict feedback second-order multi-agent systems with unmeasured states and unknown nonlinearities.To eliminate nonlinearities,neural networks are applied to approximate the inherent dynamics of the system.In addition,due to the limitations of the actual working conditions,each follower agent can only obtain the locally measurable partial state information of the leader agent.To address this problem,a neural network state observer based on the leader state information is designed.Then,a finite-time prescribed performance adaptive output feedback control strategy is proposed by restricting the sliding mode surface to a prescribed region,which ensures that the closed-loop system has practical finite-time stability and that formation errors of the multi-agent systems converge to the prescribed performance bound in finite time.Finally,a numerical simulation is provided to demonstrate the practicality and effectiveness of the developed algorithm.

Discovering Latent Variables for the Tasks With Confounders in Multi-Agent Reinforcement Learning

Efficient exploration in complex coordination tasks has been considered a challenging problem in multi-agent reinforcement learning(MARL). It is significantly more difficult for those tasks with latent variables that agents cannot directly observe. However, most of the existing latent variable discovery methods lack a clear representation of latent variables and an effective evaluation of the influence of latent variables on the agent. In this paper, we propose a new MARL algorithm based on the soft actor-critic method for complex continuous control tasks with confounders. It is called the multi-agent soft actor-critic with latent variable(MASAC-LV) algorithm, which uses variational inference theory to infer the compact latent variables representation space from a large amount of offline experience.Besides, we derive the counterfactual policy whose input has no latent variables and quantify the difference between the actual policy and the counterfactual policy via a distance function. This quantified difference is considered an intrinsic motivation that gives additional rewards based on how much the latent variable affects each agent. The proposed algorithm is evaluated on two collaboration tasks with confounders, and the experimental results demonstrate the effectiveness of MASAC-LV compared to other baseline algorithms.

A survey on multi-agent reinforcement learning and its application

Multi-agent reinforcement learning(MARL)has been a rapidly evolving field.This paper presents a comprehensive survey of MARL and its applications.We trace the historical evolution of MARL,highlight its progress,and discuss related survey works.Then,we review the existing works addressing inherent challenges and those focusing on diverse applications.Some representative stochastic games,MARL means,spatial forms of MARL,and task classification are revisited.We then conduct an in-depth exploration of a variety of challenges encountered in MARL applications.We also address critical operational aspects,such as hyperparameter tuning and computational complexity,which are pivotal in practical implementations of MARL.Afterward,we make a thorough overview of the applications of MARL to intelligent machines and devices,chemical engineering,biotechnology,healthcare,and societal issues,which highlights the extensive potential and relevance of MARL within both current and future technological contexts.Our survey also encompasses a detailed examination of benchmark environments used in MARL research,which are instrumental in evaluating MARL algorithms and demonstrate the adaptability of MARL to diverse application scenarios.In the end,we give our prospect for MARL and discuss their related techniques and potential future applications.

CoopAI-Route: DRL Empowered Multi-Agent Cooperative System for Efficient QoS-Aware Routing for Network Slicing in Multi-Domain SDN

The emergence of beyond 5G networks has the potential for seamless and intelligent connectivity on a global scale.Network slicing is crucial in delivering services for different,demanding vertical applications in this context.Next-generation applications have time-sensitive requirements and depend on the most efficient routing path to ensure packets reach their intended destinations.However,the existing IP(Internet Protocol)over a multi-domain network faces challenges in enforcing network slicing due to minimal collaboration and information sharing among network operators.Conventional inter-domain routing methods,like Border Gateway Protocol(BGP),cannot make routing decisions based on performance,which frequently results in traffic flowing across congested paths that are never optimal.To address these issues,we propose CoopAI-Route,a multi-agent cooperative deep reinforcement learning(DRL)system utilizing hierarchical software-defined networks(SDN).This framework enforces network slicing in multi-domain networks and cooperative communication with various administrators to find performance-based routes in intra-and inter-domain.CoopAI-Route employs the Distributed Global Topology(DGT)algorithm to define inter-domain Quality of Service(QoS)paths.CoopAI-Route uses a DRL agent with a message-passing multi-agent Twin-Delayed Deep Deterministic Policy Gradient method to ensure optimal end-to-end routes adapted to the specific requirements of network slicing applications.Our evaluation demonstrates CoopAI-Route’s commendable performance in scalability,link failure handling,and adaptability to evolving topologies compared to state-of-the-art methods.

Dynamics and synchronization of neural models with memristive membranes under energy coupling

Dynamical modeling of neural systems plays an important role in explaining and predicting some features of biophysical mechanisms.The electrophysiological environment inside and outside of the nerve cell is different.Due to the continuous and periodical properties of electromagnetic fields in the cell during its operation,electronic components involving two capacitors and a memristor are effective in mimicking these physical features.In this paper,a neural circuit is reconstructed by two capacitors connected by a memristor with periodical mem-conductance.It is found that the memristive neural circuit can present abundant firing patterns without stimulus.The Hamilton energy function is deduced using the Helmholtz theorem.Further,a neuronal network consisting of memristive neurons is proposed by introducing energy coupling.The controllability and flexibility of parameters give the model the ability to describe the dynamics and synchronization behavior of the system.

Transient Damping of Virtual Synchronous Generator for Enhancing Synchronization Stability during Voltage Dips

Virtual synchronous generators(VSGs)are widely introduced to the renewable power generation,the variablespeed pumped storage units,and so on,as a promising gridforming solution.It is noted that VSGs can provide virtual inertia for frequency support,but the larger inertia would worsen the synchronization stability,referring to keeping synchronization with the grid during voltage dips.Thus,this paper presents a transient damping method of VSGs for enhancing the synchronization stability during voltage dips.It is revealed that the loss of synchronization(LOS)of VSGs always accompanies with the positive frequency deviation and the damping is the key factor to remove LOS when the equilibrium point exists.In order to enhance synchronization stability during voltage dips,the transient damping is proposed,which is generated by the frequency deviation in active power loop.Additionally,the proposed method can realize seamless switching between normal state and grid fault.Moreover,detailed control design for transient damping gain is given to ensure the synchronization stability under different inertia requirements during voltage dips.Finally,the experimental results are presented to validate the analysis and the effectiveness of the improved transient damping method.

A frequency domain estimation and compensation method for system synchronization parameters of distributed-HFSWR

To analyze the influence of time synchronization error,phase synchronization error,frequency synchronization error,internal delay of the transceiver system,and range error and angle error between the unit radars on the target detection performance,firstly,a spatial detection model of distributed high-frequency surface wave radar(distributed-HFSWR)is established in this paper.In this model,a method for accurate extraction of direct wave spectrum based on curve fitting is proposed to obtain accurate system internal delay and frequency synchronization error under complex electromagnetic environment background and low signal to noise ratio(SNR),and to compensate for the shift of range and Doppler frequency caused by time-frequency synchronization error.The direct wave component is extracted from the spectrum,the range estimation error and Doppler estimation error are reduced by the method of curve fitting,and the fitting accuracy of the parameters is improved.Then,the influence of frequency synchronization error on target range and radial Doppler velocity is quantitatively analyzed.The relationship between frequency synchronization error and radial Doppler velocity shift and range shift is given.Finally,the system synchronization parameters of the trial distributed-HFSWR are obtained by the proposed spectrum extraction method based on curve fitting,the experimental data is compensated to correct the shift of the target,and finally the correct target parameter information is obtained.Simulations and experimental results demonstrate the superiority and correctness of the proposed method,theoretical derivation and detection model proposed in this paper.

Synchronization and firing mode transition of two neurons in a bilateral auditory system driven by a high–low frequency signal

The FitzHugh–Nagumo neuron circuit integrates a piezoelectric ceramic to form a piezoelectric sensing neuron,which can capture external sound signals and simulate the auditory neuron system.Two piezoelectric sensing neurons are coupled by a parallel circuit consisting of a Josephson junction and a linear resistor,and a binaural auditory system is established.Considering the non-singleness of external sound sources,the high–low frequency signal is used as the input signal to study the firing mode transition and synchronization of this system.It is found that the angular frequency of the high–low frequency signal is a key factor in determining whether the dynamic behaviors of two coupled neurons are synchronous.When they are in synchronization at a specific angular frequency,the changes in physical parameters of the input signal and the coupling strength between them will not destroy their synchronization.In addition,the firing mode of two coupled auditory neurons in synchronization is affected by the characteristic parameters of the high–low frequency signal rather than the coupling strength.The asynchronous dynamic behavior and variations in firing modes will harm the auditory system.These findings could help determine the causes of hearing loss and devise functional assistive devices for patients.

Bipartite consensus problems of Lurie multi-agent systems over signed graphs: A contraction approach

This paper examines the bipartite consensus problems for the nonlinear multi-agent systems in Lurie dynamics form with cooperative and competitive communication between different agents. Based on the contraction theory, some new conditions for the nonlinear Lurie multi-agent systems reaching bipartite leaderless consensus and bipartite tracking consensus are presented. Compared with the traditional methods, this approach degrades the dimensions of the conditions, eliminates some restrictions of the system matrix, and extends the range of the nonlinear function. Finally, two numerical examples are provided to illustrate the efficiency of our results.

Quantum synchronization with correlated baths

We study quantum synchronization under the nonequilibrium reservoirs.We consider a two-qubit XXZ chain coupled independently to their own reservoirs modeled by the collisional model.Two reservoir particles,initially prepared in a thermal state or a state with coherence,are correlated through a unitary transformation and afterward interact locally with the two quantum subsystems.We study the quantum effect of reservoir on synchronous dynamics of system.By preparing different reservoir initial states or manipulating the reservoir particles coupling and the temperature gradient,we find that quantum entanglement of reservoir is the key to control quantum synchronization of system qubits.

Hyperbolic Tangent Function-Based Protocols for Global/Semi-Global Finite-Time Consensus of Multi-Agent Systems

This paper investigates the problem of global/semi-global finite-time consensus for integrator-type multi-agent sys-tems.New hyperbolic tangent function-based protocols are pro-posed to achieve global and semi-global finite-time consensus for both single-integrator and double-integrator multi-agent systems with leaderless undirected and leader-following directed commu-nication topologies.These new protocols not only provide an explicit upper-bound estimate for the settling time,but also have a user-prescribed bounded control level.In addition,compared to some existing results based on the saturation function,the pro-posed approach considerably simplifies the protocol design and the stability analysis.Illustrative examples and an application demonstrate the effectiveness of the proposed protocols.

Designing Proportional-Integral Consensus Protocols for Second-Order Multi-Agent Systems Using Delayed and Memorized State Information

This paper is concerned with consensus of a secondorder linear time-invariant multi-agent system in the situation that there exists a communication delay among the agents in the network.A proportional-integral consensus protocol is designed by using delayed and memorized state information.Under the proportional-integral consensus protocol,the consensus problem of the multi-agent system is transformed into the problem of asymptotic stability of the corresponding linear time-invariant time-delay system.Note that the location of the eigenvalues of the corresponding characteristic function of the linear time-invariant time-delay system not only determines the stability of the system,but also plays a critical role in the dynamic performance of the system.In this paper,based on recent results on the distribution of roots of quasi-polynomials,several necessary conditions for Hurwitz stability for a class of quasi-polynomials are first derived.Then allowable regions of consensus protocol parameters are estimated.Some necessary and sufficient conditions for determining effective protocol parameters are provided.The designed protocol can achieve consensus and improve the dynamic performance of the second-order multi-agent system.Moreover,the effects of delays on consensus of systems of harmonic oscillators/double integrators under proportional-integral consensus protocols are investigated.Furthermore,some results on proportional-integral consensus are derived for a class of high-order linear time-invariant multi-agent systems.

A Synchronization Acquisition Algorithm Based on the Frequency Hopping Pulses Combining

As modern electromagnetic environments are more and more complex,the anti-interference performance of the synchronization acquisition is becoming vital in wireless communications.With the rapid development of the digital signal processing technologies,some synchronization acquisition algorithms for hybrid direct-sequence(DS)/frequency hopping(FH)spread spectrum communications have been proposed.However,these algorithms do not focus on the analysis and the design of the synchronization acquisition under typical interferences.In this paper,a synchronization acquisition algorithm based on the frequency hopping pulses combining(FHPC)is proposed.Specifically,the proposed algorithm is composed of two modules:an adaptive interference suppression(IS)module and an adaptive combining decision module.The adaptive IS module mitigates the effect of the interfered samples in the time-domain or the frequencydomain,and the adaptive combining decision module can utilize each frequency hopping pulse to construct an anti-interference decision metric and generate an adaptive acquisition decision threshold to complete the acquisition.Theory and simulation demonstrate that the proposed algorithm significantly enhances the antiinterference and anti-noise performances of the synchronization acquisition for hybrid DS/FH communications.

Dynamics and synchronization in a memristor-coupled discrete heterogeneous neuron network considering noise

Research on discrete memristor-based neural networks has received much attention.However,current research mainly focuses on memristor–based discrete homogeneous neuron networks,while memristor-coupled discrete heterogeneous neuron networks are rarely reported.In this study,a new four-stable discrete locally active memristor is proposed and its nonvolatile and locally active properties are verified by its power-off plot and DC V–I diagram.Based on two-dimensional(2D)discrete Izhikevich neuron and 2D discrete Chialvo neuron,a heterogeneous discrete neuron network is constructed by using the proposed discrete memristor as a coupling synapse connecting the two heterogeneous neurons.Considering the coupling strength as the control parameter,chaotic firing,periodic firing,and hyperchaotic firing patterns are revealed.In particular,multiple coexisting firing patterns are observed,which are induced by different initial values of the memristor.Phase synchronization between the two heterogeneous neurons is discussed and it is found that they can achieve perfect synchronous at large coupling strength.Furthermore,the effect of Gaussian white noise on synchronization behaviors is also explored.We demonstrate that the presence of noise not only leads to the transition of firing patterns,but also achieves the phase synchronization between two heterogeneous neurons under low coupling strength.

Proposal for a realtime Einstein-synchronization-defined satellite virtual clock

Realization of high performance satellite onboard clock is vital for various positioning, navigation, and timing applications. For further improvement of the synchronization-based satellite time and frequency references, we propose a geosynchronous(GEO) satellite virtual clock concept based on ground–satellite synchronization and present a beacon transponder structure for its implementation(scheduled for launch in 2025), which does not require atomic clocks to be mounted on the satellite. Its high performance relies only on minor modifications to the existing transponder structure of GEO satellites. We carefully model the carrier phase link and analyze the factors causing link asymmetry within the special relativity. Considering that performance of such synchronization-based satellite clocks is primarily limited by the link's random phase noise, which cannot be adequately modeled, we design a closed-loop experiment based on commercial GEO satellites for pre-evaluation. This experiment aims at extracting the zero-means random part of the ground-satellite Ku-band carrier phase via a feedback loop. Ultimately, we obtain a 1σ value of 0.633 ps(two-way link), following the Gaussian distribution. From this result, we conclude that the proposed real-time Einstein-synchronization-defined satellite virtual clock can achieve picosecond-level replication of onboard time and frequency.

Unleashing the Power of Multi-Agent Reinforcement Learning for Algorithmic Trading in the Digital Financial Frontier and Enterprise Information Systems

In the rapidly evolving landscape of today’s digital economy,Financial Technology(Fintech)emerges as a trans-formative force,propelled by the dynamic synergy between Artificial Intelligence(AI)and Algorithmic Trading.Our in-depth investigation delves into the intricacies of merging Multi-Agent Reinforcement Learning(MARL)and Explainable AI(XAI)within Fintech,aiming to refine Algorithmic Trading strategies.Through meticulous examination,we uncover the nuanced interactions of AI-driven agents as they collaborate and compete within the financial realm,employing sophisticated deep learning techniques to enhance the clarity and adaptability of trading decisions.These AI-infused Fintech platforms harness collective intelligence to unearth trends,mitigate risks,and provide tailored financial guidance,fostering benefits for individuals and enterprises navigating the digital landscape.Our research holds the potential to revolutionize finance,opening doors to fresh avenues for investment and asset management in the digital age.Additionally,our statistical evaluation yields encouraging results,with metrics such as Accuracy=0.85,Precision=0.88,and F1 Score=0.86,reaffirming the efficacy of our approach within Fintech and emphasizing its reliability and innovative prowess.