MADDPG-D2: An Intelligent Dynamic Task Allocation Algorithm Based on Multi-Agent Architecture Driven by Prior Knowledge

Aiming at the problems of low solution accuracy and high decision pressure when facing large-scale dynamic task allocation(DTA)and high-dimensional decision space with single agent,this paper combines the deep reinforce-ment learning(DRL)theory and an improved Multi-Agent Deep Deterministic Policy Gradient(MADDPG-D2)algorithm with a dual experience replay pool and a dual noise based on multi-agent architecture is proposed to improve the efficiency of DTA.The algorithm is based on the traditional Multi-Agent Deep Deterministic Policy Gradient(MADDPG)algorithm,and considers the introduction of a double noise mechanism to increase the action exploration space in the early stage of the algorithm,and the introduction of a double experience pool to improve the data utilization rate;at the same time,in order to accelerate the training speed and efficiency of the agents,and to solve the cold-start problem of the training,the a priori knowledge technology is applied to the training of the algorithm.Finally,the MADDPG-D2 algorithm is compared and analyzed based on the digital battlefield of ground and air confrontation.The experimental results show that the agents trained by the MADDPG-D2 algorithm have higher win rates and average rewards,can utilize the resources more reasonably,and better solve the problem of the traditional single agent algorithms facing the difficulty of solving the problem in the high-dimensional decision space.The MADDPG-D2 algorithm based on multi-agent architecture proposed in this paper has certain superiority and rationality in DTA.

Multi-Agent Architecture for the Design of WSN Applications

Complex and distributed systems are more and more associated with the application of WSN (Wireless Sensor Network) technology. The design of such applications presents important challenges and requires the assistance of several methodologies and tools. Multi-Agent systems (MAS) have been identified as one of the most suitable technologies to contribute to this domain due to their appropriateness for modeling distributed and autonomous complex systems. This work aims to contribute in the help of the design of WSN applications. The proposed architecture exploits the advantages of MAS for modeling WSN services, network topologies and sensor device architectures.

基于Multi-Agent的无人机集群体系自主作战系统设计

针对无人集群自主作战体系设计中的关键问题,提出基于Multi-Agent的无人集群自主作战系统设计方法。建立无人集群各节点的Agent模型及其推演规则;对于仿真系统模块化和通用化的需求,设计系统互操作式接口和无人集群自主作战的交互关系;开展无人集群系统仿真推演验证。仿真结果表明,所提设计方案不仅能够有效开展并完成自主作战网络生成-集群演化-效能评估的全过程动态演示验证,而且能够通过重复随机试验进一步评估无人集群的协同作战效能,最后总结了集群协同作战的策略和经验。

基于Multi-Agent的水电站变压器故障诊断系统

为了精准、快速完成水电站变压器的故障诊断,设计基于Multi-Agent的水电站变压器故障诊断系统。变压器状态监控agent将检测到的变压器故障信息发送给系统管理agent,系统管理agent通过通信agent将变压器故障信息发送给变压器故障诊断agent,变压器故障诊断agent利用小波变换方法提取变压器故障特征,并将其作为IFOA-SVM模型输入,完成变压器故障分类后,获取变压器故障诊断结果,该结果通过通信agent显示给用户。实验表明,该系统可有效诊断变压器故障诊断,诊断成功率受系统故障信息丢失率的影响较小,诊断耗时、耗能小,并具有较高故障诊断成功率。

UAV-Assisted Dynamic Avatar Task Migration for Vehicular Metaverse Services: A Multi-Agent Deep Reinforcement Learning Approach

Avatars, as promising digital representations and service assistants of users in Metaverses, can enable drivers and passengers to immerse themselves in 3D virtual services and spaces of UAV-assisted vehicular Metaverses. However, avatar tasks include a multitude of human-to-avatar and avatar-to-avatar interactive applications, e.g., augmented reality navigation,which consumes intensive computing resources. It is inefficient and impractical for vehicles to process avatar tasks locally. Fortunately, migrating avatar tasks to the nearest roadside units(RSU)or unmanned aerial vehicles(UAV) for execution is a promising solution to decrease computation overhead and reduce task processing latency, while the high mobility of vehicles brings challenges for vehicles to independently perform avatar migration decisions depending on current and future vehicle status. To address these challenges, in this paper, we propose a novel avatar task migration system based on multi-agent deep reinforcement learning(MADRL) to execute immersive vehicular avatar tasks dynamically. Specifically, we first formulate the problem of avatar task migration from vehicles to RSUs/UAVs as a partially observable Markov decision process that can be solved by MADRL algorithms. We then design the multi-agent proximal policy optimization(MAPPO) approach as the MADRL algorithm for the avatar task migration problem. To overcome slow convergence resulting from the curse of dimensionality and non-stationary issues caused by shared parameters in MAPPO, we further propose a transformer-based MAPPO approach via sequential decision-making models for the efficient representation of relationships among agents. Finally, to motivate terrestrial or non-terrestrial edge servers(e.g., RSUs or UAVs) to share computation resources and ensure traceability of the sharing records, we apply smart contracts and blockchain technologies to achieve secure sharing management. Numerical results demonstrate that the proposed approach outperforms the MAPPO approach by around 2% and effectively reduces approximately 20% of the latency of avatar task execution in UAV-assisted vehicular Metaverses.

Evolutionary Neural Architecture Search and Its Applications in Healthcare

Most of the neural network architectures are based on human experience,which requires a long and tedious trial-and-error process.Neural architecture search(NAS)attempts to detect effective architectures without human intervention.Evolutionary algorithms(EAs)for NAS can find better solutions than human-designed architectures by exploring a large search space for possible architectures.Using multiobjective EAs for NAS,optimal neural architectures that meet various performance criteria can be explored and discovered efficiently.Furthermore,hardware-accelerated NAS methods can improve the efficiency of the NAS.While existing reviews have mainly focused on different strategies to complete NAS,a few studies have explored the use of EAs for NAS.In this paper,we summarize and explore the use of EAs for NAS,as well as large-scale multiobjective optimization strategies and hardware-accelerated NAS methods.NAS performs well in healthcare applications,such as medical image analysis,classification of disease diagnosis,and health monitoring.EAs for NAS can automate the search process and optimize multiple objectives simultaneously in a given healthcare task.Deep neural network has been successfully used in healthcare,but it lacks interpretability.Medical data is highly sensitive,and privacy leaks are frequently reported in the healthcare industry.To solve these problems,in healthcare,we propose an interpretable neuroevolution framework based on federated learning to address search efficiency and privacy protection.Moreover,we also point out future research directions for evolutionary NAS.Overall,for researchers who want to use EAs to optimize NNs in healthcare,we analyze the advantages and disadvantages of doing so to provide detailed guidance,and propose an interpretable privacy-preserving framework for healthcare applications.

Exploration of Graduate Student Cultivation Mode of Landscape Architecture under the Background of“Artificial Intelligence+X”

Under the background of“artificial intelligence+X”,the development of landscape architecture industry ushers in new opportunities,and professional talents need to be updated to meet the social demand.This paper analyzes the cultivation demand of landscape architecture graduate students in the context of the new era,and identifies the problems by comparing the original professional graduate training mode.The new cultivation mode of graduate students in landscape architecture is proposed,including updating the target orientation of the discipline,optimizing the teaching system,building a“dualteacher”tutor team,and improving the“industry-university-research-utilization”integrated cultivation,so as to cultivate high-quality compound talents with disciplinary characteristics.

Discovering Latent Variables for the Tasks With Confounders in Multi-Agent Reinforcement Learning

Efficient exploration in complex coordination tasks has been considered a challenging problem in multi-agent reinforcement learning(MARL). It is significantly more difficult for those tasks with latent variables that agents cannot directly observe. However, most of the existing latent variable discovery methods lack a clear representation of latent variables and an effective evaluation of the influence of latent variables on the agent. In this paper, we propose a new MARL algorithm based on the soft actor-critic method for complex continuous control tasks with confounders. It is called the multi-agent soft actor-critic with latent variable(MASAC-LV) algorithm, which uses variational inference theory to infer the compact latent variables representation space from a large amount of offline experience.Besides, we derive the counterfactual policy whose input has no latent variables and quantify the difference between the actual policy and the counterfactual policy via a distance function. This quantified difference is considered an intrinsic motivation that gives additional rewards based on how much the latent variable affects each agent. The proposed algorithm is evaluated on two collaboration tasks with confounders, and the experimental results demonstrate the effectiveness of MASAC-LV compared to other baseline algorithms.

Intelligent architecture modeling for multilevel fluvial reservoirs based on architecture interface and elements

At present,the architecture modeling method of fluvial reservoirs are still developing.Traditional methods usually use grids to characterize architecture interbeds within the reservoir.Due to the thin thickness of this type of the interlayers,the number of the model grids must be greatly expanded.The number of grids in the tens of millions often makes an expensive computation;however,upscaling the model will generate a misleading model.The above confusion is the major reason that restricts the largescale industrialization of fluvial reservoir architecture models in oilfield development and production.Therefore,this paper explores an intelligent architecture modeling method for multilevel fluvial reservoirs based on architecture interface and element.Based on the superpositional relationship of different architectural elements within the fluvial reservoir,this method uses a combination of multilevel interface constraints and non-uniform grid techniques to build a high-resolution 3D geological model for reservoir architecture.Through the grid upscaling technology of heterogeneous architecture elements,different upscaling densities are given to the lateral-accretion bedding and lateral-accretion bodies to simplify the model gridding.This new method greatly reduces the number of model grids while ensuring the accuracy of lateral-accretion bedding models,laying a foundation for large-scale numerical simulation of the subsequent industrialization of the architecture model.This method has been validated in A layer of X oilfield with meandering fluvial channel sands as reservoirs and B layer of Y oilfield with braided river sands as reservoirs.The simulation results show that it has a higher accuracy of production history matching and remaining oil distribution forecast of the targeted sand body.The numerical simulation results show that in the actual development process of oilfield,the injected water will not displace oil in a uniform diffusive manner as traditionally assumed,but in a more complex pattern with oil in upper part of sand body being left behind as residual oil due to the influences of different levels of architecture interfaces.This investigation is important to guiding reservoir evaluation,remaining oil analysis,profile control and potential tapping and well pattern adjustment.

A survey on multi-agent reinforcement learning and its application

Multi-agent reinforcement learning(MARL)has been a rapidly evolving field.This paper presents a comprehensive survey of MARL and its applications.We trace the historical evolution of MARL,highlight its progress,and discuss related survey works.Then,we review the existing works addressing inherent challenges and those focusing on diverse applications.Some representative stochastic games,MARL means,spatial forms of MARL,and task classification are revisited.We then conduct an in-depth exploration of a variety of challenges encountered in MARL applications.We also address critical operational aspects,such as hyperparameter tuning and computational complexity,which are pivotal in practical implementations of MARL.Afterward,we make a thorough overview of the applications of MARL to intelligent machines and devices,chemical engineering,biotechnology,healthcare,and societal issues,which highlights the extensive potential and relevance of MARL within both current and future technological contexts.Our survey also encompasses a detailed examination of benchmark environments used in MARL research,which are instrumental in evaluating MARL algorithms and demonstrate the adaptability of MARL to diverse application scenarios.In the end,we give our prospect for MARL and discuss their related techniques and potential future applications.

CoopAI-Route: DRL Empowered Multi-Agent Cooperative System for Efficient QoS-Aware Routing for Network Slicing in Multi-Domain SDN

The emergence of beyond 5G networks has the potential for seamless and intelligent connectivity on a global scale.Network slicing is crucial in delivering services for different,demanding vertical applications in this context.Next-generation applications have time-sensitive requirements and depend on the most efficient routing path to ensure packets reach their intended destinations.However,the existing IP(Internet Protocol)over a multi-domain network faces challenges in enforcing network slicing due to minimal collaboration and information sharing among network operators.Conventional inter-domain routing methods,like Border Gateway Protocol(BGP),cannot make routing decisions based on performance,which frequently results in traffic flowing across congested paths that are never optimal.To address these issues,we propose CoopAI-Route,a multi-agent cooperative deep reinforcement learning(DRL)system utilizing hierarchical software-defined networks(SDN).This framework enforces network slicing in multi-domain networks and cooperative communication with various administrators to find performance-based routes in intra-and inter-domain.CoopAI-Route employs the Distributed Global Topology(DGT)algorithm to define inter-domain Quality of Service(QoS)paths.CoopAI-Route uses a DRL agent with a message-passing multi-agent Twin-Delayed Deep Deterministic Policy Gradient method to ensure optimal end-to-end routes adapted to the specific requirements of network slicing applications.Our evaluation demonstrates CoopAI-Route’s commendable performance in scalability,link failure handling,and adaptability to evolving topologies compared to state-of-the-art methods.

Long-Term Impacts of Tree Architectures and Branch Configurations on Tree Growth, Yield, Fruit Quality Attributes, and Leaf Minerals in “Aztec Fuji” Apple

Canopy and branch architectures in high-density orchards can be crucial in production and fruit quality. The influence of two canopy orientations (Upright and Tilted) in combination with two arm (branch) architectures (Shortened or Overlapped) on tree growth, yield components, fruit quality, and leaf mineral nutrients in an “Aztec Fuji” apple (Malus domestica Bork.) high-density orchard was studied over five years. Tilted trees with shortened arm configuration (TilShArm) always had significantly larger trunk cross-sectional area (TCSA) than Upright trees with an Overlapped arm configuration (UpOverArm) every year from 2012 to 2016. Trees with a TilShArm system had more cumulative fruit per tree than those with an Upright orientation. Trees with a tilted canopy (TilShArm and TilOverArm) tended to have higher yield per tree and yield per hectare than those with an upright system. Trees with a TilShArm system were more precocious and had more yield per tree than those with an upright canopy orientation in 2012. When values were polled over five years, trees with an upright canopy-shortened arm system (UpShArm) treatment had a lower biennial bearing index (BBI) than those with an upright canopy-overlapped system (UpOverArm). Trees receiving an arm shortening (UpShArm or TilShArm) configuration often had larger fruits than those with overlapped arms (UpOverArm and TilOverArm). Fruit from trees receiving an UpOverArm had higher fruit firmness than those from trees with other canopy-branch arrangements at harvest due to their smaller size. Fruit from trees with a TilShArm and TilOverArm had significantly higher water core and bitter pit but lower sunburn than trees with an upright canopy (UpShArm and UpOverArm). Leaves from trees with an UpOverArm canopy-branch configuration had the lowest leaf Ca but the highest leaf K and Fe concentrations among all treatments.

Air-Ground Collaborative Mobile Edge Computing:Architecture,Challenges,and Opportunities

By pushing computation,cache,and network control to the edge,mobile edge computing(MEC)is expected to play a leading role in fifth generation(5G)and future sixth generation(6G).Nevertheless,facing ubiquitous fast-growing computational demands,it is impossible for a single MEC paradigm to effectively support high-quality intelligent services at end user equipments(UEs).To address this issue,we propose an air-ground collaborative MEC(AGCMEC)architecture in this article.The proposed AGCMEC integrates all potentially available MEC servers within air and ground in the envisioned 6G,by a variety of collaborative ways to provide computation services at their best for UEs.Firstly,we introduce the AGC-MEC architecture and elaborate three typical use cases.Then,we discuss four main challenges in the AGC-MEC as well as their potential solutions.Next,we conduct a case study of collaborative service placement for AGC-MEC to validate the effectiveness of the proposed collaborative service placement strategy.Finally,we highlight several potential research directions of the AGC-MEC.

Bipartite consensus problems of Lurie multi-agent systems over signed graphs: A contraction approach

This paper examines the bipartite consensus problems for the nonlinear multi-agent systems in Lurie dynamics form with cooperative and competitive communication between different agents. Based on the contraction theory, some new conditions for the nonlinear Lurie multi-agent systems reaching bipartite leaderless consensus and bipartite tracking consensus are presented. Compared with the traditional methods, this approach degrades the dimensions of the conditions, eliminates some restrictions of the system matrix, and extends the range of the nonlinear function. Finally, two numerical examples are provided to illustrate the efficiency of our results.

Hyperbolic Tangent Function-Based Protocols for Global/Semi-Global Finite-Time Consensus of Multi-Agent Systems

This paper investigates the problem of global/semi-global finite-time consensus for integrator-type multi-agent sys-tems.New hyperbolic tangent function-based protocols are pro-posed to achieve global and semi-global finite-time consensus for both single-integrator and double-integrator multi-agent systems with leaderless undirected and leader-following directed commu-nication topologies.These new protocols not only provide an explicit upper-bound estimate for the settling time,but also have a user-prescribed bounded control level.In addition,compared to some existing results based on the saturation function,the pro-posed approach considerably simplifies the protocol design and the stability analysis.Illustrative examples and an application demonstrate the effectiveness of the proposed protocols.

Timing effect of high temperature exposure on the plasticity of internode and plant architecture in maize

The occurrence of high temperature(HT)in crop production is becoming more frequent and unpredictable with global warming,severely threatening food security.The state of an organ’s growth and development is largely determined by the temperature conditions it is exposed to over time.Maize is the main cereal crop,and its stem growth and plant architecture are closely related to lodging resistance,and especially sensitive to temperature.However,systematic research on the timing effect of HT on the sequentially developing internode and stem is currently lacking.To identify the timing effect of HT on the morphology and plasticity of the stem in maize,two hybrids(Zhengdan 958(ZD958),Xianyu 335(XY335))characterized by distinct morphological traits in the stem were exposed to a 7-day HT treatment from the V6 to V17 stages(Vn presents the vegetative stage with n leaves fully expanded)in 2019-2020.The results demonstrated that exposure to HT during V6-V12 accelerated the rapid elongation of stems.For instance,HT occurring at V7 and V12 specifically promoted the lengths and weights of the 3rd-5th and 9th-11th internodes,respectively.Meanwhile,HT slowed the growth of internodes adjacent to the promoted internodes.Interestingly,compared with control,the plant height was significantly increased soon after HT treatment,but the promotion effect became narrower at the subsequent flowering stage,demonstrating a self-adjusting mechanism in the maize plant in response to HT.Importantly,HT altered the plant architectures,including a rising of the ear position and increase in the ear position coefficient.XY335 exhibited greater sensitivity in stem development than ZD958 under HT treatment.These findings improve our systematic understanding of the plasticity of internode and plant architecture in response to the timing of HT exposure.

Designing Proportional-Integral Consensus Protocols for Second-Order Multi-Agent Systems Using Delayed and Memorized State Information

This paper is concerned with consensus of a secondorder linear time-invariant multi-agent system in the situation that there exists a communication delay among the agents in the network.A proportional-integral consensus protocol is designed by using delayed and memorized state information.Under the proportional-integral consensus protocol,the consensus problem of the multi-agent system is transformed into the problem of asymptotic stability of the corresponding linear time-invariant time-delay system.Note that the location of the eigenvalues of the corresponding characteristic function of the linear time-invariant time-delay system not only determines the stability of the system,but also plays a critical role in the dynamic performance of the system.In this paper,based on recent results on the distribution of roots of quasi-polynomials,several necessary conditions for Hurwitz stability for a class of quasi-polynomials are first derived.Then allowable regions of consensus protocol parameters are estimated.Some necessary and sufficient conditions for determining effective protocol parameters are provided.The designed protocol can achieve consensus and improve the dynamic performance of the second-order multi-agent system.Moreover,the effects of delays on consensus of systems of harmonic oscillators/double integrators under proportional-integral consensus protocols are investigated.Furthermore,some results on proportional-integral consensus are derived for a class of high-order linear time-invariant multi-agent systems.

Auxin-brassinosteroid crosstalk:Regulating rice plant architecture and grain shape

Rice(Oryza sativa)plant architecture and grain shape,which determine grain quality and yield,are modulatedby auxin and brassinosteroid via regulation of cell elongation and proliferation.We review the signaltransduction of these hormones and the crosstalk between their signals on the regulation of rice plantarchitecture and grain shape.

A 48-bp deletion upstream of LIGULELESS 1 alters rice panicle architecture

Panicle architecture is an agronomic determinant of crop yield and a target for cereal crop improvement.To investigate its molecular mechanisms in rice,we performed map-based cloning and characterization of OPEN PANICLE 1(OP1),a gain-of-function allele of LIGULELESS 1(LG1),controlling the spread-panicle phenotype.This allele results from a 48-bp deletion in the LG1 upstream region and promotes pulvinus development at the base of the primary branch.Increased OP1 expression and altered panicle phenotype in chimeric transgenic plants and upstream-region knockout mutants indicated that the deletion regulates spread-panicle architecture in the mutant spread panicle 1(sp1).Knocking out BRASSINOSTEROID UPREGULATED1(BU1)gene in the background of OP1 complementary plants resulted in compact panicles,suggesting OP1 may regulate inflorescence architecture via the brassinosteroid signaling pathway.We regard that manipulating the upstream regulatory region of OP1 or genes involved in BR signal pathway could be an efficient way to improve rice inflorescence architecture.

Apical meristem transcriptome analysis identifies a role for the blue light receptor gene GhFKF1 in cotton architecture development

Cotton architecture is determined by the differentiation fate transition of axillary meristem(AM),and influences cotton yield and the efficiency of mechanized harvesting.We observed that the initiation of flowering primordium was earlier in early-maturing than that in late-maturing cultivars during the differentiation and development of AM.The RNA-Seq and expression level analyses showed that genes FLAVIN BINDING,KELCH REPEAT,F-BOX1(GhFKF1),and GIGANTEA(GhGI)were in response to circadian rhythms,and involved in the regulation of cotton flowering.The gene structure,predicted protein structure,and motif content analyses showed that in Arabidopsis,cotton,rapseed,and soybean,proteins GhFKF1 and GhGI were functionally conserved and share evolutionary origins.Compared to the wild type,in GhFKF1 mutants that were created by the CRISPR/Cas9 system,the initiation of branch primordium was inhibited.Conversely,the knocking out of GhGI increased the number of AM differentiating into flower primordium,and there were much more lateral branch differentiation and development.Besides,we investigated that proteins GhFKF1 and GhGI can interact with each other.These results suggest that GhFKF1 and GhGI are key regulators of cotton architecture development,and may collaborate to regulate the differentiation fate transition of AM,ultimately influencing plant architecture.We describe a strategy for using the CRISPR/Cas9 system to increase cotton adaptation and productivity by optimizing plant architecture.