A Multi-Agent System for Environmental Monitoring Using Boolean Networks and Reinforcement Learning

Distributed wireless sensor networks have been shown to be effective for environmental monitoring tasks,in which multiple sensors are deployed in a wide range of the environments to collect information or monitor a particular event,Wireless sensor networks,consisting of a large number of interacting sensors,have been successful in a variety of applications where they are able to share information using different transmission protocols through the communication network.However,the irregular and dynamic environment requires traditional wireless sensor networks to have frequent communications to exchange the most recent information,which can easily generate high communication cost through the collaborative data collection and data transmission.High frequency communication also has high probability of failure because of long distance data transmission.In this paper,we developed a novel approach to multi-sensor environment monitoring network using the idea of distributed system.Its communication network can overcome the difficulties of high communication cost and Single Point of Failure(SPOF)through the decentralized approach,which performs in-network computation.Our approach makes use of Boolean networks that allows for a non-complex method of corroboration and retains meaningful information regarding the dynamics of the communication network.Our approach also reduces the complexity of data aggregation process and employee a reinforcement learning algorithm to predict future event inside the environment through the pattern recognition.

COLLISION AVOIDANCE DECISION-MAKING MODEL OF MULTI-AGENTS IN VIRTUAL DRIVING ENVIRONMENT WITH ANALYTIC HIERARCHY PROCESS

Collision avoidance decision-making models of multiple agents in virtual driving environment are studied. Based on the behavioral characteristics and hierarchical structure of the collision avoidance decision-making in real life driving, delphi approach and mathematical statistics method are introduced to construct pair-wise comparison judgment matrix of collision avoidance decision choices to each collision situation. Analytic hierarchy process （AHP） is adopted to establish the agents＇ collision avoidance decision-making model. To simulate drivers＇ characteristics, driver factors are added to categorize driving modes into impatient mode, normal mode, and the cautious mode. The results show that this model can simulate human＇s thinking process, and the agents in the virtual environment can deal with collision situations and make decisions to avoid collisions without intervention. The model can also reflect diversity and uncertainly of real life driving behaviors, and solves the multi-objective, multi-choice ranking priority problem in multi-vehicle collision scenarios. This collision avoidance model of multi-agents model is feasible and effective, and can provide richer and closer-to-life virtual scene for driving simulator, reflecting real-life traffic environment more truly, this model can also promote the practicality of driving simulator.

Univector field method-based multi-agent navigation for pursuit problem in obstacle environments

The pursuit problem is a well-known problem in computer science. In this problem, a group of predator agents attempt to capture a prey agent in an environment with various obstacle types, partial observation, and an infinite grid-world. Predator agents are applied algorithms that use the univector field method to reach the prey agent, strategies for avoiding obstacles and strategies for cooperation between predator agents. Obstacle avoidance strategies are generalized and presented through strategies called hitting and following boundary(HFB); trapped and following shortest path(TFSP); and predicted and following shortest path(PFSP). In terms of cooperation, cooperation strategies are employed to more quickly reach and capture the prey agent. Experimental results are shown to illustrate the efficiency of the method in the pursuit problem.

Stochastic bounded consensus of second-order multi-agent systems in noisy environment

This paper investigates the stochastic bounded consensus of leader-following second-order multi-agent systems in a noisy environment. It is assumed that each agent received the information of its neighbors corrupted by noises and time delays. Based on the graph theory, stochastic tools, and the Lyapunov function method, we derive the sufficient conditions under which the systems would reach stochastic bounded consensus in mean square with the protocol we designed. Finally, a numerical simulation is illustrated to check the effectiveness of the proposed algorithms.

Multi-agent Blind Model and Its Application to Regional Eco-environmental Quality Assessment

Through denoting each expert as an agent and viewing a multiple criteria decision-making as a synthesis problem of aggregating experts＇ ratings, a multi-agent blind model （MABM） is developed for regional eco-environmental quality assessment. In this model, the ratings of the evaluated object under an index, given by expert group, are first utilized to construct a series of blind numbers. In general, each index will correspond to different blind numbers. On the basis of aggregating index weights, the rank score in the form of a blind number is obtained for the evaluated object. Then, by means of calculating expected value of the above blind number, its rank score is further converted into a crisp value. By way of comparing the expected value with classification standards, eco-environmental quality of the evaluated sample could he identified successfully in the end. As a case, the MABM is used to evaluate the eco-environmental quality of Chaohu Lake basin. Study result shows that the MABM is a useful model for regional eco-environmental quality assessment.

Multi-agent-based Approach for Determination of Time-quota in Integrated Environment

Time-quota is one of important factors in producti on system. It is affected by various factors. time-quota is studied in CAPP and p roduction schedule integration environment in this paper. An agent-based time- quota method is put forward and the structure model is established by means of i ntelligent agent in integrated environment. The method can map the influencing t ime-quota factors into part agent related to process state and machine method a gent, resorting to the function of agent rule-based reasoning, the agents can t ransform these factors into data mode that artificial neural network (ANN) can a ccept and recognize. As a tool, ANN agent can calculate time-quota quickly. A b lackboard method is used as the means of communication and collaborative control between agents. The experiments show that precise process time-quota can be obtained rapidly with proper samples selected, continuous self-study and self -organization in system, and multi-agent approach is an effective method for d etermination of time-quota.

Human Centered Design in Multi-agent Virtual Environment

The efficient and reliable human centered design of products and processes is a major goal in manufacturing industries for numerous human factors must be taken into account during the entire life cycle of products. A multi-agents intelligent design system is presented for manufacturing process simulation and products＇ ergonomic analysis. In virtual design environment, the virtual human with high-level intelligence performs tasks＇ operation autonomously and shows optimum posture configuration with ergonomic assessment results in real time. The functions are realized by intelligent agents architecture based on a modem approach derived from fuzzy multi-objects decision-making theory. A case study is presented to demonstrate the feasibility of the suggested methodology.

Load Balancing Based on Multi-Agent Framework to Enhance Cloud Environment

According to the advances in users’service requirements,physical hardware accessibility,and speed of resource delivery,Cloud Computing(CC)is an essential technology to be used in many fields.Moreover,the Internet of Things(IoT)is employed for more communication flexibility and richness that are required to obtain fruitful services.A multi-agent system might be a proper solution to control the load balancing of interaction and communication among agents.This paper proposes a multi-agent load balancing framework that consists of two phases to optimize the workload among different servers with large-scale CC power with various utilities and a significant number of IoT devices with low resources.Different agents are integrated based on relevant features of behavioral interaction using classification techniques to balance the workload.Aload balancing algorithm is developed to serve users’requests to improve the solution of workload problems with an efficient distribution.The activity task from IoT devices has been classified by feature selection methods in the preparatory phase to optimize the scalability ofCC.Then,the server’s availability is checked and the classified task is assigned to its suitable server in the main phase to enhance the cloud environment performance.Multi-agent load balancing framework is succeeded to cope with the importance of using large-scale requirements of CC and(low resources and large number)of IoT.

A Multi-Agent Resource Selection Model in Manufacturing Grid Environment

A group of agents are intimately cooperated to set the assessment indices, establish the weight of each index in overall result of evaluation, collect the experts＇ scores given to each available resource, and the manufacturing resource whose overall assessment value is highest is taken as the optimal choice. Architecture of the proposed system is outlined and an example is offered to show the process of accomplishing the assessment.

Nonlinear Control for Multi-agent Formations with Delays in Noisy Environments

In this paper, we investigate the nonlinear control problem for multi-agent formations with communication delays in noisy environments and in directed interconnection topologies. A stable theory of stochastic delay differential equations is established and then some sufficient conditions are obtained based on this theory, which allow the required formations to be gained at exponentially converging speeds with probability one for time-invariant formations, time-varying formations, and time-varying formations for trajectory tracking under a special"multiple leaders" framework. Some numerical simulations are also given to illustrate the effectiveness of the theoretical results.

基于Multi-Agent的无人机集群体系自主作战系统设计

针对无人集群自主作战体系设计中的关键问题,提出基于Multi-Agent的无人集群自主作战系统设计方法。建立无人集群各节点的Agent模型及其推演规则;对于仿真系统模块化和通用化的需求,设计系统互操作式接口和无人集群自主作战的交互关系;开展无人集群系统仿真推演验证。仿真结果表明,所提设计方案不仅能够有效开展并完成自主作战网络生成-集群演化-效能评估的全过程动态演示验证,而且能够通过重复随机试验进一步评估无人集群的协同作战效能,最后总结了集群协同作战的策略和经验。

基于Multi-Agent的水电站变压器故障诊断系统

为了精准、快速完成水电站变压器的故障诊断,设计基于Multi-Agent的水电站变压器故障诊断系统。变压器状态监控agent将检测到的变压器故障信息发送给系统管理agent,系统管理agent通过通信agent将变压器故障信息发送给变压器故障诊断agent,变压器故障诊断agent利用小波变换方法提取变压器故障特征,并将其作为IFOA-SVM模型输入,完成变压器故障分类后,获取变压器故障诊断结果,该结果通过通信agent显示给用户。实验表明,该系统可有效诊断变压器故障诊断,诊断成功率受系统故障信息丢失率的影响较小,诊断耗时、耗能小,并具有较高故障诊断成功率。

Effects of thinning on the understory light environment of different stands and the photosynthetic performance and growth of the reforestation species Phoebe bournei

Light levels determine regeneration in stands and a key concern is how to regulate the light environment of different stand types to the requirements of the understory.In this study,we selected three stands typical in south China(a Cryptomeria japonica plantation,a Quercus acutissima plantation,and a mixed stand of both)and three thinning intensities to determine the best understory light environ-ment for 3-year-old Phoebe bournei seedlings.The canopy structure,understory light environment,and photosynthe-sis and growth indicators were assessed following thin-ning.Thinning improved canopy structure and understory light availability of each stand;species composition was the reason for differences in the understory light environ-ment.Under the same thinning intensity,the mixed stand had the greatest light radiation and most balanced spectral composition.P.bournei photosynthesis and growth were closely related to the light environment;all three stands required heavy thinning to create an effective and sustained understory light environment.In a suitable understory light environment,the efficiency of light interception,absorption,and use by seedlings was enhanced,resulting in a higher carbon assimilation the main limiting factor was stomatal conductance.As a shade-avoidance signal,red/far-red radia-tion is a critical factor driving changes in photosynthesis and growth of P.bournei seedlings,and a reduction increased light absorption and use capacity and height:diameter ratios.The growth advantage transformed from diameter to height,enabling seedlings to access more light.Our findings suggest that the regeneration of shade-tolerant species such as P.bournei could be enhanced if a targeted approach to thinning based on stand type was adopted.

UAV-Assisted Dynamic Avatar Task Migration for Vehicular Metaverse Services: A Multi-Agent Deep Reinforcement Learning Approach

Avatars, as promising digital representations and service assistants of users in Metaverses, can enable drivers and passengers to immerse themselves in 3D virtual services and spaces of UAV-assisted vehicular Metaverses. However, avatar tasks include a multitude of human-to-avatar and avatar-to-avatar interactive applications, e.g., augmented reality navigation,which consumes intensive computing resources. It is inefficient and impractical for vehicles to process avatar tasks locally. Fortunately, migrating avatar tasks to the nearest roadside units(RSU)or unmanned aerial vehicles(UAV) for execution is a promising solution to decrease computation overhead and reduce task processing latency, while the high mobility of vehicles brings challenges for vehicles to independently perform avatar migration decisions depending on current and future vehicle status. To address these challenges, in this paper, we propose a novel avatar task migration system based on multi-agent deep reinforcement learning(MADRL) to execute immersive vehicular avatar tasks dynamically. Specifically, we first formulate the problem of avatar task migration from vehicles to RSUs/UAVs as a partially observable Markov decision process that can be solved by MADRL algorithms. We then design the multi-agent proximal policy optimization(MAPPO) approach as the MADRL algorithm for the avatar task migration problem. To overcome slow convergence resulting from the curse of dimensionality and non-stationary issues caused by shared parameters in MAPPO, we further propose a transformer-based MAPPO approach via sequential decision-making models for the efficient representation of relationships among agents. Finally, to motivate terrestrial or non-terrestrial edge servers(e.g., RSUs or UAVs) to share computation resources and ensure traceability of the sharing records, we apply smart contracts and blockchain technologies to achieve secure sharing management. Numerical results demonstrate that the proposed approach outperforms the MAPPO approach by around 2% and effectively reduces approximately 20% of the latency of avatar task execution in UAV-assisted vehicular Metaverses.

Finite-time Prescribed Performance Time-Varying Formation Control for Second-Order Multi-Agent Systems With Non-Strict Feedback Based on a Neural Network Observer

This paper studies the problem of time-varying formation control with finite-time prescribed performance for nonstrict feedback second-order multi-agent systems with unmeasured states and unknown nonlinearities.To eliminate nonlinearities,neural networks are applied to approximate the inherent dynamics of the system.In addition,due to the limitations of the actual working conditions,each follower agent can only obtain the locally measurable partial state information of the leader agent.To address this problem,a neural network state observer based on the leader state information is designed.Then,a finite-time prescribed performance adaptive output feedback control strategy is proposed by restricting the sliding mode surface to a prescribed region,which ensures that the closed-loop system has practical finite-time stability and that formation errors of the multi-agent systems converge to the prescribed performance bound in finite time.Finally,a numerical simulation is provided to demonstrate the practicality and effectiveness of the developed algorithm.

Study on Distribution of Four Pseudomonas Species in Living Environment Using Multiplex PCR

Purpose: The genus Pseudomonas is a ubiquitous microorganism frequently detected from immunocompromised patients. The inherent resistance to numerous antimicrobial agents contributes to the opportunistic character of this pathogen exhaustive monitoring of this pathogen is considered of critical importance to public health organizations. The reliable identification method able to distinguish genetic close Pseudomonas species is needed, because these organisms are difficult to differentiate by phenotypic or biochemical methods. The purpose of the present study was to design species-specific primers in order to identify and detect four Pseudomonas species which are frequently detected from the human oral cavities, and to investigate the distribution of these organisms in the living environment using a multiplex PCR. Methods: Polymerase chain reaction (PCR) primers were designed based on partial sequences of the rpoD gene of four Pseudomonas species. Swab samples were collected from fifty washstands, and the distribution of Pseudomonas species was investigated using a conventional PCR at genus level and a multiplex PCR at species level. Results: Multiplex PCR method developed in this study was able to distinguish four Pseudomonas species clearly. The genus Pseudomonas was detected from all samples (100%), whereas P. putida, P, aeruginosa, P. stutzeri and P. fluorescens were detected at 44%, 8%, 4% and 2% in fifty swab samples, respectively. Conclusion: Our developed one-step multiplex PCR method is accurate, specific, cost-effective, time-saving, and works without requiring DNA extraction. It was indicated that washstands were the uninhabitable environment for P. putida, P, aeruginosa, P. stutzeri and P. fluorescens.

A survey on multi-agent reinforcement learning and its application

Multi-agent reinforcement learning(MARL)has been a rapidly evolving field.This paper presents a comprehensive survey of MARL and its applications.We trace the historical evolution of MARL,highlight its progress,and discuss related survey works.Then,we review the existing works addressing inherent challenges and those focusing on diverse applications.Some representative stochastic games,MARL means,spatial forms of MARL,and task classification are revisited.We then conduct an in-depth exploration of a variety of challenges encountered in MARL applications.We also address critical operational aspects,such as hyperparameter tuning and computational complexity,which are pivotal in practical implementations of MARL.Afterward,we make a thorough overview of the applications of MARL to intelligent machines and devices,chemical engineering,biotechnology,healthcare,and societal issues,which highlights the extensive potential and relevance of MARL within both current and future technological contexts.Our survey also encompasses a detailed examination of benchmark environments used in MARL research,which are instrumental in evaluating MARL algorithms and demonstrate the adaptability of MARL to diverse application scenarios.In the end,we give our prospect for MARL and discuss their related techniques and potential future applications.

Influence of static cartoons combined with dynamic virtual environments on preoperative anxiety of preschool-aged children undergoing surgery

BACKGROUND Preschoolers become anxious when they are about to undergo anesthesia and surgery,warranting the development of more appropriate and effective interventions.AIM To explore the effect of static cartoons combined with dynamic virtual environments on preoperative anxiety and anesthesia induction compliance in preschool-aged children undergoing surgery.METHODS One hundred and sixteen preschool-aged children were selected and assigned to the drug(n=37),intervention(n=40),and control(n=39)groups.All the children received routine preoperative checkups and nursing before being transferred to the preoperative preparation room on the day of the operation.The drug group received 0.5 mg/kg midazolam and the intervention group treatment consisting of static cartoons combined with dynamic virtual environments.The control group received no intervention.The modified Yale Preoperative Anxiety Scale was used to evaluate the children’s anxiety level on the day before surgery(T0),before leaving the preoperative preparation room(T1),when entering the operating room(T2),and at anesthesia induction(T3).Compliance during anesthesia induction(T3)was evaluated using the Induction Compliance Checklist(ICC).Changes in mean arterial pressure(MAP),heart rate(HR),and respiratory rate(RR)were also recorded at each time point.RESULTS The anxiety scores of the three groups increased variously at T1 and T2.At T3,both the drug and intervention groups had similar anxiety scores,both of which were lower than those in the control group.At T1 and T2,MAP,HR,and RR of the three groups increased.The drug and control groups had significantly higher MAP and RR than the intervention group at T2.At T3,the MAP,HR,and RR of the drug group decreased and were significantly lower than those in the control group but were comparable to those in the intervention group.Both the drug and intervention groups had similar ICC scores and duration of anesthesia induction(T3),both of which were higher than those of the control group.CONCLUSION Combining static cartoons with dynamic virtual environments as effective as medication,specifically midazolam,in reducing preoperative anxiety and fear in preschool-aged children.This approach also improve their compliance during anesthesia induction and helped maintain their stable vital signs.

Corrosion behavior of HVOF Inconel 625 coating in the simulated marine environment

High velocity oxygen fuel(HVOF)spraying process is commonly used to produce superalloy coatings.Inconel 625 coating was prepared on Q235B low carbon steel by HVOF.A series of experiments were conducted to examine the surface and corrosion resistance properties of Inconel 625 HVOF coating.In this paper,potentiodynamic polarization tests and electrochemical impedance spectroscopy(EIS)tests were carried out to evaluate the corrosion resistance of Inconel 625 coating under simulated marine environment.The experiment-al results showed that Inconel 625 coating revealed low porosity and desired coating thickness.Shift in the corrosion potential(E_(corr))to-wards the noble direction combined with much low corrosion current density(i_(corr))indicating a significant improvement of HVOF Inconel 625 coating compared with the substrate.