Safety-Constrained Multi-Agent Reinforcement Learning for Power Quality Control in Distributed Renewable Energy Networks

This paper examines the difficulties of managing distributed power systems,notably due to the increasing use of renewable energy sources,and focuses on voltage control challenges exacerbated by their variable nature in modern power grids.To tackle the unique challenges of voltage control in distributed renewable energy networks,researchers are increasingly turning towards multi-agent reinforcement learning(MARL).However,MARL raises safety concerns due to the unpredictability in agent actions during their exploration phase.This unpredictability can lead to unsafe control measures.To mitigate these safety concerns in MARL-based voltage control,our study introduces a novel approach:Safety-ConstrainedMulti-Agent Reinforcement Learning(SC-MARL).This approach incorporates a specialized safety constraint module specifically designed for voltage control within the MARL framework.This module ensures that the MARL agents carry out voltage control actions safely.The experiments demonstrate that,in the 33-buses,141-buses,and 322-buses power systems,employing SC-MARL for voltage control resulted in a reduction of the Voltage Out of Control Rate(%V.out)from0.43,0.24,and 2.95 to 0,0.01,and 0.03,respectively.Additionally,the Reactive Power Loss(Q loss)decreased from 0.095,0.547,and 0.017 to 0.062,0.452,and 0.016 in the corresponding systems.

基于Multi-Agent的无人机集群体系自主作战系统设计

针对无人集群自主作战体系设计中的关键问题,提出基于Multi-Agent的无人集群自主作战系统设计方法。建立无人集群各节点的Agent模型及其推演规则;对于仿真系统模块化和通用化的需求,设计系统互操作式接口和无人集群自主作战的交互关系;开展无人集群系统仿真推演验证。仿真结果表明,所提设计方案不仅能够有效开展并完成自主作战网络生成-集群演化-效能评估的全过程动态演示验证,而且能够通过重复随机试验进一步评估无人集群的协同作战效能,最后总结了集群协同作战的策略和经验。

基于Multi-Agent的水电站变压器故障诊断系统

为了精准、快速完成水电站变压器的故障诊断,设计基于Multi-Agent的水电站变压器故障诊断系统。变压器状态监控agent将检测到的变压器故障信息发送给系统管理agent,系统管理agent通过通信agent将变压器故障信息发送给变压器故障诊断agent,变压器故障诊断agent利用小波变换方法提取变压器故障特征,并将其作为IFOA-SVM模型输入,完成变压器故障分类后,获取变压器故障诊断结果,该结果通过通信agent显示给用户。实验表明,该系统可有效诊断变压器故障诊断,诊断成功率受系统故障信息丢失率的影响较小,诊断耗时、耗能小,并具有较高故障诊断成功率。

The influence of resistance exercise training prescription variables on skeletal muscle mass,strength,and physical function in healthy adults:An umbrella review

Purpose:The aim of this umbrella review was to determine the impact of resistance training(RT)and individual RT prescription variables on muscle mass,strength,and physical function in healthy adults.Methods:Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses(PRISMA)guidelines,we systematically searched and screened eligible systematic reviews reporting the effects of differing RT prescription variables on muscle mass(or its proxies),strength,and/or physical function in healthy adults aged>18 years.Results:We identified 44 systematic reviews that met our inclusion criteria.The methodological quality of these reviews was assessed using A Measurement Tool to Assess Systematic Reviews;standardized effectiveness statements were generated.We found that RT was consistently a potent stimulus for increasing skeletal muscle mass(4/4 reviews provide some or sufficient evidence),strength(4/6 reviews provided some or sufficient evidence),and physical function(1/1 review provided some evidence).RT load(6/8 reviews provided some or sufficient evidence),weekly frequency(2/4 reviews provided some or sufficient evidence),volume(3/7 reviews provided some or sufficient evidence),and exercise order(1/1 review provided some evidence)impacted RT-induced increases in muscular strength.We discovered that 2/3 reviews provided some or sufficient evidence that RT volume and contraction velocity influenced skeletal muscle mass,while 4/7 reviews provided insufficient evidence in favor of RT load impacting skeletal muscle mass.There was insufficient evidence to conclude that time of day,periodization,inter-set rest,set configuration,set end point,contraction velocity/time under tension,or exercise order(only pertaining to hypertrophy)influenced skeletal muscle adaptations.A paucity of data limited insights into the impact of RT prescription variables on physical function.Conclusion:Overall,RT increased muscle mass,strength,and physical function compared to no exercise.RT intensity(load)and weekly frequency impacted RT-induced increases in muscular strength but not muscle hypertrophy.RT volume(number of sets)influenced muscular strength and hypertrophy.

Surgical intervention combined with weight-bearing walking training promotes recovery in patients with chronic spinal cord injury:a randomized controlled study

For patients with chronic spinal cord injury,the co nventional treatment is rehabilitation and treatment of spinal cord injury complications such as urinary tract infection,pressure sores,osteoporosis,and deep vein thrombosis.Surgery is rarely perfo rmed on spinal co rd injury in the chronic phase,and few treatments have been proven effective in chronic spinal cord injury patients.Development of effective therapies fo r chronic spinal co rd injury patients is needed.We conducted a randomized controlled clinical trial in patients with chronic complete thoracic spinal co rd injury to compare intensive rehabilitation(weight-bearing walking training)alone with surgical intervention plus intensive rehabilitation.This clinical trial was registered at ClinicalTrials.gov(NCT02663310).The goal of surgical intervention was spinal cord detethering,restoration of cerebrospinal fluid flow,and elimination of residual spinal cord compression.We found that surgical intervention plus weight-bearing walking training was associated with a higher incidence of American Spinal Injury Association Impairment Scale improvement,reduced spasticity,and more rapid bowel and bladder functional recovery than weight-bearing walking training alone.Overall,the surgical procedures and intensive rehabilitation were safe.American Spinal Injury Association Impairment Scale improvement was more common in T7-T11 injuries than in T2-T6 injuries.Surgery combined with rehabilitation appears to have a role in treatment of chronic spinal cord injury patients.

UAV-Assisted Dynamic Avatar Task Migration for Vehicular Metaverse Services: A Multi-Agent Deep Reinforcement Learning Approach

Avatars, as promising digital representations and service assistants of users in Metaverses, can enable drivers and passengers to immerse themselves in 3D virtual services and spaces of UAV-assisted vehicular Metaverses. However, avatar tasks include a multitude of human-to-avatar and avatar-to-avatar interactive applications, e.g., augmented reality navigation,which consumes intensive computing resources. It is inefficient and impractical for vehicles to process avatar tasks locally. Fortunately, migrating avatar tasks to the nearest roadside units(RSU)or unmanned aerial vehicles(UAV) for execution is a promising solution to decrease computation overhead and reduce task processing latency, while the high mobility of vehicles brings challenges for vehicles to independently perform avatar migration decisions depending on current and future vehicle status. To address these challenges, in this paper, we propose a novel avatar task migration system based on multi-agent deep reinforcement learning(MADRL) to execute immersive vehicular avatar tasks dynamically. Specifically, we first formulate the problem of avatar task migration from vehicles to RSUs/UAVs as a partially observable Markov decision process that can be solved by MADRL algorithms. We then design the multi-agent proximal policy optimization(MAPPO) approach as the MADRL algorithm for the avatar task migration problem. To overcome slow convergence resulting from the curse of dimensionality and non-stationary issues caused by shared parameters in MAPPO, we further propose a transformer-based MAPPO approach via sequential decision-making models for the efficient representation of relationships among agents. Finally, to motivate terrestrial or non-terrestrial edge servers(e.g., RSUs or UAVs) to share computation resources and ensure traceability of the sharing records, we apply smart contracts and blockchain technologies to achieve secure sharing management. Numerical results demonstrate that the proposed approach outperforms the MAPPO approach by around 2% and effectively reduces approximately 20% of the latency of avatar task execution in UAV-assisted vehicular Metaverses.

Application value research of swallowing treatment device combined with swallowing rehabilitation training in the treatment of swallowing disorders after stroke

BACKGROUND Stroke is a common disabling disease,whether it is ischemic stroke or hemorrhagic stroke,both can result in neuronal damage,leading to various manifestations of neurological dysfunction.AIM To explore of the application value of swallowing treatment device combined with swallowing rehabilitation training in the treatment of swallowing disorders after stroke.METHODS This study selected 86 patients with swallowing disorders after stroke admitted to our rehabilitation department from February 2022 to December 2023 as research subjects.They were divided into a control group(n=43)and an observation group(n=43)according to the treatment.The control group received swallowing rehabilitation training,while the observation group received swallowing treatment device in addition to the training.Both groups underwent continuous intervention for two courses of treatment.RESULTS The total effective rate in the observation group(93.02%)was higher than that in the control group(76.74%)(P=0.035).After intervention,the oral transit time,swallowing response time,pharyngeal transit time,and laryngeal closure time decreased in both groups compared to before intervention.In the observation group,the oral transit time,swallowing response time,and pharyngeal transit time were shorter than those in the control group after intervention.However,the laryngeal closure time after intervention in the observation group was compared with that in the control group(P=0.142).After intervention,average amplitude value and duration of the genioglossus muscle group during empty swallowing and swallowing 5 mL of water are reduced compared to before intervention in both groups.After intervention,the scores of the chin-tuck swallowing exercise and the Standardized Swallowing Assessment are both reduced compared to pre-intervention levels in both groups.However,the observation group scores lower than the control group after intervention.Additionally,the Functional Oral Intake Scale scores of both groups are increased after intervention compared to pre-intervention levels,with the observation group scoring higher than the control group after intervention(P<0.001).The cumulative incidence of complications in the observation group is 9.30%,which is lower than the 27.91%in the control group(P=0.027).CONCLUSION The combination of swallowing therapy equipment with swallowing rehabilitation training can improve the muscle movement level of the genioglossus muscle group,enhance swallowing function,and prevent the occurrence of swallowing-related complications after stroke.

Finite-time Prescribed Performance Time-Varying Formation Control for Second-Order Multi-Agent Systems With Non-Strict Feedback Based on a Neural Network Observer

This paper studies the problem of time-varying formation control with finite-time prescribed performance for nonstrict feedback second-order multi-agent systems with unmeasured states and unknown nonlinearities.To eliminate nonlinearities,neural networks are applied to approximate the inherent dynamics of the system.In addition,due to the limitations of the actual working conditions,each follower agent can only obtain the locally measurable partial state information of the leader agent.To address this problem,a neural network state observer based on the leader state information is designed.Then,a finite-time prescribed performance adaptive output feedback control strategy is proposed by restricting the sliding mode surface to a prescribed region,which ensures that the closed-loop system has practical finite-time stability and that formation errors of the multi-agent systems converge to the prescribed performance bound in finite time.Finally,a numerical simulation is provided to demonstrate the practicality and effectiveness of the developed algorithm.

Robust adaptive leaderless consensus of unknown non-minimum phase linear multi-agent systems subject to disturbances and/or unmodeled dynamics

This article investigates the problem of robust adaptive leaderless consensus for heterogeneous uncertain nonminimumphase linear multi-agent systems over directed communication graphs. Each agent is assumed tobe of unknown nominal dynamics and also subject to external disturbances and/or unmodeled dynamics. Anovel distributed robust adaptive control strategy is proposed. It is shown that the robust adaptive leaderlessconsensus problem is solved with the proposed control strategy under some sufficient conditions. Two examplesare provided to demonstrate the efficacy of the proposed control strategy.

Discovering Latent Variables for the Tasks With Confounders in Multi-Agent Reinforcement Learning

Efficient exploration in complex coordination tasks has been considered a challenging problem in multi-agent reinforcement learning(MARL). It is significantly more difficult for those tasks with latent variables that agents cannot directly observe. However, most of the existing latent variable discovery methods lack a clear representation of latent variables and an effective evaluation of the influence of latent variables on the agent. In this paper, we propose a new MARL algorithm based on the soft actor-critic method for complex continuous control tasks with confounders. It is called the multi-agent soft actor-critic with latent variable(MASAC-LV) algorithm, which uses variational inference theory to infer the compact latent variables representation space from a large amount of offline experience.Besides, we derive the counterfactual policy whose input has no latent variables and quantify the difference between the actual policy and the counterfactual policy via a distance function. This quantified difference is considered an intrinsic motivation that gives additional rewards based on how much the latent variable affects each agent. The proposed algorithm is evaluated on two collaboration tasks with confounders, and the experimental results demonstrate the effectiveness of MASAC-LV compared to other baseline algorithms.

Research and Practice of Cardiopulmonary Rehabilitation Training Programs

In today’s society, the incidence of cardiopulmonary diseases is increasing annually, seriously affecting patients’ quality of life. Therefore, developing a scientific and effective rehabilitation training program is of great significance. This study first analyzes the theoretical basis of cardiopulmonary rehabilitation training, including the effects of aerobic exercise, interval training, and strength training on cardiopulmonary function. Based on this, a comprehensive rehabilitation training program is designed, which includes personalized training plans, comprehensive interventions, multidisciplinary collaboration, patient education, and regular follow-up visits. The cardiopulmonary rehabilitation training plan developed in this study has certain scientific practicability, which provides a theoretical basis for cardiopulmonary rehabilitation training, and also provides a reference for medical institutions, rehabilitation centers and communities, which is helpful for promotion and application to a wider range of patients with cardiopulmonary diseases.

Development of Multi-Agent-Based Indoor 3D Reconstruction

Large-scale indoor 3D reconstruction with multiple robots faces challenges in core enabling technologies.This work contributes to a framework addressing localization,coordination,and vision processing for multi-agent reconstruction.A system architecture fusing visible light positioning,multi-agent path finding via reinforcement learning,and 360°camera techniques for 3D reconstruction is proposed.Our visible light positioning algorithm leverages existing lighting for centimeter-level localization without additional infrastructure.Meanwhile,a decentralized reinforcement learning approach is developed to solve the multi-agent path finding problem,with communications among agents optimized.Our 3D reconstruction pipeline utilizes equirectangular projection from 360°cameras to facilitate depth-independent reconstruction from posed monocular images using neural networks.Experimental validation demonstrates centimeter-level indoor navigation and 3D scene reconstruction capabilities of our framework.The challenges and limitations stemming from the above enabling technologies are discussed at the end of each corresponding section.In summary,this research advances fundamental techniques for multi-robot indoor 3D modeling,contributing to automated,data-driven applications through coordinated robot navigation,perception,and modeling.

GeoNER:Geological Named Entity Recognition with Enriched Domain Pre-Training Model and Adversarial Training

As important geological data,a geological report contains rich expert and geological knowledge,but the challenge facing current research into geological knowledge extraction and mining is how to render accurate understanding of geological reports guided by domain knowledge.While generic named entity recognition models/tools can be utilized for the processing of geoscience reports/documents,their effectiveness is hampered by a dearth of domain-specific knowledge,which in turn leads to a pronounced decline in recognition accuracy.This study summarizes six types of typical geological entities,with reference to the ontological system of geological domains and builds a high quality corpus for the task of geological named entity recognition(GNER).In addition,Geo Wo BERT-adv BGP(Geological Word-base BERTadversarial training Bi-directional Long Short-Term Memory Global Pointer)is proposed to address the issues of ambiguity,diversity and nested entities for the geological entities.The model first uses the fine-tuned word granularitybased pre-training model Geo Wo BERT(Geological Word-base BERT)and combines the text features that are extracted using the Bi LSTM(Bi-directional Long Short-Term Memory),followed by an adversarial training algorithm to improve the robustness of the model and enhance its resistance to interference,the decoding finally being performed using a global association pointer algorithm.The experimental results show that the proposed model for the constructed dataset achieves high performance and is capable of mining the rich geological information.

A survey on multi-agent reinforcement learning and its application

Multi-agent reinforcement learning(MARL)has been a rapidly evolving field.This paper presents a comprehensive survey of MARL and its applications.We trace the historical evolution of MARL,highlight its progress,and discuss related survey works.Then,we review the existing works addressing inherent challenges and those focusing on diverse applications.Some representative stochastic games,MARL means,spatial forms of MARL,and task classification are revisited.We then conduct an in-depth exploration of a variety of challenges encountered in MARL applications.We also address critical operational aspects,such as hyperparameter tuning and computational complexity,which are pivotal in practical implementations of MARL.Afterward,we make a thorough overview of the applications of MARL to intelligent machines and devices,chemical engineering,biotechnology,healthcare,and societal issues,which highlights the extensive potential and relevance of MARL within both current and future technological contexts.Our survey also encompasses a detailed examination of benchmark environments used in MARL research,which are instrumental in evaluating MARL algorithms and demonstrate the adaptability of MARL to diverse application scenarios.In the end,we give our prospect for MARL and discuss their related techniques and potential future applications.

CoopAI-Route: DRL Empowered Multi-Agent Cooperative System for Efficient QoS-Aware Routing for Network Slicing in Multi-Domain SDN

The emergence of beyond 5G networks has the potential for seamless and intelligent connectivity on a global scale.Network slicing is crucial in delivering services for different,demanding vertical applications in this context.Next-generation applications have time-sensitive requirements and depend on the most efficient routing path to ensure packets reach their intended destinations.However,the existing IP(Internet Protocol)over a multi-domain network faces challenges in enforcing network slicing due to minimal collaboration and information sharing among network operators.Conventional inter-domain routing methods,like Border Gateway Protocol(BGP),cannot make routing decisions based on performance,which frequently results in traffic flowing across congested paths that are never optimal.To address these issues,we propose CoopAI-Route,a multi-agent cooperative deep reinforcement learning(DRL)system utilizing hierarchical software-defined networks(SDN).This framework enforces network slicing in multi-domain networks and cooperative communication with various administrators to find performance-based routes in intra-and inter-domain.CoopAI-Route employs the Distributed Global Topology(DGT)algorithm to define inter-domain Quality of Service(QoS)paths.CoopAI-Route uses a DRL agent with a message-passing multi-agent Twin-Delayed Deep Deterministic Policy Gradient method to ensure optimal end-to-end routes adapted to the specific requirements of network slicing applications.Our evaluation demonstrates CoopAI-Route’s commendable performance in scalability,link failure handling,and adaptability to evolving topologies compared to state-of-the-art methods.

Evaluating the feasibility and preliminary effects of an online compassion training program for nursing students:A pilot randomized controlled trial

Objectives:This study aimed to assess the feasibility of an online compassion training program for nursing students and preliminarily investigate its effects on mindfulness,self-compassion,and stress reduction.Methods:This study employed a randomized controlled trial design.Second-year students from a nursing college in Guangzhou,China,were recruited as research participants in August 2023.The intervention group participated in an 8-week online compassion training program via the WeChat platform,comprising three stages:mindfulness(weeks 1e2),self-compassion(weeks 3e5),and compassion for others(weeks 6 e8).Each stage included four activities:psychoeducation,mindfulness practice,weekly diary,and emotional support.Program feasibility was assessed through recruitment and retention rates,program engagement,and participant acceptability.Program effectiveness was measured with the Mindful Attention Awareness Scale,Self-Compassion Scale-Short Form,and Perceived Stress Scale.Results:A total of 28 students completed the study(13 in the intervention group,15 in the control group).The recruitment rate was 36.46%,with a high retention rate of 93.3%.Participants demonstrated high engagement:69.2%accessed learning materials every 1e2 days,93.3%practiced mindfulness at least weekly,with an average of 4.69 diary entries submitted per person and 23.30 WeChat interactions with instructors.Regarding acceptability,all participants expressed satisfaction with the program,with 92.4%finding it“very helpful”or“extremely helpful.”In terms of intervention effects,the intervention group showed a significant increase in mindfulness levels from pre-intervention(51.54±10.93)to postintervention(62.46±13.58)(P<0.05),while no significant change was observed in the control group.Although there were no statistically significant differences between the two groups in post-intervention self-compassion and perceived stress levels,the intervention group showed positive trends:selfcompassion levels increased(35.85±8.60 vs.40.85±5.54),and perceived stress levels slightly decreased(44.77±8.65 vs.42.00±5.77).Conclusions:This pilot study demonstrated the feasibility of an online compassion training program for nursing students and suggested its potential effectiveness in enhancing mindfulness,self-compassion,and stress reduction.Despite limitations such as small sample size and lack of long-term follow-up,preliminary evidence indicates promising prospects for integrating such training into nursing education.Further research is warranted to confirm thesefindings and assess the sustained impact of this approach on nursing education and practice.

Impact of task-oriented training based on acupuncture rehabilitation on upper extremity function and quality of life of patients with early stroke

BACKGROUND Eighty percent of stroke patients develop upper limb dysfunction,especially hand dysfunction,which has a very slow recovery,resulting in economic burden to families and society.AIM To investigate the impact of task-oriented training based on acupuncture therapy on upper extremity function in patients with early stroke.METHODS Patients with early stroke hemiplegia who visited our hospital between January 2021 and October 2022 were divided into a control group and an observation group,each with 50 cases.The control group underwent head acupuncture plus routine upper limb rehabilitation training(acupuncture therapy).In addition to acupuncture and rehabilitation,the observation group underwent upper limb task-oriented training(30 min).Each group underwent treatment 5 d/wk for 4 wk.Upper extremity function was assessed in both groups using the Fugl-Meyer Assessment-Upper Extremity(FMA-UE),Wolf Motor Function Rating Scale(WMFT),modified Barthel Index(MBI),and Canadian Occupational Performance Measure(COPM).Quality of life was evaluated using the Short-Form 36-Item Health Survey(SF-36).Clinical efficacy of the interventions was also evaluated.RESULTS Before intervention,no significant differences were observed in the FMA-UE,MBI,and WMFT scores between the two groups(P>0.05).After intervention,the FMA-UE,WMFT,MBI,COPM-Functional Mobility and Satisfaction,and SF-36 scores increased in both groups(P<0.05),with even higher scores in the observation group(P<0.05).The observation group also obtained a higher total effective rate than the control group(P<0.05).CONCLUSION Task-oriented training based on acupuncture rehabilitation significantly enhanced upper extremity mobility,quality of life,and clinical efficacy in patients with early stroke.

Research on consensus of multi-agent systems with and without input saturation constraints

In recent years,with the continuous development of multi-agent technology represented by unmanned aerial vehicle(UAV)swarm,consensus control has become a hot spot in academic research.In this paper,we put forward a discrete-time consensus protocol and obtain the necessary and sufficient conditions for the second-order consensus of the second-order multi-agent system with a fixed structure under the condition of no saturation input.The theoretical derivation verifies that the two eigenvalues of the Laplacian of the communication network matrix and the sampling period have an important effect on achieving consensus.Then we construct and verify sufficient conditions to achieve consensus under the condition of input saturation constraints.The results show that consensus can be achieved if velocity,position gain,and sampling period satisfy a set of inequalities related to the eigenvalues of the Laplacian matrix.Finally,the accuracy and validity of the theoretical results are proved by numerical simulations.

Bipartite consensus problems of Lurie multi-agent systems over signed graphs: A contraction approach

This paper examines the bipartite consensus problems for the nonlinear multi-agent systems in Lurie dynamics form with cooperative and competitive communication between different agents. Based on the contraction theory, some new conditions for the nonlinear Lurie multi-agent systems reaching bipartite leaderless consensus and bipartite tracking consensus are presented. Compared with the traditional methods, this approach degrades the dimensions of the conditions, eliminates some restrictions of the system matrix, and extends the range of the nonlinear function. Finally, two numerical examples are provided to illustrate the efficiency of our results.

Hyperbolic Tangent Function-Based Protocols for Global/Semi-Global Finite-Time Consensus of Multi-Agent Systems

This paper investigates the problem of global/semi-global finite-time consensus for integrator-type multi-agent sys-tems.New hyperbolic tangent function-based protocols are pro-posed to achieve global and semi-global finite-time consensus for both single-integrator and double-integrator multi-agent systems with leaderless undirected and leader-following directed commu-nication topologies.These new protocols not only provide an explicit upper-bound estimate for the settling time,but also have a user-prescribed bounded control level.In addition,compared to some existing results based on the saturation function,the pro-posed approach considerably simplifies the protocol design and the stability analysis.Illustrative examples and an application demonstrate the effectiveness of the proposed protocols.