Autonomous Vehicle Platoons In Urban Road Networks:A Joint Distributed Reinforcement Learning and Model Predictive Control Approach

In this paper, platoons of autonomous vehicles operating in urban road networks are considered. From a methodological point of view, the problem of interest consists of formally characterizing vehicle state trajectory tubes by means of routing decisions complying with traffic congestion criteria. To this end, a novel distributed control architecture is conceived by taking advantage of two methodologies: deep reinforcement learning and model predictive control. On one hand, the routing decisions are obtained by using a distributed reinforcement learning algorithm that exploits available traffic data at each road junction. On the other hand, a bank of model predictive controllers is in charge of computing the more adequate control action for each involved vehicle. Such tasks are here combined into a single framework:the deep reinforcement learning output(action) is translated into a set-point to be tracked by the model predictive controller;conversely, the current vehicle position, resulting from the application of the control move, is exploited by the deep reinforcement learning unit for improving its reliability. The main novelty of the proposed solution lies in its hybrid nature: on one hand it fully exploits deep reinforcement learning capabilities for decisionmaking purposes;on the other hand, time-varying hard constraints are always satisfied during the dynamical platoon evolution imposed by the computed routing decisions. To efficiently evaluate the performance of the proposed control architecture, a co-design procedure, involving the SUMO and MATLAB platforms, is implemented so that complex operating environments can be used, and the information coming from road maps(links,junctions, obstacles, semaphores, etc.) and vehicle state trajectories can be shared and exchanged. Finally by considering as operating scenario a real entire city block and a platoon of eleven vehicles described by double-integrator models, several simulations have been performed with the aim to put in light the main f eatures of the proposed approach. Moreover, it is important to underline that in different operating scenarios the proposed reinforcement learning scheme is capable of significantly reducing traffic congestion phenomena when compared with well-reputed competitors.

Observer-based dynamic event-triggered control for distributed parameter systems over mobile sensor-plus-actuator networks

We develop a policy of observer-based dynamic event-triggered state feedback control for distributed parameter systems over a mobile sensor-plus-actuator network.It is assumed that the mobile sensing devices that provide spatially averaged state measurements can be used to improve state estimation in the network.For the purpose of decreasing the update frequency of controller and unnecessary sampled data transmission, an efficient dynamic event-triggered control policy is constructed.In an event-triggered system, when an error signal exceeds a specified time-varying threshold, it indicates the occurrence of a typical event.The global asymptotic stability of the event-triggered closed-loop system and the boundedness of the minimum inter-event time can be guaranteed.Based on the linear quadratic optimal regulator, the actuator selects the optimal displacement only when an event occurs.A simulation example is finally used to verify that the effectiveness of such a control strategy can enhance the system performance.

Virtually coupled train set control subject to space-time separation:A distributed economic MPC approach with emergency braking configuration

The emerging virtual coupling technology aims to operate multiple train units in a Virtually Coupled Train Set(VCTS)at a minimal but safe distance.To guarantee collision avoidance,the safety distance should be calculated using the state-of-the-art space-time separation principle that separates the Emergency Braking(EB)trajectories of two successive units during the whole EB process.In this case,the minimal safety distance is usually numerically calculated without an analytic formulation.Thus,the constrained VCTS control problem is hard to address with space-time separation,which is still a gap in the existing literature.To solve this problem,we propose a Distributed Economic Model Predictive Control(DEMPC)approach with computation efficiency and theoretical guarantee.Specifically,to alleviate the computation burden,we transform implicit safety constraints into explicitly linear ones,such that the optimal control problem in DEMPC is a quadratic programming problem that can be solved efficiently.For theoretical analysis,sufficient conditions are derived to guarantee the recursive feasibility and stability of DEMPC,employing compatibility constraints,tube techniques and terminal ingredient tuning.Moreover,we extend our approach with globally optimal and distributed online EB configuration methods to shorten the minimal distance among VCTS.Finally,experimental results demonstrate the performance and advantages of the proposed approaches.

Distributed Economic MPC for Synergetic Regulation of the Voltage of an Island DC Micro-Grid

In this paper,distributed model predictive control(DMPC) for island DC micro-grids(MG) with wind/photovoltaic(PV)/battery power is proposed,which coordinates all distributed generations(DG) to stabilize the bus voltage together with the insurance of having computational efficiency under a real-time requirement.Based on the feedback of the bus voltage,the deviation of the current is dispatched to each DG according to cost over the prediction horizon.Moreover,to avoid the excessive fluctuation of the battery power,both the discharge-charge switching times and costs are considered in the model predictive control(MPC) optimization problems.A Lyapunov constraint with a time-varying steady-state is designed in each local MPC to guarantee the stabilization of the entire system.The voltage stabilization of the MG is achieved by this strategy with the cooperation of DGs.The numeric results of applying the proposed method to a MG of the Shanghai Power Supply Company shows the effectiveness of the distributed economic MPC.

Data-Driven Control of Distributed Event-Triggered Network Systems

The present paper deals with data-driven event-triggered control of a class of unknown discrete-time interconnected systems(a.k.a.network systems).To this end,we start by putting forth a novel distributed event-triggering transmission strategy based on periodic sampling,under which a model-based stability criterion for the closed-loop network system is derived,by leveraging a discrete-time looped-functional approach.Marrying the model-based criterion with a data-driven system representation recently developed in the literature,a purely data-driven stability criterion expressed in the form of linear matrix inequalities(LMIs)is established.Meanwhile,the data-driven stability criterion suggests a means for co-designing the event-triggering coefficient matrix and the feedback control gain matrix using only some offline collected state-input data.Finally,numerical results corroborate the efficacy of the proposed distributed data-driven event-triggered network system(ETS)in cutting off data transmissions and the co-design procedure.

An Optimal Control-Based Distributed Reinforcement Learning Framework for A Class of Non-Convex Objective Functionals of the Multi-Agent Network

This paper studies a novel distributed optimization problem that aims to minimize the sum of the non-convex objective functionals of the multi-agent network under privacy protection, which means that the local objective of each agent is unknown to others. The above problem involves complexity simultaneously in the time and space aspects. Yet existing works about distributed optimization mainly consider privacy protection in the space aspect where the decision variable is a vector with finite dimensions. In contrast, when the time aspect is considered in this paper, the decision variable is a continuous function concerning time. Hence, the minimization of the overall functional belongs to the calculus of variations. Traditional works usually aim to seek the optimal decision function. Due to privacy protection and non-convexity, the Euler-Lagrange equation of the proposed problem is a complicated partial differential equation.Hence, we seek the optimal decision derivative function rather than the decision function. This manner can be regarded as seeking the control input for an optimal control problem, for which we propose a centralized reinforcement learning(RL) framework. In the space aspect, we further present a distributed reinforcement learning framework to deal with the impact of privacy protection. Finally, rigorous theoretical analysis and simulation validate the effectiveness of our framework.

Differential flatness-based distributed control of underactuated robot swarms

This paper proposes a distributed control method based on the differential flatness(DF) property of robot swarms. The swarm DF mapping is established for underactuated differentially flat dynamics, according to the control objective. The DF mapping refers to the fact that the system state and input of each robot can be derived algebraically from the flat outputs of the leaders and the cooperative errors and their finite order derivatives. Based on the proposed swarm DF mapping, a distributed controller is designed. The distributed implementation of swarm DF mapping is achieved through observer design. The effectiveness of the proposed method is validated through a numerical simulation of quadrotor swarm synchronization.

Operation Control Method of Relay Protection in Flexible DC Distribution Network Compatible with Distributed Power Supply

A novel operation control method for relay protection in flexible DC distribution networks with distributed power supply is proposed to address the issue of inaccurate fault location during relay protection,leading to poor performance.The method combines a fault-tolerant fault location method based on long-term and short-term memory networks to accurately locate the fault section.Then,an operation control method for relay protection based on adaptive weight and whale optimization algorithm(WOA)is used to construct an objective function considering the shortest relay protection action time and the smallest impulse current.The adaptive weight and WOA are employed to obtain the optimal strategy for relay protection operation control,reducing the action time and impulse current.Experimental results demonstrate the effectiveness of the proposed method in accurately locating faults and improving relay protection performance.The longest operation time is reduced by 4.7023 s,and the maximum impulse current is limited to 0.3 A,effectively controlling the impact of large impulse currents and enhancing control efficiency.

STABILITY ANALYSIS AND COOPERATIVE CONTROL OF DISTRIBUTED MULTI-AGENT SYSTEM WITH SAMPLED COMMUNICATION

The cooperative control and stability analysis problems for the multi-agent system with sampled com- munication are investigated. Distributed state feedback controllers are adopted for the cooperation of networked agents. A theorem in the form of linear matrix inequalities（LMI） is derived to analyze the system stability. An- other theorem in the form of optimization problem subject to LMI constraints is proposed to design the controller, and then the algorithm is presented. The simulation results verify the validity and the effectiveness of the pro- posed approach.

Neural stem cell-derived exosomes promote mitochondrial biogenesis and restore abnormal protein distribution in a mouse model of Alzheimer's disease

Mitochondrial dysfunction is a hallmark of Alzheimer’s disease.We previously showed that neural stem cell-derived extracellular vesicles improved mitochondrial function in the cortex of AP P/PS1 mice.Because Alzheimer’s disease affects the entire brain,further research is needed to elucidate alterations in mitochondrial metabolism in the brain as a whole.Here,we investigated the expression of several important mitochondrial biogenesis-related cytokines in multiple brain regions after treatment with neural stem cell-derived exosomes and used a combination of whole brain clearing,immunostaining,and lightsheet imaging to clarify their spatial distribution.Additionally,to clarify whether the sirtuin 1(SIRT1)-related pathway plays a regulatory role in neural stem cell-de rived exosomes interfering with mitochondrial functional changes,we generated a novel nervous system-SIRT1 conditional knoc kout AP P/PS1mouse model.Our findings demonstrate that neural stem cell-de rived exosomes significantly increase SIRT1 levels,enhance the production of mitochondrial biogenesis-related fa ctors,and inhibit astrocyte activation,but do not suppress amyloid-βproduction.Thus,neural stem cell-derived exosomes may be a useful therapeutic strategy for Alzheimer’s disease that activates the SIRT1-PGC1αsignaling pathway and increases NRF1 and COXIV synthesis to improve mitochondrial biogenesis.In addition,we showed that the spatial distribution of mitochondrial biogenesis-related factors is disrupted in Alzheimer’s disease,and that neural stem cell-derived exosome treatment can reverse this effect,indicating that neural stem cell-derived exosomes promote mitochondrial biogenesis.

A PI+R Control Scheme Based on Multi-Agent Systems for Economic Dispatch in Isolated BESSs

Battery energy storage systems(BESSs)are widely used in smart grids.However,power consumed by inner impedance and the capacity degradation of each battery unit become particularly severe,which has resulted in an increase in operating costs.The general economic dispatch(ED)algorithm based on marginal cost(MC)consensus is usually a proportional(P)controller,which encounters the defects of slow convergence speed and low control accuracy.In order to solve the distributed ED problem of the isolated BESS network with excellent dynamic and steady-state performance,we attempt to design a proportional integral(PI)controller with a reset mechanism(PI+R)to asymptotically promote MC consensus and total power mismatch towards 0 in this paper.To be frank,the integral term in the PI controller is reset to 0 at an appropriate time when the proportional term undergoes a zero crossing,which accelerates convergence,improves control accuracy,and avoids overshoot.The eigenvalues of the system under a PI+R controller is well analyzed,ensuring the regularity of the system and enabling the reset mechanism.To ensure supply and demand balance within the isolated BESSs,a centralized reset mechanism is introduced,so that the controller is distributed in a flow set and centralized in a jump set.To cope with Zeno behavior and input delay,a dwell time that the system resides in a flow set is given.Based on this,the system with input delays can be reduced to a time-delay free system.Considering the capacity limitation of the battery,a modified MC scheme with PI+R controller is designed.The correctness of the designed scheme is verified through relevant simulations.

Optimal Cooperative Secondary Control for Islanded DC Microgrids via a Fully Actuated Approach

DC-DC converter-based multi-bus DC microgrids(MGs) in series have received much attention, where the conflict between voltage recovery and current balancing has been a hot topic. The lack of models that accurately portray the electrical characteristics of actual MGs while is controller design-friendly has kept the issue active. To this end, this paper establishes a large-signal model containing the comprehensive dynamical behavior of the DC MGs based on the theory of high-order fully actuated systems, and proposes distributed optimal control based on this. The proposed secondary control method can achieve the two goals of voltage recovery and current sharing for multi-bus DC MGs. Additionally, the simple structure of the proposed approach is similar to one based on droop control, which allows this control technique to be easily implemented in a variety of modern microgrids with different configurations. In contrast to existing studies, the process of controller design in this paper is closely tied to the actual dynamics of the MGs. It is a prominent feature that enables engineers to customize the performance metrics of the system. In addition, the analysis of the stability of the closed-loop DC microgrid system, as well as the optimality and consensus of current sharing are given. Finally, a scaled-down solar and battery-based microgrid prototype with maximum power point tracking controller is developed in the laboratory to experimentally test the efficacy of the proposed control method.

Real-Time Co-optimization of Gear Shifting and Engine Torque for Predictive Cruise Control of Heavy-Duty Trucks

Fuel consumption is one of the main concerns for heavy-duty trucks.Predictive cruise control(PCC)provides an intriguing opportunity to reduce fuel consumption by using the upcoming road information.In this study,a real-time implementable PCC,which simultaneously optimizes engine torque and gear shifting,is proposed for heavy-duty trucks.To minimize fuel consumption,the problem of the PCC is formulated as a nonlinear model predictive control(MPC),in which the upcoming road elevation information is used.Finding the solution of the nonlinear MPC is time consuming;thus,a real-time implementable solver is developed based on Pontryagin’s maximum principle and indirect shooting method.Dynamic programming(DP)algorithm,as a global optimization algorithm,is used as a performance benchmark for the proposed solver.Simulation,hardware-in-the-loop and real-truck experiments are conducted to verify the performance of the proposed controller.The results demonstrate that the MPC-based solution performs nearly as well as the DP-based solution,with less than 1%deviation for testing roads.Moreover,the proposed co-optimization controller is implementable in a real-truck,and the proposed MPC-based PCC algorithm achieves a fuel-saving rate of 7.9%without compromising the truck’s travel time.

Differential distribution of PINK1 and Parkin in the primate brain implies distinct roles

The vast majority of in vitro studies have demonstrated that PINK1 phosphorylates Parkin to work together in mitophagy to protect against neuronal degeneration.However,it remains largely unclear how PINK1 and Parkin are expressed in mammalian brains.This has been difficult to address because of the intrinsically low levels of PINK1 and undetectable levels of phosphorylated Parkin in small animals.Understanding this issue is critical for elucidating the in vivo roles of PINK1 and Parkin.Recently,we showed that the PINK1 kinase is selectively expressed as a truncated form(PINK1–55)in the primate brain.In the present study,we used multiple antibodies,including our recently developed monoclonal anti-PINK1,to validate the selective expression of PINK1 in the primate brain.We found that PINK1 was stably expressed in the monkey brain at postnatal and adulthood stages,which is consistent with the findings that depleting PINK1 can cause neuronal loss in developing and adult monkey brains.PINK1 was enriched in the membrane-bound fractionations,whereas Parkin was soluble with a distinguishable distribution.Immunofluorescent double staining experiments showed that PINK1 and Parkin did not colocalize under physiological conditions in cultured monkey astrocytes,though they did colocalize on mitochondria when the cells were exposed to mitochondrial stress.These findings suggest that PINK1 and Parkin may have distinct roles beyond their well-known function in mitophagy during mitochondrial damage.

Spatiotemporal Distributions of Bacterioplankton Communities in the Qiantang River(Hangzhou Section),China

The spatial and temporal distribution of bacterioplankton communities plays a vital role in understanding the ecological dynamics and health of aquatic ecosystems.In this study,we conducted a comprehensive investigation of the bacterioplankton communities in the Qiantang River(Hangzhou section).Water samples were collected quarterly from seven sites over a one-year period.Physical and chemical parameters,including dissolved oxygen(DO),water temperature(WT),chemical oxygen demand(COD),nitrite(NO_(2)^(-)),active phosphate(PO_(4)^(3-))and other indices were determined.In this study,theαdiversity,βdiversity and abundance differences of bacterial communities were investigated using 16S rRNA high-throughput sequencing analysis.The spatial and temporal distribution characteristics and main driving factors of the bacterioplankton community structure and diversity were discussed.The results showed that a total of 57 phyla were detected in the bacterioplankton community,among which Proteobacteria and Actinomycetes were the two dominant groups with the highest relative abundance.The results of PCoA based on Bray-Curtis distance showed that the sampling season had a slightly greater effect on the changes in bacterioplankton community structure in the Qiantang River.Mantel and partial Mantel test showed that environmental variables(Mantel r=0.6739,P<0.0001;partial Mantel r=0.507,P=0.0001)were more important than geographical distance(Mantel r=0.5322,P<0.001;partial Mantel r=0.1563,P=0.001).The distance attenuation model showed that there was significant distance attenuation in all four seasons,and the maximum limit of bacterial community diffusion was found in spring.RDA analysis showed that nine environmental factors,including COD,WT and DO,significantly affected community distribution(P<0.05).This study provides valuable insights into the spatial and temporal distribution characteristics of bacterioplankton communities,shedding light on their ecological roles in the Qiantang River.The information obtained can guide future research on the interactions between bacterioplankton and their environment,enabling the development of effective conservation strategies and sustainable management practices for aquatic ecosystems.

Exploring Variability in Sea Level at a Tide Gauge Station through Control Charts

Monitoring temporal changes in sea level is important in assessing coastal risk.Sea level anomalies at a tide gauge station,if kinematically conceived,include systematic variations such as trend,acceleration,periodic oscillations,and random disturbances.Among them,the non-stationary nature of the random sea level variations of known or unknown origin at coastal regions has been long recognized by the sea level community.This study proposes the analyses of subgroups of random residual statistics of a rigorously formulated kinematic model solution of tide gauge variations using X-bar and S control charts.The approach is demonstrated using Key West,Florida tide gauge records.The mean and standard errors of 5-year-long subgroups of the residuals revealed that sea level changes at this location have been progressively intensifying from 1913 to the present.Increasing oscillations in sea level at this locality may be attributed partly to the thermal expansion of seawater with increasing temperatures causing larger buoyancy-related sea level fluctuations as well as the intensification of atmospheric events including wind patterns and the impact of changes in inverted barometer effects that will alter coastal risk assessments for the future.

Analysis of debris flow control effect and hazard assessment in Xinqiao Gully,Wenchuan M_(s)8.0 earthquake area based on numerical simulation

Xinqiao Gully is located in the area of the 2008 Wenchuan M_(s)8.0 earthquake in Sichuan province,China.Based on the investigation of the 2023"6-26"Xinqiao Gully debris flow event,this study assessed the effectiveness of the debris flow control project and evaluated the debris flow hazards.Through field investigation and numerical simulation methods,the indicators of flow intensity reduction rate and storage capacity fullness were proposed to quantify the effectiveness of the engineering measures in the debris flow event.The simulation results show that the debris flow control project reduced the flow intensity by41.05%to 64.61%.The storage capacity of the dam decreases gradually from upstream to the mouth of the gully,thus effectively intercepting and controlling the debris flow.By evaluating the debris flow of different recurrence intervals,further measures are recommended for managing debris flow events.

Research on Volt/Var Control of Distribution Networks Based on PPO Algorithm

In this paper,a model free volt/var control(VVC)algorithm is developed by using deep reinforcement learning(DRL).We transform the VVC problem of distribution networks into the network framework of PPO algorithm,in order to avoid directly solving a large-scale nonlinear optimization problem.We select photovoltaic inverters as agents to adjust system voltage in a distribution network,taking the reactive power output of inverters as action variables.An appropriate reward function is designed to guide the interaction between photovoltaic inverters and the distribution network environment.OPENDSS is used to output system node voltage and network loss.This method realizes the goal of optimal VVC in distribution network.The IEEE 13-bus three phase unbalanced distribution system is used to verify the effectiveness of the proposed algorithm.Simulation results demonstrate that the proposed method has excellent performance in voltage and reactive power regulation of a distribution network.

Stability Analysis and Control of DC Distribution System with Electric Vehicles

The DC distribution network system equipped with a large number of power electronic equipment exhibits weak damping characteristics and is prone to low-frequency and high-frequency unstable oscillations.The current interpretation of the oscillation mechanism has not been unified.Firstly,this paper established the complete statespace model of the distribution system consisting of a large number of electric vehicles,characteristic equation of the distribution network system is derived by establishing a state-space model,and simplified reduced-order equations describing the low-frequency oscillation and the high-frequency oscillation are obtained.Secondly,based on eigenvalue analysis,the oscillation modes and the influence of the key system parameters on the oscillation mode are studied.Besides,impacts of key factors,such as distribution network connection topology and number of dynamic loads,have been discussed to suppress oscillatory instability caused by inappropriate design or dynamic interactions.Finally,using the DC distribution example system,through model calculation and time-domain simulation analysis,the correctness of the aforementioned analysis is verified.

Distributed Secondary Control and Optimal Power Sharing in Microgrids

We address the control problem of microgrids and present a fully distributed control system which consists of primary controller, secondary controller, and optimal active power sharing controller. Different from the existing control structure in microgrids, all these controllers are implemented as local controllers at each distributed generator. Thus, the requirement for a central controller is obviated. The performance analysis of the proposed control systems is provided, and the finite-time convergence properties for distributed secondary frequency and voltage controllers are achieved. Moreover, the distributed control system possesses the optimal active power sharing property. In the end, a microgrid test system is investigated to validate the effectiveness of the proposed control strategies. © 2014 Chinese Association of Automation.