Regional Multi-Agent Cooperative Reinforcement Learning for City-Level Traffic Grid Signal Control

This article studies the effective traffic signal control problem of multiple intersections in a city-level traffic system.A novel regional multi-agent cooperative reinforcement learning algorithm called RegionSTLight is proposed to improve the traffic efficiency.Firstly a regional multi-agent Q-learning framework is proposed,which can equivalently decompose the global Q value of the traffic system into the local values of several regions Based on the framework and the idea of human-machine cooperation,a dynamic zoning method is designed to divide the traffic network into several strong-coupled regions according to realtime traffic flow densities.In order to achieve better cooperation inside each region,a lightweight spatio-temporal fusion feature extraction network is designed.The experiments in synthetic real-world and city-level scenarios show that the proposed RegionS TLight converges more quickly,is more stable,and obtains better asymptotic performance compared to state-of-theart models.

Neural Dynamics for Cooperative Motion Control of Omnidirectional Mobile Manipulators in the Presence of Noises: A Distributed Approach

This paper presents a distributed scheme with limited communications, aiming to achieve cooperative motion control for multiple omnidirectional mobile manipulators(MOMMs).The proposed scheme extends the existing single-agent motion control to cater to scenarios involving the cooperative operation of MOMMs. Specifically, squeeze-free cooperative load transportation is achieved for the end-effectors of MOMMs by incorporating cooperative repetitive motion planning(CRMP), while guiding each individual to desired poses. Then, the distributed scheme is formulated as a time-varying quadratic programming(QP) and solved online utilizing a noise-tolerant zeroing neural network(NTZNN). Theoretical analysis shows that the NTZNN model converges globally to the optimal solution of QP in the presence of noise. Finally, the effectiveness of the control design is demonstrated by numerical simulations and physical platform experiments.

A Lane Change Model Considering the Stability of Cooperative Adaptive Cruise Control Platoon Fleet

In this article,lane change models for mixed traffic flow under cooperative adaptive cruise control(CACC)platoon formation are established.The analysis begins by examining the impact of lane changes on traffic flow stability.The influences of various factors such as lane change locations,timing,and the current traffic state on stability are discussed.In this analysis,it is assumed that the lane change location and the entry position in the adjacent lane have already been selected,without considering the specific intention behind the lane change.The speeds of the involved vehicles are adjusted based on an existing lane change model,and various conditions are analyzed for traffic flow disturbances,including duration,shock amplitude,and driving delays.Numerical calculations are provided to illustrate these effects.Additionally,traffic flow stability is factored into the lane change decision-making process.By incorporating disturbances to the fleet into the lane change income model,both a lane change intention model and a lane change execution model are constructed.These models are then compared with a model that does not account for stability,leading to the corresponding conclusions.

Cooperative Adaptive Cruise Control Platoon Fleet Formation Model Considering Efficiency and Stability

In the existing formation model,vehicles in the same lane or adjacent lane are regarded as the structure,and the driving behavior of vehicles is studied from the perspectives of safety,speed consistency,and stability,and the speed control model is proposed from the perspective of vehicles themselves,to obtain a stable fleet with the same distance and speed.However,in this process,the initial condition of the vehicle,the traffic flow environment,and the efficiency of the fleet formation are less considered.Therefore,based on summarizing the existing fleet building model,this paper puts forward the rapid construction model and algorithm of a cooperative adaptive cruise control platoon fleet.One of the important goals of forming a team is to enter the team with the smoothest trajectory in the shortest time.Therefore,this chapter studies the trajectory optimization of the vehicle formation process from the perspective of vehicle dynamics.

Analysis of International Cooperation Management Strategies of Enterprises under the Background of the“Belt and Road”

With the deepening of the“Belt and Road”initiative,Chinese enterprises are facing unprecedented opportunities and challenges for international cooperation.This paper mainly analyzes the current situation,existing problems,and coping strategies of enterprise international cooperation management in the context of the“Belt and Road”.This article expounds on the importance of the“Belt and Road”initiative for international cooperation of enterprises and analyzes the key links of international cooperation management of enterprises from the aspects of internationalization strategy,cross-cultural management,risk prevention and control,and resource integration.Finally,combined with the case,this paper puts forward the management strategies and suggestions that enterprises should adopt in international cooperation,to provide a useful reference for Chinese enterprises in international cooperation in the construction of the“Belt and Road.”

Power Optimization Cooperative Control Strategy for Flexible Fast Interconnection Device with Energy Storage

With the wide application of renewable energy power generation technology,the distribution network presents the characteristics of multi-source and complex structure.There are potential risks in the stability of power system,and the problem of power quality is becoming more and more serious.This paper studies and proposes a power optimization cooperative control strategy for flexible fast interconnection device with energy storage,which combines the flexible interconnection technology with the energy storage device.The primary technology is to regulate the active and reactive power of the converter.By comparing the actual power value of the converter with the reference value,the proportional integral(PI)controller is used for correction,and the current components of d and q axes are obtained and input to the converter as the reference value of the current inner loop.The control strategy in this paper can realize power mutual aid between feeders,and at the same time,the energy storage device can provide or absorb a certain amount of power for feeders,so that the power grid can realize stable operation in a certain range.

Communication Resource-Efficient Vehicle Platooning Control With Various Spacing Policies

Platooning represents one of the key features that connected automated vehicles may possess as it allows multiple automated vehicles to be maneuvered cooperatively with small headways on roads. However, a critical challenge in accomplishing automated vehicle platoons is to deal with the effects of intermittent and sporadic vehicle-to-vehicle data transmissions caused by limited wireless communication resources. This paper addresses the co-design problem of dynamic event-triggered communication scheduling and cooperative adaptive cruise control for a convoy of automated vehicles with diverse spacing policies. The central aim is to achieve automated vehicle platooning under various gap references with desired platoon stability and spacing performance requirements, while simultaneously improving communication efficiency. Toward this aim, a dynamic event-triggered scheduling mechanism is developed such that the intervehicle data transmissions are scheduled dynamically and efficiently over time. Then, a tractable co-design criterion on the existence of both the admissible event-driven cooperative adaptive cruise control law and the desired scheduling mechanism is derived. Finally, comparative simulation results are presented to substantiate the effectiveness and merits of the obtained results.

THE OPTIMAL CONTROL FOR PROMOTING THE COOPERATION IN EVOLUTION GAME GENERATED BY PRISONER'S DILEMMA

Natural selection opposes the evolution of cooperation unless specific mechanisms are at work in Prisoner＇s Dilemma. By taking advantage of the modern control theory, the controller design is discussed and the optimal control is designed for promoting cooperation based on the recent advances in mechanisms for the evolution of cooperation. Two con- trol strategies are proposed： compensation control strategy for the cooperator when playing against a defector and reward control strategy for cooperator when playing against a coop- erator. The feasibility and effectiveness of these control strategies for promoting cooperation in different stages are analyzed. The reward for cooperation can＇t prevent defection from being evolutionary stable strategy （ESS）. On the other hand, compensation for the coopera- tor can＇t prevent defection from emerging and sustaining. By considering the effect and the cost, an optimal control scheme with constraint on the admissible control set is put forward. By analyzing the special nonlinear system of replicator dynamics, the exact analytic solution of the optimal control scheme is obtained based on the maximum principle. Finally, the effectiveness of the proposed method is illustrated by examples.

Distributed Model Predictive Control for Networked Plant-wide Systems With Neighborhood Cooperation

For large-scale networked plant-wide systems composed by physically(or geographically) divided subsystems, only limited information is available for local controllers on account of region and communication restrictions. Concerning the optimal control problem of such subsystems, a neighbor-based distributed model predictive control(NDMPC) strategy is presented to improve the global system performance. In this scheme, the performance index of local subsystems and that of its neighbors are minimized together in the determination of the optimal control input, which makes the local control decision also beneficial to its neighboring subsystems and further contributes to improving the convergence and control performance of overall system.The stability of the closed-loop system is proved. Moreover, the parameter designing method for distributed synthesis is provided.Finally, the simulation results illustrate the main characteristics and effectiveness of the proposed control scheme.

Research on the cooperative train control method in the metro system for energy saving

Purpose–To address the problem that the current train operation mode that train selects one of several offline pre-generated control schemes before the departure and operates following the scheme after the departure,energy-saving performance of the whole metro system cannot be guaranteed.Design/methodology/approach–A cooperative train control framework is formulated to regulate a novel train operation mode.The classic train four-phase control strategy is improved for generating specific energy-efficient control schemes for each train.An improved brute force(BF)algorithm with a two-layer searching idea is designed to solve the optimisation model of energy-efficient train control schemes.Findings–Case studies on the actual metro line in Guangzhou,China verify the effectiveness of the proposed train control methods compared with four-phase control strategy under different kinds of train operation scenarios and calculation parameters.The verification on the computation efficiency as well as accuracy of the proposed algorithm indicates that it meets the requirement of online optimisation.Originality/value–Most existing studies optimised energy-efficient train timetable or train control strategies through an offline process,which has a defect in coping with the disturbance or delays effectively and promptly during real-time train operation.This paper studies an online optimisation of cooperative train control based on the rolling optimisation idea,where energy-efficient train operation can be realised once train running time is determined,thus mitigating the impact of unpredictable operation situations on the energy-saving performance of trains.

Multi-UAV cooperative maneuver decision-making for pursuitevasion using improved MADRL

Aiming at the problem of multi-UAV pursuit-evasion confrontation, a UAV cooperative maneuver method based on an improved multi-agent deep reinforcement learning(MADRL) is proposed. In this method, an improved Comm Net network based on a communication mechanism is introduced into a deep reinforcement learning algorithm to solve the multi-agent problem. A layer of gated recurrent unit(GRU) is added to the actor-network structure to remember historical environmental states. Subsequently,another GRU is designed as a communication channel in the Comm Net core network layer to refine communication information between UAVs. Finally, the simulation results of the algorithm in two sets of scenarios are given, and the results show that the method has good effectiveness and applicability.

High-speed train cooperativecontrol based on fractional-ordersliding mode adaptive algorithm

Purpose–This study aims to propose an adaptive fractional-order sliding mode controller to solve the problem of train speed tracking control and position interval control under disturbance environment in moving block system,so as to improve the tracking efficiency and collision avoidance performance.Design/methodology/approach–The mathematical model of information interaction between trains is established based on algebraic graph theory,so that the train can obtain the state information of adjacent trains,and then realize the distributed cooperative control of each train.In the controller design,the sliding mode control and fractional calculus are combined to avoid the discontinuous switching phenomenon,so as to suppress the chattering of sliding mode control,and a parameter adaptive law is constructed to approximate the time-varying operating resistance coefficient.Findings–The simulation results show that compared with proportional integral derivative(PID)control and ordinary sliding mode control,the control accuracy of the proposed algorithm in terms of speed is,respectively,improved by 25%and 75%.The error frequency and fluctuation range of the proposed algorithm are reduced in the position error control,the error value tends to 0,and the operation trend tends to be consistent.Therefore,the control method can improve the control accuracy of the system and prove that it has strong immunity.Originality/value–The algorithm can reduce the influence of external interference in the actual operating environment,realize efficient and stable tracking of trains,and ensure the safety of train control.

Cooperative Rate Splitting Transmit Design for Full-Duplex-Enabled Multiple Multicast Communication Systems

This paper examines the performance of Full-Duplex Cooperative Rate Splitting(FD-CRS)with Simultaneous Wireless Information and Power Transfer(SWIPT)support in Multiple Input Single Output(MISO)networks.In a Rate Splitting Multiple Access(RSMA)multicast system with two local users and one remote user,the common data stream contains the needs of all users,and all users can decode the common data stream.Therefore,each user can receive some information that other users need,and local users with better channel conditions can use this information to further enhance the reception reliability and data rate of users with poor channel quality.Even using Cell-Center-Users(CCUs)as a cooperative relay to assist the transmission of common data can improve the average system speed.To maximize the minimum achievable rate,we optimize the beamforming vector of Base Station(BS),the common streamsplitting vector,the cooperative distributed beamvector and the strong user transmission power under the power budget constraints of BS and relay devices and the service quality requirements constraints of users.Since the whole problem is not convex,we cannot solve it directly.Therefore,we propose a low complexity algorithm based on Successive Convex Approximation(SCA)technology to find the optimal solution to the problemunder consideration.The simulation results show that FD C-RSMA has better gain andmore powerful than FD C-NOMA,HD C-RSMA,RSMA and NOMA.

S^2 BHCA-Multiple AUVs cooperation oriented control architecture

Oceanographic survey, or other similar applications should be the applications of multiple AUVs. In this paper, the skill & simulation based hybrid control architecture (S2BHCA) as the controller's design reference was proposed. It is a multi-robot cooperation oriented intelligent control architecture based on hybrid ideas. The S2 BHCA attempts to incorporate the virtues of the reactive controller and of the deliberative controller by introducing the concept of the "skill". The additional online task simulation ability for cooperation is supported, too. As an application, a multiple AUV control system was developed with three "skills" for the MCM mission including two different cooperative tasks. The simulation and the sea trials show that simple task expression, fast reaction and better cooperation support can be achieved by realizing the AUV controller based on the S2 BHCA.

Receiver-Controlled Multi-Node Cooperation

Since wireless links in Ad hoc networks are more fragile than those in traditional wireless networks due to route flapping,multi-node cooperation plays an important role in ensuring the quality of service( QoS). Based on the authors' previous work,this paper proposes a receiver-controlled multi-node cooperation routing protocol,known as AODV-RCC. In this protocol,nodes form a cooperation group based on signal power. In a cooperation group,signal power between a partner and a transmitter,as well as signal power between the partner and the receiver,must be larger than the signal power between the transmitter and the receiver. Otherwise,the transmission will not benefit from cooperation. To avoid collision or congestion,each cooperation group only contains one partner. This partner offers both data and ACK cooperative retransmission. Its retransmission time should be shorter than the internal retry time of the transmitter's MAC layer,because it is better for the partner to retransmit firstly,as it offers a more reliable cooperative link. In AODV-RCC,it is the receiver that chooses the partner,because the link between the partner and the receiver is the most important. According to our simulation results,AODV-RCC shortens the end-to-end delay and increases the packet delivery ratio.

STABILITY ANALYSIS AND COOPERATIVE CONTROL OF DISTRIBUTED MULTI-AGENT SYSTEM WITH SAMPLED COMMUNICATION

The cooperative control and stability analysis problems for the multi-agent system with sampled com- munication are investigated. Distributed state feedback controllers are adopted for the cooperation of networked agents. A theorem in the form of linear matrix inequalities（LMI） is derived to analyze the system stability. An- other theorem in the form of optimization problem subject to LMI constraints is proposed to design the controller, and then the algorithm is presented. The simulation results verify the validity and the effectiveness of the pro- posed approach.

Connect & Drive:design and evaluation of cooperative adaptive cruise control for congestion reduction

Road throughput can be increased by driving at small inter-vehicle time gaps. The amplification of velocity disturbances in upstream direction, however, poses limitations to the minimum feasible time gap. This effect is covered by the notion of string stability. String-stable behavior is thus considered an essential requirement for the design of automatic distance control systems, which are needed to allow for safe driving at time gaps well below 1 s. Using wireless inter-vehicle communications to provide real-time information of the preceding vehicle, in addition to the information obtained by common Adaptive Cruise Control （ACC） sensors, appears to significantly decrease the feasible time gap, which is shown by practical experiments with a test fleet consisting of six passenger vehicles. The large-scale deployment of this system, known as Cooperative ACC （CACC）, however, poses challenges with respect to the reliability of the wireless communication system. A solution for this scalability problem can be found in decreasing the transmission power and/or beaconing rate, or adapting the communications protocol. Although the main CACC objective is to increase road throughput, the first commercial application of CACC is foreseen to be in truck platooning, since short distance following is expected to yield significant fuel savings in this case.

Distributed cooperative control of autonomous multi-agent UAV systems using smooth control

This paper addresses the cooperative control problem of multiple unmanned aerial vehicles(multi-UAV)systems.First,a new distributed consensus algorithm for second-order nonlinear multi-agent systems(MAS)is formulated under the leader-following approach.The algorithm provides smooth input signals to the agents’control channels,which avoids the chattering effect generated by the conventional sliding mode-based control protocols.Second,a new formation control scheme is developed by integrating smooth distributed consensus control protocols into the geometric pattern model to achieve three-dimensional formation tracking.The Lyapunov theory is used to prove the stability and convergence of both distributed consensus and formation controllers.The effectiveness of the proposed algorithms is demonstrated through simulation results.

Cooperative adaptive bidirectional control of a train platoon for efficient utility and string stability

This paper proposes cooperative adaptive control schemes for a train platoon to improve efficient utility and guarantee string stability. The control schemes are developed based on a bidirectional strategy, i.e., the information of proximal（preceding and following） trains is used in the controller design. Based on available proximal information（prox-info） of location, speed, and acceleration, a direct adaptive control is designed to maintain the tracking interval at the minimum safe distance. Based on available prox-info of location, an observer-based adaptive control is designed to achieve the same target, which alleviates the requirements of equipped sensors to measure prox-info of speed and acceleration. The developed schemes are capable of on-line estimating of the unknown system parameters and stabilizing the closed-loop system, the string stability of train platoon is guaranteed on the basis of Lyapunov stability theorem. Numerical simulation results are presented to verify the effectiveness of the proposed control laws.

Distributed Cooperative Control Algorithm for Multi-UAV Mission Rendezvous

Multiple unmanned aerial vehicles(UAVs)cooperative operation is the main form for UAVs fighting in battlefield,and multi-UAV mission rendezvous is the premise of cooperative reconnaissance and attack missions.We propose a rendezvous control strategy,which divides the rendezvous process into two parts:The loose formation rendezvous and the close formation rendezvous.In the first stage,UAVs are supposed to reach the specific target locations simultaneously and form a loose formation.A distributed control strategy based on first-order consensus algorithm is presented to achieve this goal.Then the second stage is designed based on the second-order consensus algorithm to complete the transition from the loose formation to the close formation.This process needs the speeds and heading angles of UAVs to reach an agreement.Besides,control algorithms with a virtual leader are proposed,by which the formation states can reach a specific value.Finally,simulation results show that the control algorithms are capable of realizing the mission rendezvous of multi-UAV and the consistence of UAVs′final states,which verify the effectiveness and feasibility of the designed control strategy.