Fixed-Time Antidisturbance Consensus Tracking for Nonlinear Multiagent Systems With Matching and Mismatching Disturbances

In this paper,fixed-time consensus tracking for mul-tiagent systems(MASs)with dynamics in the form of strict feed-back affine nonlinearity is addressed.A fixed-time antidistur-bance consensus tracking protocol is proposed,which consists of a distributed fixed-time observer,a fixed-time disturbance observer,a nonsmooth antidisturbance backstepping controller,and the fixed-time stability analysis is conducted by using the Lyapunov theory correspondingly.This paper includes three main improvements.First,a distributed fixed-time observer is developed for each follower to obtain an estimate of the leader’s output by utilizing the topology of the communication network.Second,a fixed-time disturbance observer is given to estimate the lumped disturbances for feedforward compensation.Finally,a nonsmooth antidisturbance backstepping tracking controller with feedforward compensation for lumped disturbances is designed.In order to mitigate the“explosion of complexity”in the tradi-tional backstepping approach,we have implemented a modified nonsmooth command filter to enhance the performance of the closed-loop system.The simulation results show that the pro-posed method is effective.

Deep Reinforcement Learning or Lyapunov Analysis?A Preliminary Comparative Study on Event-Triggered Optimal Control

Dear Editor,This letter develops a novel method to implement event-triggered optimal control(ETOC) for discrete-time nonlinear systems using parallel control and deep reinforcement learning(DRL), referred to as Deep-ETOC. The developed Deep-ETOC method introduces the communication cost into the performance index through parallel control, so that the developed method enables control systems to learn ETOC policies directly without triggering conditions.

A new optimal adaptive backstepping control approach for nonlinear systems under deception attacks via reinforcement learning

In this paper,a new optimal adaptive backstepping control approach for nonlinear systems under deception attacks via reinforcement learning is presented in this paper.The existence of nonlinear terms in the studied system makes it very difficult to design the optimal controller using traditional methods.To achieve optimal control,RL algorithm based on critic–actor architecture is considered for the nonlinear system.Due to the significant security risks of network transmission,the system is vulnerable to deception attacks,which can make all the system state unavailable.By using the attacked states to design coordinate transformation,the harm brought by unknown deception attacks has been overcome.The presented control strategy can ensure that all signals in the closed-loop system are semi-globally ultimately bounded.Finally,the simulation experiment is shown to prove the effectiveness of the strategy.

Parallel Control for Optimal Tracking via Adaptive Dynamic Programming

This paper studies the problem of optimal parallel tracking control for continuous-time general nonlinear systems.Unlike existing optimal state feedback control,the control input of the optimal parallel control is introduced into the feedback system.However,due to the introduction of control input into the feedback system,the optimal state feedback control methods can not be applied directly.To address this problem,an augmented system and an augmented performance index function are proposed firstly.Thus,the general nonlinear system is transformed into an affine nonlinear system.The difference between the optimal parallel control and the optimal state feedback control is analyzed theoretically.It is proven that the optimal parallel control with the augmented performance index function can be seen as the suboptimal state feedback control with the traditional performance index function.Moreover,an adaptive dynamic programming(ADP)technique is utilized to implement the optimal parallel tracking control using a critic neural network(NN)to approximate the value function online.The stability analysis of the closed-loop system is performed using the Lyapunov theory,and the tracking error and NN weights errors are uniformly ultimately bounded(UUB).Also,the optimal parallel controller guarantees the continuity of the control input under the circumstance that there are finite jump discontinuities in the reference signals.Finally,the effectiveness of the developed optimal parallel control method is verified in two cases.

Optimal Constrained Self-learning Battery Sequential Management in Microgrid Via Adaptive Dynamic Programming

This paper concerns a novel optimal self-learning battery sequential control scheme for smart home energy systems. The main idea is to use the adaptive dynamic programming U+0028 ADP U+0029 technique to obtain the optimal battery sequential control iteratively. First, the battery energy management system model is established, where the power efficiency of the battery is considered. Next, considering the power constraints of the battery, a new non-quadratic form performance index function is established, which guarantees that the value of the iterative control law cannot exceed the maximum charging/discharging power of the battery to extend the service life of the battery. Then, the convergence properties of the iterative ADP algorithm are analyzed, which guarantees that the iterative value function and the iterative control law both reach the optimums. Finally, simulation and comparison results are given to illustrate the performance of the presented method. © 2017 Chinese Association of Automation.

Parallel Control for Continuous-Time Linear Systems:A Case Study

In this paper,a new parallel controller is developed for continuous-time linear systems.The main contribution of the method is to establish a new parallel control law,where both state and control are considered as the input.The structure of the parallel control is provided,and the relationship between the parallel control and traditional feedback controls is presented.Considering the situations that the systems are controllable and incompletely controllable,the properties of the parallel control law are analyzed.The parallel controller design algorithms are given under the conditions that the systems are controllable and incompletely controllable.Finally,numerical simulations are carried out to demonstrate the effectiveness and applicability of the present method.Index Terms-Continuous-time linear systems,digital twin,parallel controller,parallel intelligence,parallel systems.

Residential Energy Scheduling for Variable Weather Solar Energy Based on Adaptive Dynamic Programming

The residential energy scheduling of solar energy is an important research area of smart grid. On the demand side, factors such as household loads, storage batteries, the outside public utility grid and renewable energy resources, are combined together as a nonlinear, time-varying, indefinite and complex system, which is difficult to manage or optimize. Many nations have already applied the residential real-time pricing to balance the burden on their grid. In order to enhance electricity efficiency of the residential micro grid, this paper presents an action dependent heuristic dynamic programming(ADHDP) method to solve the residential energy scheduling problem. The highlights of this paper are listed below. First,the weather-type classification is adopted to establish three types of programming models based on the features of the solar energy. In addition, the priorities of different energy resources are set to reduce the loss of electrical energy transmissions.Second, three ADHDP-based neural networks, which can update themselves during applications, are designed to manage the flows of electricity. Third, simulation results show that the proposed scheduling method has effectively reduced the total electricity cost and improved load balancing process. The comparison with the particle swarm optimization algorithm further proves that the present method has a promising effect on energy management to save cost.

Consensus Control of Leader-Following Multi-Agent Systems in Directed Topology With Heterogeneous Disturbances

This paper investigates the consensus problem for linear multi-agent systems with the heterogeneous disturbances generated by the Brown motion.Its main contribution is that a control scheme is designed to achieve the dynamic consensus for the multi-agent systems in directed topology interfered by stochastic noise.In traditional ways,the coupling weights depending on the communication structure are static.A new distributed controller is designed based on Riccati inequalities,while updating the coupling weights associated with the gain matrix by state errors between adjacent agents.By introducing time-varying coupling weights into this novel control law,the state errors between leader and followers asymptotically converge to the minimum value utilizing the local interaction.Through the Lyapunov directed method and It?formula,the stability of the closed-loop system with the proposed control law is analyzed.Two simulation results conducted by the new and traditional schemes are presented to demonstrate the effectiveness and advantage of the developed control method.

PDP: Parallel Dynamic Programming

Deep reinforcement learning is a focus research area in artificial intelligence. The principle of optimality in dynamic programming is a key to the success of reinforcement learning methods. The principle of adaptive dynamic programming ADP is first presented instead of direct dynamic programming DP , and the inherent relationship between ADP and deep reinforcement learning is developed. Next, analytics intelligence, as the necessary requirement, for the real reinforcement learning, is discussed. Finally, the principle of the parallel dynamic programming, which integrates dynamic programming and analytics intelligence, is presented as the future computational intelligence. © 2014 Chinese Association of Automation.

Parallel cognition:hybrid intelligence for human-machine interaction and management

As an interdisciplinary research approach,traditional cognitive science adopts mainly the experiment,induction,modeling,and validation paradigm.Such models are sometimes not applicable in cyber-physical-socialsystems (CPSSs),where the large number of human users involves severe heterogeneity and dynamics.To reduce the decision-making conflicts between people and machines in human-centered systems,we propose a new research paradigm called parallel cognition that uses the system of intelligent techniques to investigate cognitive activities and functionals in three stages:descriptive cognition based on artificial cognitive systems (ACSs),predictive cognition with computational deliberation experiments,and prescriptive cognition via parallel behavioral prescription.To make iteration of these stages constantly on-line,a hybrid learning method based on both a psychological model and user behavioral data is further proposed to adaptively learn an individual’s cognitive knowledge.Preliminary experiments on two representative scenarios,urban travel behavioral prescription and cognitive visual reasoning,indicate that our parallel cognition learning is effective and feasible for human behavioral prescription,and can thus facilitate human-machine cooperation in both complex engineering and social systems.

Optimal synchronization control formulti-agent systems with input saturation:a nonzero-sum game

This paper presents a novel optimal synchronization control method for multi-agent systems with input saturation.The multi-agent game theory is introduced to transform the optimal synchronization control problem into a multi-agent nonzero-sum game.Then,the Nash equilibrium can be achieved by solving the coupled Hamilton–Jacobi–Bellman(HJB)equations with nonquadratic input energy terms.A novel off-policy reinforcement learning method is presented to obtain the Nash equilibrium solution without the system models,and the critic neural networks(NNs)and actor NNs are introduced to implement the presented method.Theoretical analysis is provided,which shows that the iterative control laws converge to the Nash equilibrium.Simulation results show the good performance of the presented method.

Institutionalized and systematized gaming for multi-agent systems

Multi-agent system gaming(MASG)is widely applied in military intelligence,information networks,unmanned systems,intelligent transportation,and smart grids,exhibiting systematic and organizational characteristics.It requires the multi-agent system perceive and act in a complex dynamic environment and at the same time achieve a balance between individual interests and the maximization of group interests within the system.