Autonomous Vehicle Platoons In Urban Road Networks:A Joint Distributed Reinforcement Learning and Model Predictive Control Approach

In this paper, platoons of autonomous vehicles operating in urban road networks are considered. From a methodological point of view, the problem of interest consists of formally characterizing vehicle state trajectory tubes by means of routing decisions complying with traffic congestion criteria. To this end, a novel distributed control architecture is conceived by taking advantage of two methodologies: deep reinforcement learning and model predictive control. On one hand, the routing decisions are obtained by using a distributed reinforcement learning algorithm that exploits available traffic data at each road junction. On the other hand, a bank of model predictive controllers is in charge of computing the more adequate control action for each involved vehicle. Such tasks are here combined into a single framework:the deep reinforcement learning output(action) is translated into a set-point to be tracked by the model predictive controller;conversely, the current vehicle position, resulting from the application of the control move, is exploited by the deep reinforcement learning unit for improving its reliability. The main novelty of the proposed solution lies in its hybrid nature: on one hand it fully exploits deep reinforcement learning capabilities for decisionmaking purposes;on the other hand, time-varying hard constraints are always satisfied during the dynamical platoon evolution imposed by the computed routing decisions. To efficiently evaluate the performance of the proposed control architecture, a co-design procedure, involving the SUMO and MATLAB platforms, is implemented so that complex operating environments can be used, and the information coming from road maps(links,junctions, obstacles, semaphores, etc.) and vehicle state trajectories can be shared and exchanged. Finally by considering as operating scenario a real entire city block and a platoon of eleven vehicles described by double-integrator models, several simulations have been performed with the aim to put in light the main f eatures of the proposed approach. Moreover, it is important to underline that in different operating scenarios the proposed reinforcement learning scheme is capable of significantly reducing traffic congestion phenomena when compared with well-reputed competitors.

Data-Driven Control of Distributed Event-Triggered Network Systems

The present paper deals with data-driven event-triggered control of a class of unknown discrete-time interconnected systems(a.k.a.network systems).To this end,we start by putting forth a novel distributed event-triggering transmission strategy based on periodic sampling,under which a model-based stability criterion for the closed-loop network system is derived,by leveraging a discrete-time looped-functional approach.Marrying the model-based criterion with a data-driven system representation recently developed in the literature,a purely data-driven stability criterion expressed in the form of linear matrix inequalities(LMIs)is established.Meanwhile,the data-driven stability criterion suggests a means for co-designing the event-triggering coefficient matrix and the feedback control gain matrix using only some offline collected state-input data.Finally,numerical results corroborate the efficacy of the proposed distributed data-driven event-triggered network system(ETS)in cutting off data transmissions and the co-design procedure.

Operation Control Method of Relay Protection in Flexible DC Distribution Network Compatible with Distributed Power Supply

A novel operation control method for relay protection in flexible DC distribution networks with distributed power supply is proposed to address the issue of inaccurate fault location during relay protection,leading to poor performance.The method combines a fault-tolerant fault location method based on long-term and short-term memory networks to accurately locate the fault section.Then,an operation control method for relay protection based on adaptive weight and whale optimization algorithm(WOA)is used to construct an objective function considering the shortest relay protection action time and the smallest impulse current.The adaptive weight and WOA are employed to obtain the optimal strategy for relay protection operation control,reducing the action time and impulse current.Experimental results demonstrate the effectiveness of the proposed method in accurately locating faults and improving relay protection performance.The longest operation time is reduced by 4.7023 s,and the maximum impulse current is limited to 0.3 A,effectively controlling the impact of large impulse currents and enhancing control efficiency.

Optimal distributed resource allocation in a wireless sensor network for control systems

Wireless technology is applied increasingly in networked control systems. A new form of wireless network called wireless sensor network can bring control systems some advantages, such as flexibility and feasibility of network deployment at low costs, while it also raises some new challenges. First, the communication resources shared by all the control loops are limited. Second, the wireless and multi-hop character of sensor network makes the resources scheduling more difficult. Thus, how to effectively allocate the limited communication resources for those control loops is an important problem. In this paper, this problem is formulated as an optimal sampling frequency assignment problem, where the objective function is to maximize the utility of control systems, subject to channel capacity constraints. Then an iterative distributed algorithm based on local buffer information is proposed. Finally, the simulation results show that the proposed algorithm can effectively allocate the limited communication resource in a distributed way. It can achieve the optimal quality of the control system and adapt to the network load changes.

Resilience Against Replay Attacks:A Distributed Model Predictive Control Scheme for Networked Multi-Agent Systems

In this paper,a resilient distributed control scheme against replay attacks for multi-agent networked systems subject to input and state constraints is proposed.The methodological starting point relies on a smart use of predictive arguments with a twofold aim:1)Promptly detect malicious agent behaviors affecting normal system operations;2)Apply specific control actions,based on predictive ideas,for mitigating as much as possible undesirable domino effects resulting from adversary operations.Specifically,the multi-agent system is topologically described by a leader-follower digraph characterized by a unique leader and set-theoretic receding horizon control ideas are exploited to develop a distributed algorithm capable to instantaneously recognize the attacked agent.Finally,numerical simulations are carried out to show benefits and effectiveness of the proposed approach.

Distributed Fault-Tolerant Containment Control for Nonlinear Multi-Agent Systems Under Directed Network Topology via Hierarchical Approach

This paper investigates the distributed fault-tolerant containment control(FTCC)problem of nonlinear multi-agent systems(MASs)under a directed network topology.The proposed control framework which is independent on the global information about the communication topology consists of two layers.Different from most existing distributed fault-tolerant control(FTC)protocols where the fault in one agent may propagate over network,the developed control method can eliminate the phenomenon of fault propagation.Based on the hierarchical control strategy,the FTCC problem with a directed graph can be simplified to the distributed containment control of the upper layer and the fault-tolerant tracking control of the lower layer.Finally,simulation results are given to demonstrate the effectiveness of the proposed control protocol.

Aperiodic Sampled-Data Control of Distributed Networked Control Systems Under Stochastic Cyber-Attacks

This paper examines the stabilization problem of a distributed networked control system under the effect of cyberattacks by employing a hybrid aperiodic triggering mechanism.The cyber-attack considered in the paper is a stochastic deception attack at the sensor-controller end. The probability of the occurrence of attack on a subsystem is represented using a random variable. A decentralized hybrid sampled-data strategy is introduced to save energy consumption and reduce the transmission load of the network. In the proposed decentralized strategy, each subsystem can decide independently whether its state should be transmitted to the controller or not. The scheme of the hybrid triggering mechanism for each subsystem composed of two stages: In the first stage, the next sampling instant is computed using a self-triggering strategy. Subsequently, in the second stage, an event-triggering condition is checked at these sampling instants and the control signal is computed only if the event-triggering condition is violated. The self-triggering condition used in the first stage is dependent on the selection of eventtriggering condition of the second stage. Finally, a comparison of the proposed approach with other triggering mechanisms existing in the literature is presented in terms of the sampling instants,transmission frequency and performance measures through simulation examples.

Distributed control and optimization of process system networks:A review and perspective

Large-scale and complex process systems are essentially interconnected networks.The automated operation of such process networks requires the solution of control and optimization problems in a distributed manner.In this approach,the network is decomposed into several subsystems,each of which is under the supervision of a corresponding computing agent(controller,optimizer).The agents coordinate their control and optimization decisions based on information communication among them.In recent years,algorithms and methods for distributed control and optimization are undergoing rapid development.In this paper,we provide a comprehensive,up-to-date review with perspectives and discussions on possible future directions.

Distributed Optimal Control for Traffic Networks with Fog Computing

This paper presents a distributed optimization strategy for large-scale traffic network based on fog computing. Different from the traditional cloud-based centralized optimization strategy, the fog-based distributed optimization strategy distributes its computing tasks to individual sub-processors, thus significantly reducing computation time. A traffic model is built and a series of communication rules between subsystems are set to ensure that the entire transportation network can be globally optimized while the subsystem is achieving its local optimization. Finally, this paper numerically simulates the operation of the traffic network by mixed-Integer programming, also, compares the advantages and disadvantages of the two optimization strategies.

Developing Environment of Hierarchical and Distributed Real-Time Fire-Control Network

This paper introduces a hierarchical real-time environment for developing ship-bornefire-control system. Advanced computer networks are used to simulate the system with the requiredengagement scenario, including own-ship and parameters, and data processing and transmission,mission calculation, graphical supervision and gunnery ballistics outputting. The simulation systemis able to receive instruction from, or send information to the command-control center. Furthermore,the system can also be used to compare various designed schemes and analyze the accuracy andeffectiveness of the system.

Cosimulation Platform for Distributed Control System via Heterogeneous Network

A cosimulation platform was established for distributed control systems via heterogeneous network,which integrated OPNET and Matlab/Simulink.The communication node in this cosimulation platform was built based on OSI model and UDP protocol,which was adopted as the transportation layer protocol.Data exchanged between the data source module and the specified node.It was fulfilled by revising the corresponding protocol modules based on the characteristics of UDP.The effectiveness of the constructed simulation platform was demonstrated by a numerical example.

Designing Neural Network Model in Distributed Control System

The realizing of Artificial Neural Network(ANN) in Distributed Control System (DCS) is discussed. The model of ANN designed can be called as easily as conventional algorithm. It can act as an ANN controller or as an identifier in adaptive control system.

STUDY ON OPTIMAL CONTROL OF MUNICIPAL WATER DISTRIBUTION NETWORK

A systematic investigation is made on the problems which are related to the optimal control of the municipal water distribution network.A mathematical model of forecasting the water short term demand is proposed using the time series trigonometric function analysis method;the service discharge based macroscopic model of network performance is established using the network structuring method;a relatively satisfactory mathematical model for the optimal control of water distribution network is put forward in view of security and economy,and solved by the constrained mixed discrete variable complex arithmetic.The model is applied in many examples and the results are satisfactory.

Distributed Secondary Control and Optimal Power Sharing in Microgrids

We address the control problem of microgrids and present a fully distributed control system which consists of primary controller, secondary controller, and optimal active power sharing controller. Different from the existing control structure in microgrids, all these controllers are implemented as local controllers at each distributed generator. Thus, the requirement for a central controller is obviated. The performance analysis of the proposed control systems is provided, and the finite-time convergence properties for distributed secondary frequency and voltage controllers are achieved. Moreover, the distributed control system possesses the optimal active power sharing property. In the end, a microgrid test system is investigated to validate the effectiveness of the proposed control strategies. © 2014 Chinese Association of Automation.

A Novel Distributed Optimal Adaptive Control Algorithm for Nonlinear Multi-Agent Differential Graphical Games

In this paper, an online optimal distributed learning algorithm is proposed to solve leader-synchronization problem of nonlinear multi-agent differential graphical games. Each player approximates its optimal control policy using a single-network approximate dynamic programming(ADP) where only one critic neural network(NN) is employed instead of typical actorcritic structure composed of two NNs. The proposed distributed weight tuning laws for critic NNs guarantee stability in the sense of uniform ultimate boundedness(UUB) and convergence of control policies to the Nash equilibrium. In this paper, by introducing novel distributed local operators in weight tuning laws, there is no more requirement for initial stabilizing control policies. Furthermore, the overall closed-loop system stability is guaranteed by Lyapunov stability analysis. Finally, Simulation results show the effectiveness of the proposed algorithm.

A Distributed Decision Mechanism for Controller Load Balancing Based on Switch Migration in SDN

Software Defined Networking(SDN) provides flexible network management by decoupling control plane from data plane. And multiple controllers are deployed to improve the scalability and reliability of the control plane, which could divide the network into several subdomains with separate controllers. However, such deployment introduces a new problem of controller load imbalance due to the dynamic traffic and the static configuration between switches and controllers. To address this issue, this paper proposes a Distribution Decision Mechanism(DDM) based on switch migration in the multiple subdomains SDN network. Firstly, through collecting network information, it constructs distributed migration decision fields based on the controller load condition. Then we choose the migrating switches according to the selection probability, and the target controllers are determined by integrating three network costs, including data collection, switch migration and controller state synchronization. Finally, we set the migrating countdown to achieve the ordered switch migration. Through verifying several evaluation indexes, results show that the proposed mechanism can achieve controller load balancing with better performance.

A novel fractional uplink power control framework for self-organizing networks

Internet of things and network densification bring significant challenges to uplink management.Only depending on optimization algorithm enhancements is not enough for uplink transmission.To control intercell interference,Fractional Uplink Power Control(FUPC)should be optimized from network-wide perspective,which has to find a better traffic distribution model.Conventionally,traffic distribution is geographic-based,and ineffective due to tricky locating efforts.This paper proposes a novel uplink power management framework for Self-Organizing Networks(SON),which firstly builds up pathloss-based traffic distribution model and then makes the decision of FUPC based on the model.PathLoss-based Traffic Distribution(PLTD)aggregates traffic based on the propagation condition of traffic that is defined as the pathloss between the position generating the traffic and surrounding cells.Simulations show that the improvement in optimization efficiency of FUPC with PLTD can be up to 40%compared to conventional GeoGraphic-based Traffic Distribution(GGTD).

Stability Analysis and Control of DC Distribution System with Electric Vehicles

The DC distribution network system equipped with a large number of power electronic equipment exhibits weak damping characteristics and is prone to low-frequency and high-frequency unstable oscillations.The current interpretation of the oscillation mechanism has not been unified.Firstly,this paper established the complete statespace model of the distribution system consisting of a large number of electric vehicles,characteristic equation of the distribution network system is derived by establishing a state-space model,and simplified reduced-order equations describing the low-frequency oscillation and the high-frequency oscillation are obtained.Secondly,based on eigenvalue analysis,the oscillation modes and the influence of the key system parameters on the oscillation mode are studied.Besides,impacts of key factors,such as distribution network connection topology and number of dynamic loads,have been discussed to suppress oscillatory instability caused by inappropriate design or dynamic interactions.Finally,using the DC distribution example system,through model calculation and time-domain simulation analysis,the correctness of the aforementioned analysis is verified.

Performance Analysis of WLAN Medium Access Control Protocols in Simulcast Radio-Over-Fiber-Based Distributed Antenna Systems

The performance of three wireless local-area network(WLAN) media access control(MAC) protocols is investigated and compared in the context of simulcast radioover-fiber-based distributed antenna systems(RoF-DASs) where multiple remote antenna units(RAUs) are connected to one access point(AP) with different-length fiber links.The three WLAN MAC protocols under investigation are distributed coordination function(DCF) in basic access mode,DCF in request/clear to send(RTS/CTS) exchange mode,and point coordination function(PCF).In the analysis,the inter-RAU hidden nodes problems and fiber-length difference effect are both taken into account.Results show that adaptive PCF mechanism has better throughput performances than the other two DCF modes,especially when the inserted fiber length is short.

An Approach of Distributed Joint Optimization for Cluster-based Wireless Sensor Networks

Wireless sensor networks (WSNs) are energyconstrained, so energy saving is one of the most important issues in typical applications. The clustered WSN topology is considered in this paper. To achieve the balance of energy consumption and utility of network resources, we explicitly model and factor the effect of power and rate. A novel joint optimization model is proposed with the protection for cluster head. By the mean of a choice of two appropriate sub-utility functions, the distributed iterative algorithm is obtained. The convergence of the proposed iterative algorithm is proved analytically. We consider general dual decomposition method to realize variable separation and distributed computation, which is practical in large-scale sensor networks. Numerical results show that the proposed joint optimal algorithm converges to the optimal power allocation and rate transmission, and validate the performance in terms of prolonging of network lifetime and improvement of throughput. © 2014 Chinese Association of Automation.