Autonomous Vehicle Platoons In Urban Road Networks:A Joint Distributed Reinforcement Learning and Model Predictive Control Approach

In this paper, platoons of autonomous vehicles operating in urban road networks are considered. From a methodological point of view, the problem of interest consists of formally characterizing vehicle state trajectory tubes by means of routing decisions complying with traffic congestion criteria. To this end, a novel distributed control architecture is conceived by taking advantage of two methodologies: deep reinforcement learning and model predictive control. On one hand, the routing decisions are obtained by using a distributed reinforcement learning algorithm that exploits available traffic data at each road junction. On the other hand, a bank of model predictive controllers is in charge of computing the more adequate control action for each involved vehicle. Such tasks are here combined into a single framework:the deep reinforcement learning output(action) is translated into a set-point to be tracked by the model predictive controller;conversely, the current vehicle position, resulting from the application of the control move, is exploited by the deep reinforcement learning unit for improving its reliability. The main novelty of the proposed solution lies in its hybrid nature: on one hand it fully exploits deep reinforcement learning capabilities for decisionmaking purposes;on the other hand, time-varying hard constraints are always satisfied during the dynamical platoon evolution imposed by the computed routing decisions. To efficiently evaluate the performance of the proposed control architecture, a co-design procedure, involving the SUMO and MATLAB platforms, is implemented so that complex operating environments can be used, and the information coming from road maps(links,junctions, obstacles, semaphores, etc.) and vehicle state trajectories can be shared and exchanged. Finally by considering as operating scenario a real entire city block and a platoon of eleven vehicles described by double-integrator models, several simulations have been performed with the aim to put in light the main f eatures of the proposed approach. Moreover, it is important to underline that in different operating scenarios the proposed reinforcement learning scheme is capable of significantly reducing traffic congestion phenomena when compared with well-reputed competitors.

Distributed fiber optic sensors for tunnel monitoring:A state-of-the-art review

Distributed fiber optic sensors(DFOSs)possess the capability to measure strain and temperature variations over long distances,demonstrating outstanding potential for monitoring underground infrastructure.This study presents a state-of-the-art review of the DFOS applications for monitoring and assessing the deformation behavior of typical tunnel infrastructure,including bored tunnels,conventional tunnels,as well as immersed and cut-and-cover tunnels.DFOS systems based on Brillouin and Rayleigh scattering principles are both considered.When implementing DFOS monitoring,the fiber optic cable can be primarily installed along transverse and longitudinal directions to(1)measure distributed strains by continuously adhering the fiber to the structure’s surface or embedding it in the lining,or(2)measure point displacements by spot-anchoring it on the lining surface.There are four critical aspects of DFOS monitoring,including proper selection of the sensing fiber,selection of the measuring principle for the specific application,design of an effective sensor layout,and establishment of robust field sensor instrumentation.These four issues are comprehensively discussed,and practical suggestions are provided for the implementation of DFOS in tunnel infrastructure monitoring.

GRU-integrated constrained soft actor-critic learning enabled fully distributed scheduling strategy for residential virtual power plant

In this study,a novel residential virtual power plant(RVPP)scheduling method that leverages a gate recurrent unit(GRU)-integrated deep reinforcement learning(DRL)algorithm is proposed.In the proposed scheme,the GRU-integrated DRL algorithm guides the RVPP to participate effectively in both the day-ahead and real-time markets,lowering the electricity purchase costs and consumption risks for end-users.The Lagrangian relaxation technique is introduced to transform the constrained Markov decision process(CMDP)into an unconstrained optimization problem,which guarantees that the constraints are strictly satisfied without determining the penalty coefficients.Furthermore,to enhance the scalability of the constrained soft actor-critic(CSAC)-based RVPP scheduling approach,a fully distributed scheduling architecture was designed to enable plug-and-play in the residential distributed energy resources(RDER).Case studies performed on the constructed RVPP scenario validated the performance of the proposed methodology in enhancing the responsiveness of the RDER to power tariffs,balancing the supply and demand of the power grid,and ensuring customer comfort.

Distributed Nash Equilibrium Seeking Strategies Under Quantized Communication

This paper is concerned with distributed Nash equi librium seeking strategies under quantized communication. In the proposed seeking strategy, a projection operator is synthesized with a gradient search method to achieve the optimization o players' objective functions while restricting their actions within required non-empty, convex and compact domains. In addition, a leader-following consensus protocol, in which quantized informa tion flows are utilized, is employed for information sharing among players. More specifically, logarithmic quantizers and uniform quantizers are investigated under both undirected and connected communication graphs and strongly connected digraphs, respec tively. Through Lyapunov stability analysis, it is shown that play ers' actions can be steered to a neighborhood of the Nash equilib rium with logarithmic and uniform quantizers, and the quanti fied convergence error depends on the parameter of the quan tizer for both undirected and directed cases. A numerical exam ple is given to verify the theoretical results.

Monitoring Surface Deformation Using Distributed Scatterers InSAR

In the past two decades,extensive and in-depth research has been conducted on Time Series InSAR technology with the advancement of high-performance SAR satellites and the accumulation of big SAR data.The introduction of distributed scatterers in Distributed Scatterers InSAR(DS-InSAR)has significantly expanded the application scenarios of InSAR geodetic measurement by increasing the number of measurement points.This study traces the history of DS-InSAR,presents the definition and characteristics of distributed scatterers,and focuses on exploring the relationships and distinctions among proposed algorithms in two crucial steps:statistically homogeneous pixel selection and phase optimization.Additionally,the latest research progress in this field is tracked and the possible development direction in the future is discussed.Through simulation experiments and two real InSAR case studies,the proposed algorithms are compared and verified,and the advantages of DS-InSAR in deformation measurement practice are demonstrated.This work not only offers insights into current trends and focal points for theoretical research on DS-InSAR but also provides practical cases and guidance for applied research.

A Novel Defender-Attacker-Defender Model for Resilient Distributed Generator Planning with Network Reconfiguration and Demand Response

To improve the resilience of a distribution system against extreme weather,a fuel-based distributed generator(DG)allocation model is proposed in this study.In this model,the DGs are placed at the planning stage.When an extreme event occurs,the controllable generators form temporary microgrids(MGs)to restore the load maximally.Simultaneously,a demand response program(DRP)mitigates the imbalance between the power supply and demand during extreme events.To cope with the fault uncertainty,a robust optimization(RO)method is applied to reduce the long-term investment and short-term operation costs.The optimization is formulated as a tri-level defenderattacker-defender(DAD)framework.At the first level,decision-makers work out the DG allocation scheme;at the second level,the attacker finds the optimal attack strategy with maximum damage;and at the third level,restoration measures,namely distribution network reconfiguration(DNR)and demand response are performed.The problem is solved by the nested column and constraint generation(NC&CG)method and the model is validated using an IEEE 33-node system.Case studies validate the effectiveness and superiority of the proposed model according to the enhanced resilience and reduced cost.

Distributed Multicircular Circumnavigation Control for UAVs with Desired Angular Spacing

This paper addresses a multicircular circumnavigation control for UAVs with desired angular spacing around a nonstationary target.By defining a coordinated error relative to neighboring angular spacing,under the premise that target information is perfectly accessible by all nodes,a centralized circular enclosing control strategy is derived for multiple UAVs connected by an undirected graph to allow for formation behaviors concerning the moving target.Besides,to avoid the requirement of target’s states being accessible for each UAV,fixed-time distributed observers are introduced to acquire the state estimates in a fixed-time sense,and the upper boundary of settling time can be determined offline irrespective of initial properties,greatly releasing the burdensome communication traffic.Then,with the aid of fixed-time distributed observers,a distributed circular circumnavigation controller is derived to force all UAVs to collaboratively evolve along the preset circles while keeping a desired angular spacing.It is inferred from Lyapunov stability that all errors are demonstrated to be convergent.Simulations are offered to verify the utility of proposed protocol.

Enhancing Security and Privacy in Distributed Face Recognition Systems through Blockchain and GAN Technologies

The use of privacy-enhanced facial recognition has increased in response to growing concerns about data securityand privacy in the digital age. This trend is spurred by rising demand for face recognition technology in a varietyof industries, including access control, law enforcement, surveillance, and internet communication. However,the growing usage of face recognition technology has created serious concerns about data monitoring and userprivacy preferences, especially in context-aware systems. In response to these problems, this study provides a novelframework that integrates sophisticated approaches such as Generative Adversarial Networks (GANs), Blockchain,and distributed computing to solve privacy concerns while maintaining exact face recognition. The framework’spainstaking design and execution strive to strike a compromise between precise face recognition and protectingpersonal data integrity in an increasingly interconnected environment. Using cutting-edge tools like Dlib for faceanalysis,Ray Cluster for distributed computing, and Blockchain for decentralized identity verification, the proposedsystem provides scalable and secure facial analysis while protecting user privacy. The study’s contributions includethe creation of a sustainable and scalable solution for privacy-aware face recognition, the implementation of flexibleprivacy computing approaches based on Blockchain networks, and the demonstration of higher performanceover previous methods. Specifically, the proposed StyleGAN model has an outstanding accuracy rate of 93.84%while processing high-resolution images from the CelebA-HQ dataset, beating other evaluated models such asProgressive GAN 90.27%, CycleGAN 89.80%, and MGAN 80.80%. With improvements in accuracy, speed, andprivacy protection, the framework has great promise for practical use in a variety of fields that need face recognitiontechnology. This study paves the way for future research in privacy-enhanced face recognition systems, emphasizingthe significance of using cutting-edge technology to meet rising privacy issues in digital identity.

Distributed Dynamic Load in Structural Dynamics by the Impulse-Based Force Estimation Algorithm

This paper proposes a novel approach for identifying distributed dynamic loads in the time domain.Using polynomial andmodal analysis,the load is transformed intomodal space for coefficient identification.This allows the distributed dynamic load with a two-dimensional form in terms of time and space to be simultaneously identified in the form of modal force,thereby achieving dimensionality reduction.The Impulse-based Force Estimation Algorithm is proposed to identify dynamic loads in the time domain.Firstly,the algorithm establishes a recursion scheme based on convolution integral,enabling it to identify loads with a long history and rapidly changing forms over time.Secondly,the algorithm introduces moving mean and polynomial fitting to detrend,enhancing its applicability in load estimation.The aforementioned methodology successfully accomplishes the reconstruction of distributed,instead of centralized,dynamic loads on the continuum in the time domain by utilizing acceleration response.To validate the effectiveness of the method,computational and experimental verification were conducted.

Observer-based dynamic event-triggered control for distributed parameter systems over mobile sensor-plus-actuator networks

We develop a policy of observer-based dynamic event-triggered state feedback control for distributed parameter systems over a mobile sensor-plus-actuator network.It is assumed that the mobile sensing devices that provide spatially averaged state measurements can be used to improve state estimation in the network.For the purpose of decreasing the update frequency of controller and unnecessary sampled data transmission, an efficient dynamic event-triggered control policy is constructed.In an event-triggered system, when an error signal exceeds a specified time-varying threshold, it indicates the occurrence of a typical event.The global asymptotic stability of the event-triggered closed-loop system and the boundedness of the minimum inter-event time can be guaranteed.Based on the linear quadratic optimal regulator, the actuator selects the optimal displacement only when an event occurs.A simulation example is finally used to verify that the effectiveness of such a control strategy can enhance the system performance.

Q-Learning-Assisted Meta-Heuristics for Scheduling Distributed Hybrid Flow Shop Problems

The flow shop scheduling problem is important for the manufacturing industry.Effective flow shop scheduling can bring great benefits to the industry.However,there are few types of research on Distributed Hybrid Flow Shop Problems(DHFSP)by learning assisted meta-heuristics.This work addresses a DHFSP with minimizing the maximum completion time(Makespan).First,a mathematical model is developed for the concerned DHFSP.Second,four Q-learning-assisted meta-heuristics,e.g.,genetic algorithm(GA),artificial bee colony algorithm(ABC),particle swarm optimization(PSO),and differential evolution(DE),are proposed.According to the nature of DHFSP,six local search operations are designed for finding high-quality solutions in local space.Instead of randomselection,Q-learning assists meta-heuristics in choosing the appropriate local search operations during iterations.Finally,based on 60 cases,comprehensive numerical experiments are conducted to assess the effectiveness of the proposed algorithms.The experimental results and discussions prove that using Q-learning to select appropriate local search operations is more effective than the random strategy.To verify the competitiveness of the Q-learning assistedmeta-heuristics,they are compared with the improved iterated greedy algorithm(IIG),which is also for solving DHFSP.The Friedman test is executed on the results by five algorithms.It is concluded that the performance of four Q-learning-assisted meta-heuristics are better than IIG,and the Q-learning-assisted PSO shows the best competitiveness.

Big Data Application Simulation Platform Design for Onboard Distributed Processing of LEO Mega-Constellation Networks

Due to the restricted satellite payloads in LEO mega-constellation networks(LMCNs),remote sensing image analysis,online learning and other big data services desirably need onboard distributed processing(OBDP).In existing technologies,the efficiency of big data applications(BDAs)in distributed systems hinges on the stable-state and low-latency links between worker nodes.However,LMCNs with high-dynamic nodes and long-distance links can not provide the above conditions,which makes the performance of OBDP hard to be intuitively measured.To bridge this gap,a multidimensional simulation platform is indispensable that can simulate the network environment of LMCNs and put BDAs in it for performance testing.Using STK's APIs and parallel computing framework,we achieve real-time simulation for thousands of satellite nodes,which are mapped as application nodes through software defined network(SDN)and container technologies.We elaborate the architecture and mechanism of the simulation platform,and take the Starlink and Hadoop as realistic examples for simulations.The results indicate that LMCNs have dynamic end-to-end latency which fluctuates periodically with the constellation movement.Compared to ground data center networks(GDCNs),LMCNs deteriorate the computing and storage job throughput,which can be alleviated by the utilization of erasure codes and data flow scheduling of worker nodes.

Combined CNN-LSTM Deep Learning Algorithms for Recognizing Human Physical Activities in Large and Distributed Manners:A Recommendation System

Recognizing human activity(HAR)from data in a smartphone sensor plays an important role in the field of health to prevent chronic diseases.Daily and weekly physical activities are recorded on the smartphone and tell the user whether he is moving well or not.Typically,smartphones and their associated sensing devices operate in distributed and unstable environments.Therefore,collecting their data and extracting useful information is a significant challenge.In this context,the aimof this paper is twofold:The first is to analyze human behavior based on the recognition of physical activities.Using the results of physical activity detection and classification,the second part aims to develop a health recommendation system to notify smartphone users about their healthy physical behavior related to their physical activities.This system is based on the calculation of calories burned by each user during physical activities.In this way,conclusions can be drawn about a person’s physical behavior by estimating the number of calories burned after evaluating data collected daily or even weekly following a series of physical workouts.To identify and classify human behavior our methodology is based on artificial intelligence models specifically deep learning techniques like Long Short-Term Memory(LSTM),stacked LSTM,and bidirectional LSTM.Since human activity data contains both spatial and temporal information,we proposed,in this paper,to use of an architecture allowing the extraction of the two types of information simultaneously.While Convolutional Neural Networks(CNN)has an architecture designed for spatial information,our idea is to combine CNN with LSTM to increase classification accuracy by taking into consideration the extraction of both spatial and temporal data.The results obtained achieved an accuracy of 96%.On the other side,the data learned by these algorithms is prone to error and uncertainty.To overcome this constraint and improve performance(96%),we proposed to use the fusion mechanisms.The last combines deep learning classifiers tomodel non-accurate and ambiguous data to obtain synthetic information to aid in decision-making.The Voting and Dempster-Shafer(DS)approaches are employed.The results showed that fused classifiers based on DS theory outperformed individual classifiers(96%)with the highest accuracy level of 98%.Also,the findings disclosed that participants engaging in physical activities are healthy,showcasing a disparity in the distribution of physical activities between men and women.

Recurrent neural network decoding of rotated surface codes based on distributed strategy

Quantum error correction is a crucial technology for realizing quantum computers.These computers achieve faulttolerant quantum computing by detecting and correcting errors using decoding algorithms.Quantum error correction using neural network-based machine learning methods is a promising approach that is adapted to physical systems without the need to build noise models.In this paper,we use a distributed decoding strategy,which effectively alleviates the problem of exponential growth of the training set required for neural networks as the code distance of quantum error-correcting codes increases.Our decoding algorithm is based on renormalization group decoding and recurrent neural network decoder.The recurrent neural network is trained through the ResNet architecture to improve its decoding accuracy.Then we test the decoding performance of our distributed strategy decoder,recurrent neural network decoder,and the classic minimum weight perfect matching(MWPM)decoder for rotated surface codes with different code distances under the circuit noise model,the thresholds of these three decoders are about 0.0052,0.0051,and 0.0049,respectively.Our results demonstrate that the distributed strategy decoder outperforms the other two decoders,achieving approximately a 5%improvement in decoding efficiency compared to the MWPM decoder and approximately a 2%improvement compared to the recurrent neural network decoder.

Distributed Fault Estimation for Nonlinear Systems With Sensor Saturation and Deception Attacks Using Stochastic Communication Protocols

This paper is aimed at the distributed fault estimation issue associated with the potential loss of actuator efficiency for a type of discrete-time nonlinear systems with sensor saturation.For the distributed estimation structure under consideration,an estimation center is not necessary,and the estimator derives its information from itself and neighboring nodes,which fuses the state vector and the measurement vector.In an effort to cut down data conflicts in communication networks,the stochastic communication protocol(SCP)is employed so that the output signals from sensors can be selected.Additionally,a recursive security estimator scheme is created since attackers randomly inject malicious signals into the selected data.On this basis,sufficient conditions for a fault estimator with less conservatism are presented which ensure an upper bound of the estimation error covariance and the mean-square exponential boundedness of the estimating error.Finally,a numerical example is used to show the reliability and effectiveness of the considered distributed estimation algorithm.

Sparse Adversarial Learning for FDIA Attack Sample Generation in Distributed Smart

False data injection attack(FDIA)is an attack that affects the stability of grid cyber-physical system(GCPS)by evading the detecting mechanism of bad data.Existing FDIA detection methods usually employ complex neural networkmodels to detect FDIA attacks.However,they overlook the fact that FDIA attack samples at public-private network edges are extremely sparse,making it difficult for neural network models to obtain sufficient samples to construct a robust detection model.To address this problem,this paper designs an efficient sample generative adversarial model of FDIA attack in public-private network edge,which can effectively bypass the detectionmodel to threaten the power grid system.A generative adversarial network(GAN)framework is first constructed by combining residual networks(ResNet)with fully connected networks(FCN).Then,a sparse adversarial learning model is built by integrating the time-aligned data and normal data,which is used to learn the distribution characteristics between normal data and attack data through iterative confrontation.Furthermore,we introduce a Gaussian hybrid distributionmatrix by aggregating the network structure of attack data characteristics and normal data characteristics,which can connect and calculate FDIA data with normal characteristics.Finally,efficient FDIA attack samples can be sequentially generated through interactive adversarial learning.Extensive simulation experiments are conducted with IEEE 14-bus and IEEE 118-bus system data,and the results demonstrate that the generated attack samples of the proposed model can present superior performance compared to state-of-the-art models in terms of attack strength,robustness,and covert capability.

Evaluation of Multi-Temporal-Spatial Scale Adjustment Capability and Cluster Optimization Operation Method for Distribution Networks with Distributed Photovoltaics

Themassive integration of high-proportioned distributed photovoltaics into distribution networks poses significant challenges to the flexible regulation capabilities of distribution stations.To accurately assess the flexible regulation capabilities of distribution stations,amulti-temporal and spatial scale regulation capability assessment technique is proposed for distribution station areas with distributed photovoltaics,considering different geographical locations,coverage areas,and response capabilities.Firstly,the multi-temporal scale regulation characteristics and response capabilities of different regulation resources in distribution station areas are analyzed,and a resource regulation capability model is established to quantify the adjustable range of different regulation resources.On this basis,considering the limitations of line transmission capacity,a regulation capability assessment index for distribution stations is proposed to evaluate their regulation capabilities.Secondly,considering different geographical locations and coverage areas,a comprehensive performance index based on electrical distance modularity and active power balance is established,and a cluster division method based on genetic algorithms is proposed to fully leverage the coordination and complementarity among nodes and improve the active power matching degree within clusters.Simultaneously,an economic optimization model with the objective of minimizing the economic cost of the distribution station is established,comprehensively considering the safety constraints of the distribution network and the regulation constraints of resources.This model can provide scientific guidance for the economic dispatch of the distribution station area.Finally,case studies demonstrate that the proposed assessment and optimization methods effectively evaluate the regulation capabilities of distribution stations,facilitate the consumption of distributed photovoltaics,and enhance the economic efficiency of the distribution station area.

A Distributed Photovoltaics Ordering Grid-Connected Method for Analyzing Voltage Impact in Radial Distribution Networks

In recent years,distributed photovoltaics(DPV)has ushered in a good development situation due to the advantages of pollution-free power generation,full utilization of the ground or roof of the installation site,and balancing a large number of loads nearby.However,under the background of a large-scale DPV grid-connected to the county distribution network,an effective analysis method is needed to analyze its impact on the voltage of the distribution network in the early development stage of DPV.Therefore,a DPV orderly grid-connected method based on photovoltaics grid-connected order degree(PGOD)is proposed.This method aims to orderly analyze the change of voltage in the distribution network when large-scale DPV will be connected.Firstly,based on the voltagemagnitude sensitivity(VMS)index of the photovoltaics permitted grid-connected node and the acceptance of grid-connected node(AoGCN)index of other nodes in the network,thePGODindex is constructed to determine the photovoltaics permitted grid-connected node of the current photovoltaics grid-connected state network.Secondly,a photovoltaics orderly grid-connected model with a continuous updating state is constructed to obtain an orderly DPV grid-connected order.The simulation results illustrate that the photovoltaics grid-connected order determined by this method based on PGOD can effectively analyze the voltage impact of large-scale photovoltaics grid-connected,and explore the internal factors and characteristics of the impact.

Distributed Optimal Variational GNE Seeking in Merely Monotone Games

In this paper, the optimal variational generalized Nash equilibrium(v-GNE) seeking problem in merely monotone games with linearly coupled cost functions is investigated, in which the feasible strategy domain of each agent is coupled through an affine constraint. A distributed algorithm based on the hybrid steepest descent method is first proposed to seek the optimal v-GNE. Then, an accelerated algorithm with relaxation is proposed and analyzed, which has the potential to further improve the convergence speed to the optimal v-GNE. Some sufficient conditions in both algorithms are obtained to ensure the global convergence towards the optimal v-GNE. To illustrate the performance of the algorithms, numerical simulation is conducted based on a networked Nash-Cournot game with bounded market capacities.

Integrated wellbore-reservoir-geomechanics modeling for enhanced interpretation of distributed fiber-optic strain sensing data in hydraulicfracture analysis

Fiber-optic distributed strain sensing(FO-DSS)has been successful in monitoring strain changes along horizontal wellbores in hydraulically fractured reservoirs.However,the mechanism driving the various FO-DSS responses associated with near-wellbore hydraulic fracture properties is still unclear.To address this knowledge gap,we use coupled wellbore-reservoir-geomechanics simulations to study measured strain-change behavior and infer hydraulic fracture characteristics.The crossflow among fractures is captured through explicit modeling of the transient wellbore flow.In addition,local grid refinement is applied to accurately capture strain changes along the fiber.A Base Case model was designed with four fractures of varying properties,simulating strain change signals when the production well is shut-in for 10 d after 240 d of production and reopened for 2 d.Strain-pressure plots for different fracture clusters were used to gain insights into inferring fracture properties using DSS data.When comparing the model with and without the wellbore,distinct strain change signals were observed,emphasizing the importance of incorporating the wellbore in FO-DSS modeling.The effects of fracture spacing and matrix permeability on strain change signals were thoroughly investigated.The results of our numerical study can improve the understanding of the relation between DSS signals and fracture hydraulic properties,thus maximizing the value of the dataset for fracture diagnostics and characterization.