Failure-Informed Adaptive Sampling for PINNs,Part II:Combining with Re-sampling and Subset Simulation

This is the second part of our series works on failure-informed adaptive sampling for physic-informed neural networks(PINNs).In our previous work(SIAM J.Sci.Comput.45:A1971–A1994),we have presented an adaptive sampling framework by using the failure probability as the posterior error indicator,where the truncated Gaussian model has been adopted for estimating the indicator.Here,we present two extensions of that work.The first extension consists in combining with a re-sampling technique,so that the new algorithm can maintain a constant training size.This is achieved through a cosine-annealing,which gradually transforms the sampling of collocation points from uniform to adaptive via the training progress.The second extension is to present the subset simulation(SS)algorithm as the posterior model(instead of the truncated Gaussian model)for estimating the error indicator,which can more effectively estimate the failure probability and generate new effective training points in the failure region.We investigate the performance of the new approach using several challenging problems,and numerical experiments demonstrate a significant improvement over the original algorithm.

Boosting Adversarial Training with Learnable Distribution

In recent years,various adversarial defense methods have been proposed to improve the robustness of deep neural networks.Adversarial training is one of the most potent methods to defend against adversarial attacks.However,the difference in the feature space between natural and adversarial examples hinders the accuracy and robustness of the model in adversarial training.This paper proposes a learnable distribution adversarial training method,aiming to construct the same distribution for training data utilizing the Gaussian mixture model.The distribution centroid is built to classify samples and constrain the distribution of the sample features.The natural and adversarial examples are pushed to the same distribution centroid to improve the accuracy and robustness of the model.The proposed method generates adversarial examples to close the distribution gap between the natural and adversarial examples through an attack algorithm explicitly designed for adversarial training.This algorithm gradually increases the accuracy and robustness of the model by scaling perturbation.Finally,the proposed method outputs the predicted labels and the distance between the sample and the distribution centroid.The distribution characteristics of the samples can be utilized to detect adversarial cases that can potentially evade the model defense.The effectiveness of the proposed method is demonstrated through comprehensive experiments.

Multi-source information fusion:Progress and future

Multi-Source Information Fusion(MSIF),as a comprehensive interdisciplinary field based on modern information technology,has gained significant research value and extensive application prospects in various domains,attracting high attention and interest from scholars,engineering experts,and practitioners worldwide.Despite achieving fruitful results in both theoretical and applied aspects over the past five decades,there remains a lack of comprehensive and systematic review articles that provide an overview of recent development in MSIF.In light of this,this paper aims to assist researchers and individuals interested in gaining a quick understanding of the relevant theoretical techniques and development trends in MSIF,which conducts a statistical analysis of academic reports and related application achievements in the field of MSIF over the past two decades,and provides a brief overview of the relevant theories,methodologies,and application domains,as well as key issues and challenges currently faced.Finally,an analysis and outlook on the future development directions of MSIF are presented.

Integrated Model for Resilience Evaluation of Power-Gas Systems Under Windstorms

Integrated power-gas systems(IPGS)have developed critical infrastructure in integrated energy systems.Moreover,various extreme weather events with low probability and high risk have seriously affected the stable operation of IPGSs.Due to close interconnectedness through coupling elements between the power system(PS)and natural gas system(NGS)when a disturbance happens in one system,a series of complicated sequences of dependent events may follow in another system.Especially under extreme conditions,this coupling can lead to a dramatic degradation of system performance,resulting in catastrophic failures.Therefore,there is an urgent need to model and evaluate resilience of IPGSs under extreme weather.Following this development trend,an integrated model for resilience evaluation of IPGS is proposed under extreme weather events focusing on windstorms.First,a framework of IPGS is proposed to describe states of the system at different stages under disaster conditions.Furthermore,an evaluation model considering cascading effects is used to quantify the impact of windstorms on NGS and PS.Meanwhile,a Monte Carlo simulation(MCS)technique is utilized to characterize chaotic fault of components.Moreover,time-dependent nodal and system resilience indices for IPGS are proposed to display impacts of windstorms.Numerical results on the IPGS test system demonstrate the proposed methods.

Reliability Assessment for Integrated Power-gas Systems Considering Renewable Energy Uncertainty and Cascading Effects

Integrated power-gas systems(IPGSs)make the power system and natural gas system(NGS)as a whole,and strengthen interdependence between the two systems.Due to bidirectional energy conversion in IPGS,a disturbance may turn into a catastrophic outage.Meanwhile,increasing proportion of renewable energy brings challenges to reliability of IPGS.Moreover,partial failure or degradation of system performance leads IPGS operate at multiple performance levels.Therefore,this paper proposes a reliability assessment model of IPGSs which represents multiple performance of components and considers cascading effects,as well as renewable energy uncertainty.First,a framework of IPGS reliability assessment is proposed:multistate models for main elements in the IPGS are represented.Especially a gas-power-generation calculation operator and a power-to-gas calculation operator are utilized to bi-directionally convert a multi-state model between NGS and power systems.Furthermore,nodal reliability indices for IPGS are given to display impacts of cascading effects and renewable energy uncertainty on reliabilities of IPGSs.Numerical results on IPGS test system demonstrate the proposed methods.

Nearly optimal stochastic approximation for online principal subspace estimation

Principal component analysis(PCA) has been widely used in analyzing high-dimensional data. It converts a set of observed data points of possibly correlated variables into a set of linearly uncorrelated variables via an orthogonal transformation. To handle streaming data and reduce the complexities of PCA,(subspace)online PCA iterations were proposed to iteratively update the orthogonal transformation by taking one observed data point at a time. Existing works on the convergence of(subspace) online PCA iterations mostly focus on the case where the samples are almost surely uniformly bounded. In this paper, we analyze the convergence of a subspace online PCA iteration under more practical assumptions and obtain a nearly optimal finite-sample error bound. Our convergence rate almost matches the minimax information lower bound. We prove that the convergence is nearly global in the sense that the subspace online PCA iteration is convergent with high probability for random initial guesses. This work also leads to a simpler proof of the recent work on analyzing online PCA for the first principal component only.

Event-triggered and consensus-based attitude tracking alignment for discrete-time multiple spacecraft system exploiting interference

This paper presents a discrete-time attitude control strategy with equi-global practical stabilizability for aligning the attitude of multiple spacecraft to a predesigned configuration according to a time-variant reference.By utilizing the interference of the wireless channel,the communication scheme designed in this paper can save communication resources,amount of computation,and energy proportionally to the number of spacecraft.The exact discrete-time model and approximate discrete-time model of the consensus-based spacecraft tracking system are given.Then the framework for the design of an event-triggered control scheme for the exact discrete-time system via its approximate models is developed,which avoids the periodic actuation,and Zeno behavior is proved to be excluded.Furthermore,the control scheme can handle the presence of the unknown fading channel.Finally,simulation results are presented to demonstrate the effectiveness of the control strategy.

AN ACCELERATION STRATEGY FOR RANDOMIZE-THEN-OPTIMIZE SAMPLING VIA DEEP NEURAL NETWORKS

Randomize-then-optimize (RTO) is widely used for sampling from posterior distributions in Bayesian inverse problems. However, RTO can be computationally intensive forcomplexity problems due to repetitive evaluations of the expensive forward model and itsgradient. In this work, we present a novel goal-oriented deep neural networks (DNN) surrogate approach to substantially reduce the computation burden of RTO. In particular,we propose to drawn the training points for the DNN-surrogate from a local approximatedposterior distribution – yielding a flexible and efficient sampling algorithm that convergesto the direct RTO approach. We present a Bayesian inverse problem governed by ellipticPDEs to demonstrate the computational accuracy and efficiency of our DNN-RTO approach, which shows that DNN-RTO can significantly outperform the traditional RTO.

Adaptive Ensemble Kalman Inversion with Statistical Linearization

The ensemble Kalman inversion(EKI),inspired by the well-known ensemble Kalman filter,is a derivative-free and parallelizable method for solving inverse problems.The method is appealing for applications in a variety of fields due to its low computational cost and simple implementation.In this paper,we propose an adaptive ensemble Kalman inversion with statistical linearization(AEKI-SL)method for solving inverse problems from a hierarchical Bayesian perspective.Specifically,by adaptively updating the unknown with an EKI and updating the hyper-parameter in the prior model,the method can improve the accuracy of the solutions to the inverse problem.To avoid semi-convergence,we employ Morozov’s discrepancy principle as a stopping criterion.Furthermore,we extend the method to simultaneous estimation of noise levels in order to reduce the randomness of artificially ensemble noise levels.The convergence of the hyper-parameter in prior model is investigated theoretically.Numerical experiments show that our proposed methods outperform the traditional EKI and EKI with statistical linearization(EKI-SL)methods.

Intermittent Behaviors in Coupled Piecewise Expanding Map Lattices

In this paper,we propose a new method to study intermittent behaviors of coupled piecewise-expanding map lattices.We show that the successive transition between ordered and disordered phases occurs for almost every orbit when the coupling is small.That is,lim inf n→∞∑1≤i,j≤m|x_(i)(n)−x_(j)(n)|=0,lim sup n→∞∑1≤i,j≤m|x_(i)(n)−x_(j)(n)|≥c_(0)>0,where xi(n)correspond to the coordinates of m nodes at the iterative step n.Moreover,when the uncoupled system is generated by the tent map and the lattice consists of two nodes,we prove a phase transition occurs between synchronization and intermittent behaviors.That is,limn→∞|x_(1)(n)−x_(2)(n)|=0 for c−1/2<1/4 and intermittent behaviors occur for|c−1/2|>1/4,where 0≤c≤1 is the coupling.