Adaptive Ensemble Kalman Inversion with Statistical Linearization

The ensemble Kalman inversion(EKI),inspired by the well-known ensemble Kalman filter,is a derivative-free and parallelizable method for solving inverse problems.The method is appealing for applications in a variety of fields due to its low computational cost and simple implementation.In this paper,we propose an adaptive ensemble Kalman inversion with statistical linearization(AEKI-SL)method for solving inverse problems from a hierarchical Bayesian perspective.Specifically,by adaptively updating the unknown with an EKI and updating the hyper-parameter in the prior model,the method can improve the accuracy of the solutions to the inverse problem.To avoid semi-convergence,we employ Morozov’s discrepancy principle as a stopping criterion.Furthermore,we extend the method to simultaneous estimation of noise levels in order to reduce the randomness of artificially ensemble noise levels.The convergence of the hyper-parameter in prior model is investigated theoretically.Numerical experiments show that our proposed methods outperform the traditional EKI and EKI with statistical linearization(EKI-SL)methods.

VAE-KRnet and its Applications to Variational Bayes

In this work,we have proposed a generative model,called VAE-KRnet,for density estimation or approximation,which combines the canonical variational autoencoder(VAE)with our recently developed flow-based generativemodel,called KRnet.VAE is used as a dimension reduction technique to capture the latent space,and KRnet is used to model the distribution of the latent variable.Using a linear model between the data and the latent variable,we show that VAE-KRnet can be more effective and robust than the canonical VAE.VAE-KRnet can be used as a density model to approximate either data distribution or an arbitrary probability density function(PDF)known up to a constant.VAE-KRnet is flexible in terms of dimensionality.When the number of dimensions is relatively small,KRnet can effectively approximate the distribution in terms of the original random variable.For high-dimensional cases,we may use VAE-KRnet to incorporate dimension reduction.One important application of VAE-KRnet is the variational Bayes for the approximation of the posterior distribution.The variational Bayes approaches are usually based on the minimization of the Kullback-Leibler(KL)divergence between the model and the posterior.For highdimensional distributions,it is very challenging to construct an accurate densitymodel due to the curse of dimensionality,where extra assumptions are often introduced for efficiency.For instance,the classical mean-field approach assumes mutual independence between dimensions,which often yields an underestimated variance due to oversimplification.To alleviate this issue,we include into the loss the maximization of the mutual information between the latent random variable and the original random variable,which helps keep more information from the region of low density such that the estimation of variance is improved.Numerical experiments have been presented to demonstrate the effectiveness of our model.

AN ACCELERATION STRATEGY FOR RANDOMIZE-THEN-OPTIMIZE SAMPLING VIA DEEP NEURAL NETWORKS

Randomize-then-optimize (RTO) is widely used for sampling from posterior distributions in Bayesian inverse problems. However, RTO can be computationally intensive forcomplexity problems due to repetitive evaluations of the expensive forward model and itsgradient. In this work, we present a novel goal-oriented deep neural networks (DNN) surrogate approach to substantially reduce the computation burden of RTO. In particular,we propose to drawn the training points for the DNN-surrogate from a local approximatedposterior distribution – yielding a flexible and efficient sampling algorithm that convergesto the direct RTO approach. We present a Bayesian inverse problem governed by ellipticPDEs to demonstrate the computational accuracy and efficiency of our DNN-RTO approach, which shows that DNN-RTO can significantly outperform the traditional RTO.