Reducing the Prediction Uncertainties of High-Impact Weather and Climate Events： An Overview of Studies at LASG

This paper summarizes recent progress at the State Key Laboratory of Numerical Modeling for Atmospheric Sci- ences and Geophysical Fluid Dynamics （LASG）, Institute of Atmospheric Physics, Chinese Academy of Sciences in studies on targeted observations, data assimilation, and ensemble prediction, which are three effective strategies to re- duce the prediction uncertainties and improve the forecast skill of weather and climate events. Considering the limita- tions of traditional targeted observation approaches, LASG researchers have developed a conditional nonlinear op- timal perturbation-based targeted observation strategy to optimize the design of the observing network. This strategy has been employed to identify sensitive areas for targeted observations of the E1 Nifio-Southern Oscillation, Indian Ocean dipole, and tropical cyclones, and has been demonstrated to be effective in improving the forecast skill of these events. To assimilate the targeted observations into the initial state of a numerical model, a dimension-reduced- projection-based four-dimensional variational data assimilation （DRP-4DVar） approach has been proposed and is used operationally to supply accurate initial conditions in numerical forecasts. The performance of DRP-4DVar is good, and its computational cost is much lower than the standard 4DVar approach. Besides, ensemble prediction, which is a practical approach to generate probabilistic forecasts of the future state of a particular system, can be used to reduce the prediction uncertainties of single forecasts by taking the ensemble mean of forecast members. In this field, LASG researchers have proposed an ensemble forecast method that uses nonlinear local Lyapunov vectors （NLLVs） to yield ensemble initial perturbations. Its application in simple models has shown that NLLVs are more useful than bred vectors and singular vectors in improving the skill of the ensemble forecast. Therefore, NLLVs rep- resent a candidate for possible development as an ensemble method in operational forecasts. Despite the consider- able efforts made towards developing these methods to reduce prediction uncertainties, much challenging but highly important work remains in terms of improving the methods to further increase the skill in forecasting such weather and climate events.

Fourth-Order Predictive Modelling: II. 4th-BERRU-PM Methodology for Combining Measurements with Computations to Obtain Best-Estimate Results with Reduced Uncertainties

This work presents a comprehensive fourth-order predictive modeling (PM) methodology that uses the MaxEnt principle to incorporate fourth-order moments (means, covariances, skewness, kurtosis) of model parameters, computed and measured model responses, as well as fourth (and higher) order sensitivities of computed model responses to model parameters. This new methodology is designated by the acronym 4th-BERRU-PM, which stands for “fourth-order best-estimate results with reduced uncertainties.” The results predicted by the 4th-BERRU-PM incorporates, as particular cases, the results previously predicted by the second-order predictive modeling methodology 2nd-BERRU-PM, and vastly generalizes the results produced by extant data assimilation and data adjustment procedures.

Using Cross Entropy as a Performance Metric for Quantifying Uncertainty in DNN Image Classifiers: An Application to Classification of Lung Cancer on CT Images

Cross entropy is a measure in machine learning and deep learning that assesses the difference between predicted and actual probability distributions. In this study, we propose cross entropy as a performance evaluation metric for image classifier models and apply it to the CT image classification of lung cancer. A convolutional neural network is employed as the deep neural network (DNN) image classifier, with the residual network (ResNet) 50 chosen as the DNN archi-tecture. The image data used comprise a lung CT image set. Two classification models are built from datasets with varying amounts of data, and lung cancer is categorized into four classes using 10-fold cross-validation. Furthermore, we employ t-distributed stochastic neighbor embedding to visually explain the data distribution after classification. Experimental results demonstrate that cross en-tropy is a highly useful metric for evaluating the reliability of image classifier models. It is noted that for a more comprehensive evaluation of model perfor-mance, combining with other evaluation metrics is considered essential. .

Interval finite difference method for steady-state temperature field prediction with interval parameters

A new numerical technique named interval finite difference method is proposed for the steady-state temperature field prediction with uncertainties in both physical parameters and boundary conditions. Interval variables are used to quantitatively describe the uncertain parameters with limited information. Based on different Taylor and Neumann series, two kinds of parameter perturbation methods are presented to approximately yield the ranges of the uncertain temperature field. By comparing the results with traditional Monte Carlo simulation, a numerical example is given to demonstrate the feasibility and effectiveness of the proposed method for solving steady-state heat conduction problem with uncertain-but-bounded parameters.

Fourth-Order Predictive Modelling: I. General-Purpose Closed-Form Fourth-Order Moments-Constrained MaxEnt Distribution

This work (in two parts) will present a novel predictive modeling methodology aimed at obtaining “best-estimate results with reduced uncertainties” for the first four moments (mean values, covariance, skewness and kurtosis) of the optimally predicted distribution of model results and calibrated model parameters, by combining fourth-order experimental and computational information, including fourth (and higher) order sensitivities of computed model responses to model parameters. Underlying the construction of this fourth-order predictive modeling methodology is the “maximum entropy principle” which is initially used to obtain a novel closed-form expression of the (moments-constrained) fourth-order Maximum Entropy (MaxEnt) probability distribution constructed from the first four moments (means, covariances, skewness, kurtosis), which are assumed to be known, of an otherwise unknown distribution of a high-dimensional multivariate uncertain quantity of interest. This fourth-order MaxEnt distribution provides optimal compatibility of the available information while simultaneously ensuring minimal spurious information content, yielding an estimate of a probability density with the highest uncertainty among all densities satisfying the known moment constraints. Since this novel generic fourth-order MaxEnt distribution is of interest in its own right for applications in addition to predictive modeling, its construction is presented separately, in this first part of a two-part work. The fourth-order predictive modeling methodology that will be constructed by particularizing this generic fourth-order MaxEnt distribution will be presented in the accompanying work (Part-2).

Second-Order MaxEnt Predictive Modelling Methodology. III: Illustrative Application to a Reactor Physics Benchmark

This work illustrates the innovative results obtained by applying the recently developed the 2nd-order predictive modeling methodology called “2nd- BERRU-PM”, where the acronym BERRU denotes “best-estimate results with reduced uncertainties” and “PM” denotes “predictive modeling.” The physical system selected for this illustrative application is a polyethylene-reflected plutonium (acronym: PERP) OECD/NEA reactor physics benchmark. This benchmark is modeled using the neutron transport Boltzmann equation (involving 21,976 uncertain parameters), the solution of which is representative of “large-scale computations.” The results obtained in this work confirm the fact that the 2nd-BERRU-PM methodology predicts best-estimate results that fall in between the corresponding computed and measured values, while reducing the predicted standard deviations of the predicted results to values smaller than either the experimentally measured or the computed values of the respective standard deviations. The obtained results also indicate that 2nd-order response sensitivities must always be included to quantify the need for including (or not) the 3rd- and/or 4th-order sensitivities. When the parameters are known with high precision, the contributions of the higher-order sensitivities diminish with increasing order, so that the inclusion of the 1st- and 2nd-order sensitivities may suffice for obtaining accurate predicted best- estimate response values and best-estimate standard deviations. On the other hand, when the parameters’ standard deviations are sufficiently large to approach (or be outside of) the radius of convergence of the multivariate Taylor-series which represents the response in the phase-space of model parameters, the contributions stemming from the 3rd- and even 4th-order sensitivities are necessary to ensure consistency between the computed and measured response. In such cases, the use of only the 1st-order sensitivities erroneously indicates that the computed results are inconsistent with the respective measured response. Ongoing research aims at extending the 2nd-BERRU-PM methodology to fourth-order, thus enabling the computation of third-order response correlations (skewness) and fourth-order response correlations (kurtosis).

Second-Order MaxEnt Predictive Modelling Methodology. II: Probabilistically Incorporated Computational Model (2nd-BERRU-PMP)

This work presents a comprehensive second-order predictive modeling (PM) methodology based on the maximum entropy (MaxEnt) principle for obtaining best-estimate mean values and correlations for model responses and parameters. This methodology is designated by the acronym 2nd-BERRU-PMP, where the attribute “2nd” indicates that this methodology incorporates second- order uncertainties (means and covariances) and second (and higher) order sensitivities of computed model responses to model parameters. The acronym BERRU stands for “Best-Estimate Results with Reduced Uncertainties” and the last letter (“P”) in the acronym indicates “probabilistic,” referring to the MaxEnt probabilistic inclusion of the computational model responses. This is in contradistinction to the 2nd-BERRU-PMD methodology, which deterministically combines the computed model responses with the experimental information, as presented in the accompanying work (Part I). Although both the 2nd-BERRU-PMP and the 2nd-BERRU-PMD methodologies yield expressions that include second (and higher) order sensitivities of responses to model parameters, the respective expressions for the predicted responses, for the calibrated predicted parameters and for their predicted uncertainties (covariances), are not identical to each other. Nevertheless, the results predicted by both the 2nd-BERRU-PMP and the 2nd-BERRU-PMD methodologies encompass, as particular cases, the results produced by the extant data assimilation and data adjustment procedures, which rely on the minimization, in a least-square sense, of a user-defined functional meant to represent the discrepancies between measured and computed model responses.

Second-Order MaxEnt Predictive Modelling Methodology. I: Deterministically Incorporated Computational Model (2nd-BERRU-PMD)

This work presents a comprehensive second-order predictive modeling (PM) methodology designated by the acronym 2nd-BERRU-PMD. The attribute “2nd” indicates that this methodology incorporates second-order uncertainties (means and covariances) and second-order sensitivities of computed model responses to model parameters. The acronym BERRU stands for “Best- Estimate Results with Reduced Uncertainties” and the last letter (“D”) in the acronym indicates “deterministic,” referring to the deterministic inclusion of the computational model responses. The 2nd-BERRU-PMD methodology is fundamentally based on the maximum entropy (MaxEnt) principle. This principle is in contradistinction to the fundamental principle that underlies the extant data assimilation and/or adjustment procedures which minimize in a least-square sense a subjective user-defined functional which is meant to represent the discrepancies between measured and computed model responses. It is shown that the 2nd-BERRU-PMD methodology generalizes and extends current data assimilation and/or data adjustment procedures while overcoming the fundamental limitations of these procedures. In the accompanying work (Part II), the alternative framework for developing the “second- order MaxEnt predictive modelling methodology” is presented by incorporating probabilistically (as opposed to “deterministically”) the computed model responses.

The rapid uncertainty prediction of the ocean-acoustic coupled model

Focusing on the rapid prediction of acoustic field uncertainty in environment with temporal and spatial sound speed perturbation, evolvement of sound speed structure over time is predicted based on the ocean-acoustic coupled model to obtain the uncertainty distribution of the vertical structure of sound speed. Further, a method combining the arbitrary polynomial chaos expansion with the empirical orthogonal function is proposed to reduce the dimensionality of uncertain parameters and to obtain the uncertainty distribution of the acoustic field. Simulations have shown that the computational complexity can be reduced by 2 orders of magnitude compared to the conventional polynomial chaos expansion while ensures the same precision.Moreover, the computational complexity is not influenced by the complexity of the sound speed profile. The acoustic field and uncertainty predicted in uncertain environment by proposed method also have been tested with the experimental data.

Static Frame Model Validation with Small Samples Solution Using Improved Kernel Density Estimation and Confidence Level Method

An improved method using kernel density estimation （KDE） and confidence level is presented for model validation with small samples. Decision making is a challenging problem because of input uncertainty and only small samples can be used due to the high costs of experimental measurements. However, model validation provides more confidence for decision makers when improving prediction accuracy at the same time. The confidence level method is introduced and the optimum sample variance is determined using a new method in kernel density estimation to increase the credibility of model validation. As a numerical example, the static frame model validation challenge problem presented by Sandia National Laboratories has been chosen. The optimum bandwidth is selected in kernel density estimation in order to build the probability model based on the calibration data. The model assessment is achieved using validation and accreditation experimental data respectively based on the probability model. Finally, the target structure prediction is performed using validated model, which are consistent with the results obtained by other researchers. The results demonstrate that the method using the improved confidence level and kernel density estimation is an effective approach to solve the model validation problem with small samples.