Augmented Industrial Data-Driven Modeling Under the Curse of Dimensionality

The curse of dimensionality refers to the problem o increased sparsity and computational complexity when dealing with high-dimensional data.In recent years,the types and vari ables of industrial data have increased significantly,making data driven models more challenging to develop.To address this prob lem,data augmentation technology has been introduced as an effective tool to solve the sparsity problem of high-dimensiona industrial data.This paper systematically explores and discusses the necessity,feasibility,and effectiveness of augmented indus trial data-driven modeling in the context of the curse of dimen sionality and virtual big data.Then,the process of data augmen tation modeling is analyzed,and the concept of data boosting augmentation is proposed.The data boosting augmentation involves designing the reliability weight and actual-virtual weigh functions,and developing a double weighted partial least squares model to optimize the three stages of data generation,data fusion and modeling.This approach significantly improves the inter pretability,effectiveness,and practicality of data augmentation in the industrial modeling.Finally,the proposed method is verified using practical examples of fault diagnosis systems and virtua measurement systems in the industry.The results demonstrate the effectiveness of the proposed approach in improving the accu racy and robustness of data-driven models,making them more suitable for real-world industrial applications.

DEEP RELU NETWORKS OVERCOME THE CURSE OF DIMENSIONALITY FOR GENERALIZED BANDLIMITED FUNCTIONS

We prove a theorem concerning the approximation of generalized bandlimited multivariate functions by deep ReLU networks for which the curse of the dimensionality is overcome.Our theorem is based on a result by Maurey and on the ability of deep ReLU networks to approximate Chebyshev polynomials and analytic functions efficiently.

Similarity measurement method of high-dimensional data based on normalized net lattice subspace

The performance of conventional similarity measurement methods is affected seriously by the curse of dimensionality of high-dimensional data.The reason is that data difference between sparse and noisy dimensionalities occupies a large proportion of the similarity,leading to the dissimilarities between any results.A similarity measurement method of high-dimensional data based on normalized net lattice subspace is proposed.The data range of each dimension is divided into several intervals,and the components in different dimensions are mapped onto the corresponding interval.Only the component in the same or adjacent interval is used to calculate the similarity.To validate this method,three data types are used,and seven common similarity measurement methods are compared.The experimental result indicates that the relative difference of the method is increasing with the dimensionality and is approximately two or three orders of magnitude higher than the conventional method.In addition,the similarity range of this method in different dimensions is [0,1],which is fit for similarity analysis after dimensionality reduction.

User Association and Power Allocation for UAV-Assisted Networks: A Distributed Reinforcement Learning Approach

Unmanned aerial vehicles(UAVs)can be employed as aerial base stations(BSs)due to their high mobility and flexible deployment.This paper focuses on a UAV-assisted wireless network,where users can be scheduled to get access to either an aerial BS or a terrestrial BS for uplink transmission.In contrast to state-of-the-art designs focusing on the instantaneous cost of the network,this paper aims at minimizing the long-term average transmit power consumed by the users by dynamically optimizing user association and power allocation in each time slot.Such a joint user association scheduling and power allocation problem can be formulated as a Markov decision process(MDP).Unfortunately,solving such an MDP problem with the conventional relative value iteration(RVI)can suffer from the curses of dimensionality,in the presence of a large number of users.As a countermeasure,we propose a distributed RVI algorithm to reduce the dimension of the MDP problem,such that the original problem can be decoupled into multiple solvable small-scale MDP problems.Simulation results reveal that the proposed algorithm can yield lower longterm average transmit power consumption than both the conventional RVI algorithm and a baseline algorithm with myopic policies.

Improved wavelet neural network combined with particle swarm optimization algorithm and its application

An improved wavelet neural network algorithm which combines with particle swarm optimization was proposed to avoid encountering the curse of dimensionality and overcome the shortage in the responding speed and learning ability brought about by the traditional models. Based on the operational data provided by a regional power grid in the south of China, the method was used in the actual short term load forecasting. The results show that the average time cost of the proposed method in the experiment process is reduced by 12.2 s, and the precision of the proposed method is increased by 3.43% compared to the traditional wavelet network. Consequently, the improved wavelet neural network forecasting model is better than the traditional wavelet neural network forecasting model in both forecasting effect and network function.

Global sensitivity analysis based on high-dimensional sparse surrogate construction

Surrogate models are usually used to perform global sensitivity analysis （GSA） by avoiding a large ensemble of deterministic simulations of the Monte Carlo method to provide a reliable estimate of GSA indices. However, most surrogate models such as polynomial chaos （PC） expansions suffer from the curse of dimensionality due to the high-dimensional input space. Thus, sparse surrogate models have been proposed to alleviate the curse of dimensionality. In this paper, three techniques of sparse reconstruc- tion are used to construct sparse PC expansions that are easily applicable to computing variance-based sensitivity indices （Sobol indices）. These are orthogonal matching pursuit （OMP）, spectral projected gradient for L1 minimization （SPGL1）, and Bayesian compressive sensing with Laplace priors. By computing Sobol indices for several benchmark response models including the Sobol function, the Morris function, and the Sod shock tube problem, effective implementations of high-dimensional sparse surrogate construction are exhibited for GSA.

Variable selection in identification of a high dimensional nonlinear non-parametric system

The problem of variable selection in system identification of a high dimensional nonlinear non-parametric system is described. The inherent difficulty, the curse of dimensionality, is introduced. Then its connections to various topics and research areas are briefly discussed, including order determination, pattern recognition, data mining, machine learning, statistical regression and manifold embedding. Finally, some results of variable selection in system identification in the recent literature are presented.

Numerical Solution to Optimal Feedback Control by Dynamic Programming Approach:A Local Approximation Algorithm

This paper considers optimal feedback control for a general continuous time finite-dimensional deterministic system with finite horizon cost functional. A practically feasible algorithm to calculate the numerical solution of the optimal feedback control by dynamic programming approach is developed. The highlights of this algorithm are: a) It is based on a convergent constructive algorithm for optimal feedback control law which was proposed by the authors before through an approximation for the viscosity solution of the time-space discretization scheme developed by dynamic programming method; b) The computation complexity is significantly reduced since only values of viscosity solution on some local cones around the optimal trajectory are calculated. Two numerical experiments are presented to illustrate the effectiveness and fastness of the algorithm.

Cooperative learning with joint state value approximation for multi-agent systems

This paper relieves the ＇curse of dimensionality＇ problem, which becomes intractable when scaling rein- forcement learning to multi-agent systems. This problem is aggravated exponentially as the number of agents increases, resulting in large memory requirement and slowness in learning speed. For cooperative systems which widely exist in multi-agent systems, this paper proposes a new multi-agent Q-learning algorithm based on decomposing the joint state and joint action learning into two learning processes, which are learning individual action and the maximum value of the joint state approximately. The latter process considers others＇ actions to insure that the joint action is optimal and supports the updating of the former one. The simulation results illustrate that the proposed algorithm can learn the optimal joint behavior with smaller memory and faster leamin~ soeed comoared with friend-O learnin~ and indet^endent learning.

Variable-fidelity optimization with design space reduction

Advanced engineering systems, like aircraft, are defined by tens or even hundreds of design variables. Building an accurate surrogate model for use in such high-dimensional optimization problems is a difficult task owing to the curse of dimensionality. This paper presents a new algorithm to reduce the size of a design space to a smaller region of interest allowing a more accurate surrogate model to be generated. The framework requires a set of models of different physical or numerical fidelities. The low-fidelity （LF） model provides physics-based approximation of the high-fidelity （HF） model at a fraction of the computational cost. It is also instrumental in identifying the small region of interest in the design space that encloses the high-fidelity optimum. A surrogate model is then constructed to match the low-fidelity model to the high-fidelity model in the identified region of interest. The optimization process is managed by an update strategy to prevent convergence to false optima. The algorithm is applied on mathematical problems and a two-dimen-sional aerodynamic shape optimization problem in a variable-fidelity context. Results obtained are in excellent agreement with high-fidelity results, even with lower-fidelity flow solvers, while showing up to 39% time savings.

Nearest neighbour imputation under single index models

A popular imputation method used to compensate for item nonresponse in sample surveys is thenearest neighbour imputation (NNI) method utilising a covariate to defined neighbours. Whenthe covariate is multivariate, however, NNI suffers the well-known curse of dimensionality andgives unstable results. As a remedy, we propose a single-index NNI when the conditional meanof response given covariates follows a single index model. For estimating the population meanor quantiles, we establish the consistency and asymptotic normality of the single-index NNI estimators. Some limited simulation results are presented to examine the finite-sample performanceof the proposed estimator of population mean.

A Consensus-Based Global Optimization Method with Adaptive Momentum Estimation

Objective functions in large-scalemachine-learning and artificial intelligence applications often live in high dimensions with strong non-convexity and massive local minima.Gradient-based methods,such as the stochastic gradient method and Adam[15],and gradient-freemethods,such as the consensus-based optimization(CBO)method,can be employed to find minima.In this work,based on the CBO method and Adam,we propose a consensus-based global optimization method with adaptive momentum estimation(Adam-CBO).Advantages of the Adam-CBO method include:It is capable of finding global minima of non-convex objective functions with high success rates and low costs.This is verified by finding the global minimizer of the 1000 dimensional Rastrigin function with 100%success rate at a cost only growing linearly with respect to the dimensionality.It can handle non-differentiable activation functions and thus approximate lowregularity functions with better accuracy.This is confirmed by solving a machine learning task for partial differential equations with low-regularity solutions where the Adam-CBO method provides better results than Adam.It is robust in the sense that its convergence is insensitive to the learning rate by a linear stability analysis.This is confirmed by finding theminimizer of a quadratic function.

Nonlinear Time Series Analysis Since 1990:Some Personal Reflections

I reflect upon the development of nonlinear time series analysis since 1990 by focusing on five major areas of development. These areas include the interface between nonlinear time series analysis and chaos, the nonparametric/semiparametric approach, nonlinear state space modelling, financial time series and nonlinear modelling of panels of time series.