The curse of dimensionality refers to the problem o increased sparsity and computational complexity when dealing with high-dimensional data.In recent years,the types and vari ables of industrial data have increased si...The curse of dimensionality refers to the problem o increased sparsity and computational complexity when dealing with high-dimensional data.In recent years,the types and vari ables of industrial data have increased significantly,making data driven models more challenging to develop.To address this prob lem,data augmentation technology has been introduced as an effective tool to solve the sparsity problem of high-dimensiona industrial data.This paper systematically explores and discusses the necessity,feasibility,and effectiveness of augmented indus trial data-driven modeling in the context of the curse of dimen sionality and virtual big data.Then,the process of data augmen tation modeling is analyzed,and the concept of data boosting augmentation is proposed.The data boosting augmentation involves designing the reliability weight and actual-virtual weigh functions,and developing a double weighted partial least squares model to optimize the three stages of data generation,data fusion and modeling.This approach significantly improves the inter pretability,effectiveness,and practicality of data augmentation in the industrial modeling.Finally,the proposed method is verified using practical examples of fault diagnosis systems and virtua measurement systems in the industry.The results demonstrate the effectiveness of the proposed approach in improving the accu racy and robustness of data-driven models,making them more suitable for real-world industrial applications.展开更多
The performance of conventional similarity measurement methods is affected seriously by the curse of dimensionality of high-dimensional data.The reason is that data difference between sparse and noisy dimensionalities...The performance of conventional similarity measurement methods is affected seriously by the curse of dimensionality of high-dimensional data.The reason is that data difference between sparse and noisy dimensionalities occupies a large proportion of the similarity,leading to the dissimilarities between any results.A similarity measurement method of high-dimensional data based on normalized net lattice subspace is proposed.The data range of each dimension is divided into several intervals,and the components in different dimensions are mapped onto the corresponding interval.Only the component in the same or adjacent interval is used to calculate the similarity.To validate this method,three data types are used,and seven common similarity measurement methods are compared.The experimental result indicates that the relative difference of the method is increasing with the dimensionality and is approximately two or three orders of magnitude higher than the conventional method.In addition,the similarity range of this method in different dimensions is [0,1],which is fit for similarity analysis after dimensionality reduction.展开更多
Unmanned aerial vehicles(UAVs)can be employed as aerial base stations(BSs)due to their high mobility and flexible deployment.This paper focuses on a UAV-assisted wireless network,where users can be scheduled to get ac...Unmanned aerial vehicles(UAVs)can be employed as aerial base stations(BSs)due to their high mobility and flexible deployment.This paper focuses on a UAV-assisted wireless network,where users can be scheduled to get access to either an aerial BS or a terrestrial BS for uplink transmission.In contrast to state-of-the-art designs focusing on the instantaneous cost of the network,this paper aims at minimizing the long-term average transmit power consumed by the users by dynamically optimizing user association and power allocation in each time slot.Such a joint user association scheduling and power allocation problem can be formulated as a Markov decision process(MDP).Unfortunately,solving such an MDP problem with the conventional relative value iteration(RVI)can suffer from the curses of dimensionality,in the presence of a large number of users.As a countermeasure,we propose a distributed RVI algorithm to reduce the dimension of the MDP problem,such that the original problem can be decoupled into multiple solvable small-scale MDP problems.Simulation results reveal that the proposed algorithm can yield lower longterm average transmit power consumption than both the conventional RVI algorithm and a baseline algorithm with myopic policies.展开更多
We prove a theorem concerning the approximation of generalized bandlimited multivariate functions by deep ReLU networks for which the curse of the dimensionality is overcome.Our theorem is based on a result by Maurey ...We prove a theorem concerning the approximation of generalized bandlimited multivariate functions by deep ReLU networks for which the curse of the dimensionality is overcome.Our theorem is based on a result by Maurey and on the ability of deep ReLU networks to approximate Chebyshev polynomials and analytic functions efficiently.展开更多
Surrogate models are usually used to perform global sensitivity analysis (GSA) by avoiding a large ensemble of deterministic simulations of the Monte Carlo method to provide a reliable estimate of GSA indices. Howev...Surrogate models are usually used to perform global sensitivity analysis (GSA) by avoiding a large ensemble of deterministic simulations of the Monte Carlo method to provide a reliable estimate of GSA indices. However, most surrogate models such as polynomial chaos (PC) expansions suffer from the curse of dimensionality due to the high-dimensional input space. Thus, sparse surrogate models have been proposed to alleviate the curse of dimensionality. In this paper, three techniques of sparse reconstruc- tion are used to construct sparse PC expansions that are easily applicable to computing variance-based sensitivity indices (Sobol indices). These are orthogonal matching pursuit (OMP), spectral projected gradient for L1 minimization (SPGL1), and Bayesian compressive sensing with Laplace priors. By computing Sobol indices for several benchmark response models including the Sobol function, the Morris function, and the Sod shock tube problem, effective implementations of high-dimensional sparse surrogate construction are exhibited for GSA.展开更多
This paper relieves the 'curse of dimensionality' problem, which becomes intractable when scaling rein- forcement learning to multi-agent systems. This problem is aggravated exponentially as the number of agents inc...This paper relieves the 'curse of dimensionality' problem, which becomes intractable when scaling rein- forcement learning to multi-agent systems. This problem is aggravated exponentially as the number of agents increases, resulting in large memory requirement and slowness in learning speed. For cooperative systems which widely exist in multi-agent systems, this paper proposes a new multi-agent Q-learning algorithm based on decomposing the joint state and joint action learning into two learning processes, which are learning individual action and the maximum value of the joint state approximately. The latter process considers others' actions to insure that the joint action is optimal and supports the updating of the former one. The simulation results illustrate that the proposed algorithm can learn the optimal joint behavior with smaller memory and faster leamin~ soeed comoared with friend-O learnin~ and indet^endent learning.展开更多
Advanced engineering systems, like aircraft, are defined by tens or even hundreds of design variables. Building an accurate surrogate model for use in such high-dimensional optimization problems is a difficult task ow...Advanced engineering systems, like aircraft, are defined by tens or even hundreds of design variables. Building an accurate surrogate model for use in such high-dimensional optimization problems is a difficult task owing to the curse of dimensionality. This paper presents a new algorithm to reduce the size of a design space to a smaller region of interest allowing a more accurate surrogate model to be generated. The framework requires a set of models of different physical or numerical fidelities. The low-fidelity (LF) model provides physics-based approximation of the high-fidelity (HF) model at a fraction of the computational cost. It is also instrumental in identifying the small region of interest in the design space that encloses the high-fidelity optimum. A surrogate model is then constructed to match the low-fidelity model to the high-fidelity model in the identified region of interest. The optimization process is managed by an update strategy to prevent convergence to false optima. The algorithm is applied on mathematical problems and a two-dimen-sional aerodynamic shape optimization problem in a variable-fidelity context. Results obtained are in excellent agreement with high-fidelity results, even with lower-fidelity flow solvers, while showing up to 39% time savings.展开更多
A popular imputation method used to compensate for item nonresponse in sample surveys is thenearest neighbour imputation (NNI) method utilising a covariate to defined neighbours. Whenthe covariate is multivariate, how...A popular imputation method used to compensate for item nonresponse in sample surveys is thenearest neighbour imputation (NNI) method utilising a covariate to defined neighbours. Whenthe covariate is multivariate, however, NNI suffers the well-known curse of dimensionality andgives unstable results. As a remedy, we propose a single-index NNI when the conditional meanof response given covariates follows a single index model. For estimating the population meanor quantiles, we establish the consistency and asymptotic normality of the single-index NNI estimators. Some limited simulation results are presented to examine the finite-sample performanceof the proposed estimator of population mean.展开更多
The problem of variable selection in system identification of a high dimensional nonlinear non-parametric system is described. The inherent difficulty, the curse of dimensionality, is introduced. Then its connections ...The problem of variable selection in system identification of a high dimensional nonlinear non-parametric system is described. The inherent difficulty, the curse of dimensionality, is introduced. Then its connections to various topics and research areas are briefly discussed, including order determination, pattern recognition, data mining, machine learning, statistical regression and manifold embedding. Finally, some results of variable selection in system identification in the recent literature are presented.展开更多
Objective functions in large-scalemachine-learning and artificial intelligence applications often live in high dimensions with strong non-convexity and massive local minima.Gradient-based methods,such as the stochasti...Objective functions in large-scalemachine-learning and artificial intelligence applications often live in high dimensions with strong non-convexity and massive local minima.Gradient-based methods,such as the stochastic gradient method and Adam[15],and gradient-freemethods,such as the consensus-based optimization(CBO)method,can be employed to find minima.In this work,based on the CBO method and Adam,we propose a consensus-based global optimization method with adaptive momentum estimation(Adam-CBO).Advantages of the Adam-CBO method include:It is capable of finding global minima of non-convex objective functions with high success rates and low costs.This is verified by finding the global minimizer of the 1000 dimensional Rastrigin function with 100%success rate at a cost only growing linearly with respect to the dimensionality.It can handle non-differentiable activation functions and thus approximate lowregularity functions with better accuracy.This is confirmed by solving a machine learning task for partial differential equations with low-regularity solutions where the Adam-CBO method provides better results than Adam.It is robust in the sense that its convergence is insensitive to the learning rate by a linear stability analysis.This is confirmed by finding theminimizer of a quadratic function.展开更多
I reflect upon the development of nonlinear time series analysis since 1990 by focusing on five major areas of development. These areas include the interface between nonlinear time series analysis and chaos, the nonpa...I reflect upon the development of nonlinear time series analysis since 1990 by focusing on five major areas of development. These areas include the interface between nonlinear time series analysis and chaos, the nonparametric/semiparametric approach, nonlinear state space modelling, financial time series and nonlinear modelling of panels of time series.展开更多
基金supported in part by the National Natural Science Foundation of China(NSFC)(92167106,61833014)Key Research and Development Program of Zhejiang Province(2022C01206)。
文摘The curse of dimensionality refers to the problem o increased sparsity and computational complexity when dealing with high-dimensional data.In recent years,the types and vari ables of industrial data have increased significantly,making data driven models more challenging to develop.To address this prob lem,data augmentation technology has been introduced as an effective tool to solve the sparsity problem of high-dimensiona industrial data.This paper systematically explores and discusses the necessity,feasibility,and effectiveness of augmented indus trial data-driven modeling in the context of the curse of dimen sionality and virtual big data.Then,the process of data augmen tation modeling is analyzed,and the concept of data boosting augmentation is proposed.The data boosting augmentation involves designing the reliability weight and actual-virtual weigh functions,and developing a double weighted partial least squares model to optimize the three stages of data generation,data fusion and modeling.This approach significantly improves the inter pretability,effectiveness,and practicality of data augmentation in the industrial modeling.Finally,the proposed method is verified using practical examples of fault diagnosis systems and virtua measurement systems in the industry.The results demonstrate the effectiveness of the proposed approach in improving the accu racy and robustness of data-driven models,making them more suitable for real-world industrial applications.
基金Supported by the National Natural Science Foundation of China(No.61502475)the Importation and Development of High-Caliber Talents Project of the Beijing Municipal Institutions(No.CIT&TCD201504039)
文摘The performance of conventional similarity measurement methods is affected seriously by the curse of dimensionality of high-dimensional data.The reason is that data difference between sparse and noisy dimensionalities occupies a large proportion of the similarity,leading to the dissimilarities between any results.A similarity measurement method of high-dimensional data based on normalized net lattice subspace is proposed.The data range of each dimension is divided into several intervals,and the components in different dimensions are mapped onto the corresponding interval.Only the component in the same or adjacent interval is used to calculate the similarity.To validate this method,three data types are used,and seven common similarity measurement methods are compared.The experimental result indicates that the relative difference of the method is increasing with the dimensionality and is approximately two or three orders of magnitude higher than the conventional method.In addition,the similarity range of this method in different dimensions is [0,1],which is fit for similarity analysis after dimensionality reduction.
基金This work was supported in part by the National Natural Science Foundation of China under Grant 61901216,61631020 and 61827801the Natural Science Foundation of Jiangsu Province under Grant BK20190400+1 种基金the open research fund of National Mobile Communications Research Laboratory,Southeast University(No.2020D08)the Foundation of Graduate Innovation Center in NUAA under Grant No.KFJJ20190408.
文摘Unmanned aerial vehicles(UAVs)can be employed as aerial base stations(BSs)due to their high mobility and flexible deployment.This paper focuses on a UAV-assisted wireless network,where users can be scheduled to get access to either an aerial BS or a terrestrial BS for uplink transmission.In contrast to state-of-the-art designs focusing on the instantaneous cost of the network,this paper aims at minimizing the long-term average transmit power consumed by the users by dynamically optimizing user association and power allocation in each time slot.Such a joint user association scheduling and power allocation problem can be formulated as a Markov decision process(MDP).Unfortunately,solving such an MDP problem with the conventional relative value iteration(RVI)can suffer from the curses of dimensionality,in the presence of a large number of users.As a countermeasure,we propose a distributed RVI algorithm to reduce the dimension of the MDP problem,such that the original problem can be decoupled into multiple solvable small-scale MDP problems.Simulation results reveal that the proposed algorithm can yield lower longterm average transmit power consumption than both the conventional RVI algorithm and a baseline algorithm with myopic policies.
基金The research of the second author was partially supported by US NSF under the grant award DMS-1945029The research of the third author is supported in part by US NSF DMS-1719699the NSF TRIPODS program CCF-1704833.
文摘We prove a theorem concerning the approximation of generalized bandlimited multivariate functions by deep ReLU networks for which the curse of the dimensionality is overcome.Our theorem is based on a result by Maurey and on the ability of deep ReLU networks to approximate Chebyshev polynomials and analytic functions efficiently.
基金Project supported by the National Natural Science Foundation of China(Nos.11172049 and11472060)the Science Foundation of China Academy of Engineering Physics(Nos.2015B0201037and 2013A0101004)
文摘Surrogate models are usually used to perform global sensitivity analysis (GSA) by avoiding a large ensemble of deterministic simulations of the Monte Carlo method to provide a reliable estimate of GSA indices. However, most surrogate models such as polynomial chaos (PC) expansions suffer from the curse of dimensionality due to the high-dimensional input space. Thus, sparse surrogate models have been proposed to alleviate the curse of dimensionality. In this paper, three techniques of sparse reconstruc- tion are used to construct sparse PC expansions that are easily applicable to computing variance-based sensitivity indices (Sobol indices). These are orthogonal matching pursuit (OMP), spectral projected gradient for L1 minimization (SPGL1), and Bayesian compressive sensing with Laplace priors. By computing Sobol indices for several benchmark response models including the Sobol function, the Morris function, and the Sod shock tube problem, effective implementations of high-dimensional sparse surrogate construction are exhibited for GSA.
基金supported by National Nature Science Foundation of China(Nos.61074058,60874042)the Chinese Postdoctoral Science Foundation(No.200902483)+1 种基金the Specialized Research Fund for the Doctoral Program of Higher Education of China(No.20090162120068)the Central South University Innovation Project(No.2011ssxt221)
文摘This paper relieves the 'curse of dimensionality' problem, which becomes intractable when scaling rein- forcement learning to multi-agent systems. This problem is aggravated exponentially as the number of agents increases, resulting in large memory requirement and slowness in learning speed. For cooperative systems which widely exist in multi-agent systems, this paper proposes a new multi-agent Q-learning algorithm based on decomposing the joint state and joint action learning into two learning processes, which are learning individual action and the maximum value of the joint state approximately. The latter process considers others' actions to insure that the joint action is optimal and supports the updating of the former one. The simulation results illustrate that the proposed algorithm can learn the optimal joint behavior with smaller memory and faster leamin~ soeed comoared with friend-O learnin~ and indet^endent learning.
文摘Advanced engineering systems, like aircraft, are defined by tens or even hundreds of design variables. Building an accurate surrogate model for use in such high-dimensional optimization problems is a difficult task owing to the curse of dimensionality. This paper presents a new algorithm to reduce the size of a design space to a smaller region of interest allowing a more accurate surrogate model to be generated. The framework requires a set of models of different physical or numerical fidelities. The low-fidelity (LF) model provides physics-based approximation of the high-fidelity (HF) model at a fraction of the computational cost. It is also instrumental in identifying the small region of interest in the design space that encloses the high-fidelity optimum. A surrogate model is then constructed to match the low-fidelity model to the high-fidelity model in the identified region of interest. The optimization process is managed by an update strategy to prevent convergence to false optima. The algorithm is applied on mathematical problems and a two-dimen-sional aerodynamic shape optimization problem in a variable-fidelity context. Results obtained are in excellent agreement with high-fidelity results, even with lower-fidelity flow solvers, while showing up to 39% time savings.
基金This work was partially supported by the National Natural Science Foundation of China grants 11831008 and 11871287the U.S.National Science Foundation grants DMS-1612873 and DMS-1914411the Natural Science Foundation of Tianjin(18JCYBJC41100)and the Fundamental Research Funds for the Central Universities.
文摘A popular imputation method used to compensate for item nonresponse in sample surveys is thenearest neighbour imputation (NNI) method utilising a covariate to defined neighbours. Whenthe covariate is multivariate, however, NNI suffers the well-known curse of dimensionality andgives unstable results. As a remedy, we propose a single-index NNI when the conditional meanof response given covariates follows a single index model. For estimating the population meanor quantiles, we establish the consistency and asymptotic normality of the single-index NNI estimators. Some limited simulation results are presented to examine the finite-sample performanceof the proposed estimator of population mean.
基金supported by the National Science Foundation(No.CNS-1239509)the National Key Basic Research Program of China(973 program)(No.2014CB845301)+1 种基金the National Natural Science Foundation of China(Nos.61104052,61273193,61227902,61134013)the Australian Research Council(No.DP120104986)
文摘The problem of variable selection in system identification of a high dimensional nonlinear non-parametric system is described. The inherent difficulty, the curse of dimensionality, is introduced. Then its connections to various topics and research areas are briefly discussed, including order determination, pattern recognition, data mining, machine learning, statistical regression and manifold embedding. Finally, some results of variable selection in system identification in the recent literature are presented.
基金J.Chen was supported by National Natural Science Foundation of China via grant 11971021S.Jin was supported by Natural Science Foundation of China under grant 12031013.
文摘Objective functions in large-scalemachine-learning and artificial intelligence applications often live in high dimensions with strong non-convexity and massive local minima.Gradient-based methods,such as the stochastic gradient method and Adam[15],and gradient-freemethods,such as the consensus-based optimization(CBO)method,can be employed to find minima.In this work,based on the CBO method and Adam,we propose a consensus-based global optimization method with adaptive momentum estimation(Adam-CBO).Advantages of the Adam-CBO method include:It is capable of finding global minima of non-convex objective functions with high success rates and low costs.This is verified by finding the global minimizer of the 1000 dimensional Rastrigin function with 100%success rate at a cost only growing linearly with respect to the dimensionality.It can handle non-differentiable activation functions and thus approximate lowregularity functions with better accuracy.This is confirmed by solving a machine learning task for partial differential equations with low-regularity solutions where the Adam-CBO method provides better results than Adam.It is robust in the sense that its convergence is insensitive to the learning rate by a linear stability analysis.This is confirmed by finding theminimizer of a quadratic function.
基金Supported by Biological & Biotechnology Research Council and the Engineering & Physical Science Research Council of the United Kingdom,and by the Research Grant Council of Hong Kong.
文摘I reflect upon the development of nonlinear time series analysis since 1990 by focusing on five major areas of development. These areas include the interface between nonlinear time series analysis and chaos, the nonparametric/semiparametric approach, nonlinear state space modelling, financial time series and nonlinear modelling of panels of time series.