Target Controllability of Multi-Layer Networks With High-Dimensional

This paper studies the target controllability of multilayer complex networked systems,in which the nodes are highdimensional linear time invariant(LTI)dynamical systems,and the network topology is directed and weighted.The influence of inter-layer couplings on the target controllability of multi-layer networks is discussed.It is found that even if there exists a layer which is not target controllable,the entire multi-layer network can still be target controllable due to the inter-layer couplings.For the multi-layer networks with general structure,a necessary and sufficient condition for target controllability is given by establishing the relationship between uncontrollable subspace and output matrix.By the derived condition,it can be found that the system may be target controllable even if it is not state controllable.On this basis,two corollaries are derived,which clarify the relationship between target controllability,state controllability and output controllability.For the multi-layer networks where the inter-layer couplings are directed chains and directed stars,sufficient conditions for target controllability of networked systems are given,respectively.These conditions are easier to verify than the classic criterion.

Physics-informed neural network-based petroleum reservoir simulation with sparse data using domain decomposition

Recent advances in deep learning have expanded new possibilities for fluid flow simulation in petroleum reservoirs.However,the predominant approach in existing research is to train neural networks using high-fidelity numerical simulation data.This presents a significant challenge because the sole source of authentic wellbore production data for training is sparse.In response to this challenge,this work introduces a novel architecture called physics-informed neural network based on domain decomposition(PINN-DD),aiming to effectively utilize the sparse production data of wells for reservoir simulation with large-scale systems.To harness the capabilities of physics-informed neural networks(PINNs)in handling small-scale spatial-temporal domain while addressing the challenges of large-scale systems with sparse labeled data,the computational domain is divided into two distinct sub-domains:the well-containing and the well-free sub-domain.Moreover,the two sub-domains and the interface are rigorously constrained by the governing equations,data matching,and boundary conditions.The accuracy of the proposed method is evaluated on two problems,and its performance is compared against state-of-the-art PINNs through numerical analysis as a benchmark.The results demonstrate the superiority of PINN-DD in handling large-scale reservoir simulation with limited data and show its potential to outperform conventional PINNs in such scenarios.

Sparse Seismic Data Reconstruction Based on a Convolutional Neural Network Algorithm

At present,the acquisition of seismic data is developing toward high-precision and high-density methods.However,complex natural environments and cultural factors in many exploration areas cause difficulties in achieving uniform and intensive acquisition,which makes complete seismic data collection impossible.Therefore,data reconstruction is required in the processing link to ensure imaging accuracy.Deep learning,as a new field in rapid development,presents clear advantages in feature extraction and modeling.In this study,the convolutional neural network deep learning algorithm is applied to seismic data reconstruction.Based on the convolutional neural network algorithm and combined with the characteristics of seismic data acquisition,two training strategies of supervised and unsupervised learning are designed to reconstruct sparse acquisition seismic records.First,a supervised learning strategy is proposed for labeled data,wherein the complete seismic data are segmented as the input of the training set and are randomly sampled before each training,thereby increasing the number of samples and the richness of features.Second,an unsupervised learning strategy based on large samples is proposed for unlabeled data,and the rolling segmentation method is used to update(pseudo)labels and training parameters in the training process.Through the reconstruction test of simulated and actual data,the deep learning algorithm based on a convolutional neural network shows better reconstruction quality and higher accuracy than compressed sensing based on Curvelet transform.

Application of graph neural network and feature information enhancement in relation inference of sparse knowledge graph

At present,knowledge embedding methods are widely used in the field of knowledge graph(KG)reasoning,and have been successfully applied to those with large entities and relationships.However,in research and production environments,there are a large number of KGs with a small number of entities and relations,which are called sparse KGs.Limited by the performance of knowledge extraction methods or some other reasons(some common-sense information does not appear in the natural corpus),the relation between entities is often incomplete.To solve this problem,a method of the graph neural network and information enhancement is proposed.The improved method increases the mean reciprocal rank(MRR)and Hit@3 by 1.6%and 1.7%,respectively,when the sparsity of the FB15K-237 dataset is 10%.When the sparsity is 50%,the evaluation indexes MRR and Hit@10 are increased by 0.8%and 1.8%,respectively.

Randomized Latent Factor Model for High-dimensional and Sparse Matrices from Industrial Applications

Latent factor(LF) models are highly effective in extracting useful knowledge from High-Dimensional and Sparse(HiDS) matrices which are commonly seen in various industrial applications. An LF model usually adopts iterative optimizers,which may consume many iterations to achieve a local optima,resulting in considerable time cost. Hence, determining how to accelerate the training process for LF models has become a significant issue. To address this, this work proposes a randomized latent factor(RLF) model. It incorporates the principle of randomized learning techniques from neural networks into the LF analysis of HiDS matrices, thereby greatly alleviating computational burden. It also extends a standard learning process for randomized neural networks in context of LF analysis to make the resulting model represent an HiDS matrix correctly.Experimental results on three HiDS matrices from industrial applications demonstrate that compared with state-of-the-art LF models, RLF is able to achieve significantly higher computational efficiency and comparable prediction accuracy for missing data.I provides an important alternative approach to LF analysis of HiDS matrices, which is especially desired for industrial applications demanding highly efficient models.

Robust Latent Factor Analysis for Precise Representation of High-Dimensional and Sparse Data

High-dimensional and sparse(HiDS)matrices commonly arise in various industrial applications,e.g.,recommender systems(RSs),social networks,and wireless sensor networks.Since they contain rich information,how to accurately represent them is of great significance.A latent factor(LF)model is one of the most popular and successful ways to address this issue.Current LF models mostly adopt L2-norm-oriented Loss to represent an HiDS matrix,i.e.,they sum the errors between observed data and predicted ones with L2-norm.Yet L2-norm is sensitive to outlier data.Unfortunately,outlier data usually exist in such matrices.For example,an HiDS matrix from RSs commonly contains many outlier ratings due to some heedless/malicious users.To address this issue,this work proposes a smooth L1-norm-oriented latent factor(SL-LF)model.Its main idea is to adopt smooth L1-norm rather than L2-norm to form its Loss,making it have both strong robustness and high accuracy in predicting the missing data of an HiDS matrix.Experimental results on eight HiDS matrices generated by industrial applications verify that the proposed SL-LF model not only is robust to the outlier data but also has significantly higher prediction accuracy than state-of-the-art models when they are used to predict the missing data of HiDS matrices.

Chip-Based High-Dimensional Optical Neural Network

Parallel multi-thread processing in advanced intelligent processors is the core to realize high-speed and high-capacity signal processing systems.Optical neural network(ONN)has the native advantages of high parallelization,large bandwidth,and low power consumption to meet the demand of big data.Here,we demonstrate the dual-layer ONN with Mach-Zehnder interferometer(MZI)network and nonlinear layer,while the nonlinear activation function is achieved by optical-electronic signal conversion.Two frequency components from the microcomb source carrying digit datasets are simultaneously imposed and intelligently recognized through the ONN.We successfully achieve the digit classification of different frequency components by demultiplexing the output signal and testing power distribution.Efficient parallelization feasibility with wavelength division multiplexing is demonstrated in our high-dimensional ONN.This work provides a high-performance architecture for future parallel high-capacity optical analog computing.

Learning sparse and smooth functions by deep Sigmoid nets

To pursue the outperformance of deep nets in learning,we construct a deep net with three hidden layers and prove that,implementing the empirical risk minimization(ERM)on this deep net,the estimator can theoretically realize the optimal learning rates without the classical saturation problem.In other words,deepening the networks with only three hidden layers can overcome the saturation and not degrade the optimal learning rates.The obtained results underlie the success of deep nets and provide a theoretical guidance for deep learning.

Multi-Layer Deep Sparse Representation for Biological Slice Image Inpainting

Biological slices are an effective tool for studying the physiological structure and evolutionmechanism of biological systems.However,due to the complexity of preparation technology and the presence of many uncontrollable factors during the preparation processing,leads to problems such as difficulty in preparing slice images and breakage of slice images.Therefore,we proposed a biological slice image small-scale corruption inpainting algorithm with interpretability based on multi-layer deep sparse representation,achieving the high-fidelity reconstruction of slice images.We further discussed the relationship between deep convolutional neural networks and sparse representation,ensuring the high-fidelity characteristic of the algorithm first.A novel deep wavelet dictionary is proposed that can better obtain image prior and possess learnable feature.And multi-layer deep sparse representation is used to implement dictionary learning,acquiring better signal expression.Compared with methods such as NLABH,Shearlet,Partial Differential Equation(PDE),K-Singular Value Decomposition(K-SVD),Convolutional Sparse Coding,and Deep Image Prior,the proposed algorithm has better subjective reconstruction and objective evaluation with small-scale image data,which realized high-fidelity inpainting,under the condition of small-scale image data.And theOn2-level time complexitymakes the proposed algorithm practical.The proposed algorithm can be effectively extended to other cross-sectional image inpainting problems,such as magnetic resonance images,and computed tomography images.

Underwater Noise Target Recognition Based on Sparse Adversarial Co-Training Model with Vertical Line Array

The automatic identification of underwater noncooperative targets without label records remains an arduous task considering the marine noise interference and the shortage of labeled samples.In particular,the data-driven mechanism of deep learning cannot identify false samples,aggravating the difficulty in noncooperative underwater target recognition.A semi-supervised ensemble framework based on vertical line array fusion and the sparse adversarial co-training algorithm is proposed to identify noncooperative targets effectively.The sound field cross-correlation compression(SCC)feature is developed to reduce noise and computational redundancy.Starting from an incomplete dataset,a joint adversarial autoencoder is constructed to extract the sparse features with source depth sensitivity,aiming to discover the unknown underwater targets.The adversarial prediction label is converted to initialize the joint co-forest,whose evaluation function is optimized by introducing adaptive confidence.The experiments prove the strong denoising performance,low mean square error,and high separability of SCC features.Compared with several state-of-the-art approaches,the numerical results illustrate the superiorities of the proposed method due to feature compression,secondary recognition,and decision fusion.

Alzheimer’s Disease Stage Classification Using a Deep Transfer Learning and Sparse Auto Encoder Method

Alzheimer’s Disease(AD)is a progressive neurological disease.Early diagnosis of this illness using conventional methods is very challenging.Deep Learning(DL)is one of the finest solutions for improving diagnostic procedures’performance and forecast accuracy.The disease’s widespread distribution and elevated mortality rate demonstrate its significance in the older-onset and younger-onset age groups.In light of research investigations,it is vital to consider age as one of the key criteria when choosing the subjects.The younger subjects are more susceptible to the perishable side than the older onset.The proposed investigation concentrated on the younger onset.The research used deep learning models and neuroimages to diagnose and categorize the disease at its early stages automatically.The proposed work is executed in three steps.The 3D input images must first undergo image pre-processing using Weiner filtering and Contrast Limited Adaptive Histogram Equalization(CLAHE)methods.The Transfer Learning(TL)models extract features,which are subsequently compressed using cascaded Auto Encoders(AE).The final phase entails using a Deep Neural Network(DNN)to classify the phases of AD.The model was trained and tested to classify the five stages of AD.The ensemble ResNet-18 and sparse autoencoder with DNN model achieved an accuracy of 98.54%.The method is compared to state-of-the-art approaches to validate its efficacy and performance.

Anomaly-Based Intrusion DetectionModel Using Deep Learning for IoT Networks

The rapid growth of Internet of Things(IoT)devices has brought numerous benefits to the interconnected world.However,the ubiquitous nature of IoT networks exposes them to various security threats,including anomaly intrusion attacks.In addition,IoT devices generate a high volume of unstructured data.Traditional intrusion detection systems often struggle to cope with the unique characteristics of IoT networks,such as resource constraints and heterogeneous data sources.Given the unpredictable nature of network technologies and diverse intrusion methods,conventional machine-learning approaches seem to lack efficiency.Across numerous research domains,deep learning techniques have demonstrated their capability to precisely detect anomalies.This study designs and enhances a novel anomaly-based intrusion detection system(AIDS)for IoT networks.Firstly,a Sparse Autoencoder(SAE)is applied to reduce the high dimension and get a significant data representation by calculating the reconstructed error.Secondly,the Convolutional Neural Network(CNN)technique is employed to create a binary classification approach.The proposed SAE-CNN approach is validated using the Bot-IoT dataset.The proposed models exceed the performance of the existing deep learning approach in the literature with an accuracy of 99.9%,precision of 99.9%,recall of 100%,F1 of 99.9%,False Positive Rate(FPR)of 0.0003,and True Positive Rate(TPR)of 0.9992.In addition,alternative metrics,such as training and testing durations,indicated that SAE-CNN performs better.

Sparse Autoencoder-based Multi-head Deep Neural Networks for Machinery Fault Diagnostics with Detection of Novelties

Supervised fault diagnosis typically assumes that all the types of machinery failures are known.However,in practice unknown types of defect,i.e.,novelties,may occur,whose detection is a challenging task.In this paper,a novel fault diagnostic method is developed for both diagnostics and detection of novelties.To this end,a sparse autoencoder-based multi-head Deep Neural Network(DNN)is presented to jointly learn a shared encoding representation for both unsupervised reconstruction and supervised classification of the monitoring data.The detection of novelties is based on the reconstruction error.Moreover,the computational burden is reduced by directly training the multi-head DNN with rectified linear unit activation function,instead of performing the pre-training and fine-tuning phases required for classical DNNs.The addressed method is applied to a benchmark bearing case study and to experimental data acquired from a delta 3D printer.The results show that its performance is satisfactory both in detection of novelties and fault diagnosis,outperforming other state-of-the-art methods.This research proposes a novel fault diagnostics method which can not only diagnose the known type of defect,but also detect unknown types of defects.

Pre-Training Physics-Informed Neural Network with Mixed Sampling and Its Application in High-Dimensional Systems

Recently,the physics-informed neural network shows remarkable ability in the context of solving the low-dimensional nonlinear partial differential equations.However,for some cases of high-dimensional systems,such technique may be time-consuming and inaccurate.In this paper,the authors put forward a pre-training physics-informed neural network with mixed sampling(pPINN)to address these issues.Just based on the initial and boundary conditions,the authors design the pre-training stage to filter out the set of the misfitting points,which is regarded as part of the training points in the next stage.The authors further take the parameters of the neural network in Stage 1 as the initialization in Stage 2.The advantage of the proposed approach is that it takes less time to transfer the valuable information from the first stage to the second one to improve the calculation accuracy,especially for the high-dimensional systems.To verify the performance of the pPINN algorithm,the authors first focus on the growing-and-decaying mode of line rogue wave in the Davey-Stewartson I equation.Another case is the accelerated motion of lump in the inhomogeneous Kadomtsev-Petviashvili equation,which admits a more complex evolution than the uniform equation.The exact solution provides a perfect sample for data experiments,and can also be used as a reference frame to identify the performance of the algorithm.The experiments confirm that the pPINN algorithm can improve the prediction accuracy and training efficiency well,and reduce the training time to a large extent for simulating nonlinear waves of high-dimensional equations.

Multi-task Joint Sparse Representation Classification Based on Fisher Discrimination Dictionary Learning

Recently,sparse representation classification(SRC)and fisher discrimination dictionary learning(FDDL)methods have emerged as important methods for vehicle classification.In this paper,inspired by recent breakthroughs of discrimination dictionary learning approach and multi-task joint covariate selection,we focus on the problem of vehicle classification in real-world applications by formulating it as a multi-task joint sparse representation model based on fisher discrimination dictionary learning to merge the strength of multiple features among multiple sensors.To improve the classification accuracy in complex scenes,we develop a new method,called multi-task joint sparse representation classification based on fisher discrimination dictionary learning,for vehicle classification.In our proposed method,the acoustic and seismic sensor data sets are captured to measure the same physical event simultaneously by multiple heterogeneous sensors and the multi-dimensional frequency spectrum features of sensors data are extracted using Mel frequency cepstral coefficients(MFCC).Moreover,we extend our model to handle sparse environmental noise.We experimentally demonstrate the benefits of joint information fusion based on fisher discrimination dictionary learning from different sensors in vehicle classification tasks.

基于sparse group Lasso方法的脑功能超网络构建与特征融合分析

功能超网络广泛地应用于脑疾病诊断和分类研究中,而现有的关于超网络创建的研究缺乏解释分组效应的能力或者仅考虑到脑区间组级的信息,这样构建的脑功能超网络会丢失一些有用的连接或包含一些虚假的信息,因此,考虑到脑区间的组结构问题,引入sparse group Lasso(sgLasso)方法进一步改善超网络的创建。首先,利用sgLasso方法进行超网络创建;然后,引入两组超网络特有的属性指标进行特征提取以及特征选择,这些指标分别是基于单一节点的聚类系数和基于一对节点的聚类系数;最后,将特征选择后得到的两组有显著差异的特征通过多核学习进行特征融合和分类。实验结果表明,所提方法经过多特征融合取得了87.88%的分类准确率。该结果表明为了改善脑功能超网络的创建,需要考虑到组信息,但不能逼迫使用整组信息,可以适当地对组结构进行扩展。

Slope reliability analysis based on Monte Carlo simulation and sparse grid method

In order to solve the problem of the reliability of slope engineering due to complex uncertainties, the Monte Carlo simulation method is adopted. Based on the characteristics of sparse grid, an interpolation algorithm, which can be applied to high dimensional problems, is introduced. A surrogate model of high dimensional implicit function is established, which makes Monte Carlo method more adaptable. Finally, a reliability analysis method is proposed to evaluate the reliability of the slope engineering, and is applied in the Sau Mau Ping slope project in Hong Kong. The reliability analysis method has great theoretical and practical significance for engineering quality evaluation and natural disaster assessment.

Image segmentation algorithm based on high-dimension fuzzy character and restrained clustering network

An image segmentation algorithm of the restrained fuzzy Kohonen clustering network （RFKCN） based on high- dimension fuzzy character is proposed. The algorithm includes two steps. The first step is the fuzzification of pixels in which two redundant images are built by fuzzy mean value and fuzzy median value. The second step is to construct a three-dimensional （3-D） feature vector of redundant images and their original images and cluster the feature vector through RFKCN, to realize image seg- mentation. The proposed algorithm fully takes into account not only gray distribution information of pixels, but also relevant information and fuzzy information among neighboring pixels in constructing 3- D character space. Based on the combination of competitiveness, redundancy and complementary of the information, the proposed algorithm improves the accuracy of clustering. Theoretical anal- yses and experimental results demonstrate that the proposed algorithm has a good segmentation performance.

Physics-informed neural networks with residual/gradient-based adaptive sampling methods for solving partial differential equations with sharp solutions

We consider solving the forward and inverse partial differential equations(PDEs)which have sharp solutions with physics-informed neural networks(PINNs)in this work.In particular,to better capture the sharpness of the solution,we propose the adaptive sampling methods(ASMs)based on the residual and the gradient of the solution.We first present a residual only-based ASM denoted by ASMⅠ.In this approach,we first train the neural network using a small number of residual points and divide the computational domain into a certain number of sub-domains,then we add new residual points in the sub-domain which has the largest mean absolute value of the residual,and those points which have the largest absolute values of the residual in this sub-domain as new residual points.We further develop a second type of ASM(denoted by ASMⅡ)based on both the residual and the gradient of the solution due to the fact that only the residual may not be able to efficiently capture the sharpness of the solution.The procedure of ASMⅡis almost the same as that of ASMⅠ,and we add new residual points which have not only large residuals but also large gradients.To demonstrate the effectiveness of the present methods,we use both ASMⅠand ASMⅡto solve a number of PDEs,including the Burger equation,the compressible Euler equation,the Poisson equation over an Lshape domain as well as the high-dimensional Poisson equation.It has been shown from the numerical results that the sharp solutions can be well approximated by using either ASMⅠor ASMⅡ,and both methods deliver much more accurate solutions than the original PINNs with the same number of residual points.Moreover,the ASMⅡalgorithm has better performance in terms of accuracy,efficiency,and stability compared with the ASMⅠalgorithm.This means that the gradient of the solution improves the stability and efficiency of the adaptive sampling procedure as well as the accuracy of the solution.Furthermore,we also employ the similar adaptive sampling technique for the data points of boundary conditions(BCs)if the sharpness of the solution is near the boundary.The result of the L-shape Poisson problem indicates that the present method can significantly improve the efficiency,stability,and accuracy.

The Study and Applications of Sparse Methods Technology in Yunnan Mountain Substation

As one of the divisions in China Southern Power Grid, Yunnan Power Grid Corporation has conducted research and demonstration projects on multiple smart grid technologies to improve the power system reliability, save operation cost and enhance measurement accuracy. In this paper, we will introduce The Study of Yunnan Mountain Substation Data Aggregation Technology based on Sparse Methods. Most substations are built in the mountain, the complex geological conditions and poor natural conditions put forward higher requirements on the substation running and real-time comprehensive monitoring of substation system. Processing and polymerization research of large amounts of the monitoring data and information is studied in this article. This paper introduces the sparse methods and then explains the thinning algorithm, especially new algorithm is proposed. Finally, the substation sparse method architecture is put forward and the simulation experiment was carried out to prove the feasibility and effectiveness of the proposed method.