Applying Analytical Derivative and Sparse Matrix Techniques to Large-Scale Process Optimization Problems

The performance of analytical derivative and sparse matrix techniques applied to a traditional dense sequential quadratic programming (SQP) is studied, and the strategy utilizing those techniques is also presented.Computational results on two typical chemical optimization problems demonstrate significant enhancement in efficiency, which shows this strategy is promising and suitable for large-scale process optimization problems.

Cache performance optimization of irregular sparse matrix multiplication on modern multi-core CPU and GPU

This paper focuses on how to optimize the cache performance of sparse matrix-matrix multiplication(SpGEMM).It classifies the cache misses into two categories;one is caused by the irregular distribution pattern of the multiplier-matrix,and the other is caused by the multiplicand.For each of them,the paper puts forward an optimization method respectively.The first hash based method removes cache misses of the 1 st category effectively,and improves the performance by a factor of 6 on an Intel 8-core CPU for the best cases.For cache misses of the 2nd category,it proposes a new cache replacement algorithm,which achieves a cache hit rate much higher than other historical knowledge based algorithms,and the algorithm is applicable on CELL and GPU.To further verify the effectiveness of our methods,we implement our algorithm on GPU,and the performance perfectly scales with the size of on-chip storage.

Robust Principal Component Analysis Integrating Sparse and Low-Rank Priors

Principal Component Analysis (PCA) is a widely used technique for data analysis and dimensionality reduction, but its sensitivity to feature scale and outliers limits its applicability. Robust Principal Component Analysis (RPCA) addresses these limitations by decomposing data into a low-rank matrix capturing the underlying structure and a sparse matrix identifying outliers, enhancing robustness against noise and outliers. This paper introduces a novel RPCA variant, Robust PCA Integrating Sparse and Low-rank Priors (RPCA-SL). Each prior targets a specific aspect of the data’s underlying structure and their combination allows for a more nuanced and accurate separation of the main data components from outliers and noise. Then RPCA-SL is solved by employing a proximal gradient algorithm for improved anomaly detection and data decomposition. Experimental results on simulation and real data demonstrate significant advancements.

PERFORMANCE OF SIMPLE-ENCODING IRREGULAR LDPC CODES BASED ON SPARSE GENERATOR MATRIX

A new method for the construction of the high performance systematic irregular low-density paritycheck （LDPC） codes based on the sparse generator matrix （G-LDPC） is introduced. The code can greatly reduce the encoding complexity while maintaining the same decoding complexity as traditional regular LDPC （H-LDPC） codes defined by the sparse parity check matrix. Simulation results show that the performance of the proposed irregular LDPC codes can offer significant gains over traditional LDPC codes in low SNRs with a few decoding iterations over an additive white Gaussian noise （AWGN） channel.

Robust Latent Factor Analysis for Precise Representation of High-Dimensional and Sparse Data

High-dimensional and sparse(HiDS)matrices commonly arise in various industrial applications,e.g.,recommender systems(RSs),social networks,and wireless sensor networks.Since they contain rich information,how to accurately represent them is of great significance.A latent factor(LF)model is one of the most popular and successful ways to address this issue.Current LF models mostly adopt L2-norm-oriented Loss to represent an HiDS matrix,i.e.,they sum the errors between observed data and predicted ones with L2-norm.Yet L2-norm is sensitive to outlier data.Unfortunately,outlier data usually exist in such matrices.For example,an HiDS matrix from RSs commonly contains many outlier ratings due to some heedless/malicious users.To address this issue,this work proposes a smooth L1-norm-oriented latent factor(SL-LF)model.Its main idea is to adopt smooth L1-norm rather than L2-norm to form its Loss,making it have both strong robustness and high accuracy in predicting the missing data of an HiDS matrix.Experimental results on eight HiDS matrices generated by industrial applications verify that the proposed SL-LF model not only is robust to the outlier data but also has significantly higher prediction accuracy than state-of-the-art models when they are used to predict the missing data of HiDS matrices.

Randomized Latent Factor Model for High-dimensional and Sparse Matrices from Industrial Applications

Latent factor(LF) models are highly effective in extracting useful knowledge from High-Dimensional and Sparse(HiDS) matrices which are commonly seen in various industrial applications. An LF model usually adopts iterative optimizers,which may consume many iterations to achieve a local optima,resulting in considerable time cost. Hence, determining how to accelerate the training process for LF models has become a significant issue. To address this, this work proposes a randomized latent factor(RLF) model. It incorporates the principle of randomized learning techniques from neural networks into the LF analysis of HiDS matrices, thereby greatly alleviating computational burden. It also extends a standard learning process for randomized neural networks in context of LF analysis to make the resulting model represent an HiDS matrix correctly.Experimental results on three HiDS matrices from industrial applications demonstrate that compared with state-of-the-art LF models, RLF is able to achieve significantly higher computational efficiency and comparable prediction accuracy for missing data.I provides an important alternative approach to LF analysis of HiDS matrices, which is especially desired for industrial applications demanding highly efficient models.

A NEW HIGH PERFORMANCE SPARSE STATIC SOLVER IN FINITE ELEMENT ANALYSIS WITH LOOP-UNROLLING

In the previous papers, a high performance sparse static solver with two-level unrolling based on a cell-sparse storage scheme was reported. Although the solver reaches quite a high efficiency for a big percentage of finite element analysis benchmark tests, the MFLOPS （million floating operations per second） of LDL^T factorization of benchmark tests vary on a Dell Pentium IV 850 MHz machine from 100 to 456 depending on the average size of the super-equations, i.e., on the average depth of unrolling. In this paper, a new sparse static solver with two-level unrolling that employs the concept of master-equations and searches for an appropriate depths of unrolling is proposed. The new solver provides higher MFLOPS for LDL^T factorization of benchmark tests, and therefore speeds up the solution process.

A Fast LDL-factorization Approach for Large Sparse Positive Definite System and Its Application to One-to-one Marketing Optimization Computation

LDL-factorization is an efficient way of solving Ax = b for a large symmetric positive definite sparse matrix A. This paper presents a new method that further improves the efficiency of LDL-factorization. It is based on the theory of elimination trees for the factorization factor. It breaks the computations involved in LDL-factorization down into two stages： 1） the pattern of nonzero entries of the factor is predicted, and 2） the numerical values of the nonzero entries of the factor are computed. The factor is stored using the form of an elimination tree so as to reduce memory usage and avoid unnecessary numerical operations. The calculation results for some typical numerical examples demonstrate that this method provides a significantly higher calculation efficiency for the one-to-one marketing optimization algorithm.

General dynamic model for educational assembling-type robot and its fast simulation algorithm

To realize automatic modeling and dynamic simulation of the educational assembling-type robot with open structure, a general dynamic model for the educational assembling-type robot and a fast simulation algorithm are put forward. First, the educational robot system is abstracted to a multibody system and a general dynamic model of the educational robot is constructed by the Newton-Euler method. Then the dynamic model is simplified by a combination of components with fixed connections according to the structural characteristics of the educational robot. Secondly, in order to obtain a high efficiency simulation algorithm, based on the sparse matrix technique, the augmentation algorithm and the direct projective constraint stabilization algorithm are improved. Finally, a numerical example is given. The results show that the model and the fast algorithm are valid and effective. This study lays a dynamic foundation for realizing the simulation platform of the educational robot.

Fast precise integration method for hyperbolic heat conduction problems

A fast precise integration method is developed for the time integral of the hyperbolic heat conduction problem. The wave nature of heat transfer is used to analyze the structure of the matrix exponential, leading to the fact that the matrix exponential is sparse. The presented method employs the sparsity of the matrix exponential to improve the original precise integration method. The merits are that the proposed method is suitable for large hyperbolic heat equations and inherits the accuracy of the original version and the good computational efficiency, which are verified by two numerical examples.

Benchmarking Performance of Document Level Classification and Topic Modeling

Text classification of low resource language is always a trivial and challenging problem.This paper discusses the process of Urdu news classification and Urdu documents similarity.Urdu is one of the most famous spoken languages in Asia.The implementation of computational methodologies for text classification has increased over time.However,Urdu language has not much experimented with research,it does not have readily available datasets,which turn out to be the primary reason behind limited research and applying the latest methodologies to the Urdu.To overcome these obstacles,a mediumsized dataset having six categories is collected from authentic Pakistani news sources.Urdu is a rich but complex language.Text processing can be challenging for Urdu due to its complex features as compared to other languages.Term frequency-inverse document frequency(TFIDF)based term weighting scheme for extracting features,chi-2 for selecting essential features,and Linear discriminant analysis(LDA)for dimensionality reduction have been used.TFIDF matrix and cosine similarity measure have been used to identify similar documents in a collection and find the semantic meaning of words in a document FastText model has been applied.The training-test split evaluation methodology is used for this experimentation,which includes 70%for training data and 30%for testing data.State-of-the-art machine learning and deep dense neural network approaches for Urdu news classification have been used.Finally,we trained Multinomial Naïve Bayes,XGBoost,Bagging,and Deep dense neural network.Bagging and deep dense neural network outperformed the other algorithms.The experimental results show that deep dense achieves 92.0%mean f1 score,and Bagging 95.0%f1 score.

Wavelet Numerical Solutions for Weakly Singular Fredholm Integral Equations of the Second Kind

Daubechies interval cally weakly singular Fredholm kind. Utilizing the orthogonality equation is reduced into a linear wavelet is used to solve nurneriintegral equations of the second of the wavelet basis, the integral system of equations. The vanishing moments of the wavelet make the wavelet coefficient matrices sparse, while the continuity of the derivative functions of basis overcomes naturally the singular problem of the integral solution. The uniform convergence of the approximate solution by the wavelet method is proved and the error bound is given. Finally, numerical example is presented to show the application of the wavelet method.

Time-dependent Density Functional-based Tight-bind Method Efficiently Implemented with OpenMP Parallel and GPU Acceleration

The time-dependent density functional-based tight-bind （TD-DFTB） method is implemented on the multi-core and the graphical processing unit （GPU） system for excited state calcu-lations of large system with hundreds or thousands of atoms. Sparse matrix and OpenMP multithreaded are used for building the Hamiltonian matrix. The diagonal of the eigenvalue problem in the ground state is implemented on the GPUs with double precision. The GPU- based acceleration fully preserves all the properties, and a considerable total speedup of 8.73 can be achieved. A Krylov-space-based algorithm with the OpenMP parallel and CPU acceleration is used for finding the lowest eigenvalue and eigenvector of the large TDDFT matrix, which greatly reduces the iterations taken and the time spent on the excited states eigenvalue problem. The Krylov solver with the GPU acceleration of matrix-vector product can converge quickly to obtain the final result and a notable speed-up of 206 times can be observed for system size of 812 atoms. The calculations on serials of small and large systems show that the fast TD-DFTB code can obtain reasonable result with a much cheaper computational requirement compared with the first-principle results of CIS and full TDDFT calculation.

AINV AND BILUM PRECONDITIONING TECHNIQUES

It was proposed that a robust and efficient parallelizable preconditioner for solving general sparse linear systems of equations, in which the use of sparse approximate inverse (AINV) techniques in a multi-level block ILU (BILUM) preconditioner were investigated. The resulting preconditioner retains robustness of BILUM preconditioner and has two advantages over the standard BILUM preconditioner: the ability to control sparsity and increased parallelism. Numerical experiments are used to show the effectiveness and efficiency of the new preconditioner.

Clustering-based hyperspectral band selection using sparse nonnegative matrix factorization

Hyperspectral imagery generally contains a very large amount of data due to hundreds of spectral bands.Band selection is often applied firstly to reduce computational cost and facilitate subsequent tasks such as land-cover classification and higher level image analysis.In this paper,we propose a new band selection algorithm using sparse nonnegative matrix factorization (sparse NMF).Though acting as a clustering method for band selection,sparse NMF need not consider the distance metric between different spectral bands,which is often the key step for most common clustering-based band selection methods.By imposing sparsity on the coefficient matrix,the bands' clustering assignments can be easily indicated through the largest entry in each column of the matrix.Experimental results showed that sparse NMF provides considerable insight into the clustering-based band selection problem and the selected bands are good for land-cover classification.

Encoding of rat working memory by power of multi-channel local field potentials via sparse non-negative matrix factorization

Working memory plays an important role in human cognition. This study investigated how working memory was encoded by the power of multichannel local field potentials （LFPs） based on sparse non negative matrix factorization （SNMF）. SNMF was used to extract features from LFPs recorded from the prefrontal cortex of four SpragueDawley rats during a memory task in a Y maze, with 10 trials for each rat. Then the powerincreased LFP components were selected as working memoryrelated features and the other components were removed. After that, the inverse operation of SNMF was used to study the encoding of working memory in the time frequency domain. We demonstrated that theta and gamma power increased significantly during the working memory task. The results suggested that postsynaptic activity was simulated well by the sparse activity model. The theta and gamma bands were meaningful for encoding working memory.

Robust signal recovery algorithm for structured perturbation compressive sensing

It is understood that the sparse signal recovery with a standard compressive sensing（CS） strategy requires the measurement matrix known as a priori. The measurement matrix is, however, often perturbed in a practical application.In order to handle such a case, an optimization problem by exploiting the sparsity characteristics of both the perturbations and signals is formulated. An algorithm named as the sparse perturbation signal recovery algorithm（SPSRA） is then proposed to solve the formulated optimization problem. The analytical results show that our SPSRA can simultaneously recover the signal and perturbation vectors by an alternative iteration way, while the convergence of the SPSRA is also analytically given and guaranteed. Moreover, the support patterns of the sparse signal and structured perturbation shown are the same and can be exploited to improve the estimation accuracy and reduce the computation complexity of the algorithm. The numerical simulation results verify the effectiveness of analytical ones.

Efficient Distributed Estimation of High-dimensional Sparse Precision Matrix for Transelliptical Graphical Models

In this paper,distributed estimation of high-dimensional sparse precision matrix is proposed based on the debiased D-trace loss penalized lasso and the hard threshold method when samples are distributed into different machines for transelliptical graphical models.At a certain level of sparseness,this method not only achieves the correct selection of non-zero elements of sparse precision matrix,but the error rate can be comparable to the estimator in a non-distributed setting.The numerical results further prove that the proposed distributed method is more effective than the usual average method.

Matrix Decomposition and Calculation of Chrestenson Spectra

Calculation of a variation of discrete Fourier transform.Chrestenson spectraof functions of n indeterminates over integer modulo m(composite integer),is con-sidered.Based on sparse matrix decomposition,two fast algorithms with complexityO(mnn∑ri=1pi)are given to calculate the Chrestenson spectra,where p1p2…p2 is theprime factor decomposition of m.

Parallel Solutions for Large-Scale General Sparse Nonlinear Systems of Equations

In solving application problems, many largesscale nonlinear systems of equations result in sparse Jacobian matrices. Such nonlinear systems are called sparse nonlinear systems. The irregularity of the locations of nonzero elements of a general sparse matrix makes it very difficult to generally map sparse matrix computations to multiprocessors for parallel processing in a well balanced manner. To overcome this difficulty, we define a new storage scheme for general sparse matrices in this paper. With the new storage scheme, we develop parallel algorithms to solve large-scale general sparse systems of equations by interval Newton/Generalized bisection methods which reliably find all numerical solutions within a given domain.In Section 1, we provide an introduction to the addressed problem and the interval Newton's methods. In Section 2, some currently used storage schemes for sparse sys-terns are reviewed. In Section 3, new index schemes to store general sparse matrices are reported. In Section 4, we present a parallel algorithm to evaluate a general sparse Jarobian matrix. In Section 5, we present a parallel algorithm to solve the correspond-ing interval linear 8ystem by the all-row preconditioned scheme. Conclusions and future work are discussed in Section 6.