Robust Principal Component Analysis Integrating Sparse and Low-Rank Priors

Principal Component Analysis (PCA) is a widely used technique for data analysis and dimensionality reduction, but its sensitivity to feature scale and outliers limits its applicability. Robust Principal Component Analysis (RPCA) addresses these limitations by decomposing data into a low-rank matrix capturing the underlying structure and a sparse matrix identifying outliers, enhancing robustness against noise and outliers. This paper introduces a novel RPCA variant, Robust PCA Integrating Sparse and Low-rank Priors (RPCA-SL). Each prior targets a specific aspect of the data’s underlying structure and their combination allows for a more nuanced and accurate separation of the main data components from outliers and noise. Then RPCA-SL is solved by employing a proximal gradient algorithm for improved anomaly detection and data decomposition. Experimental results on simulation and real data demonstrate significant advancements.

A new image processing method for discriminating internal layers from radio echo sounding data of ice sheets via a combined robust principal component analysis and total variation approach

Discriminating internal layers by radio echo sounding is important in analyzing the thickness and ice deposits in the Antarctic ice sheet.The signal processing method of synthesis aperture radar(SAR)has been widely used for improving the signal to noise ratio(SNR)and discriminating internal layers by radio echo sounding data of ice sheets.This method is not efficient when we use edge detection operators to obtain accurate information of the layers,especially the ice-bed interface.This paper presents a new image processing method via a combined robust principal component analysis-total variation(RPCA-TV)approach for discriminating internal layers of ice sheets by radio echo sounding data.The RPCA-based method is adopted to project the high-dimensional observations to low-dimensional subspace structure to accelerate the operation of the TV-based method,which is used to discriminate the internal layers.The efficiency of the presented method has been tested on simulation data and the dataset of the Institute of Electronics,Chinese Academy of Sciences,collected during CHINARE 28.The results show that the new method is more efficient than the previous method in discriminating internal layers of ice sheets by radio echo sounding data.

Applications of gauge duality in robust principal component analysis and semidefinite programming

Gauge duality theory was originated by Preund （1987）, and was recently further investigated by Friedlander et al. （2014）. When solving some matrix optimization problems via gauge dual, one is usually able to avoid full matrix decompositions such as singular value and/or eigenvalue decompositions. In such an approach, a gauge dual problem is solved in the first stage, and then an optimal solution to the primal problem can be recovered from the dual optimal solution obtained in the first stage. Recently, this theory has been applied to a class of semidefinite programming （SDP） problems with promising numerical results by Friedlander and Mac^to （2016）. We establish some theoretical results on applying the gauge duality theory to robust principal component analysis （PCA） and general SDP. For each problem, we present its gauge dual problem, characterize the optimality conditions for the primal-dual gauge pair, and validate a way to recover a primal optimal solution from a dual one. These results are extensions of Friedlander and Macedo （2016） from nuclear norm regularization to robust PCA and from a special class of SDP which requires the coefficient matrix in the linear objective to be positive definite to SDP problems without this restriction. Our results provide further understanding in the potential advantages and disadvantages of the gauge duality theory.

Robust Principal Component Analysis via Truncated Nuclear Norm Minimization

Robust principal component analysis(PCA) is widely used in many applications, such as image processing, data mining and bioinformatics. The existing methods for solving the robust PCA are mostly based on nuclear norm minimization. Those methods simultaneously minimize all the singular values, and thus the rank cannot be well approximated in practice. We extend the idea of truncated nuclear norm regularization(TNNR) to the robust PCA and consider truncated nuclear norm minimization(TNNM) instead of nuclear norm minimization(NNM). This method only minimizes the smallest N-r singular values to preserve the low-rank components, where N is the number of singular values and r is the matrix rank. Moreover, we propose an effective way to determine r via the shrinkage operator. Then we develop an effective iterative algorithm based on the alternating direction method to solve this optimization problem. Experimental results demonstrate the efficiency and accuracy of the TNNM method. Moreover, this method is much more robust in terms of the rank of the reconstructed matrix and the sparsity of the error.

A Robust Statistical Batch Process Monitoring Framework and Its Application

In order to reduce the variations of the product quality in batch processes, multivariate statistical process control methods according to multi-way principal component analysis (MPCA) or multi-way projection to latent structure (MPLS) were proposed for on-line batch process monitoring. However, they are based on the decomposition of relative covariance matrix and strongly affected by outlying observations. In this paper, in view of an efficient projection pursuit algorithm, a robust statistical batch process monitoring (RSBPM) framework,which is resistant to outliers, is proposed to reduce the high demand for modeling data. The construction of robust normal operating condition model and robust control limits are discussed in detail. It is evaluated on monitoring an industrial streptomycin fermentation process and compared with the conventional MPCA. The results show that the RSBPM framework is resistant to possible outliers and the robustness is confirmed.

Local singularity and S–A methods for analyzing ore-producing anomalies in the Jianbiannongchang area of Heilongjiang,China

The Heilongjiang Jianbiannongchang area is located at the confluence of the Great and Lesser Xing’an Ranges.This area has a complex magmatic and tectonic evolutionary history that has resulted in a complex and diverse geological background for mineralization.In this study,isometric logarithmic ratio(ILR)transformations of Au,Cu,Pb,Zn,and Sb contents were performed in the1:50,000 soil geochemical data of the Jianbiannongchang area.Robust principal component analysis(RPCA)was conducted based on ILR transformation.The local singularity and spectrum-area(S-A)methods were used to extract information on mineralogic anomalies.The results showed that:(1)the transformed data eliminated the influence of the original data closure effect,and the PC1and PC2 information obtained by applying RPCA reflected ore-producing element anomalies dominated by Au and Cu.(2)The local singularity method can enhance the information of the local strong and weak slow anomalies.After performing local singularity analysis on PC1 and PC2,the obtained local anomalies reflected the local singularity spatial anomaly patterns related to Cu and Au mineralization in this area,which is an effective method for trapping ore-producing anomalies.(3)Furthermore,the composite anomaly decomposition of PC1 and PC2 was performed using the S-A method,and the screened anomalous and background fields reflect the ore-producing anomalies related to Cu and Au mineralization.This information is in agreement with known Cu and Au mineralization.(4)The geochemical anomalies with mineralization potential were obtained outside the known mineralization sites by integrating the information of oreproducing anomalies extracted by the local singularity and S-A methods,providing the theoretical basis and exploration direction for future exploration in the study area.

Multivariate Statistical Process Monitoring Using Robust Nonlinear Principal Component Analysis

The principal component analysis （PCA） algorithm is widely applied in a diverse range of fields for performance assessment, fault detection, and diagnosis. However, in the presence of noise and gross errors, the nonlinear PCA （NLPCA） using autoassociative bottle-neck neural networks is so sensitive that the obtained model differs significantly from the underlying system. In this paper, a robust version of NLPCA is introduced by replacing the generally used error criterion mean squared error with a mean log squared error. This is followed by a concise analysis of the corresponding training method. A novel multivariate statistical process monitoring （MSPM） scheme incorporating the proposed robust NLPCA technique is then investigated and its efficiency is assessed through application to an industrial fluidized catalytic cracking plant. The results demonstrate that, compared with NLPCA, the proposed approach can effectively reduce the number of false alarms and is, hence, expected to better monitor real-world processes.

Identification of Heterogeneity of Social and Economic Environment of Land Uses in China

The robust principal component analysis （RPCA） is a technique of multivariate statistics to assess the social and economic environment quality. This paper aims to explore a RPCA algorithm to analyze the spatial heterogeneity of social and economic environment of land uses （SEELU）. RPCA supplies one of the most efficient methods to derive the most important components or factors affecting the regional difference of the social and economic environment. According to the spatial distributions of the levels of SEELU,the total land resources of China were divided into eight zones numbered by Ⅰ to Ⅷ which spatially referred to the eight levels of SEELU.

Accelerated Matrix Recovery via Random Projection Based on Inexact Augmented Lagrange Multiplier Method

In this paper, a unified matrix recovery model was proposed for diverse corrupted matrices. Resulting from the separable structure of the proposed model, the convex optimization problem can be solved efficiently by adopting an inexact augmented Lagrange multiplier (IALM) method. Additionally, a random projection accelerated technique (IALM+RP) was adopted to improve the success rate. From the preliminary numerical comparisons, it was indicated that for the standard robust principal component analysis (PCA) problem, IALM+RP was at least two to six times faster than IALM with an insignificant reduction in accuracy; and for the outlier pursuit (OP) problem, IALM+RP was at least 6.9 times faster, even up to 8.3 times faster when the size of matrix was 2 000×2 000.

Mainlobe jamming suppression via improved BSS method for rotated array radar

This study deals with the problem of mainlobe jamming suppression for rotated array radar.The interference becomes spatially nonstationary while the radar array rotates,which causes the mismatch between the weight and the snapshots and thus the loss of target signal to noise ratio(SNR)of pulse compression.In this paper,we explore the spatial divergence of interference sources and consider the rotated array radar anti-mainlobe jamming problem as a generalized rotated array mixed signal(RAMS)model firstly.Then the corresponding algorithm improved blind source separation(BSS)using the frequency domain of robust principal component analysis(FDRPCA-BSS)is proposed based on the established rotating model.It can eliminate the influence of the rotating parts and address the problem of loss of SNR.Finally,the measured peakto-average power ratio(PAPR)of each separated channel is performed to identify the target echo channel among the separated channels.Simulation results show that the proposed method is practically feasible and can suppress the mainlobe jamming with lower loss of SNR.

Robust PCA-Based Abnormal Traffic Flow Pattern Isolation and Loop Detector Fault Detection

One key function of intelligent transportation systems is to automatically detect abnormal traffic phenomena and to help further investigations of the cause of the abnormality. This paper describes a robust principal components analysis （RPCA）-based abnormal traffic flow pattern isolation and loop detector fault detection method. The results show that RPCA is a useful tool to distinguish regular traffic flow from abnormal traffic flow patterns caused by accidents and loop detector faults. This approach gives an effective traffic flow data pre-processing method to reduce the human effort in finding potential loop detector faults. The method can also be used to further investigate the causes of the abnormality.

Identifying porphyry-Cu geochemical footprints using local neighborhood statistics in Baft area, Iran

Identifying geochemical anomalies related to ore deposition processes facilitates the practice of vectoring toward undiscovered mineral deposit sites.In districtscale exploration studies,analysis of dispersion patterns of ore-forming elements results in more-reliable targets.Therefore,deriving significant geochemical footprints and mapping the ensuing geochemical anomalies are of important issues that lead exploration geologists toward anomaly sources,e.g.,mineralization.This paper aims to examine the effectiveness of local relative enrichment index and singularity mapping technique,as two methods of local neighborhood statistics,in the delineation of anomalous areas for further exploration.A data set of element contents obtained from stream sediment samples in Baft area,Iran,therefore was applied to illustrate the procedure proposed.The close relationship between anomalous patterns recognized and known Cu-occurrences demonstrated that the procedures proposed can efficiently model complex dispersion patterns of geochemical anomalies in the study area.The results showed that singularity mapping method is a better technique,compared to local relative enrichment index,to delineate targets for follow-up exploration in the area.We made this comparison because,as pointed out by exploration geochemists,dispersion patterns of geochemical indicators in stream sediments vary in different areas even for the same deposit type.The variety in the dispersion patterns is due to the operation of post-mineralization subsystems,which are affected by local factors such as landscape of the areas under study.Therefore,the effectiveness of the methods should be evaluated in every area for every targeted deposit.