Optimizing data aggregation and clustering in Internet of things networks using principal component analysis and Q-learning

The Internet of things(IoT)is a wireless network designed to perform specific tasks and plays a crucial role in various fields such as environmental monitoring,surveillance,and healthcare.To address the limitations imposed by inadequate resources,energy,and network scalability,this type of network relies heavily on data aggregation and clustering algorithms.Although various conventional studies have aimed to enhance the lifespan of a network through robust systems,they do not always provide optimal efficiency for real-time applications.This paper presents an approach based on state-of-the-art machine-learning methods.In this study,we employed a novel approach that combines an extended version of principal component analysis(PCA)and a reinforcement learning algorithm to achieve efficient clustering and data reduction.The primary objectives of this study are to enhance the service life of a network,reduce energy usage,and improve data aggregation efficiency.We evaluated the proposed methodology using data collected from sensors deployed in agricultural fields for crop monitoring.Our proposed approach(PQL)was compared to previous studies that utilized adaptive Q-learning(AQL)and regional energy-aware clustering(REAC).Our study outperformed in terms of both network longevity and energy consumption and established a fault-tolerant network.

NONLINEAR DATA RECONCILIATION METHOD BASED ON KERNEL PRINCIPAL COMPONENT ANALYSIS

In the industrial process situation, principal component analysis (PCA) is ageneral method in data reconciliation. However, PCA sometime is unfeasible to nonlinear featureanalysis and limited in application to nonlinear industrial process. Kernel PCA (KPCA) is extensionof PCA and can be used for nonlinear feature analysis. A nonlinear data reconciliation method basedon KPCA is proposed. The basic idea of this method is that firstly original data are mapped to highdimensional feature space by nonlinear function, and PCA is implemented in the feature space. Thennonlinear feature analysis is implemented and data are reconstructed by using the kernel. The datareconciliation method based on KPCA is applied to ternary distillation column. Simulation resultsshow that this method can filter the noise in measurements of nonlinear process and reconciliateddata can represent the true information of nonlinear process.

Polarimetric Meteorological Satellite Data Processing Software Classification Based on Principal Component Analysis and Improved K-Means Algorithm

With the increasing variety of application software of meteorological satellite ground system, how to provide reasonable hardware resources and improve the efficiency of software is paid more and more attention. In this paper, a set of software classification method based on software operating characteristics is proposed. The method uses software run-time resource consumption to describe the software running characteristics. Firstly, principal component analysis (PCA) is used to reduce the dimension of software running feature data and to interpret software characteristic information. Then the modified K-means algorithm was used to classify the meteorological data processing software. Finally, it combined with the results of principal component analysis to explain the significance of various types of integrated software operating characteristics. And it is used as the basis for optimizing the allocation of software hardware resources and improving the efficiency of software operation.

Identification and classification of transient pulses observed in magnetometer array data by time-domain principal component analysis filtering

A method for identification of pulsations in time series of magnetic field data which are simultaneously present in multiple channels of data at one or more sensor locations is described. Candidate pulsations of interest are first identified in geomagnetic time series by inspection. Time series of these ＂training events＂ are represented in matrix form and transpose-multiplied to generate time- domain covariance matrices. The ranked eigenvectors of this matrix are stored as a feature of the pulsation. In the second stage of the algorithm, a sliding window （approxi- mately the width of the training event） is moved across the vector-valued time-series comprising the channels on which the training event was observed. At each window position, the data covariance matrix and associated eigen- vectors are calculated. We compare the orientation of the dominant eigenvectors of the training data to those from the windowed data and flag windows where the dominant eigenvectors directions are similar. This was successful in automatically identifying pulses which share polarization and appear to be from the same source process. We apply the method to a case study of continuously sampled （50 Hz） data from six observatories, each equipped with three- component induction coil magnetometers. We examine a 90-day interval of data associated with a cluster of four observatories located within 50 km of Napa, California, together with two remote reference stations-one 100 km to the north of the cluster and the other 350 km south. When the training data contains signals present in the remote reference observatories, we are reliably able to identify and extract global geomagnetic signals such as solar-generated noise. When training data contains pulsations only observed in the cluster of local observatories, we identify several types of non-plane wave signals having similar polarization.

Functional Data Analysis of Spectroscopic Data with Application to Classification of Colon Polyps

In this study, two functional logistic regression models with functional principal component basis (FPCA) and functional partial least squares basis (FPLS) have been developed to distinguish precancerous adenomatous polyps from hyperplastic polyps for the purpose of classification and interpretation. The classification performances of the two functional models have been compared with two widely used multivariate methods, principal component discriminant analysis (PCDA) and partial least squares discriminant analysis (PLSDA). The results indicated that classification abilities of FPCA and FPLS models outperformed those of the PCDA and PLSDA models by using a small number of functional basis components. With substantial reduction in model complexity and improvement of classification accuracy, it is particularly helpful for interpretation of the complex spectral features related to precancerous colon polyps.

Functional Analysis of Chemometric Data

The objective of this paper is to present a review of different calibration and classification methods for functional data in the context of chemometric applications. In chemometric, it is usual to measure certain parameters in terms of a set of spectrometric curves that are observed in a finite set of points (functional data). Although the predictor variable is clearly functional, this problem is usually solved by using multivariate calibration techniques that consider it as a finite set of variables associated with the observed points (wavelengths or times). But these explicative variables are highly correlated and it is therefore more informative to reconstruct first the true functional form of the predictor curves. Although it has been published in several articles related to the implementation of functional data analysis techniques in chemometric, their power to solve real problems is not yet well known. Because of this the extension of multivariate calibration techniques (linear regression, principal component regression and partial least squares) and classification methods (linear discriminant analysis and logistic regression) to the functional domain and some relevant chemometric applications are reviewed in this paper.

Loss-of-Main Monitoring and Detection for Distributed Generations Using Dynamic Principal Component Analysis

In this paper, our previous work on Principal Component Analysis (PCA) based fault detection method is extended to the dynamic monitoring and detection of loss-of-main in power systems using wide-area synchrophasor measurements. In the previous work, a static PCA model was built and verified to be capable of detecting and extracting system faulty events;however the false alarm rate is high. To address this problem, this paper uses a well-known ‘time lag shift’ method to include dynamic behavior of the PCA model based on the synchronized measurements from Phasor Measurement Units (PMU), which is named as the Dynamic Principal Component Analysis (DPCA). Compared with the static PCA approach as well as the traditional passive mechanisms of loss-of-main detection, the proposed DPCA procedure describes how the synchrophasors are linearly auto- and cross-correlated, based on conducting the singular value decomposition on the augmented time lagged synchrophasor matrix. Similar to the static PCA method, two statistics, namely T2 and Q with confidence limits are calculated to form intuitive charts for engineers or operators to monitor the loss-of-main situation in real time. The effectiveness of the proposed methodology is evaluated on the loss-of-main monitoring of a real system, where the historic data are recorded from PMUs installed in several locations in the UK/Ireland power system.

Data Mining Based on Principal Component Analysis Application to the Nitric Oxide Response in Escherichia coli

This work evaluates a recently developed multivariate statistical method based on the creation of pseudo or latent variables using principal component analysis （PCA）. The application is the data mining of gene expression data to find a small subset of the most important genes in a set of thousand or tens of thousands of genes from a relatively small number of experimental runs. The method was previously developed and evaluated on artificially generated data and real data sets. Its evaluations consisted of its ability to rank the genes against known truth in simulated data studies and to identify known important genes in real data studies. The purpose of the work described here is to identify a ranked set of genes in an experimental study and then for a few of the most highly ranked unverified genes, experimentally verify their importance.This method was evaluated using the transcriptional response of Escherichia coli to treatment with four distinct inhibitory compounds： nitric oxide, S-nitrosoglutathione, serine hydroxamate and potassium cyanide. Our analysis identified genes previously recognized in the response to these compounds and also identified new genes.Three of these new genes, ycbR, yJhA and yahN, were found to significantly （p-values〈0.002） affect the sensitivityofE, coli to nitric oxide-mediated growth inhibition. Given that the three genes were not highly ranked in the selected ranked set （RS）, these results support strong sensitivity in the ability of the method to successfully identify genes related to challenge by NO and GSNO. This ability to identify genes related to the response to an inhibitory compound is important for engineering tolerance to inhibitory metabolic products, such as biofuels, and utilization of cheap sugar streams, such as biomass-derived sugars or hydrolysate.

Selection of Spectral Data for Classification of Steels Using Laser-Induced Breakdown Spectroscopy

Principal component analysis （PCA） combined with artificial neural networks was used to classify the spectra of 27 steel samples acquired using laser-induced breakdown spectroscopy. Three methods of spectral data selection, selecting all the peak lines of the spectra, selecting intensive spectral partitions and the whole spectra, were utilized to compare the infiuence of different inputs of PCA on the classification of steels. Three intensive partitions were selected based on experience and prior knowledge to compare the classification, as the partitions can obtain the best results compared to all peak lines and the whole spectra. We also used two test data sets, mean spectra after being averaged and raw spectra without any pretreatment, to verify the results of the classification. The results of this comprehensive comparison show that a back propagation network trained using the principal components of appropriate, carefully selecred spectral partitions can obtain the best results accuracy can be achieved using the intensive spectral A perfect result with 100% classification partitions ranging of 357-367 nm.

Derringer desirability and kinetic plot LC-column comparison approach for MS-compatible lipopeptide analysis

Lipopeptides are currently re-emerging as an interesting subgroup in the peptide research field, having historical applications as antibacterial and antifungal agents and new potential applications as antiviral, antitumor, immune-modulating and cell-penetrating compounds. However, due to their specific structure, chromatographic analysis often requires special buffer systems or the use of trifluoroacetic acid, limiting mass spectrometry detection. Therefore, we used a traditional aqueous/acetonitrile based gradient system, containing 0.1% （m/v） formic acid, to separate four pharmaceutically relevant lipopeptides （polymyxin B1, caspofungin, daptomycin and gramicidin A1）, which were selected based upon hierarchical cluster analysis （HCA） and principal component analysis （PCA）.In total, the performance of four different C18 columns, including one UPLC column, were evaluated using two parallel approaches. First, a Derringer desirability function was used, whereby six single and multiple chromatographic response values were rescaled into one overall D-value per column. Using this approach, the YMC Pack Pro C18 column was ranked as the best column for general MS-compatible lipopeptide separation. Secondly, the kinetic plot approach was used to compare the different columns at different flow rate ranges. As the optimal kinetic column performance is obtained at its maximal pressure, the length elongation factor λ（Pmax/Pexp） was used to transform the obtained experimental data （retention times and peak capacities） and construct kinetic performance limit （KPL） curves, allowing a direct visual and unbiased comparison of the selected columns, whereby the YMC Triart C18 UPLC and ACE C18 columns performed as best. Finally, differences in column performance and the （dis）advantages of both approaches are discussed.

Effect of Two Kinds of Similarity Factors on Principal Component Analysis Fault Detection in Air Conditioning Systems

Screening similar historical fault-free candidate data would greatly affect the effectiveness of fault detection results based on principal component analysis(PCA).In order to find out the candidate data,this study compares unweighted and weighted similarity factors(SFs),which measure the similarity of the principal component subspace corresponding to the first k main components of two datasets.The fault detection employs the principal component subspace corresponding to the current measured data and the historical fault-free data.From the historical fault-free database,the load parameters are employed to locate the candidate data similar to the current operating data.Fault detection method for air conditioning systems is based on principal component.The results show that the weighted principal component SF can improve the effects of the fault-free detection and the fault detection.Compared with the unweighted SF,the average fault-free detection rate of the weighted SF is 17.33%higher than that of the unweighted,and the average fault detection rate is 7.51%higher than unweighted.

Study on Factors Affecting Springback and Application of Data Mining in Springback Analysis

Springback of sheet metal induced by elastic recovery is one of major defects in sheet metal forming processed. Springback is influenced by many factors including properties of the sheet material and processing conditions. In this paper, a springback simulation was conducted and comparisons between the results based on different processing variables were illustrated. The discovery of knowledge of the effects of geometry and process parameters on springback from FEM results becomes increasingly important, as the number of numerical simulation has grown exponentially. Data mining is an effective tool to realize knowledge discovery in simulation results. A data-mining algorithm, rough sets theory (RST), was applied to analyze the effects of process parameters on springback in U-bending.

A DDoS Attack Information Fusion Method Based on CNN for Multi-Element Data

Traditional distributed denial of service(DDoS)detection methods need a lot of computing resource,and many of them which are based on single element have high missing rate and false alarm rate.In order to solve the problems,this paper proposes a DDoS attack information fusion method based on CNN for multi-element data.Firstly,according to the distribution,concentration and high traffic abruptness of DDoS attacks,this paper defines six features which are respectively obtained from the elements of source IP address,destination IP address,source port,destination port,packet size and the number of IP packets.Then,we propose feature weight calculation algorithm based on principal component analysis to measure the importance of different features in different network environment.The algorithm of weighted multi-element feature fusion proposed in this paper is used to fuse different features,and obtain multi-element fusion feature(MEFF)value.Finally,the DDoS attack information fusion classification model is established by using convolutional neural network and support vector machine respectively based on the MEFF time series.Experimental results show that the information fusion method proposed can effectively fuse multi-element data,reduce the missing rate and total error rate,memory resource consumption,running time,and improve the detection rate.

Data Masking for Chinese Electronic Medical Records with Named Entity Recognition

With the rapid development of information technology,the electronifi-cation of medical records has gradually become a trend.In China,the population base is huge and the supporting medical institutions are numerous,so this reality drives the conversion of paper medical records to electronic medical records.Electronic medical records are the basis for establishing a smart hospital and an important guarantee for achieving medical intelligence,and the massive amount of electronic medical record data is also an important data set for conducting research in the medical field.However,electronic medical records contain a large amount of private patient information,which must be desensitized before they are used as open resources.Therefore,to solve the above problems,data masking for Chinese electronic medical records with named entity recognition is proposed in this paper.Firstly,the text is vectorized to satisfy the required format of the model input.Secondly,since the input sentences may have a long or short length and the relationship between sentences in context is not negligible.To this end,a neural network model for named entity recognition based on bidirectional long short-term memory(BiLSTM)with conditional random fields(CRF)is constructed.Finally,the data masking operation is performed based on the named entity recog-nition results,mainly using regular expression filtering encryption and principal component analysis(PCA)word vector compression and replacement.In addi-tion,comparison experiments with the hidden markov model(HMM)model,LSTM-CRF model,and BiLSTM model are conducted in this paper.The experi-mental results show that the method used in this paper achieves 92.72%Accuracy,92.30%Recall,and 92.51%F1_score,which has higher accuracy compared with other models.

The Principal Component Transform of Parametrized Functions

Many advanced mathematical models of biochemical, biophysical and other processes in systems biology can be described by parametrized systems of nonlinear differential equations. Due to complexity of the models, a problem of their simplification has become of great importance. In particular, rather challengeable methods of estimation of parameters in these models may require such simplifications. The paper offers a practical way of constructing approximations of nonlinearly parametrized functions by linearly parametrized ones. As the idea of such approximations goes back to Principal Component Analysis, we call the corresponding transformation Principal Component Transform. We show that this transform possesses the best individual fit property, in the sense that the corresponding approximations preserve most information (in some sense) about the original function. It is also demonstrated how one can estimate the error between the given function and its approximations. In addition, we apply the theory of tensor products of compact operators in Hilbert spaces to justify our method for the case of the products of parametrized functions. Finally, we provide several examples, which are of relevance for systems biology.

Reconstructing urban wind flows for urban air mobility using reduced-order data assimilation

Advancements in uncrewed aircrafts and communications technologies have led to a wave of interest and investment in unmanned aircraft systems(UASs)and urban air mobility(UAM)vehicles over the past decade.To support this emerging aviation application,concepts for UAS/UAM traffic management(UTM)systems have been explored.Accurately characterizing and predicting the microscale weather conditions,winds in particular,will be critical to safe and efficient operations of the small UASs/UAM aircrafts within the UTM.This study implements a reduced order data assimilation approach to reduce discrepancies between the predicted urban wind speed with computational fluid dynamics(CFD)Reynolds-averaged Navier Stokes(RANS)model with real-world,limited and sparse observations.The developed data assimilation system is UrbanDA.These observations are simulated using a large eddy simulation(LES).The data assimilation approach is based on the time-independent variational framework and uses space reduction to reduce the memory cost of the process.This approach leads to error reduction throughout the simulated domain and the reconstructed field is different than the initial guess by ingesting wind speeds at sensor locations and hence taking into account flow unsteadiness in a time when only the mean flow quantities are resolved.Different locations where wind sensors can be installed are discussed in terms of their impact on the resulting wind field.It is shown that near-wall locations,near turbulence generation areas with high wind speeds have the highest impact.Approximating the model error with its principal mode provides a better agreement with the truth and the hazardous areas for UAS navigation increases by more than 10%as wind hazards resulting from buildings wakes are better simulated through this process.

Application of Particle Swarm Optimization to Fault Condition Recognition Based on Kernel Principal Component Analysis

Panicle swarm optimization （PSO） is an optimization algorithm based on the swarm intelligent principle. In this paper the modified PSO is applied to a kernel principal component analysis （ KPCA ） for an optimal kernel function parameter. We first comprehensively considered within-class scatter and between-class scatter of the sample features. Then, the fitness function of an optimized kernel function parameter is constructed, and the particle swarm optimization algorithm with adaptive acceleration （CPSO） is applied to optimizing it. It is used for gearbox condi- tion recognition, and the result is compared with the recognized results based on principal component analysis （PCA）. The results show that KPCA optimized by CPSO can effectively recognize fault conditions of the gearbox by reducing bind set-up of the kernel function parameter, and its results of fault recognition outperform those of PCA. We draw the conclusion that KPCA based on CPSO has an advantage in nonlinear feature extraction of mechanical failure, and is helpful for fault condition recognition of complicated machines.

IoT Information Status Using Data Fusion and Feature Extraction Method

The Internet of Things(IoT)role is instrumental in the technological advancement of the healthcare industry.Both the hardware and the core level of software platforms are the progress resulted from the accompaniment of Medicine 4.0.Healthcare IoT systems are the emergence of this foresight.The communication systems between the sensing nodes and the processors;and the processing algorithms to produce output obtained from the data collected by the sensors are the major empowering technologies.At present,many new technologies supplement these empowering technologies.So,in this research work,a practical feature extraction and classification technique is suggested for handling data acquisition besides data fusion to enhance treatment-related data.In the initial stage,IoT devices are gathered and pre-processed for fusion processing.Dynamic Bayesian Network is considered an improved balance for tractability,a tool for CDF operations.Improved Principal Component Analysis is deployed for feature extraction along with dimension reduction.Lastly,this data learning is attained through Hybrid Learning Classifier Model for data fusion performance examination.In this research,Deep Belief Neural Network and Support VectorMachine are hybridized for healthcare data prediction.Thus,the suggested system is probably a beneficial decision support tool for multiple data sources prediction and predictive ability enhancement.

Analysis of ATR-FTIR Absorption-Reflection Data from 13 Polymeric Fabric Materials Using Chemometrics

We used both correlation and covariance-principal component analysis (PCA) to classify the same absorption-reflectance data collected from 13 different polymeric fabric materials that was obtained using Attenuated Total Reflectance-Fourier Transform Infrared spectroscopy (ATR-FTIR). The application of the two techniques, though similar, yielded results that represent different chemical properties of the polymeric substances. On one hand, correlation-PCA enabled the classification of the fabric materials according to the organic functional groups of their repeating monomer units. On the other hand, covariance-PCA was used to classify the fabric materials primarily according to their origins;natural (animal or plant) or synthetic. Hence besides major chemical functional groups of the repeat units, it appears covariance-PCA is also sensitive to other characteristic chemical (inorganic and/or organic) or biochemical material inclusions that are found in different samples. We therefore recommend the application of both covariance-PCA and correlation-PCA on datasets, whenever applicable, to enable a broader classification of spectroscopic information through data mining and exploration.

Unsupervised Functional Data Clustering Based on Adaptive Weights

In recent years, functional data has been widely used in finance, medicine, biology and other fields. The current clustering analysis can solve the problems in finite-dimensional space, but it is difficult to be directly used for the clustering of functional data. In this paper, we propose a new unsupervised clustering algorithm based on adaptive weights. In the absence of initialization parameter, we use entropy-type penalty terms and fuzzy partition matrix to find the optimal number of clusters. At the same time, we introduce a measure based on adaptive weights to reflect the difference in information content between different clustering metrics. Simulation experiments show that the proposed algorithm has higher purity than some algorithms.