Dimensionality reduction model based on integer planning for the analysis of key indicators affecting life expectancy

Purpose:Exploring a dimensionality reduction model that can adeptly eliminate outliers and select the appropriate number of clusters is of profound theoretical and practical importance.Additionally,the interpretability of these models presents a persistent challenge.Design/methodology/approach:This paper proposes two innovative dimensionality reduction models based on integer programming(DRMBIP).These models assess compactness through the correlation of each indicator with its class center,while separation is evaluated by the correlation between different class centers.In contrast to DRMBIP-p,the DRMBIP-v considers the threshold parameter as a variable aiming to optimally balances both compactness and separation.Findings:This study,getting data from the Global Health Observatory(GHO),investigates 141 indicators that influence life expectancy.The findings reveal that DRMBIP-p effectively reduces the dimensionality of data,ensuring compactness.It also maintains compatibility with other models.Additionally,DRMBIP-v finds the optimal result,showing exceptional separation.Visualization of the results reveals that all classes have a high compactness.Research limitations:The DRMBIP-p requires the input of the correlation threshold parameter,which plays a pivotal role in the effectiveness of the final dimensionality reduction results.In the DRMBIP-v,modifying the threshold parameter to variable potentially emphasizes either separation or compactness.This necessitates an artificial adjustment to the overflow component within the objective function.Practical implications:The DRMBIP presented in this paper is adept at uncovering the primary geometric structures within high-dimensional indicators.Validated by life expectancy data,this paper demonstrates potential to assist data miners with the reduction of data dimensions.Originality/value:To our knowledge,this is the first time that integer programming has been used to build a dimensionality reduction model with indicator filtering.It not only has applications in life expectancy,but also has obvious advantages in data mining work that requires precise class centers.

Nonlinear Dimensionality Reduction and Data Visualization:A Review

Dimensionality reduction and data visualization are useful and important processes in pattern recognition. Many techniques have been developed in the recent years. The self-organizing map （SOM） can be an efficient method for this purpose. This paper reviews recent advances in this area and related approaches such as multidimensional scaling （MDS）, nonlinear PC A, principal manifolds, as well as the connections of the SOM and its recent variant, the visualization induced SOM （ViSOM）, with these approaches. The SOM is shown to produce a quantized, qualitative scaling and while the ViSOM a quantitative or metric scaling and approximates principal curve/surface. The SOM can also be regarded as a generalized MDS to relate two metric spaces by forming a topological mapping between them. The relationships among various recently proposed techniques such as ViSOM, Isomap, LLE, and eigenmap are discussed and compared.

Similarity measurement method of high-dimensional data based on normalized net lattice subspace

The performance of conventional similarity measurement methods is affected seriously by the curse of dimensionality of high-dimensional data.The reason is that data difference between sparse and noisy dimensionalities occupies a large proportion of the similarity,leading to the dissimilarities between any results.A similarity measurement method of high-dimensional data based on normalized net lattice subspace is proposed.The data range of each dimension is divided into several intervals,and the components in different dimensions are mapped onto the corresponding interval.Only the component in the same or adjacent interval is used to calculate the similarity.To validate this method,three data types are used,and seven common similarity measurement methods are compared.The experimental result indicates that the relative difference of the method is increasing with the dimensionality and is approximately two or three orders of magnitude higher than the conventional method.In addition,the similarity range of this method in different dimensions is [0,1],which is fit for similarity analysis after dimensionality reduction.

Critical Evaluation of Linear Dimensionality Reduction Techniques for Cardiac Arrhythmia Classification

Embedding the original high dimensional data in a low dimensional space helps to overcome the curse of dimensionality and removes noise. The aim of this work is to evaluate the performance of three different linear dimensionality reduction techniques (DR) techniques namely principal component analysis (PCA), multi dimensional scaling (MDS) and linear discriminant analysis (LDA) on classification of cardiac arrhythmias using probabilistic neural network classifier (PNN). The design phase of classification model comprises of the following stages: preprocessing of the cardiac signal by eliminating detail coefficients that contain noise, feature extraction through daubechies wavelet transform, dimensionality reduction through linear DR techniques specified, and arrhythmia classification using PNN. Linear dimensionality reduction techniques have simple geometric representations and simple computational properties. Entire MIT-BIH arrhythmia database is used for experimentation. The experimental results demonstrates that combination of PNN classifier (spread parameter, σ = 0.08) and PCA DR technique exhibits highest sensitivity and F score of 78.84% and 78.82% respectively with a minimum of 8 dimensions.

A Power Data Anomaly Detection Model Based on Deep Learning with Adaptive Feature Fusion

With the popularisation of intelligent power,power devices have different shapes,numbers and specifications.This means that the power data has distributional variability,the model learning process cannot achieve sufficient extraction of data features,which seriously affects the accuracy and performance of anomaly detection.Therefore,this paper proposes a deep learning-based anomaly detection model for power data,which integrates a data alignment enhancement technique based on random sampling and an adaptive feature fusion method leveraging dimension reduction.Aiming at the distribution variability of power data,this paper developed a sliding window-based data adjustment method for this model,which solves the problem of high-dimensional feature noise and low-dimensional missing data.To address the problem of insufficient feature fusion,an adaptive feature fusion method based on feature dimension reduction and dictionary learning is proposed to improve the anomaly data detection accuracy of the model.In order to verify the effectiveness of the proposed method,we conducted effectiveness comparisons through elimination experiments.The experimental results show that compared with the traditional anomaly detection methods,the method proposed in this paper not only has an advantage in model accuracy,but also reduces the amount of parameter calculation of the model in the process of feature matching and improves the detection speed.

AE-TPGG:a novel autoencoder-based approach for single-cell RNA-seq data imputation and dimensionality reduction

Single-cell RNA sequencing(scRNA-seq)technology has become an effective tool for high-throughout transcriptomic study,which circumvents the averaging artifacts corresponding to bulk RNA-seq technology,yielding new perspectives on the cellular diversity of potential superficially homogeneous populations.Although various sequencing techniques have decreased the amplification bias and improved capture efficiency caused by the low amount of starting material,the technical noise and biological variation are inevitably introduced into experimental process,resulting in high dropout events,which greatly hinder the downstream analysis.Considering the bimodal expression pattern and the right-skewed characteristic existed in normalized scRNA-seq data,we propose a customized autoencoder based on a twopart-generalized-gamma distribution(AE-TPGG)for scRNAseq data analysis,which takes mixed discrete-continuous random variables of scRNA-seq data into account using a twopart model and utilizes the generalized gamma(GG)distribution,for fitting the positive and right-skewed continuous data.The adopted autoencoder enables AE-TPGG to captures the inherent relationship between genes.In addition to the ability of achieving low-dimensional representation,the AETPGG model also provides a denoised imputation according to statistical characteristic of gene expression.Results on real datasets demonstrate that our proposed model is competitive to current imputation methods and ameliorates a diverse set of typical scRNA-seq data analyses.

Global aerodynamic design optimization based on data dimensionality reduction

In aerodynamic optimization, global optimization methods such as genetic algorithms are preferred in many cases because of their advantage on reaching global optimum. However,for complex problems in which large number of design variables are needed, the computational cost becomes prohibitive, and thus original global optimization strategies are required. To address this need, data dimensionality reduction method is combined with global optimization methods, thus forming a new global optimization system, aiming to improve the efficiency of conventional global optimization. The new optimization system involves applying Proper Orthogonal Decomposition（POD） in dimensionality reduction of design space while maintaining the generality of original design space. Besides, an acceleration approach for samples calculation in surrogate modeling is applied to reduce the computational time while providing sufficient accuracy. The optimizations of a transonic airfoil RAE2822 and the transonic wing ONERA M6 are performed to demonstrate the effectiveness of the proposed new optimization system. In both cases, we manage to reduce the number of design variables from 20 to 10 and from 42 to 20 respectively. The new design optimization system converges faster and it takes 1/3 of the total time of traditional optimization to converge to a better design, thus significantly reducing the overall optimization time and improving the efficiency of conventional global design optimization method.

基于空间位置关系的轨迹数据高效降维和查询算法

由于新型信息技术的快速发展,社会处于数字化、信息化转型的关键时期,各行业对于以数据库技术为基础的信息系统的需求也日益凸显.基于位置的服务依赖于海量实时生成的轨迹数据,在处理亿万级随时间连续变化的轨迹数据时,降维算法和查询技术一直是研究的关键,通过降低轨迹数据的规模,减少查询操作时处理数据的时间,能有效提升查询的性能,而能否实现高质量、高效率查询对于数据库而言至关重要.提出了面向轨迹数据的均匀网格编码,并在进一步优化后提出非均匀网格降维算法,将轨迹数据的坐标转化为1维字符串存储,对不符合要求的网格进行合并处理;通过空间位置映射充分保留轨迹数据间复杂的相互关系,并采用范围查询与最近邻查询对降维后的数据进行性能测试.实验使用不同城市真实轨迹数据与模拟生成轨迹数据作为数据集,将提出的均匀网格算法、非均匀网格算法与3种基准方法进行对比.实验证明,优化后的非均匀网格算法降维后数据的空间位置关系相似度可高达82.50%,范围查询时间较其他查询时间提升了至少73.86%,最近邻查询时间提升了至少52.26%,与其他基准方法相比取得了更好的效果.

基于PCA的大数据降维应用

随着互联网和信息技术的飞速发展,数据源的广泛性和复杂性给获取信息准确性带来了巨大的挑战。采用合适的降维算法可以把这些海量数据从高维降低到可以接受的范围、且不失去原数据表达的含义,而计算量大大降低,更容易理解。PCA(principal component analysis)即主成份分析作为数据降维的重要算法之一,利用正交变换,把一组相关的变量转化为一组线性不相关的变量,通常这种变换会减少变量个数,计算各成份在表达数据的贡献度,选取排列最前的贡献最高的几个特征即可表达整个数据集。实验表明,主成份从多维降至二维即可表达整个数据集,在精度可控的范围内使计算量大大降低。

基于堆叠降噪自编码器的肝癌亚型分类

肝癌是威胁人类健康的常见恶性肿瘤之一。通过对基因数据使用深度学习方法进行整合来系统地获取对肝癌的认知,使用多组学的疾病分析方法来探究各组学之间的相互关系,有助于更准确的临床决策。然而,由于多组学数据具有高维稀疏性,存在大量的冗余特征和较少的可用临床标签样本。堆叠降噪编码器(SDAE)是能够从海量数据中获取有效特征的高效模型,因此基于SDAE模型提出一种层次式堆叠降噪编码器,来学习肝癌的RNA表达、miRNA表达和DNA甲基化数据的特征并进行整合和识别。实验结果表明:Hi-SDAE方法提高了对肝癌亚型分类的准确度,为肝癌针对性治疗提供了更有价值的参考依据。

基于NMI-SC的糖尿病混合数据特征选择

针对糖尿病预测精度受高维混合数据影响的问题,提出基于NMI-SC的糖尿病特征选择方法,通过邻域互信息(NMI)计算混合属性特征邻域半径内的联合概率密度,构建相似度矩阵,通过糖尿病特征之间的相似性构建无向图,基于谱聚类(SC)将糖尿病特征切分为多个特征相似组,实现非线性特征间的聚类,根据特征分类重要性选出相似组中的代表特征。并将其与原始特征集在支持向量机分类器上的准确率进行比较,该特征选择方法在删除46个冗余特征后,准确率提高了13.07%。实验结果表明,该方法能有效删除冗余特征,得到糖尿病分类性能优异的特征子集。

基于特征工程和深度自动编码器的桥梁损伤识别研究

针对桥梁监测领域中损伤识别精度较低的问题,提出了一种基于特征工程和深度自动编码器的识别方案。首先采用快速傅里叶变换分析原始数据的特征和规律,其次通过滑动窗口从频谱图中提取表现出损伤差异的模态频率,最后将经过主成分分析法选择的保留损伤信息量最大的敏感特征作为深度自动编码器的输入。实验结果表明,经过特征工程处理后的新指标提高了模型的识别能力和计算效率,在仅占原始数据集14.9%的特征维度的情况下,模型的识别精确率从81.12%提升到98.67%。

基于时间序列的局部离群数据挖掘优化算法

针对数据量较大和数据维度较高导致离群数据挖掘困难的问题,提出基于时间序列的局部离群数据挖掘优化算法。将角度优化的全局嵌入算法和共同核主成分分析法相结合构建AOCKPCA降维算法,对海量高维时间序列降维处理;在蚁群算法中引入K-means算法,提升蚁群算法运算效率,降低不稳定性;将降维后的时间序列输入到优化后算法中,实现局部离群数据挖掘。实验结果表明,采用所提方法挖掘离群数据的准确率较高,误判的离群点个数较少,说明其挖掘效果较好。

基于张量Tucker分解的高光谱图像异常目标识别

高光谱图像每个像素点的光谱信息包含数百甚至数千个波段,使得高光谱图像在维度上具有高度的复杂性,且由于光谱波段众多,其中存在大量的冗余信息,加大了异常目标识别计算的负担。为此,文中提出基于张量Tucker分解的高光谱图像异常目标识别方法。通过张量Tucker分解压缩高光谱图像后,采用依据高光谱图像数据样本学习的构造方法,构建压缩后高光谱图像的字典,获取高光谱图像数据的稀疏表示形式后,通过RX异常检测方法检测出高光谱图像中的异常目标。实验结果表明:所提方法张量分解重构高光谱图像后,可以缩短压缩时间,减少算法复杂度;重构后的高光谱图像清晰度高,且高光谱图像异常目标检测虚警率低。

区块链环境下多源大数据访问加密防护仿真

为了提高多源大数据在网络中传输和存储的安全性,提出区块链技术下多源大数据加密防护算法。对多源大数据展开降维处理,减少数据计算量,提高算法的整体效率;结合Henon映射与Logistic映射设计一种双混沌加密算法,实现多源大数据初步加密;利用区块链技术设计多源大数据加密防护方案,在智能合约的基础上,大数据所有者可以将双混沌加密算法加密后的密钥上传至区块链中,同时对数据的有效访问时间展开设定,在有效访问时间内只有满足访问策略的用户才可以访问多源大数据,以此实现多源大数据的加密防护。实验结果表明,所提算法加密后的数据不容易被攻击,且具有较高的运行效率和良好的加密效果。

基于PCA方法的移动医疗高维数据降维处理

移动医疗领域产生大数据集,数据量庞大、数据维度复杂,数据挖掘过程存在诸多冗余数据等问题,本文在此基础上使用主成分分析PCA方法进行降维处理,筛选出有效数据维度,避免数据过度拟合,将高维数据降维成低维数据,对其隐含的结构具有很大帮助,经常用于数据压缩、数据探索以及数据可视化。对移动医疗大数据进行降维处理,不仅能减少数据计算量,也能避免数据过度拟合,有利于医疗数据价值分析,为移动医疗数据集领域的数据挖掘提供新的思路和方法。

基于支持向量机的网格化电网负荷预测算法设计

针对电网负荷预测算法预测能力较差、效率偏低的问题,文中提出了一种PCA-PSO-SVM算法。其在经典粒子群算法的基础上引入主元分析法,使模型具有降低数据维度及算法冗余度的特性。同时通过改进的PCA-PSO算法对SVM模型的内置参数进行最优选取,从而使改进后的SVM模型具有最佳的分类性能。在实验测试中,采用PCA算法选取了91%贡献度内的6项数据特征进行样本数据训练。结果表明,相较于其他对比算法,该文算法预测结果的RMSE、MAE与MAPE误差值均为最小,证明其可对网格化电网负荷加以预测。此外,该算法还能提升传统算法的准确度,为电力负荷分配提供有力支持。

最大相关和最大差异的高维数据特征选择算法

针对高维数据存在冗余信息且维度过高的问题,提出基于信息量的最大相关最大差异特征选择算法(MCD)。首先,利用互信息(MI)度量特征和标签之间的相关性,对特征进行排序,选择互信息最大的特征加入特征子集;然后,引入信息距离度量特征之间的信息冗余性及差异性,设计评价准则对每个特征进行评价,使特征子集中特征和标签的相关性、特征之间的差异性最大;最后,用前向搜索策略结合评价准则进行属性约简,最优化特征子集。采用2种不同的分类器,在6个数据集上和mRMR(minimal-Redundancy-Maximal-Relevance criterion)、RReliefF等5个经典算法进行对比实验,利用分类精度验证MCD的有效性。在支持向量机(SVM)分类器下,平均分类精度提高了5.67~23.80个百分点;在K-近邻(KNN)分类器下,平均分类精度提高了2.69~25.18个百分点。可见,MCD在绝大多数情况下,能有效去除冗余特征,分类精度有明显提高。

高层建筑深基坑支护结构位移动态监测方法

针对高层建筑深基坑支护结构位移监测时,位移轨迹、位移速率以及位移时间变化监测准确性较差问题,本文研究了高层建筑深基坑支护结构位移动态监测方法。对支护结构位移影响因素进行具体分析,生成了影响指标;再对基坑数据进行采集,建立数据集并进行降维处理;计算获取目标函数,结合影响指标建立时间序列模型,依据对模型的计算建立动态变量矩阵;通过对矩阵的计算获取动态监测数据的统计量,完成支护结构的动态监测。研究结果表明:运用该方法进行监测时,位移移动轨迹监测误差为0.1,位移速率保持在0.9 mm/d以下,且与实际位移速率基本一致,纵向位移量达到202 mm,且与实际沉降量一致。本文方法能够有效应用于高层建筑深基坑支护结构的位移动态监测,为保障高层建筑的稳定性和安全性提供重要的技术支持。

面向智能电表大数据分析的数据降维方法

针对智能电表大数据聚类分析过程中出现的计算复杂度过大的问题,提出基于极坐标投影的数据降维方法。采用极坐标投影的方法将由电能总消耗量和峰值电能消耗量所组成的数据集映射成由极角和极径组成的投影矩阵,设计一个由电能总消耗量和峰值电能消耗量所组成的改进聚类距离度量,使用误差平方和方法确定最优的聚类数量,将该方法引入于k均值算法进行算例分析。结果表明,该降维方法能够有效提升聚类算法的性能,并降低聚类算法的计算复杂度。