An Enhanced Integrated Method for Healthcare Data Classification with Incompleteness

In numerous real-world healthcare applications,handling incomplete medical data poses significant challenges for missing value imputation and subsequent clustering or classification tasks.Traditional approaches often rely on statistical methods for imputation,which may yield suboptimal results and be computationally intensive.This paper aims to integrate imputation and clustering techniques to enhance the classification of incomplete medical data with improved accuracy.Conventional classification methods are ill-suited for incomplete medical data.To enhance efficiency without compromising accuracy,this paper introduces a novel approach that combines imputation and clustering for the classification of incomplete data.Initially,the linear interpolation imputation method alongside an iterative Fuzzy c-means clustering method is applied and followed by a classification algorithm.The effectiveness of the proposed approach is evaluated using multiple performance metrics,including accuracy,precision,specificity,and sensitivity.The encouraging results demonstrate that our proposed method surpasses classical approaches across various performance criteria.

Physics-Informed AI Surrogates for Day-Ahead Wind Power Probabilistic Forecasting with Incomplete Data for Smart Grid in Smart Cities

Due to the high inherent uncertainty of renewable energy,probabilistic day-ahead wind power forecasting is crucial for modeling and controlling the uncertainty of renewable energy smart grids in smart cities.However,the accuracy and reliability of high-resolution day-ahead wind power forecasting are constrained by unreliable local weather prediction and incomplete power generation data.This article proposes a physics-informed artificial intelligence(AI)surrogates method to augment the incomplete dataset and quantify its uncertainty to improve wind power forecasting performance.The incomplete dataset,built with numerical weather prediction data,historical wind power generation,and weather factors data,is augmented based on generative adversarial networks.After augmentation,the enriched data is then fed into a multiple AI surrogates model constructed by two extreme learning machine networks to train the forecasting model for wind power.Therefore,the forecasting models’accuracy and generalization ability are improved by mining the implicit physics information from the incomplete dataset.An incomplete dataset gathered from a wind farm in North China,containing only 15 days of weather and wind power generation data withmissing points caused by occasional shutdowns,is utilized to verify the proposed method’s performance.Compared with other probabilistic forecastingmethods,the proposed method shows better accuracy and probabilistic performance on the same incomplete dataset,which highlights its potential for more flexible and sensitive maintenance of smart grids in smart cities.

Power Incomplete Data Clustering Based on Fuzzy Fusion Algorithm

With the rapid development of the economy,the scale of the power grid is expanding.The number of power equipment that constitutes the power grid has been very large,which makes the state data of power equipment grow explosively.These multi-source heterogeneous data have data differences,which lead to data variation in the process of transmission and preservation,thus forming the bad information of incomplete data.Therefore,the research on data integrity has become an urgent task.This paper is based on the characteristics of random chance and the Spatio-temporal difference of the system.According to the characteristics and data sources of the massive data generated by power equipment,the fuzzy mining model of power equipment data is established,and the data is divided into numerical and non-numerical data based on numerical data.Take the text data of power equipment defects as the mining material.Then,the Apriori algorithm based on an array is used to mine deeply.The strong association rules in incomplete data of power equipment are obtained and analyzed.From the change trend of NRMSE metrics and classification accuracy,most of the filling methods combined with the two frameworks in this method usually show a relatively stable filling trend,and will not fluctuate greatly with the growth of the missing rate.The experimental results show that the proposed algorithm model can effectively improve the filling effect of the existing filling methods on most data sets,and the filling effect fluctuates greatly with the increase of the missing rate,that is,with the increase of the missing rate,the improvement effect of the model for the existing filling methods is higher than 4.3%.Through the incomplete data clustering technology studied in this paper,a more innovative state assessment of smart grid reliability operation is carried out,which has good research value and reference significance.

Damage Identification under Incomplete Mode Shape Data Using Optimization Technique Based on Generalized Flexibility Matrix

A generalized flexibility–based objective function utilized for structure damage identification is constructed for solving the constrained nonlinear least squares optimized problem. To begin with, the generalized flexibility matrix (GFM) proposed to solve the damage identification problem is recalled and a modal expansion method is introduced. Next, the objective function for iterative optimization process based on the GFM is formulated, and the Trust-Region algorithm is utilized to obtain the solution of the optimization problem for multiple damage cases. And then for computing the objective function gradient, the sensitivity analysis regarding design variables is derived. In addition, due to the spatial incompleteness, the influence of stiffness reduction and incomplete modal measurement data is discussed by means of two numerical examples with several damage cases. Finally, based on the computational results, it is evident that the presented approach provides good validity and reliability for the large and complicated engineering structures.

Fault detection and diagnosis for data incomplete industrial systems with new Bayesian network approach

For the fault detection and diagnosis problem in largescale industrial systems, there are two important issues： the missing data samples and the non-Gaussian property of the data. However, most of the existing data-driven methods cannot be able to handle both of them. Thus, a new Bayesian network classifier based fault detection and diagnosis method is proposed. At first, a non-imputation method is presented to handle the data incomplete samples, with the property of the proposed Bayesian network classifier, and the missing values can be marginalized in an elegant manner. Furthermore, the Gaussian mixture model is used to approximate the non-Gaussian data with a linear combination of finite Gaussian mixtures, so that the Bayesian network can process the non-Gaussian data in an effective way. Therefore, the entire fault detection and diagnosis method can deal with the high-dimensional incomplete process samples in an efficient and robust way. The diagnosis results are expressed in the manner of probability with the reliability scores. The proposed approach is evaluated with a benchmark problem called the Tennessee Eastman process. The simulation results show the effectiveness and robustness of the proposed method in fault detection and diagnosis for large-scale systems with missing measurements.

Analysis of Incomplete Data of Accelerated Life Testing with Competing Failure Modes

Data obtained from accelerated life testing （ALT） when there are two or more failure modes, which is commonly referred to as competing failure modes, are often incomplete. The incompleteness is mainly due to censoring, as well as masking which might be the case that the failure time is observed, but its corresponding failure mode is not identified. Because the identification of the failure mode may be expensive, or very difficult to investigate due to lack of appropriate diagnostics. A method is proposed for analyzing incomplete data of constant stress ALT with competing failure modes. It is assumed that failure modes have s-independent latent lifetimes and the log lifetime of each failure mode can be written as a linear function of stress. The parameters of the model are estimated by using the expectation maximum （EM） algorithm with incomplete data. Simulation studies are performed to check＇model validity and investigate the properties of estimates. For further validation, the method is also illustrated by an example, which shows the process of analyze incomplete data from ALT of some insulation system. Because of considering the incompleteness of data in modeling and making use of the EM algorithm in estimating, the method becomes more flexible in ALT analysis.

Energy Consumption Prediction of a CNC Machining Process With Incomplete Data

Energy consumption prediction of a CNC machining process is important for energy efficiency optimization strategies.To improve the generalization abilities,more and more parameters are acquired for energy prediction modeling.While the data collected from workshops may be incomplete because of misoperation,unstable network connections,and frequent transfers,etc.This work proposes a framework for energy modeling based on incomplete data to address this issue.First,some necessary preliminary operations are used for incomplete data sets.Then,missing values are estimated to generate a new complete data set based on generative adversarial imputation nets(GAIN).Next,the gene expression programming(GEP)algorithm is utilized to train the energy model based on the generated data sets.Finally,we test the predictive accuracy of the obtained model.Computational experiments are designed to investigate the performance of the proposed framework with different rates of missing data.Experimental results demonstrate that even when the missing data rate increases to 30%,the proposed framework can still make efficient predictions,with the corresponding RMSE and MAE 0.903 k J and 0.739 k J,respectively.

Deep learning technique for process fault detection and diagnosis in the presence of incomplete data

In modern industrial processes, timely detection and diagnosis of process abnormalities are critical for monitoring process operations. Various fault detection and diagnosis(FDD) methods have been proposed and implemented, the performance of which, however, could be drastically influenced by the common presence of incomplete or missing data in real industrial scenarios. This paper presents a new FDD approach based on an incomplete data imputation technique for process fault recognition. It employs the modified stacked autoencoder,a deep learning structure, in the phase of incomplete data treatment, and classifies data representations rather than the imputed complete data in the phase of fault identification. A benchmark process, the Tennessee Eastman process, is employed to illustrate the effectiveness and applicability of the proposed method.

Bayesian estimation of a power law process with incomplete data

Due to the simplicity and flexibility of the power law process,it is widely used to model the failures of repairable systems.Although statistical inference on the parameters of the power law process has been well developed,numerous studies largely depend on complete failure data.A few methods on incomplete data are reported to process such data,but they are limited to their specific cases,especially to that where missing data occur at the early stage of the failures.No framework to handle generic scenarios is available.To overcome this problem,from the point of view of order statistics,the statistical inference of the power law process with incomplete data is established in this paper.The theoretical derivation is carried out and the case studies demonstrate and verify the proposed method.Order statistics offer an alternative to the statistical inference of the power law process with incomplete data as they can reformulate current studies on the left censored failure data and interval censored data in a unified framework.The results show that the proposed method has more flexibility and more applicability.

A Fast and Effective Multiple Kernel Clustering Method on Incomplete Data

Multiple kernel clustering is an unsupervised data analysis method that has been used in various scenarios where data is easy to be collected but hard to be labeled.However,multiple kernel clustering for incomplete data is a critical yet challenging task.Although the existing absent multiple kernel clustering methods have achieved remarkable performance on this task,they may fail when data has a high value-missing rate,and they may easily fall into a local optimum.To address these problems,in this paper,we propose an absent multiple kernel clustering(AMKC)method on incomplete data.The AMKC method rst clusters the initialized incomplete data.Then,it constructs a new multiple-kernel-based data space,referred to as K-space,from multiple sources to learn kernel combination coefcients.Finally,it seamlessly integrates an incomplete-kernel-imputation objective,a multiple-kernel-learning objective,and a kernel-clustering objective in order to achieve absent multiple kernel clustering.The three stages in this process are carried out simultaneously until the convergence condition is met.Experiments on six datasets with various characteristics demonstrate that the kernel imputation and clustering performance of the proposed method is signicantly better than state-of-the-art competitors.Meanwhile,the proposed method gains fast convergence speed.

Average Life Prediction Based on Incomplete Data

The two-parameter exponential distribution can often be used to describe the lifetime of products for example, electronic components, engines and so on. This paper considers a prediction problem arising in the life test of key parts in high speed trains. Employing the Bayes method, a joint prior is used to describe the variability of the parameters but the form of the prior is not specified and only several moment conditions are assumed. Under the condition that the observed samples are randomly right censored, we define a statistic to predict a set of future samples which describes the average life of the second-round samples, firstly, under the condition that the censoring distribution is known and secondly, that it is unknown. For several different priors and life data sets, we demonstrate the coverage frequencies of the proposed prediction intervals as the sample size of the observed and the censoring proportion change. The numerical results show that the prediction intervals are efficient and applicable.

Belief Combination of Classifiers for Incomplete Data

Data with missing values,or incomplete information,brings some challenges to the development of classification,as the incompleteness may significantly affect the performance of classifiers.In this paper,we handle missing values in both training and test sets with uncertainty and imprecision reasoning by proposing a new belief combination of classifier(BCC)method based on the evidence theory.The proposed BCC method aims to improve the classification performance of incomplete data by characterizing the uncertainty and imprecision brought by incompleteness.In BCC,different attributes are regarded as independent sources,and the collection of each attribute is considered as a subset.Then,multiple classifiers are trained with each subset independently and allow each observed attribute to provide a sub-classification result for the query pattern.Finally,these sub-classification results with different weights(discounting factors)are used to provide supplementary information to jointly determine the final classes of query patterns.The weights consist of two aspects:global and local.The global weight calculated by an optimization function is employed to represent the reliability of each classifier,and the local weight obtained by mining attribute distribution characteristics is used to quantify the importance of observed attributes to the pattern classification.Abundant comparative experiments including seven methods on twelve datasets are executed,demonstrating the out-performance of BCC over all baseline methods in terms of accuracy,precision,recall,F1 measure,with pertinent computational costs.

NEW METHOD OF MINING INCOMPLETE DATA

The data used in the process of knowledge discovery often includes noise and incomplete information. The boundaries of different classes of these data are blur and unobvious. When these data are clustered or classified, we often get the coverings instead of the partitions, and it usually makes our information system insecure. In this paper, optimal partitioning of incomplete data is researched. Firstly, the relationship of set cover and set partition is discussed, and the distance between set cover and set partition is defined. Secondly, the optimal partitioning of given cover is researched by the combing and parting method, acquiring the optimal partition from three different partitions set family is discussed. Finally, the corresponding optimal algorithm is given. The real wireless signals offten contain a lot of noise, and there are many errors in boundaries when these data is clustered based on the tradional method. In our experimant, the proposed method improves correct rate greatly, and the experimental results demonstrate the method's validity.

On the Convergence of Observed Partial Likelihood under Incomplete Data with Two Class Possibilities

In this paper, we discuss the theoretical validity of the observed partial likelihood (OPL) constructed in a Coxtype model under incomplete data with two class possibilities, such as missing binary covariates, a cure-mixture model or doubly censored data. A main result is establishing the asymptotic convergence of the OPL. To reach this result, as it is difficult to apply some standard tools in the survival analysis, we develop tools for weak convergence based on partial-sum processes. The result of the asymptotic convergence shown here indicates that a suitable order of the number of Monte Carlo trials is less than the square of the sample size. In addition, using numerical examples, we investigate how the asymptotic properties discussed here behave in a finite sample.

Mining incomplete data-A rough set approach

Many real-life data sets are incomplete,or in different words,are affected by missing attribute values.Three interpretations of missing attribute values are discussed in the paper:lost values(erased values),attribute-concept values(such a value may be replaced by any value from the attribute domain restricted to the concept),and "do not care" conditions(a missing attribute value may be replaced by any value from the attribute domain).For incomplete data sets three definitions of lower and upper approximations are discussed.Experiments were conducted on six typical data sets with missing attribute values,using three different interpretations of missing attribute values and the same definition of concept lower and upper approximations.The conclusion is that the best approach to missing attribute values is the lost value type.

空间自回归模型下不完整大数据缺失值插补算法

针对不完整大数据因其自身结构具有不规则性,导致在进行缺失值插补时计算量大、插补精度低的问题,提出空间自回归模型下不完整大数据缺失值插补算法。利用迁移学习算法在动态权重下过滤出原始数据中冗余数据,区分异常和正常数据,提取残缺数据,采用最小二乘回归对残缺数据实施修补。将缺失值插补分为3种类型,分别为一阶空间自回归模型插补、空间自回归模型插补和多重插补法。根据实际情况将修补后数据插补到合适的位置,实现不完整大数据缺失值插补。实验结果表明,所提方法具有良好的缺失值插补能力。

Weibull和正态分布不完全数据可靠性评估方法

提出一种两参数Weibull分布和正态分布定数截尾数据可靠性评估方法,建立了其高置信度下的可靠寿命和可靠度单侧置信限计算公式。同时,给出一种能够充分开发利用以往试验数据,并与当前试验数据有机融合进行可靠性评估的方法,由于增大了信息量,从而可以显著提高当前产品可靠性评估精度。在此基础上,还进一步将上述方法推广用于两参数Weibull分布和正态分布的定时截尾数据、无失效数据以及一般不完全数据的可靠性评估,从而实现了不完全数据情况机电产品高精度小样本可靠性评估。与传统的需查表计算的BLUE和BLIE等方法相比,本文方法不但理论上更加严谨,而且评估精度更高,工程计算也更加便捷。

基于非完整点云法线滤波补偿的散货船舶舱口识别算法

自动装船系统是智能化港口建设的重要组成部分,能够大幅降低港口作业成本,提高经济效益。舱口识别作为自动装船任务的首要环节,成功率和识别精度是后续任务顺利进行的重要保障。由于港口激光雷达的数目和角度等问题,采集所得船舶点云数据时常出现缺失;此外船舶舱口附近经常有大量物料堆积,会使采集到的点云数据无法准确表达舱口的几何信息。由于上述港口实际装船作业中时常出现的问题,显著降低了现有算法的识别成功率,对自动装船作业造成了不良影响,因此迫切需要提升在船舶点云中存在物料干扰或舱口数据缺失的情况下的舱口识别成功率。基于船舶结构特征与自动装船过程中采集的点云数据分析,提出了基于非完整点云法线滤波补偿的散货船舶舱口识别算法。在使用港口实际采集点云所制作的数据集上进行了实验验证,识别成功率和识别精度较Miao和Li的舱口识别算法相比均有提升。实验结果表明,所提算法既能对舱口内物料噪声进行滤除,又能对数据缺失部分进行补偿,能够有效提升舱口识别效果。

基于一维卷积和图神经网络的配电网故障区段定位方法

快速、准确地定位故障区段对配电网的安全运行至关重要。传统故障定位方法容错率低、耗费时间长,多数深度学习算法对拓扑变动的泛化性不足。基于此,提出了一种基于一维卷积神经网络(one-dimensional convolutional neural network,1D-CNN)和图神经网络(graph neural network,GNN)的配电网故障区段定位方法。该方法将配电网原始信息与GNN等深度学习算法相结合进行建模。首先利用基于注意力的时空图卷积网络从不同的时空尺度上对遥测数据进行故障特征提取,使用图注意力网络来融合多源遥信数据。然后,利用1D-CNN来调整特征输出维度以实现节点特征到故障支路的映射。最后,通过增设全连接网络来输出故障区段定位结果。依托于Matlab/Simulink平台搭建10 kV中性点不接地配电网系统进行仿真和测试。结果表明,所提方法具有优越的定位性能,能够灵活适用于各类低、中、高阻性接地故障场景,对系统拓扑变动具有强大的泛化能力以及对故障数据不完备的鲁棒性好。

不完整多视图聚类综述

多视图聚类是近年来图数据挖掘领域的研究热点。由于数据采集技术的限制或人为因素等原因常导致视图或样本缺失问题。降低多视图的不完整性对聚类效果的影响是多视图聚类目前面临的重大挑战。因此,综合研究不完整多视图聚类(IMC)近年的发展具有重要的理论意义和实践价值。首先,归纳分析不完整多视图数据缺失类型;其次,详细比较基于多核学习(MKL)、矩阵分解(MF)学习、深度学习和图学习这4类IMC方法,分析代表性方法的技术特点和区别;再次,从数据集类型、视图和类别数量、应用领域等角度总结22个公开不完整多视图数据集;继次,总结评价指标,并系统分析现有不完整多视图聚类方法在同构和异构数据集上的性能表现;最后,归纳分析不完整多视图聚类目前存在的问题、未来的发展方向和现有应用领域。