Semi-Supervised Clustering Algorithm Based on Deep Feature Mapping

Clustering analysis is one of the main concerns in data mining.A common approach to the clustering process is to bring together points that are close to each other and separate points that are away from each other.Therefore,measuring the distance between sample points is crucial to the effectiveness of clustering.Filtering features by label information and mea-suring the distance between samples by these features is a common supervised learning method to reconstruct distance metric.However,in many application scenarios,it is very expensive to obtain a large number of labeled samples.In this paper,to solve the clustering problem in the few supervised sample and high data dimensionality scenarios,a novel semi-supervised clustering algorithm is proposed by designing an improved prototype network that attempts to reconstruct the distance metric in the sample space with a small amount of pairwise supervised information,such as Must-Link and Cannot-Link,and then cluster the data in the new metric space.The core idea is to make the similar ones closer and the dissimilar ones further away through embedding mapping.Extensive experiments on both real-world and synthetic datasets show the effectiveness of this algorithm.Average clustering metrics on various datasets improved by 8%compared to the comparison algorithm.

Improved Semi-supervised Clustering Algorithm Based on Affinity Propagation

A clustering algorithm for semi-supervised affinity propagation based on layered combination is proposed in this paper in light of existing flaws. To improve accuracy of the algorithm,it introduces the idea of layered combination, divides an affinity propagation clustering( APC) process into several hierarchies evenly,draws samples from data of each hierarchy according to weight,and executes semi-supervised learning through construction of pairwise constraints and use of submanifold label mapping,weighting and combining clustering results of all hierarchies by combined promotion. It is shown by theoretical analysis and experimental result that clustering accuracy and computation complexity of the semi-supervised affinity propagation clustering algorithm based on layered combination( SAP-LC algorithm) have been greatly improved.

Hybridization of Fuzzy and Hard Semi-Supervised Clustering Algorithms Tuned with Ant Lion Optimizer Applied to Higgs Boson Search

This paper focuses on the unsupervised detection of the Higgs boson particle using the most informative features and variables which characterize the“Higgs machine learning challenge 2014”data set.This unsupervised detection goes in this paper analysis through 4 steps:(1)selection of the most informative features from the considered data;(2)definition of the number of clusters based on the elbow criterion.The experimental results showed that the optimal number of clusters that group the considered data in an unsupervised manner corresponds to 2 clusters;(3)proposition of a new approach for hybridization of both hard and fuzzy clustering tuned with Ant Lion Optimization(ALO);(4)comparison with some existing metaheuristic optimizations such as Genetic Algorithm(GA)and Particle Swarm Optimization(PSO).By employing a multi-angle analysis based on the cluster validation indices,the confusion matrix,the efficiencies and purities rates,the average cost variation,the computational time and the Sammon mapping visualization,the results highlight the effectiveness of the improved Gustafson-Kessel algorithm optimized withALO(ALOGK)to validate the proposed approach.Even if the paper gives a complete clustering analysis,its novel contribution concerns only the Steps(1)and(3)considered above.The first contribution lies in the method used for Step(1)to select the most informative features and variables.We used the t-Statistic technique to rank them.Afterwards,a feature mapping is applied using Self-Organizing Map(SOM)to identify the level of correlation between them.Then,Particle Swarm Optimization(PSO),a metaheuristic optimization technique,is used to reduce the data set dimension.The second contribution of thiswork concern the third step,where each one of the clustering algorithms as K-means(KM),Global K-means(GlobalKM),Partitioning AroundMedoids(PAM),Fuzzy C-means(FCM),Gustafson-Kessel(GK)and Gath-Geva(GG)is optimized and tuned with ALO.

Semi-Supervised Clustering Fingerprint Positioning Algorithm Based on Distance Constraints

With the rapid development of WLAN( Wireless Local Area Network) technology,an important target of indoor positioning systems is to improve the positioning accuracy while reducing the online computation.In this paper,it proposes a novel fingerprint positioning algorithm known as semi-supervised affinity propagation clustering based on distance function constraints. We show that by employing affinity propagation techniques,it is able to use a fractional labeled data to adjust similarity matrix of signal space to cluster reference points with high accuracy. The semi-supervised APC uses a combination of machine learning,clustering analysis and fingerprinting algorithm. By collecting data and testing our algorithm in a realistic indoor WLAN environment,the experimental results indicate that the proposed algorithm can improve positioning accuracy while reduce the online localization computation,as compared with the widely used K nearest neighbor and maximum likelihood estimation algorithms.

Multiscale and Auto-Tuned Semi-Supervised Deep Subspace Clustering and Its Application in Brain Tumor Clustering

In this paper,we introduce a novel Multi-scale and Auto-tuned Semi-supervised Deep Subspace Clustering(MAS-DSC)algorithm,aimed at addressing the challenges of deep subspace clustering in high-dimensional real-world data,particularly in the field of medical imaging.Traditional deep subspace clustering algorithms,which are mostly unsupervised,are limited in their ability to effectively utilize the inherent prior knowledge in medical images.Our MAS-DSC algorithm incorporates a semi-supervised learning framework that uses a small amount of labeled data to guide the clustering process,thereby enhancing the discriminative power of the feature representations.Additionally,the multi-scale feature extraction mechanism is designed to adapt to the complexity of medical imaging data,resulting in more accurate clustering performance.To address the difficulty of hyperparameter selection in deep subspace clustering,this paper employs a Bayesian optimization algorithm for adaptive tuning of hyperparameters related to subspace clustering,prior knowledge constraints,and model loss weights.Extensive experiments on standard clustering datasets,including ORL,Coil20,and Coil100,validate the effectiveness of the MAS-DSC algorithm.The results show that with its multi-scale network structure and Bayesian hyperparameter optimization,MAS-DSC achieves excellent clustering results on these datasets.Furthermore,tests on a brain tumor dataset demonstrate the robustness of the algorithm and its ability to leverage prior knowledge for efficient feature extraction and enhanced clustering performance within a semi-supervised learning framework.

ASCFL:Accurate and Speedy Semi-Supervised Clustering Federated Learning

The influence of non-Independent Identically Distribution(non-IID)data on Federated Learning(FL)has been a serious concern.Clustered Federated Learning(CFL)is an emerging approach for reducing the impact of non-IID data,which employs the client similarity calculated by relevant metrics for clustering.Unfortunately,the existing CFL methods only pursue a single accuracy improvement,but ignore the convergence rate.Additionlly,the designed client selection strategy will affect the clustering results.Finally,traditional semi-supervised learning changes the distribution of data on clients,resulting in higher local costs and undesirable performance.In this paper,we propose a novel CFL method named ASCFL,which selects clients to participate in training and can dynamically adjust the balance between accuracy and convergence speed with datasets consisting of labeled and unlabeled data.To deal with unlabeled data,the prediction labels strategy predicts labels by encoders.The client selection strategy is to improve accuracy and reduce overhead by selecting clients with higher losses participating in the current round.What is more,the similarity-based clustering strategy uses a new indicator to measure the similarity between clients.Experimental results show that ASCFL has certain advantages in model accuracy and convergence speed over the three state-of-the-art methods with two popular datasets.

A Graph-Based Semi-Supervised Approach for Few-Shot Class-Incremental Modulation Classification

With the successive application of deep learning(DL)in classification tasks,the DL-based modulation classification method has become the preference for its state-of-the-art performance.Nevertheless,once the DL recognition model is pre-trained with fixed classes,the pre-trained model tends to predict incorrect results when identifying incremental classes.Moreover,the incremental classes are usually emergent without label information or only a few labeled samples of incremental classes can be obtained.In this context,we propose a graphbased semi-supervised approach to address the fewshot classes-incremental(FSCI)modulation classification problem.Our proposed method is a twostage learning method,specifically,a warm-up model is trained for classifying old classes and incremental classes,where the unlabeled samples of incremental classes are uniformly labeled with the same label to alleviate the damage of the class imbalance problem.Then the warm-up model is regarded as a feature extractor for constructing a similar graph to connect labeled samples and unlabeled samples,and the label propagation algorithm is adopted to propagate the label information from labeled nodes to unlabeled nodes in the graph to achieve the purpose of incremental classes recognition.Simulation results prove that the proposed method is superior to other finetuning methods and retrain methods.

Semi-supervised learning based hybrid beamforming under time-varying propagation environments

Hybrid precoding is considered as a promising low-cost technique for millimeter wave(mm-wave)massive Multi-Input Multi-Output(MIMO)systems.In this work,referring to the time-varying propagation circumstances,with semi-supervised Incremental Learning(IL),we propose an online hybrid beamforming scheme.Firstly,given the constraint of constant modulus on analog beamformer and combiner,we propose a new broadnetwork-based structure for the design model of hybrid beamforming.Compared with the existing network structure,the proposed network structure can achieve better transmission performance and lower complexity.Moreover,to enhance the efficiency of IL further,by combining the semi-supervised graph with IL,we propose a hybrid beamforming scheme based on chunk-by-chunk semi-supervised learning,where only few transmissions are required to calculate the label and all other unlabelled transmissions would also be put into a training data chunk.Unlike the existing single-by-single approach where transmissions during the model update are not taken into the consideration of model update,all transmissions,even the ones during the model update,would make contributions to model update in the proposed method.During the model update,the amount of unlabelled transmissions is very large and they also carry some information,the prediction performance can be enhanced to some extent by these unlabelled channel data.Simulation results demonstrate the spectral efficiency of the proposed method outperforms that of the existing single-by-single approach.Besides,we prove the general complexity of the proposed method is lower than that of the existing approach and give the condition under which its absolute complexity outperforms that of the existing approach.

Semi-supervised surface defect detection of wind turbine blades with YOLOv4

Timely inspection of defects on the surfaces of wind turbine blades can effectively prevent unpredictable accidents.To this end,this study proposes a semi-supervised object-detection network based on You Only Looking Once version 4(YOLOv4).A semi-supervised structure comprising a generative adversarial network(GAN)was designed to overcome the difficulty in obtaining sufficient samples and sample labeling.In a GAN,the generator is realized by an encoder-decoder network,where the backbone of the encoder is YOLOv4 and the decoder comprises inverse convolutional layers.Partial features from the generator are passed to the defect detection network.Deploying several unlabeled images can significantly improve the generalization and recognition capabilities of defect-detection models.The small-scale object detection capacity of the network can be improved by enhancing essential features in the feature map by adding the concurrent spatial and channel squeeze and excitation(scSE)attention module to the three parts of the YOLOv4 network.A balancing improvement was made to the loss function of YOLOv4 to overcome the imbalance problem of the defective species.The results for both the single-and multi-category defect datasets show that the improved model can make good use of the features of the unlabeled images.The accuracy of wind turbine blade defect detection also has a significant advantage over classical object detection algorithms,including faster R-CNN and DETR.

A Novel Graph Structure Learning Based Semi-Supervised Framework for Anomaly Identification in Fluctuating IoT Environment

With the rapid development of Internet of Things(IoT)technology,IoT systems have been widely applied in health-care,transportation,home,and other fields.However,with the continuous expansion of the scale and increasing complexity of IoT systems,the stability and security issues of IoT systems have become increasingly prominent.Thus,it is crucial to detect anomalies in the collected IoT time series from various sensors.Recently,deep learning models have been leveraged for IoT anomaly detection.However,owing to the challenges associated with data labeling,most IoT anomaly detection methods resort to unsupervised learning techniques.Nevertheless,the absence of accurate abnormal information in unsupervised learning methods limits their performance.To address these problems,we propose AS-GCN-MTM,an adaptive structural Graph Convolutional Networks(GCN)-based framework using a mean-teacher mechanism(AS-GCN-MTM)for anomaly identification.It performs better than unsupervised methods using only a small amount of labeled data.Mean Teachers is an effective semi-supervised learning method that utilizes unlabeled data for training to improve the generalization ability and performance of the model.However,the dependencies between data are often unknown in time series data.To solve this problem,we designed a graph structure adaptive learning layer based on neural networks,which can automatically learn the graph structure from time series data.It not only better captures the relationships between nodes but also enhances the model’s performance by augmenting key data.Experiments have demonstrated that our method improves the baseline model with the highest F1 value by 10.4%,36.1%,and 5.6%,respectively,on three real datasets with a 10%data labeling rate.

Model Change Active Learning in Graph-Based Semi-supervised Learning

Active learning in semi-supervised classification involves introducing additional labels for unlabelled data to improve the accuracy of the underlying classifier.A challenge is to identify which points to label to best improve performance while limiting the number of new labels."Model Change"active learning quantifies the resulting change incurred in the classifier by introducing the additional label(s).We pair this idea with graph-based semi-supervised learning(SSL)methods,that use the spectrum of the graph Laplacian matrix,which can be truncated to avoid prohibitively large computational and storage costs.We consider a family of convex loss functions for which the acquisition function can be efficiently approximated using the Laplace approximation of the posterior distribution.We show a variety of multiclass examples that illustrate improved performance over prior state-of-art.

Picture-Neutrosophic Trusted Safe Semi-Supervised Fuzzy Clustering for Noisy Data

Clustering is a crucial method for deciphering data structure and producing new information.Due to its significance in revealing fundamental connections between the human brain and events,it is essential to utilize clustering for cognitive research.Dealing with noisy data caused by inaccurate synthesis from several sources or misleading data production processes is one of the most intriguing clustering difficulties.Noisy data can lead to incorrect object recognition and inference.This research aims to innovate a novel clustering approach,named Picture-Neutrosophic Trusted Safe Semi-Supervised Fuzzy Clustering(PNTS3FCM),to solve the clustering problem with noisy data using neutral and refusal degrees in the definition of Picture Fuzzy Set(PFS)and Neutrosophic Set(NS).Our contribution is to propose a new optimization model with four essential components:clustering,outlier removal,safe semi-supervised fuzzy clustering and partitioning with labeled and unlabeled data.The effectiveness and flexibility of the proposed technique are estimated and compared with the state-of-art methods,standard Picture fuzzy clustering(FC-PFS)and Confidence-weighted safe semi-supervised clustering(CS3FCM)on benchmark UCI datasets.The experimental results show that our method is better at least 10/15 datasets than the compared methods in terms of clustering quality and computational time.

Decentralized Semi-Supervised Learning for Stochastic Configuration Networks Based on the Mean Teacher Method

The aim of this paper is to broaden the application of Stochastic Configuration Network (SCN) in the semi-supervised domain by utilizing common unlabeled data in daily life. It can enhance the classification accuracy of decentralized SCN algorithms while effectively protecting user privacy. To this end, we propose a decentralized semi-supervised learning algorithm for SCN, called DMT-SCN, which introduces teacher and student models by combining the idea of consistency regularization to improve the response speed of model iterations. In order to reduce the possible negative impact of unsupervised data on the model, we purposely change the way of adding noise to the unlabeled data. Simulation results show that the algorithm can effectively utilize unlabeled data to improve the classification accuracy of SCN training and is robust under different ground simulation environments.

XA-GANomaly: An Explainable Adaptive Semi-Supervised Learning Method for Intrusion Detection Using GANomaly

Intrusion detection involves identifying unauthorized network activity and recognizing whether the data constitute an abnormal network transmission.Recent research has focused on using semi-supervised learning mechanisms to identify abnormal network traffic to deal with labeled and unlabeled data in the industry.However,real-time training and classifying network traffic pose challenges,as they can lead to the degradation of the overall dataset and difficulties preventing attacks.Additionally,existing semi-supervised learning research might need to analyze the experimental results comprehensively.This paper proposes XA-GANomaly,a novel technique for explainable adaptive semi-supervised learning using GANomaly,an image anomalous detection model that dynamically trains small subsets to these issues.First,this research introduces a deep neural network(DNN)-based GANomaly for semi-supervised learning.Second,this paper presents the proposed adaptive algorithm for the DNN-based GANomaly,which is validated with four subsets of the adaptive dataset.Finally,this study demonstrates a monitoring system that incorporates three explainable techniques—Shapley additive explanations,reconstruction error visualization,and t-distributed stochastic neighbor embedding—to respond effectively to attacks on traffic data at each feature engineering stage,semi-supervised learning,and adaptive learning.Compared to other single-class classification techniques,the proposed DNN-based GANomaly achieves higher scores for Network Security Laboratory-Knowledge Discovery in Databases and UNSW-NB15 datasets at 13%and 8%of F1 scores and 4.17%and 11.51%for accuracy,respectively.Furthermore,experiments of the proposed adaptive learning reveal mostly improved results over the initial values.An analysis and monitoring system based on the combination of the three explainable methodologies is also described.Thus,the proposed method has the potential advantages to be applied in practical industry,and future research will explore handling unbalanced real-time datasets in various scenarios.

Radio Frequency Fingerprinting Identification Using Semi-Supervised Learning with Meta Labels

Radio frequency fingerprinting(RFF)is a remarkable lightweight authentication scheme to support rapid and scalable identification in the internet of things(IoT)systems.Deep learning(DL)is a critical enabler of RFF identification by leveraging the hardware-level features.However,traditional supervised learning methods require huge labeled training samples.Therefore,how to establish a highperformance supervised learning model with few labels under practical application is still challenging.To address this issue,we in this paper propose a novel RFF semi-supervised learning(RFFSSL)model which can obtain a better performance with few meta labels.Specifically,the proposed RFFSSL model is constituted by a teacher-student network,in which the student network learns from the pseudo label predicted by the teacher.Then,the output of the student model will be exploited to improve the performance of teacher among the labeled data.Furthermore,a comprehensive evaluation on the accuracy is conducted.We derive about 50 GB real long-term evolution(LTE)mobile phone’s raw signal datasets,which is used to evaluate various models.Experimental results demonstrate that the proposed RFFSSL scheme can achieve up to 97%experimental testing accuracy over a noisy environment only with 10%labeled samples when training samples equal to 2700.

Detecting While Accessing:A Semi-Supervised Learning-Based Approach for Malicious Traffic Detection in Internet of Things

In the upcoming large-scale Internet of Things(Io T),it is increasingly challenging to defend against malicious traffic,due to the heterogeneity of Io T devices and the diversity of Io T communication protocols.In this paper,we propose a semi-supervised learning-based approach to detect malicious traffic at the access side.It overcomes the resource-bottleneck problem of traditional malicious traffic defenders which are deployed at the victim side,and also is free of labeled traffic data in model training.Specifically,we design a coarse-grained behavior model of Io T devices by self-supervised learning with unlabeled traffic data.Then,we fine-tune this model to improve its accuracy in malicious traffic detection by adopting a transfer learning method using a small amount of labeled data.Experimental results show that our method can achieve the accuracy of 99.52%and the F1-score of 99.52%with only 1%of the labeled training data based on the CICDDoS2019 dataset.Moreover,our method outperforms the stateof-the-art supervised learning-based methods in terms of accuracy,precision,recall and F1-score with 1%of the training data.

Distance metric learning guided adaptive subspace semi-supervised clustering

Most existing semi-supervised clustering algorithms are not designed for handling high- dimensional data. On the other hand, semi-supervised dimensionality reduction methods may not necessarily improve the clustering performance, due to the fact that the inherent relationship between subspace selection and clustering is ignored. In order to mitigate the above problems, we present a semi-supervised clustering algo- rithm using adaptive distance metric learning （SCADM） which performs semi-supervised clustering and distance metric learning simultaneously. SCADM applies the clustering results to learn a distance metric and then projects the data onto a low-dimensional space where the separability of the data is maximized. Experimental results on real-world data sets show that the proposed method can effectively deal with high-dimensional data and provides an appealing clustering performance.

基于Blending-Clustering集成学习的大坝变形预测模型

【目的】变形是反映大坝结构性态最直观的效应量,构建科学合理的变形预测模型是保障大坝安全健康运行的重要手段。针对传统大坝变形预测模型预测精度低、误报率高等问题导致的错误报警现象,【方法】选取不同预测模型和聚类算法集成,构建了一种Blending-Clustering集成学习的大坝变形预测模型,该模型以Blending对单一预测模型集成提升预测精度为核心,并通过Clustering聚类优选预测值改善模型稳定性。以新疆某面板堆石坝变形监测数据为实例分析,通过多模型预测性能比较,对所提出模型的预测精度和稳定性进行全面评估。【结果】结果显示:Blending-Clustering模型将预测模型和聚类算法集成,均方根误差(RMSE)和归一化平均百分比误差(nMAPE)明显降低,模型的预测精度得到显著提高;回归相关系数(R~2)得到提升,模型具备更强的拟合能力;在面板堆石坝上22个测点变形数据集上的预测评价指标波动范围更小,模型的泛化性和稳定性得到有效增强。【结论】结果表明:Blending-Clustering集成预测模型对于预测精度、泛化性和稳定性均有明显提升,在实际工程具有一定的应用价值。

Semi-supervised multi-layered clustering model for intrusion detection

A Machine Learning (ML)-based Intrusion Detection and Prevention System (IDPS)requires a large amount of labeled up-to-date training data to effectively detect intrusions and generalize well to novel attacks.However,the labeling of data is costly and becomes infeasible when dealing with big data,such as those generated by Intemet of Things applications.To this effect,building an ML model that learns from non-labeled or partially labeled data is of critical importance.This paper proposes a Semi-supervised Mniti-Layered Clustering ((SMLC))model for the detection and prevention of network intrusion.SMLC has the capability to learn from partially labeled data while achieving a detection performance comparable to that of supervised ML-based IDPS.The performance of SMLC is compared with that of a well-known semi-supervised model (tri-training)and of supervised ensemble ML models, namely Random.Forest,Bagging,and AdaboostM1on two benchmark network-intrusion datasets,NSL and Kyoto 2006+.Experimental resnits show that SMLC is superior to tri-training,providing a comparable detection accuracy with 20%less labeled instances of training data.Furthermore,our results demonstrate that our scheme has a detection accuracy comparable to that of the supervised ensemble models.

Transfer Learning-Based Semi-Supervised Generative Adversarial Network for Malaria Classification

Malaria is a lethal disease responsible for thousands of deaths worldwide every year.Manual methods of malaria diagnosis are timeconsuming that require a great deal of human expertise and efforts.Computerbased automated diagnosis of diseases is progressively becoming popular.Although deep learning models show high performance in the medical field,it demands a large volume of data for training which is hard to acquire for medical problems.Similarly,labeling of medical images can be done with the help of medical experts only.Several recent studies have utilized deep learning models to develop efficient malaria diagnostic system,which showed promising results.However,the most common problem with these models is that they need a large amount of data for training.This paper presents a computer-aided malaria diagnosis system that combines a semi-supervised generative adversarial network and transfer learning.The proposed model is trained in a semi-supervised manner and requires less training data than conventional deep learning models.Performance of the proposed model is evaluated on a publicly available dataset of blood smear images(with malariainfected and normal class)and achieved a classification accuracy of 96.6%.