Decentralized Semi-Supervised Learning for Stochastic Configuration Networks Based on the Mean Teacher Method

The aim of this paper is to broaden the application of Stochastic Configuration Network (SCN) in the semi-supervised domain by utilizing common unlabeled data in daily life. It can enhance the classification accuracy of decentralized SCN algorithms while effectively protecting user privacy. To this end, we propose a decentralized semi-supervised learning algorithm for SCN, called DMT-SCN, which introduces teacher and student models by combining the idea of consistency regularization to improve the response speed of model iterations. In order to reduce the possible negative impact of unsupervised data on the model, we purposely change the way of adding noise to the unlabeled data. Simulation results show that the algorithm can effectively utilize unlabeled data to improve the classification accuracy of SCN training and is robust under different ground simulation environments.

Radar emitter signal recognition method based on improved collaborative semi-supervised learning

Rare labeled data are difficult to recognize by using conventional methods in the process of radar emitter recogni-tion.To solve this problem,an optimized cooperative semi-supervised learning radar emitter recognition method based on a small amount of labeled data is developed.First,a small amount of labeled data are randomly sampled by using the bootstrap method,loss functions for three common deep learning net-works are improved,the uniform distribution and cross-entropy function are combined to reduce the overconfidence of softmax classification.Subsequently,the dataset obtained after sam-pling is adopted to train three improved networks so as to build the initial model.In addition,the unlabeled data are preliminarily screened through dynamic time warping(DTW)and then input into the initial model trained previously for judgment.If the judg-ment results of two or more networks are consistent,the unla-beled data are labeled and put into the labeled data set.Lastly,the three network models are input into the labeled dataset for training,and the final model is built.As revealed by the simula-tion results,the semi-supervised learning method adopted in this paper is capable of exploiting a small amount of labeled data and basically achieving the accuracy of labeled data recognition.

Semi-supervised learning based probabilistic latent semantic analysis for automatic image annotation

In recent years,multimedia annotation problem has been attracting significant research attention in multimedia and computer vision areas,especially for automatic image annotation,whose purpose is to provide an efficient and effective searching environment for users to query their images more easily. In this paper,a semi-supervised learning based probabilistic latent semantic analysis( PLSA) model for automatic image annotation is presenred. Since it's often hard to obtain or create labeled images in large quantities while unlabeled ones are easier to collect,a transductive support vector machine( TSVM) is exploited to enhance the quality of the training image data. Then,different image features with different magnitudes will result in different performance for automatic image annotation. To this end,a Gaussian normalization method is utilized to normalize different features extracted from effective image regions segmented by the normalized cuts algorithm so as to reserve the intrinsic content of images as complete as possible. Finally,a PLSA model with asymmetric modalities is constructed based on the expectation maximization( EM) algorithm to predict a candidate set of annotations with confidence scores. Extensive experiments on the general-purpose Corel5k dataset demonstrate that the proposed model can significantly improve performance of traditional PLSA for the task of automatic image annotation.

Semi-Supervised Learning Based on Manifold in BCI

A Laplacian support vector machine （LapSVM） algorithm, a semi-supervised learning based on manifold, is introduced to brain-computer interface （BCI） to raise the classification precision and reduce the subjects＇ training complexity. The data are collected from three subjects in a three-task mental imagery experiment. LapSVM and transductive SVM （TSVM） are trained with a few labeled samples and a large number of unlabeled samples. The results confirm that LapSVM has a much better classification than TSVM.

Semi-Supervised Dimensionality Reduction of Hyperspectral Image Based on Sparse Multi-Manifold Learning

In this paper, we proposed a new semi-supervised multi-manifold learning method, called semi- supervised sparse multi-manifold embedding (S3MME), for dimensionality reduction of hyperspectral image data. S3MME exploits both the labeled and unlabeled data to adaptively find neighbors of each sample from the same manifold by using an optimization program based on sparse representation, and naturally gives relative importance to the labeled ones through a graph-based methodology. Then it tries to extract discriminative features on each manifold such that the data points in the same manifold become closer. The effectiveness of the proposed multi-manifold learning algorithm is demonstrated and compared through experiments on a real hyperspectral images.

基于Semi-Supervised LLE的人脸表情识别方法

目的为提取有效的鉴别特征和降低鉴别向量的维数来识别人脸表情图像.方法将流行学习(Manifold learning,ML)和半监督学习(Semi-Supervised learning,SSL)结合起来,利用人脸表情图像数据本身的非线性流形结构信息和部分标签信息来调整点与点之间的距离形成距离矩阵,而后基于被调整的距离矩阵进行线性近邻重建来实现维数约简,提取低维鉴别特征用于人脸表情识别.结果该方法能充分利用数据的结构信息和有限的标签信息,使具有标签信息的同类样本之间的距离最小化,不同类数据之间的距离最大化,进而可以有效地提取数据的低维鉴别子流形,使得分类性能要优于非监督的维数约简方法.结论笔者提出的半监督局部线性嵌入算法能有效地提高人脸表情识别的性能.

Enhanced vision-transformer integrating with semi-supervised transfer learning for state of health and remaining useful life estimation of lithium-ion batteries

The state of health(SOH)and remaining useful life(RUL)of lithium-ion batteries are crucial for health management and diagnosis.However,most data-driven estimation methods heavily rely on scarce labeled data,while traditional transfer learning faces challenges in handling domain shifts across various battery types.This paper proposes an enhanced vision-transformer integrating with semi-supervised transfer learning for SOH and RUL estimation of lithium-ion batteries.A depth-wise separable convolutional vision-transformer is developed to extract local aging details with depth-wise convolutions and establishes global dependencies between aging information using multi-head attention.Maximum mean discrepancy is employed to initially reduce the distribution difference between the source and target domains,providing a superior starting point for fine-tuning the target domain model.Subsequently,the abundant aging data of the same type as the target battery are labeled through semi-supervised learning,compensating for the source model's limitations in capturing target battery aging characteristics.Consistency regularization incorporates the cross-entropy between predictions with and without adversarial perturbations into the gradient backpropagation of the overall model.In particular,across the experimental groups 13–15 for different types of batteries,the root mean square error of SOH estimation was less than 0.66%,and the mean relative error of RUL estimation was 3.86%.Leveraging extensive unlabeled aging data,the proposed method could achieve accurate estimation of SOH and RUL.

Labeling Malicious Communication Samples Based on Semi-Supervised Deep Neural Network

The limited labeled sample data in the field of advanced security threats detection seriously restricts the effective development of research work.Learning the sample labels from the labeled and unlabeled data has received a lot of research attention and various universal labeling methods have been proposed.However,the labeling task of malicious communication samples targeted at advanced threats has to face the two practical challenges:the difficulty of extracting effective features in advance and the complexity of the actual sample types.To address these problems,we proposed a sample labeling method for malicious communication based on semi-supervised deep neural network.This method supports continuous learning and optimization feature representation while labeling sample,and can handle uncertain samples that are outside the concerned sample types.According to the experimental results,our proposed deep neural network can automatically learn effective feature representation,and the validity of features is close to or even higher than that of features which extracted based on expert knowledge.Furthermore,our proposed method can achieve the labeling accuracy of 97.64%~98.50%,which is more accurate than the train-then-detect,kNN and LPA methodsin any labeled-sample proportion condition.The problem of insufficient labeled samples in many network attack detecting scenarios,and our proposed work can function as a reference for the sample labeling tasks in the similar real-world scenarios.

LOCAL CORRELATION DISCRIMINANT ANALYSIS AND ITS SEMI-SUPERVISED EXTENSION

Considering limitations of Linear Discriminant Analysis (LDA) and Marginal Fisher Analysis (MFA), a novel discriminant analysis called Local Correlation Discriminant Analysis (LCDA) is proposed in this paper. The main idea behind LCDA is to use more robust similarity measure, correlation metric, to measure the local similarity between image data. This results in better classifi-cation performance. In addition, to further improve the discriminant power of LCDA, we extend LCDA to semi-supervised case, which can make use of both labeled and unlabeled data to perform dis-criminant analysis. Extensive experimental results on ORL and AR face databases demonstrate that the proposed LCDA and its semi-supervised version are superior to Principal Component Analysis (PCA), LDA, CEA, and MFA.

A Semi-Supervised WLAN Indoor Localization Method Based on l1-Graph Algorithm

For indoor location estimation based on received signal strength( RSS) in wireless local area networks( WLAN),in order to reduce the influence of noise on the positioning accuracy,a large number of RSS should be collected in offline phase. Therefore,collecting training data with positioning information is time consuming which becomes the bottleneck of WLAN indoor localization. In this paper,the traditional semisupervised learning method based on k-NN and ε-NN graph for reducing collection workload of offline phase are analyzed,and the result shows that the k-NN or ε-NN graph are sensitive to data noise,which limit the performance of semi-supervised learning WLAN indoor localization system. Aiming at the above problem,it proposes a l1-graph-algorithm-based semi-supervised learning( LG-SSL) indoor localization method in which the graph is built by l1-norm algorithm. In our system,it firstly labels the unlabeled data using LG-SSL and labeled data to build the Radio Map in offline training phase,and then uses LG-SSL to estimate user's location in online phase. Extensive experimental results show that,benefit from the robustness to noise and sparsity ofl1-graph,LG-SSL exhibits superior performance by effectively reducing the collection workload in offline phase and improving localization accuracy in online phase.

Semi-supervised Document Clustering Based on Latent Dirichlet Allocation (LDA)

To discover personalized document structure with the consideration of user preferences,user preferences were captured by limited amount of instance level constraints and given as interested and uninterested key terms.Develop a semi-supervised document clustering approach based on the latent Dirichlet allocation(LDA)model,namely,pLDA,guided by the user provided key terms.Propose a generalized Polya urn(GPU) model to integrate the user preferences to the document clustering process.A Gibbs sampler was investigated to infer the document collection structure.Experiments on real datasets were taken to explore the performance of pLDA.The results demonstrate that the pLDA approach is effective.

Global Inference Preserving Projection for Semi-supervised Discriminant Analysis

Semi-supervised dimensionality reduction is an important research area for data classification. A new linear dimensionality reduction approach, global inference preserving projection （GIPP）, was proposed to perform classification task in semi-supervised case. GIPP provided a global structure that utilized the underlying discriminative knowledge of unlabeled samples. It used path-based dissimilarity measurement to infer the class label information for unlabeled samples and transformd the diseriminant algorithm into a generalized eigenequation problem. Experimental results demonstrate the effectiveness of the proposed approach.

基于改进教师-学生模型的色情音频事件检测

为保障青少年身心健康,国家日益重视色情信息的监管工作。针对传统色情音频检测无法精准定位事件起止时间的问题,提出一种基于半监督学习的改进教师-学生模型。将无标签、弱标签、强标签数据作为训练集输入,通过多层神经网络提取音频的帧、段特征,随后迭代优化帧、段所产生的分类损失以及教师-学生模型和段分类模型之间的一致性损失。在真实数据集上,实验结果表明当时间容忍度为5 s时,色情类别召回率达到94.3%,F1得分可达到83.4%。

TS-Aug架构的半监督自训练情感分类算法

网络教学资源的普及使得资源评价的文本数据规模逐步增大.传统的有监督学习文本分类对标注数据的依赖度较高,需要足够的数据量和高质量数据才能得到良好的结果.在网络教学资源的评价文本工作中,由于标注数据难以获取且质量参差不齐,使得这一任务的难度越来越高.针对这一困难,提出一种TS-Aug半监督自训练方案,通过添加无标签数据并进行伪标签训练,能在强力数据增广的作用下大幅扩充样本集,解决数据增广中的过拟合风险.首先利用标注数据和弱增广策略进行初始化监督训练,然后利用无标注数据和强增广策略进行半监督训练,最后使用标注数据进行微调监督训练.在自建的在线课程评论数据中,能将分类F 1-Score从0.88提升至0.95,表明TS-Aug半监督自训练方案在文本分类任务中具有较好的应用前景.

基于教师-学生网络的半监督故障诊断模型

针对神经网络模型在有标签样本数量较少的情况下,容易产生网络过拟合、故障诊断精度低、不能充分利用大量无标签样本数据等问题,提出一种基于连续小波变换和教师-学生网络的半监督学习方法用于旋转机械的故障诊断。该方法以改进的LeNet5卷积神经网络模型为基础,建立具有相同结构和初始化参数的学生网络模型和教师网络模型。首先,将旋转机械振动信号进行连续小波变换,将其转换为三维时频图像。接着,利用教师模型的预测结果生成伪标签,将这些伪标签和真实标签结合起来,训练学生网络。同时,通过指数加权移动平均算法更新教师网络模型参数。试验结果表明,相对于纯监督学习模型,所提出的算法能够在有标签样本数量较少的情况下显著提高模型训练过程的稳定性和故障诊断的精度。

基于密度峰值聚类的Tri-training算法

Tri-training利用无标签数据进行分类可有效提高分类器的泛化能力,但其易将无标签数据误标,从而形成训练噪声。提出一种基于密度峰值聚类的Tri-training(Tri-training with density peaks clustering,DPC-TT)算法。密度峰值聚类通过类簇中心和局部密度可选出数据空间结构表现较好的样本。DPC-TT算法采用密度峰值聚类算法获取训练数据的类簇中心和样本的局部密度,对类簇中心的截断距离范围内的样本认定为空间结构表现较好,标记为核心数据,使用核心数据更新分类器,可降低迭代过程中的训练噪声,进而提高分类器的性能。实验结果表明:相比于标准Tritraining算法及其改进算法,DPC-TT算法具有更好的分类性能。

基于Tri-training的社交媒体药物不良反应实体抽取

社交媒体因其数据的实时性,对其充分利用可以弥补传统医疗文献药物不良反应中实体抽取的迟滞性问题,但社交媒体文本面临标注数据成本高、数据噪声大等问题,使得模型难以发挥良好的效果。针对社交媒体大量未标注语料存在标注成本高的问题,采用Tri-training半监督的方法进行社交媒体药物不良反应实体抽取,通过三个学习器Transformer+CRF、BiLSTM+CRF和IDCNN+CRF对未标注数据进行标注,再利用一致性评价函数迭代地扩展训练集,最后通过加权投票整合模型输出标签。针对社交媒体的文本不正式性(口语化严重、错别字等)问题,通过融合字与词两个粒度的向量作为整个模型嵌入层的输入,来提取更丰富的语义信息。实验结果表明,提出的模型在“好大夫在线”网站获取的数据集上取得了良好表现。

基于PU-learning的磷酸激酶预测算法

目的蛋白质磷酸化是通过激酶催化特定位点把磷酸基转移到底物蛋白质氨基酸残基的过程,是研究蛋白质活力及功能的重要机制。目前已鉴定的数千个磷酸化位点大多缺失激酶信息,为此本研究提出基于PU-learning的磷酸激酶预测算法,通过迭代标记磷酸位点,可以准确预测催化磷酸肽的磷酸激酶。方法首先该算法以PU-learning为框架,利用最大熵方差对不同种类的磷酸激酶自动筛选最佳阈值,从而提取每条磷酸肽上潜在的磷酸化位点,然后根据统计分析确定磷酸化位点对应的激酶,最后通过五折交叉验证该算法在Phospho.ELM数据库上的预测性能,并与现有算法对比。结果该算法的交叉验证特异性和灵敏度比现有最好算法在单个数据集上最高提高4%及10%,其预测Phospho.ELM中数据准确度达到79.52%。结论基于PU-learning的磷酸激酶预测算法显著优于现有算法,且可以准确预测Phospho.ELM数据库中未知激酶信息的磷酸肽,在磷酸化实验中具有较强的指导意义。

基于UNet-ResNet14^(*)半监督学习的无人机影像森林树种分类

无人机遥感在森林树种精细和高效分类制图中具有巨大的潜力。为了快速准确获取森林的优势树种分布信息,该研究探讨了半监督学习方法在树种分类方面的有效性。以福建省福州市、龙岩市和三明市的4个试验区为例,构建精简的ResNet18为主干的UNet树种分类模型(UNet-ResNet14^(*)),使用交叉熵和Dice系数的联合损失函数来优化模型参数,对比分析Self-training和Mean Teacher两种不同的半监督学习方法在无人机影像森林树种分类模型的泛化能力。结果表明,以ResNet14^(*)作为主干的分类模型与其他模型相比精度更高且预测速度更快,当联合损失函数权重值为0.5的情况下模型预测效果最好,总体精度达到了91.15%。经过Self-training的模型在木荷、马尾松、杉木3个样本充足的类别中精度均有所提升,总精度为91.08%,比原始模型略低,但在独立验证区的精度为88.50%,比原始模型高;Mean Teacher方法的总精度为88.56%,在独立验证区的精度为73.56%。因此,研究认为可以采用Self-trainin半监督方法结合UNet-ResNet14^(*)的方案快速得到试验区的树种组成信息。