Incremental support vector machine algorithm based on multi-kernel learning

A new incremental support vector machine （SVM） algorithm is proposed which is based on multiple kernel learning. Through introducing multiple kernel learning into the SVM incremental learning, large scale data set learning problem can be solved effectively. Furthermore, different punishments are adopted in allusion to the training subset and the acquired support vectors, which may help to improve the performance of SVM. Simulation results indicate that the proposed algorithm can not only solve the model selection problem in SVM incremental learning, but also improve the classification or prediction precision.

A stacked multiple kernel support vector machine for blast inducedflyrock prediction

As a widely used rock excavation method in civil and mining construction works, the blasting operations and theinduced side effects are always investigated by the existing studies. The occurrence of flyrock is regarded as one ofthe most important issues induced by blasting operations, since the accurate prediction of which is crucial fordelineating safety zone. For this purpose, this study developed a flyrock prediction model based on 234 sets ofblasting data collected from Sugun Copper Mine site. A stacked multiple kernel support vector machine (stackedMK-SVM) model was proposed for flyrock prediction. The proposed stacked structure can effectively improve themodel performance by addressing the importance level of different features. For comparison purpose, 6 othermachine learning models were developed, including SVM, MK-SVM, Lagragian Twin SVM (LTSVM), ArtificialNeural Network (ANN), Random Forest (RF) and M5 Tree. This study implemented a 5-fold cross validationprocess for hyperparameters tuning purpose. According to the evaluation results, the proposed stacked MK-SVMmodel achieved the best overall performance, with RMSE of 1.73 and 1.74, MAE of 0.58 and 1.08, VAF of 98.95and 99.25 in training and testing phase, respectively.

Elastic Multiple Kernel Learning

(MKL ) 多重核学习被建议处理核熔化。MKL 听说线性联合几个核并且解决同时与联合的核联系的支持的向量机器(SVM ) 。MKL 的当前的框架鼓励核联合系数的稀少。核的重要部分什么时候是增进知识的，强迫稀少，趋于选择仅仅一些核并且可以忽略有用信息。在这份报纸，我们建议学习的有弹性的多重核(EMKL ) 完成适应的核熔化。EMKL 使用混合规则化功能损害稀少和非稀少。MKL 和 SVM 能被认为是 EMKL 的特殊情况。为 MKL 问题基于坡度降下算法，我们建议一个快算法解决 EMKL 问题。模拟数据集上的结果证明 EMKL 的表演有利地比作 MKL 和 SVM。我们进一步把 EMKL 用于基因集合分析并且得到有希望的结果。最后，我们学习比作另外的非稀少的 MKL 的 EMKL 的理论优点。

A Novel Kernel for Least Squares Support Vector Machine

Extreme learning machine(ELM) has attracted much attention in recent years due to its fast convergence and good performance.Merging both ELM and support vector machine is an important trend,thus yielding an ELM kernel.ELM kernel based methods are able to solve the nonlinear problems by inducing an explicit mapping compared with the commonly-used kernels such as Gaussian kernel.In this paper,the ELM kernel is extended to the least squares support vector regression(LSSVR),so ELM-LSSVR was proposed.ELM-LSSVR can be used to reduce the training and test time simultaneously without extra techniques such as sequential minimal optimization and pruning mechanism.Moreover,the memory space for the training and test was relieved.To confirm the efficacy and feasibility of the proposed ELM-LSSVR,the experiments are reported to demonstrate that ELM-LSSVR takes the advantage of training and test time with comparable accuracy to other algorithms.

Word Sense Disambiguation Based Sentiment Classification Using Linear Kernel Learning Scheme

Word Sense Disambiguation has been a trending topic of research in Natural Language Processing and Machine Learning.Mining core features and performing the text classification still exist as a challenging task.Here the features of the context such as neighboring words like adjective provide the evidence for classification using machine learning approach.This paper presented the text document classification that has wide applications in information retrieval,which uses movie review datasets.Here the document indexing based on controlled vocabulary,adjective,word sense disambiguation,generating hierarchical cate-gorization of web pages,spam detection,topic labeling,web search,document summarization,etc.Here the kernel support vector machine learning algorithm helps to classify the text and feature extract is performed by cuckoo search opti-mization.Positive review and negative review of movie dataset is presented to get the better classification accuracy.Experimental results focused with context mining,feature analysis and classification.By comparing with the previous work,proposed work designed to achieve the efficient results.Overall design is per-formed with MATLAB 2020a tool.

Classification of hyperspectral remote sensing images based on simulated annealing genetic algorithm and multiple instance learning

A hybrid feature selection and classification strategy was proposed based on the simulated annealing genetic algonthrn and multiple instance learning （MIL）. The band selection method was proposed from subspace decomposition, which combines the simulated annealing algorithm with the genetic algorithm in choosing different cross-over and mutation probabilities, as well as mutation individuals. Then MIL was combined with image segmentation, clustering and support vector machine algorithms to classify hyperspectral image. The experimental results show that this proposed method can get high classification accuracy of 93.13% at small training samples and the weaknesses of the conventional methods are overcome.

ERROR ANALYSIS OF MULTICATEGORY SUPPORT VECTOR MACHINE CLASSIFIERS

The paper is related to the error analysis of Multicategory Support Vector Machine （MSVM） classifiers based on reproducing kernel Hilbert spaces. We choose the polynomial kernel as Mercer kernel and give the error estimate with De La Vall6e Poussin means. We also introduce the standard estimation of sample error, and derive the explicit learning rate.

Traffic Sign Recognition Based on CNN and Twin Support Vector Machine Hybrid Model

With the progress of deep learning research, convolutional neural networks have become the most important method in feature extraction. How to effectively classify and recognize the extracted features will directly affect the performance of the entire network. Traditional processing methods include classification models such as fully connected network models and support vector machines. In order to solve the problem that the traditional convolutional neural network is prone to over-fitting for the classification of small samples, a CNN-TWSVM hybrid model was proposed by fusing the twin support vector machine (TWSVM) with higher computational efficiency as the CNN classifier, and it was applied to the traffic sign recognition task. In order to improve the generalization ability of the model, the wavelet kernel function is introduced to deal with the nonlinear classification task. The method uses the network initialized from the ImageNet dataset to fine-tune the specific domain and intercept the inner layer of the network to extract the high abstract features of the traffic sign image. Finally, the TWSVM based on wavelet kernel function is used to identify the traffic signs, so as to effectively solve the over-fitting problem of traffic signs classification. On GTSRB and BELGIUMTS datasets, the validity and generalization ability of the improved model is verified by comparing with different kernel functions and different SVM classifiers.

基于SPSO优化Multiple Kernel-TWSVM的滚动轴承故障诊断

双子支持向量机(twin support vector machine,简称TWSVM)的核函数选择对其分类性能有着重要影响,TWSVM其核函数一般是局部核函数或者全局核函数,这两种核函数的泛化能力和分类性能不能兼顾。笔者利用综合加权的高斯局部核函数和多项式全局核函数方法组成双核函数来改进TWSVM以提高其泛化能力和分类性能,并采用简化粒子群优化(simple particle swarm optimization,简称SPSO)方法来对权值和参数进行优化,提出了SPSO优化Multiple Kernel-TWSVM模型,将该模型应用到滚动轴承故障诊断模式识别中。实验结果表明,双核TWSVM比单核TWSVM和反向传播(back propagation,简称BP)神经网络具有更高的分类准确率。

Kernel matrix learning with a general regularized risk functional criterion

Kernel-based methods work by embedding the data into a feature space and then searching linear hypothesis among the embedding data points. The performance is mostly affected by which kernel is used. A promising way is to learn the kernel from the data automatically. A general regularized risk functional （RRF） criterion for kernel matrix learning is proposed. Compared with the RRF criterion, general RRF criterion takes into account the geometric distributions of the embedding data points. It is proven that the distance between different geometric distdbutions can be estimated by their centroid distance in the reproducing kernel Hilbert space. Using this criterion for kernel matrix learning leads to a convex quadratically constrained quadratic programming （QCQP） problem. For several commonly used loss functions, their mathematical formulations are given. Experiment results on a collection of benchmark data sets demonstrate the effectiveness of the proposed method.

Kernel-based adversarial attacks and defenses on support vector classification

While malicious samples are widely found in many application fields of machine learning,suitable countermeasures have been investigated in the field of adversarial machine learning.Due to the importance and popularity of Support Vector Machines(SVMs),we first describe the evasion attack against SVM classification and then propose a defense strategy in this paper.The evasion attack utilizes the classification surface of SVM to iteratively find the minimal perturbations that mislead the nonlinear classifier.Specially,we propose what is called a vulnerability function to measure the vulnerability of the SVM classifiers.Utilizing this vulnerability function,we put forward an effective defense strategy based on the kernel optimization of SVMs with Gaussian kernel against the evasion attack.Our defense method is verified to be very effective on the benchmark datasets,and the SVM classifier becomes more robust after using our kernel optimization scheme.

Machine learning approach for estimating the human-related VOC emissions in a university classroom

Indoor air quality becomes increasingly important,partly because the COVID-19 pandemic increases the time people spend indoors.Research into the prediction of indoor volatile organic compounds(VOCs)is traditionally confined to building materials and furniture.Relatively little research focuses on estimation of human-related VOCs,which have been shown to contribute significantly to indoor air quality,especially in densely-occupied environments.This study applies a machine learning approach to accurately estimate the human-related VOC emissions in a university classroom.The time-resolved concentrations of two typical human-related(ozone-related)VOCs in the classroom over a five-day period were analyzed,i.e.,6-methyl-5-hepten-2-one(6-MHO),4-oxopentanal(4-OPA).By comparing the results for 6-MHO concentration predicted via five machine learning approaches including the random forest regression(RFR),adaptive boosting(Adaboost),gradient boosting regression tree(GBRT),extreme gradient boosting(XGboost),and least squares support vector machine(LSSVM),we find that the LSSVM approach achieves the best performance,by using multi-feature parameters(number of occupants,ozone concentration,temperature,relative humidity)as the input.The LSSVM approach is then used to predict the 4-OPA concentration,with mean absolute percentage error(MAPE)less than 5%,indicating high accuracy.By combining the LSSVM with a kernel density estimation(KDE)method,we further establish an interval prediction model,which can provide uncertainty information and viable option for decision-makers.The machine learning approach in this study can easily incorporate the impact of various factors on VOC emission behaviors,making it especially suitable for concentration prediction and exposure assessment in realistic indoor settings.

Fusion-Based Deep Learning Model for Hyperspectral Images Classification

A crucial task in hyperspectral image(HSI)taxonomy is exploring effective methodologies to effusively practice the 3-D and spectral data delivered by the statistics cube.For classification of images,3-D data is adjudged in the phases of pre-cataloging,an assortment of a sample,classifiers,post-cataloging,and accurateness estimation.Lastly,a viewpoint on imminent examination directions for proceeding 3-D and spectral approaches is untaken.In topical years,sparse representation is acknowledged as a dominant classification tool to effectually labels deviating difficulties and extensively exploited in several imagery dispensation errands.Encouraged by those efficacious solicitations,sparse representation(SR)has likewise been presented to categorize HSI’s and validated virtuous enactment.This research paper offers an overview of the literature on the classification of HSI technology and its applications.This assessment is centered on a methodical review of SR and support vector machine(SVM)grounded HSI taxonomy works and equates numerous approaches for this matter.We form an outline that splits the equivalent mechanisms into spectral aspects of systems,and spectral–spatial feature networks to methodically analyze the contemporary accomplishments in HSI taxonomy.Furthermore,cogitating the datum that accessible training illustrations in the remote distinguishing arena are generally appropriate restricted besides training neural networks(NNs)to necessitate an enormous integer of illustrations,we comprise certain approaches to increase taxonomy enactment,which can deliver certain strategies for imminent learnings on this issue.Lastly,numerous illustrative neural learning-centered taxonomy approaches are piloted on physical HSI’s in our experimentations.

Metric Learning with Relative Distance Constraints:A Modified SVM Approach

Distance metric learning plays an important role in many machine learning tasks. In this paper, we propose a method for learning a Mahanalobis distance metric. By formulating the metric learning problem with relative distance constraints, we suggest a Relative Distance Constrained Metric Learning (RDCML) model which can be easily implemented and effectively solved by a modified support vector machine (SVM) approach. Experimental results on UCI datasets and handwritten digits datasets show that RDCML achieves better or comparable classification accuracy when compared with the state-of-the-art metric learning methods.

弱监督场景下的支持向量机算法综述

支持向量机(Support Vector Machine,SVM)是一种建立在结构风险最小化原则上的统计学习方法,以其在非线性、小样本以及高维问题中的独特优势被广泛应用于图像识别、故障诊断以及文本分类等领域.但SVM是一种监督学习算法,它旨在利用大量的、唯一且明确的真值标记样本来训练学习器,在不完全监督、不确切监督以及多义监督等弱监督场景下难以取得较好的效果.本文首先阐述了弱监督场景的概念和SVM的相关理论,然后从弱监督场景角度出发,系统地梳理了目前SVM算法的研究现状和发展,包括基于半监督学习、多示例学习以及多标记学习的方法;其中基于半监督学习的方法根据数据假设可细分为基于聚类假设和基于流形假设的方法,基于多标记学习的方法根据解决方案可细分为基于示例水平空间、基于包水平空间以及基于嵌入空间的方法,基于多标记学习的方法根据处理思路可细分为基于问题转换和基于算法自适应的方法;随后,本文总结了部分代表性算法在公开数据集上的实验结果;最后,探讨并展望了未来可能的研究方向.

基于GMKL-SVM的模拟电路故障诊断方法

提出了一种新颖的基于广义多核支持向量机(GMKL-SVM)的模拟电路故障诊断方法。首先,应用Haar小波分析提取被测电路时域响应信号的小波系数作为特征参量,并生成样本数据;然后,基于样本数据,应用量子粒子群算法对GMKL-SVM的参数进行优化,并以此建立基于GMKL-SVM的故障诊断模型,用于区分模拟电路的各个故障。实例电路的单故障和双故障诊断实验结果表明,所提出的GMKL-SVM方法能较好地实现模拟电路故障诊断,与传统的GMKL-SVM方法相比,表现出了更好的性能,获得了更高的故障诊断正确率。

基于KPCA和MKL-SVM的非线性过程监控与故障诊断

利用核主元分析非线性过程监控的优势,结合多重核学习支持向量机在故障诊断方面的准确性,提出了基于核主元分析和多重核学习支持向量机的非线性过程监控与故障诊断方法。该方法运用核主元法对数据进行处理,在特征空间构建T2和SPE来检测故障的发生,若有故障发生,则计算样本的非线性主元得分向量,将其作为MKL-SVM的输入值,通过MKL-SVM的分类进行故障类型识别。将上述方法应用到Tennessee Eastman(TE)化工过程,多种故障模式的仿真结果表明该方法不但能有效地辨识故障,而且提高了故障检测和故障诊断的速度。

基于测量阻抗动态轨迹的大型调相机失磁保护

大型调相机失磁故障严重影响设备本体安全以及电网稳定,现有基于静态阈值的低电压与无功反向判据可靠性与选择性不足。文中提出一种可反映调相机运行状态的机端测量阻抗全局动态轨迹智能识别的失磁保护原理,从运动学角度建立能够准确反映失磁与其他工况下测量阻抗轨迹的特征量时间序列,基于统计学提取解释性强的特征量。利用自适应权重的全局与局部核函数组合训练多核支持向量机(multiple kernel learning support vector machine,MKL-SVM),在保证模型学习能力的同时增强其泛化能力;提出基于分类核空间距离的两阶段识别策略,可在保证可靠性的前提下提高保护速动性。基于PSCAD仿真平台搭建调相机接入电网模型进行验证,结果表明所提失磁保护方案无须采集转子侧电气量,识别准确,面对新能源接入和未知扰动时仍具有优良的适用性。

基于多核学习的单分类多示例学习算法

将多核学习引入到单分类多示例学习中,提出了一种基于多核学习的单分类多示例支持向量数据描述算法,解决了多核学习方法在实际应用中多示例数据具有比较复杂分布结构的学习问题。本文算法是将多个示例数据通过多个不同的核函数多核映射到特征空间,在特征空间中通过支持向量数据描述算法构建球形分类器。该算法采用迭代优化框架,首先,根据初始化包中的正示例来优化目标函数以此建立分类器。然后,根据上一步得到的分类器再对包中的正示例的标签进行更新。最后,在Corel、VOC 2007和Messidor数据集上的实验结果表明,所提出的算法比单核多示例方法具有更好的性能,进一步验证了算法的可行性和有效性。

基于递归定量分析与多核学习支持向量机的玻璃纤维增强复合材料缺陷识别技术

为了提高玻璃纤维增强复合材料(Glass Fiber Reinforced Polymer,GFRP)超声检测中缺陷识别技术的准确性,提出基于递归定量分析(Recurrence Quantitative Analysis,RQA)与多核学习支持向量机(MKLSVM)相结合的检测模型,以提高检测GFRP中不同类型缺陷的能力。结果表明,该模型能够准确识别GFRP中的分层缺陷与夹杂缺陷,检测识别率达到92.92%,并且与基于离散小波变换(Discrete Wavelet Transform,DWT)和经验模态分解(Empirical Mode Decomposition,EMD)的MKLSVM检测模型的识别率相比,所提出的检测模型的识别率分别提高了7.5%和3.75%。