Improved Twin Support Vector Machine Algorithm and Applications in Classification Problems

The distribution of data has a significant impact on the results of classification.When the distribution of one class is insignificant compared to the distribution of another class,data imbalance occurs.This will result in rising outlier values and noise.Therefore,the speed and performance of classification could be greatly affected.Given the above problems,this paper starts with the motivation and mathematical representing of classification,puts forward a new classification method based on the relationship between different classification formulations.Combined with the vector characteristics of the actual problem and the choice of matrix characteristics,we firstly analyze the orderly regression to introduce slack variables to solve the constraint problem of the lone point.Then we introduce the fuzzy factors to solve the problem of the gap between the isolated points on the basis of the support vector machine.We introduce the cost control to solve the problem of sample skew.Finally,based on the bi-boundary support vector machine,a twostep weight setting twin classifier is constructed.This can help to identify multitasks with feature-selected patterns without the need for additional optimizers,which solves the problem of large-scale classification that can’t deal effectively with the very low category distribution gap.

Construction and application of pre-classified smooth semi-supervised twin support vector machine

In order to handle the semi-supervised problem quickly and efficiently in the twin support vector machine (TWSVM) field, a semi-supervised twin support vector machine (S2TSVM) is proposed by adding the original unlabeled samples. In S2TSVM, the addition of unlabeled samples can easily cause the classification hyper plane to deviate from the sample points. Then a centerdistance principle is proposed to pre-classify unlabeled samples, and a pre-classified S2TSVM (PS2TSVM) is proposed. Compared with S2TSVM, PS2TSVM not only improves the problem of the samples deviating from the classification hyper plane, but also improves the training speed. Then PS2TSVM is smoothed. After smoothing the model, the pre-classified smooth S2TSVM (PS3TSVM) is obtained, and its convergence is deduced. Finally, nine datasets are selected in the UCI machine learning database for comparison with other types of semi-supervised models. The experimental results show that the proposed PS3TSVM model has better classification results.

Traffic Sign Recognition Based on CNN and Twin Support Vector Machine Hybrid Model

With the progress of deep learning research, convolutional neural networks have become the most important method in feature extraction. How to effectively classify and recognize the extracted features will directly affect the performance of the entire network. Traditional processing methods include classification models such as fully connected network models and support vector machines. In order to solve the problem that the traditional convolutional neural network is prone to over-fitting for the classification of small samples, a CNN-TWSVM hybrid model was proposed by fusing the twin support vector machine (TWSVM) with higher computational efficiency as the CNN classifier, and it was applied to the traffic sign recognition task. In order to improve the generalization ability of the model, the wavelet kernel function is introduced to deal with the nonlinear classification task. The method uses the network initialized from the ImageNet dataset to fine-tune the specific domain and intercept the inner layer of the network to extract the high abstract features of the traffic sign image. Finally, the TWSVM based on wavelet kernel function is used to identify the traffic signs, so as to effectively solve the over-fitting problem of traffic signs classification. On GTSRB and BELGIUMTS datasets, the validity and generalization ability of the improved model is verified by comparing with different kernel functions and different SVM classifiers.

TWIN SUPPORT TENSOR MACHINES FOR MCS DETECTION

Tensor representation is useful to reduce the overfitting problem in vector-based learning algorithm in pattern recognition.This is mainly because the structure information of objects in pattern analysis is a reasonable constraint to reduce the number of unknown parameters used to model a classifier.In this paper, we generalize the vector-based learning algorithm TWin Support Vector Machine(TWSVM) to the tensor-based method TWin Support Tensor Machines(TWSTM), which accepts general tensors as input.To examine the effectiveness of TWSTM, we implement the TWSTM method for Microcalcification Clusters(MCs) detection.In the tensor subspace domain, the MCs detection procedure is formulated as a supervised learning and classification problem, and TWSTM is used as a classifier to make decision for the presence of MCs or not.A large number of experiments were carried out to evaluate and compare the performance of the proposed MCs detection algorithm.By comparison with TWSVM, the tensor version reduces the overfitting problem.

Least squares twin support vector machine with asymmetric squared loss

For classification problems,the traditional least squares twin support vector machine(LSTSVM)generates two nonparallel hyperplanes directly by solving two systems of linear equations instead of a pair of quadratic programming problems(QPPs),which makes LSTSVM much faster than the original TSVM.But the standard LSTSVM adopting quadratic loss measured by the minimal distance is sensitive to noise and unstable to re-sampling.To overcome this problem,the expectile distance is taken into consideration to measure the margin between classes and LSTSVM with asymmetric squared loss(aLSTSVM)is proposed.Compared to the original LSTSVM with the quadratic loss,the proposed aLSTSVM not only has comparable computational accuracy,but also performs good properties such as noise insensitivity,scatter minimization and re-sampling stability.Numerical experiments on synthetic datasets,normally distributed clustered(NDC)datasets and University of California,Irvine(UCI)datasets with different noises confirm the great performance and validity of our proposed algorithm.

Robust least squares projection twin SVM and its sparse solution

Least squares projection twin support vector machine(LSPTSVM)has faster computing speed than classical least squares support vector machine(LSSVM).However,LSPTSVM is sensitive to outliers and its solution lacks sparsity.Therefore,it is difficult for LSPTSVM to process large-scale datasets with outliers.In this paper,we propose a robust LSPTSVM model(called R-LSPTSVM)by applying truncated least squares loss function.The robustness of R-LSPTSVM is proved from a weighted perspective.Furthermore,we obtain the sparse solution of R-LSPTSVM by using the pivoting Cholesky factorization method in primal space.Finally,the sparse R-LSPTSVM algorithm(SR-LSPTSVM)is proposed.Experimental results show that SR-LSPTSVM is insensitive to outliers and can deal with large-scale datasets fastly.

Improved twin support vector machine

We improve the twin support vector machine(TWSVM)to be a novel nonparallel hyperplanes classifier,termed as ITSVM(improved twin support vector machine),for binary classification.By introducing the diferent Lagrangian functions for the primal problems in the TWSVM,we get an improved dual formulation of TWSVM,then the resulted ITSVM algorithm overcomes the common drawbacks in the TWSVMs and inherits the essence of the standard SVMs.Firstly,ITSVM does not need to compute the large inverse matrices before training which is inevitable for the TWSVMs.Secondly,diferent from the TWSVMs,kernel trick can be applied directly to ITSVM for the nonlinear case,therefore nonlinear ITSVM is superior to nonlinear TWSVM theoretically.Thirdly,ITSVM can be solved efciently by the successive overrelaxation(SOR)technique or sequential minimization optimization(SMO)method,which makes it more suitable for large scale problems.We also prove that the standard SVM is the special case of ITSVM.Experimental results show the efciency of our method in both computation time and classification accuracy.

Intrusion Detection Model with Twin Support Vector Machines

Intrusion detection system(IDS) is becoming a critical component of network security. However,the performance of many proposed intelligent intrusion detection models is still not competent to be applied to real network security. This paper aims to explore a novel and effective approach to significantly improve the performance of IDS. An intrusion detection model with twin support vector machines(TWSVMs) is proposed.In this model, an efficient algorithm is also proposed to determine the parameter of TWSVMs. The performance of the proposed intrusion detection model is evaluated with KDD'99 dataset and is compared with those of some recent intrusion detection models. The results demonstrate that the proposed intrusion detection model achieves remarkable improvement in intrusion detection rate and more balanced performance on each type of attacks.Moreover, TWSVMs consume much less training time than standard support vector machines(SVMs).

Quadratic Kernel-Free Least Square Twin Support Vector Machine for Binary Classification Problems

In this paper,a new quadratic kernel-free least square twin support vector machine(QLSTSVM)is proposed for binary classification problems.The advantage of QLSTSVM is that there is no need to select the kernel function and related parameters for nonlinear classification problems.After using consensus technique,we adopt alternating direction method of multipliers to solve the reformulated consensus QLSTSVM directly.To reduce CPU time,the Karush-Kuhn-Tucker(KKT)conditions is also used to solve the QLSTSVM.The performance of QLSTSVM is tested on two artificial datasets and several University of California Irvine(UCI)benchmark datasets.Numerical results indicate that the QLSTSVM may outperform several existing methods for solving twin support vector machine with Gaussian kernel in terms of the classification accuracy and operation time.

基于PSO优化双子支持向量机的电商经济预测研究

为提高电商经济预测性能,解决因时序复杂、特征量多及用户需求复杂等带来的预测精度偏低的问题,采用双子支持向量机进行电商经济预测。首先,获取电商经济数据特征,接着构建双子支持向量机(TWSVM)的电商经济预测模型,提取TWSVM的正则因子等参数,随机初始化多组参数,并构建多个粒子。然后借助粒子群优化(PSO)算法搜索最优TWSVM参数,以生成适合电商经济预测的最佳TWSVM模型,通过PSO优化获得最优TWSVM参数。最后采用最佳TWSVM模型进行电商经济预测,并对预测结果进行评价。在实例仿真中,以电商经济销售金额和增长率两个指标为主,PSO优化的TWSVM算法的预测准确度均高于90%。

基于最小二乘孪生支持向量机的不确定数据学习算法

孪生支持向量机通过计算2个二次规划问题,得到2个不平行的超平面,用于解决二分类问题。然而在实际的应用中,数据通常包含不确定信息,这将会对构建模型带来困难。对此,提出了一种用于求解带有不确定数据的最小二乘孪生支持向量机模型。首先,对于每个实例,该方法都分配一个噪声向量来构建噪声信息。其次,将噪声向量结合到最小二乘孪生支持向量机,并在训练阶段得到优化。最后,采用一个2步循环迭代的启发式框架求解得到分类器和更新噪声向量。实验表明,跟其他对比方法比较,本方法采用噪声向量对不确定信息进行建模,并将孪生支持向量机的二次规划问题转化为线性方程,具有更好的分类精度和更高的训练效率。

基于特征加权混合隶属度的模糊孪生支持向量机

模糊孪生支持向量机(FTSVM)忽略了不同特征间的差异,导致核函数或距离的计算无法准确反映样本间的相似性,使FTSVM在处理含有大量不相关或弱相关特征的高维数据分类时,难以达到良好分类效果;且隶属度的设计未有效区分离群点或噪声。针对以上问题,提出了一种基于特征加权混合隶属度的FM-FTSVM。首先计算每个特征的信息增益,并依据信息增益值的大小为特征赋予权重,降低不相关或弱相关特征的作用,使其能更好地应用于高维数据分类;然后,为每一类样本构造一个最小包围球计算基于紧密度的特征加权隶属度,并结合基于距离的特征加权隶属度得到特征加权混合隶属度,综合考虑样本点到类中心的特征加权欧式距离和样本间的紧密程度,可更好识别离群点或噪声数据;最后,融合特征加权核函数,降低不相关特征对核函数或距离计算产生的影响。与对比算法在人工数据集、高维数据集和UCI数据集上进行比较,发现本文提出的方法在区分离群点、噪声和有效样本上有明显优势,且在高维数据集上可获得更好分类效果。

Intuitionistic Fuzzy Laplacian Twin Support Vector Machine for Semi-supervised Classification

In general,data contain noises which come from faulty instruments,flawed measurements or faulty communication.Learning with data in the context of classification or regression is inevitably affected by noises in the data.In order to remove or greatly reduce the impact of noises,we introduce the ideas of fuzzy membership functions and the Laplacian twin support vector machine(Lap-TSVM).A formulation of the linear intuitionistic fuzzy Laplacian twin support vector machine(IFLap-TSVM)is presented.Moreover,we extend the linear IFLap-TSVM to the nonlinear case by kernel function.The proposed IFLap-TSVM resolves the negative impact of noises and outliers by using fuzzy membership functions and is a more accurate reasonable classi-fier by using the geometric distribution information of labeled data and unlabeled data based on manifold regularization.Experiments with constructed artificial datasets,several UCI benchmark datasets and MNIST dataset show that the IFLap-TSVM has better classification accuracy than other state-of-the-art twin support vector machine(TSVM),intuitionistic fuzzy twin support vector machine(IFTSVM)and Lap-TSVM.

Structural regularized twin support vector machine based on within-class scatter and between-class scatter

Robust minimum class variance twin support vector machine(RMCV-TWSVM) presented previously gets better classification performance than the classical TWSVM. The RMCV-TWSVM introduces the class variance matrix of positive and negative samples into the construction of two hyperplanes. However, it does not consider the total structure information of all the samples, which can substantially reduce its classification accuracy. In this paper, a new algorithm named structural regularized TWSVM based on within-class scatter and between-class scatter(WSBS-STWSVM) is put forward. The WSBS-STWSVM can make full use of the total within-class distribution information and between-class structure information of all the samples. The experimental results illustrate high classification accuracy and strong generalization ability of the proposed algorithm.

A Probability Approach to Anomaly Detection with Twin Support Vector Machines

Classification of intrusion attacks and normal network flow is a critical and challenging issue in network security study. Many intelligent intrusion detection models are proposed, but their performances and efficiencies are not satisfied to real computer networks. This paper presents a novel effective intrusion detection system based on statistic reference model and twin support vector machines （TWSVMs）. Moreover, a network flow feature selection procedure has been studied and implemented with TWSVMs. The performances of proposed system are evaluated through using the fifth international conference on knowledge discovery and data mining in 1999 （KDD＇99） data set collected at MIT＇s Lincoln Labs and the results indicate that the proposed system is more efficient and effective than conventional support vector machines （SVMs） and TWSVMs.

结构化最大间隔双支持向量机在股票预测中的应用

股票价格受政策、宏观经济以及公司经营状况等多方因素的影响,且各因素之间存在较高的相关性,因此股票数据存在的高噪声、非平稳等特性使得股票预测充满困难。为了减少数据中存在的噪声对股价预测准确性的影响,基于马氏距离的类间隔可分性,提出了结构化最大间隔双支持向量机,其分别针对正类样本和负类样本,寻找两个非平行的超平面,使每一类样本离本类样本的欧式距离尽可能小,同时离异类超平面的马氏距离尽可能大。8组基准数据集的实验结果表明,该方法在含噪声数据的分类问题上具有稳定的准确率,从而提升了模型的预测性能和抗噪能力。同时将其应用到股票涨跌趋势预测中,通过对上证综指、上证A指、上证380指数以及中国平安等14只股票实证分析的结果表明,相较于其他对比模型,结构化最大间隔双支持向量机表现出了较好的预测结果,具有一定的实用价值。

基于数字孪生的铣刀磨损状态识别方法研究

实时精准地监测铣刀磨损状态对于提高加工质量与加工效率具有重要意义,提出一种基于数字孪生的铣刀磨损状态识别方法,该方法通过结合VMD-MPE特征提取方法和GA-SVM状态识别模型构建数字孪生体对铣刀磨损状态进行实时监测。首先,利用变分模态分解算法(VMD)分解铣刀振动信号得到包含磨损状态信息的模态分量;其次,引入多尺度排列熵(MPE)从包含磨损状态信息的模态分量中提取铣刀的非线性动力学特征,并取各有效模态分量的多尺度排列熵平均值作为特征矩阵;最后,通过遗传算法(GA)优化支持向量机(SVM)构建铣刀磨损状态识别模型。实验结果表明,所构建的数字孪生体具有良好识别效果,其识别精度可达97.33%。

基于优化TQWT及孪生SVM的有载分接开关机械故障诊断

为进一步提高有载分接开关(on-load tap changer,OLTC)机械状态监测的准确性,文中基于优化品质因数可调小波变换(tunable quality wavelet transform,TQWT)对OLTC切换过程中的振动信号进行了分析。即使用人工鱼群算法(artificial fish swarm algorithm,AFSA)基于分解余量与整体正交系数研究了TQWT的优化分解方法,计算得到了OLTC振动信号的多个子序列,构建了基于优化孪生支持向量机(twin support vector machine,TWSVM)的OLTC机械故障诊断模型。对某CM型OLTC正常与典型机械故障下振动信号的分析结果表明,所提优化TQWT分解方法有效提高了OLTC振动信号分解结果的准确性。相对于其他诊断模型,所构建AFSA-TWSVM的OLTC机械故障诊断模型分类效果好且收敛速度更快。

Twin-SVM和Twin-KSVC标志物检测与分类方法

针对交通标志中禁令标志和指示标志的检测和分类难题,提出一种基于Twin-SVM和Twin-KSVC的交通标志检测与分类方法。对交通标志图像的红色、蓝色和亮度3个通道进行光照归一化处理;在这3个通道上提取Haar-like特征,构建特征向量;采用Twin-SVM方法进行交通标志检测过程的特征训练与验证,采用Twin-KSVC方法进行交通标志分类过程的特征训练与验证。实验采用实测数据对算法进行测试与评价,实验结果表明,该方法可以有效地检测和识别常见的20类禁令和指示交通标志。

基于传感信号采集的电控发动机振动故障监测方法

通过调理振动信号可以更高效地监测振动故障。为此,提出基于传感信号采集的电控发动机振动故障监测方法。首先,搭建电控发电机传感信号采集与处理架构,通过放大传感信号增益、滤波和转换信号模数的方式处理待监测信号,为提高监测准确性奠定可靠的数据基础。通过小波包分解与重构,获取信号的时域参数和小波能谱熵,并构建三维特征量。然后,利用“一对一”分解策略优化孪生支持向量机,构造多元分类器,使其更适用于振动故障监测这一多类别分类问题,再输入待监测信号的特征量,通过确定故障类别实现持续性监测。仿真结果表明:该方法训练耗时的最大值仅为897 ms,对于转子摩擦振动、不平衡振动等5种类型故障的监测准确率始终在97%以上,在缩减训练样本后准确率仍保持在90%以上。