Support Vector Machines for Regression: A Succinct Review of Large-Scale and Linear Programming Formulations

Support Vector-based learning methods are an important part of Computational Intelligence techniques. Recent efforts have been dealing with the problem of learning from very large datasets. This paper reviews the most commonly used formulations of support vector machines for regression (SVRs) aiming to emphasize its usability on large-scale applications. We review the general concept of support vector machines (SVMs), address the state-of-the-art on training methods SVMs, and explain the fundamental principle of SVRs. The most common learning methods for SVRs are introduced and linear programming-based SVR formulations are explained emphasizing its suitability for large-scale learning. Finally, this paper also discusses some open problems and current trends.

New predictive control algorithms based on Least Squares Support Vector Machines

Used for industrial process with different degree of nonlinearity, the two predictive control algorithms presented in this paper are based on Least Squares Support Vector Machines (LS-SVM) model. For the weakly nonlinear system, the system model is built by using LS-SVM with linear kernel function, and then the obtained linear LS-SVM model is transformed into linear input-output relation of the controlled system. However, for the strongly nonlinear system, the off-line model of the controlled system is built by using LS-SVM with Radial Basis Function (RBF) kernel. The obtained nonlinear LS-SVM model is linearized at each sampling instant of system running, after which the on-line linear input-output model of the system is built. Based on the obtained linear input-output model, the Generalized Predictive Control (GPC) algorithm is employed to implement predictive control for the controlled plant in both algorithms. The simulation results after the presented algorithms were implemented in two different industrial processes model; respectively revealed the effectiveness and merit of both algorithms.

Weighted Proximal Support Vector Machines: Robust Classification

Despite of its great efficiency for pattern classification, proximal supportvector machines (PSVM), a new version of SVM proposed recently, is sensitive to noise and outliers.To overcome the drawback, this paper modifies PSVM by associating a weightvalue with each input dataof PSVM. The distance between each data point and the center of corresponding class is used tocalculate the weight value. In this way, the effect of noise is reduced. The experiments indicatethat new SVM, weighted proximal support vector machine (WPSVM), is much more robust to noise thanPSVM without loss of computationally attractive feature of PSVM.

Facial Expression Recognition Model Depending on Optimized Support Vector Machine

In computer vision,emotion recognition using facial expression images is considered an important research issue.Deep learning advances in recent years have aided in attaining improved results in this issue.According to recent studies,multiple facial expressions may be included in facial photographs representing a particular type of emotion.It is feasible and useful to convert face photos into collections of visual words and carry out global expression recognition.The main contribution of this paper is to propose a facial expression recognitionmodel(FERM)depending on an optimized Support Vector Machine(SVM).To test the performance of the proposed model(FERM),AffectNet is used.AffectNet uses 1250 emotion-related keywords in six different languages to search three major search engines and get over 1,000,000 facial photos online.The FERM is composed of three main phases:(i)the Data preparation phase,(ii)Applying grid search for optimization,and(iii)the categorization phase.Linear discriminant analysis(LDA)is used to categorize the data into eight labels(neutral,happy,sad,surprised,fear,disgust,angry,and contempt).Due to using LDA,the performance of categorization via SVM has been obviously enhanced.Grid search is used to find the optimal values for hyperparameters of SVM(C and gamma).The proposed optimized SVM algorithm has achieved an accuracy of 99%and a 98%F1 score.

Temperature Prediction Model Identification Using Cyclic Coordinate Descent Based Linear Support Vector Regression

Temperature prediction plays an important role in ring die granulator control,which can influence the quantity and quality of production. Temperature prediction modeling is a complicated problem with its MIMO, nonlinear, and large time-delay characteristics. Support vector machine( SVM) has been successfully based on small data. But its accuracy is not high,in contrast,if the number of data and dimension of feature increase,the training time of model will increase dramatically. In this paper,a linear SVM was applied combing with cyclic coordinate descent( CCD) to solving big data regression. It was mathematically strictly proved and validated by simulation. Meanwhile,real data were conducted to prove the linear SVM model's effect. Compared with other methods for big data in simulation, this algorithm has apparent advantage not only in fast modeling but also in high fitness.

Support vector classification algorithm based on variable parameter linear programming

To solve the problems of SVM in dealing with large sample size and asymmetric distributed samples, a support vector classification algorithm based on variable parameter linear programming is proposed. In the proposed algorithm, linear programming is employed to solve the optimization problem of classification to decrease the computation time and to reduce its complexity when compared with the original model. The adjusted punishment parameter greatly reduced the classification error resulting from asymmetric distributed samples and the detailed procedure of the proposed algorithm is given. An experiment is conducted to verify whether the proposed algorithm is suitable for asymmetric distributed samples.

Effective Classification of Synovial Sarcoma Cancer Using Structure Features and Support Vectors

In this research work,we proposed a medical image analysis framework with two separate releases whether or not Synovial Sarcoma(SS)is the cell structure for cancer.Within this framework the histopathology images are decomposed into a third-level sub-band using a two-dimensional Discrete Wavelet Transform.Subsequently,the structure features(SFs)such as PrincipalComponentsAnalysis(PCA),Independent ComponentsAnalysis(ICA)and Linear Discriminant Analysis(LDA)were extracted from this subband image representation with the distribution of wavelet coefficients.These SFs are used as inputs of the Support Vector Machine(SVM)classifier.Also,classification of PCA+SVM,ICA+SVM,and LDA+SVM with Radial Basis Function(RBF)kernel the efficiency of the process is differentiated and compared with the best classification results.Furthermore,data collected on the internet from various histopathological centres via the Internet of Things(IoT)are stored and shared on blockchain technology across a wide range of image distribution across secure data IoT devices.Due to this,the minimum and maximum values of the kernel parameter are adjusted and updated periodically for the purpose of industrial application in device calibration.Consequently,these resolutions are presented with an excellent example of a technique for training and testing the cancer cell structure prognosis methods in spindle shaped cell(SSC)histopathological imaging databases.The performance characteristics of cross-validation are evaluated with the help of the receiver operating characteristics(ROC)curve,and significant differences in classification performance between the techniques are analyzed.The combination of LDA+SVM technique has been proven to be essential for intelligent SS cancer detection in the future,and it offers excellent classification accuracy,sensitivity,specificity.

深埋长大隧道地温预测的机器学习算法对比研究

地热对隧道施工、工程结构及运营安全等均有较大的危害,随着我国基础设施建设布局西移,隧道建设的地质条件愈发复杂,隧道埋深和长度不断增加,隧道施工期高温热害问题频发。针对传统地温预测方法中预测精度不高、数据运用不充分,单一机器学习模型解译性差等问题,以A隧道为研究对象,将决策树(decision tree,DT)、支持向量机(support vector machine,SVM)、随机森林(random forest,RF)进行耦合,提出了基于DT-SVM-RF模型的深埋长大隧道地温预测方法。在分析隧道综合测井、地应力及岩石热物理试验、航空物探数据后,选取深度、声波波速等10个影响因子作为模型的输入,采用随机交叉验证和空间交叉验证对模型的鲁棒性、泛化能力进行检验,构建LASSO回归、随机森林、互信息3种回归模型,分析10个影响因子的特征重要性排序。结果表明:在测试集上多元线性回归、支持向量机、人工神经网络和决策树-支持向量机-随机森林(decision tree-support vector machinerandom forest,DT-SVM-RF)模型决定系数(R^(2))分别为0.76、0.91、0.88、0.93,均方误差MSE分别为17.64、6.25、8.46、5.20,DT-SVM-RF模型具有相对更优的预测性能,深度、岩石导温系数、岩石导热系数、最大水平主应力特征较为重要,说明DT-SVM-RF模型能有效地提高地温预测的准确率。研究结果可为类似隧道地温预测提供一种精度更高的可行新思路。

Predicting the Acute Toxicity of Aromatic Amines by Linear and Nonlinear Regression Methods

In current paper, a quantitative structure-activity relationship （QSAR） study was performed for the prediction of acute toxicity of aromatic amines. A set of 56 compounds was randomly divided into a training set of 46 compounds and a test set of 10 compounds. The electronic and topological descriptors computed by the Scigress package and Dragon software were used as predictor variables. Multiple linear regression （MLR） and support vector machine （SVM） were utilized to build the linear and nonlinear QSAR models, respectively. The obtained models with five descriptors show strong predictive ability. The linear model fits the training set with R2 = 0.71, with higher SVM values of R2 = 0.77. The validation results obtained from the test set indicate that the SVM model is comparable or superior to that obtained by MLR, both in terms of prediction ability and robustness.

Novel linear search for support vector machine parameter selection

Selecting the optimal parameters for support vector machine (SVM) has long been a hot research topic. Aiming for support vector classification/regression (SVC/SVR) with the radial basis function (RBF) kernel, we summarize the rough line rule of the penalty parameter and kernel width, and propose a novel linear search method to obtain these two optimal parameters. We use a direct-setting method with thresholds to set the epsilon parameter of SVR. The proposed method directly locates the right search field, which greatly saves computing time and achieves a stable, high accuracy. The method is more competitive for both SVC and SVR. It is easy to use and feasible for a new data set without any adjustments, since it requires no parameters to set.

Landslide susceptibility assessment in Western Henan Province based on a comparison of conventional and ensemble machine learning

Landslide is a serious natural disaster next only to earthquake and flood,which will cause a great threat to people’s lives and property safety.The traditional research of landslide disaster based on experience-driven or statistical model and its assessment results are subjective,difficult to quantify,and no pertinence.As a new research method for landslide susceptibility assessment,machine learning can greatly improve the landslide susceptibility model’s accuracy by constructing statistical models.Taking Western Henan for example,the study selected 16 landslide influencing factors such as topography,geological environment,hydrological conditions,and human activities,and 11 landslide factors with the most significant influence on the landslide were selected by the recursive feature elimination(RFE)method.Five machine learning methods[Support Vector Machines(SVM),Logistic Regression(LR),Random Forest(RF),Extreme Gradient Boosting(XGBoost),and Linear Discriminant Analysis(LDA)]were used to construct the spatial distribution model of landslide susceptibility.The models were evaluated by the receiver operating characteristic curve and statistical index.After analysis and comparison,the XGBoost model(AUC 0.8759)performed the best and was suitable for dealing with regression problems.The model had a high adaptability to landslide data.According to the landslide susceptibility map of the five models,the overall distribution can be observed.The extremely high and high susceptibility areas are distributed in the Funiu Mountain range in the southwest,the Xiaoshan Mountain range in the west,and the Yellow River Basin in the north.These areas have large terrain fluctuations,complicated geological structural environments and frequent human engineering activities.The extremely high and highly prone areas were 12043.3 km^(2)and 3087.45 km^(2),accounting for 47.61%and 12.20%of the total area of the study area,respectively.Our study reflects the distribution of landslide susceptibility in western Henan Province,which provides a scientific basis for regional disaster warning,prediction,and resource protection.The study has important practical significance for subsequent landslide disaster management.

Forecasting Flowering and Maturity Times of Barley Using Six Machine Learning Algorithms

Interior Alaska has a short growing season of 110 d.The knowledge of timings of crop flowering and maturity will provide the information for the agricultural decision making.In this study,six machine learning algorithms,namely Linear Discriminant Analysis(LDA),Support Vector Machines(SVMs),k-nearest neighbor(kNN),Naïve Bayes(NB),Recursive Partitioning and Regression Trees(RPART),and Random Forest(RF),were selected to forecast the timings of barley flowering and maturity based on the Alaska Crop Datasets and climate data from 1991 to 2016 in Fairbanks,Alaska.Among 32 models fit to forecast flowering time,two from LDA,12 from SVMs,four from NB,three from RF outperformed models from other algorithms with the highest accuracy.Models from kNN performed worst to forecast flowering time.Among 32 models fit to forecast maturity time,two models from LDA outperformed the models from other algorithms.Models from kNN and RPART performed worst to forecast maturity time.Models from machine learning methods also provided a variable importance explanation.In this study,four out of six algorithms gave the same variable importance order.Sowing date was the most important variable to forecast flowering but less important variable to forecast maturity.The daily maximum temperature may be more important than daily minimum temperature to fit flowering models while daily minimum temperature may be more important than daily maximum temperature to fit maturity models.The results indicate that models from machine learning provide a promising technique in forecasting the timings of flowering and maturity of barley.

Recognition for avian influenza virus proteins based on support vector machine and linear discriminant analysis

Total 200 properties related to structural characteristics were employed to represent structures of 400 HA coded proteins of influenza virus as training samples. Some recognition models for HA proteins of avian influenza virus (AIV) were developed using support vector machine (SVM) and linear discriminant analysis (LDA). The results obtained from LDA are as follows: the identification accuracy (Ria) for training samples is 99.8% and Ria by leave one out cross validation is 99.5%. Both Ria of 99.8% for training samples and Ria of 99.3% by leave one out cross validation are obtained using SVM model, respectively. External 200 HA proteins of influenza virus were used to validate the external predictive power of the resulting model. The external Ria for them is 95.5% by LDA and 96.5% by SVM, respectively, which shows that HA proteins of AIVs are preferably recognized by SVM and LDA, and the performances by SVM are superior to those by LDA.

A Highly Accurate Dysphonia Detection System Using Linear Discriminant Analysis

The recognition of pathological voice is considered a difficult task for speech analysis.Moreover,otolaryngologists needed to rely on oral communication with patients to discover traces of voice pathologies like dysphonia that are caused by voice alteration of vocal folds and their accuracy is between 60%–70%.To enhance detection accuracy and reduce processing speed of dysphonia detection,a novel approach is proposed in this paper.We have leveraged Linear Discriminant Analysis(LDA)to train multiple Machine Learning(ML)models for dysphonia detection.Several ML models are utilized like Support Vector Machine(SVM),Logistic Regression,and K-nearest neighbor(K-NN)to predict the voice pathologies based on features like Mel-Frequency Cepstral Coefficients(MFCC),Fundamental Frequency(F0),Shimmer(%),Jitter(%),and Harmonic to Noise Ratio(HNR).The experiments were performed using Saarbrucken Voice Data-base(SVD)and a privately collected dataset.The K-fold cross-validation approach was incorporated to increase the robustness and stability of the ML models.According to the experimental results,our proposed approach has a 70%increase in processing speed over Principal Component Analysis(PCA)and performs remarkably well with a recognition accuracy of 95.24%on the SVD dataset surpassing the previous best accuracy of 82.37%.In the case of the private dataset,our proposed method achieved an accuracy rate of 93.37%.It can be an effective non-invasive method to detect dysphonia.

Linear SVM在大数据分类中的应用

线性支持向量机是处理高维稀疏数据的有效机器学习方法之一,本文对线性支持向量机与传统支持向量机在解决大规模数据时的训练时间和分类准确率进行了对比分析,基于三个不同规模的数据集分别进行了分类实验,结果表明性支持向量机在训练速度上具有明显的优势,分类准确率也高于传统支持向量机。

Identification of Question and Non-Question Segments in Arabic Monologues Using Prosodic Features: Novel Type-2 Fuzzy Logic and Sensitivity-Based Linear Learning Approaches

In this paper, we extend our previous study of addressing the important problem of automatically identifying question and non-question segments in Arabic monologues using prosodic features. We propose here two novel classification approaches to this problem: one based on the use of the powerful type-2 fuzzy logic systems (type-2 FLS) and the other on the use of the discriminative sensitivity-based linear learning method (SBLLM). The use of prosodic features has been used in a plethora of practical applications, including speech-related applications, such as speaker and word recognition, emotion and accent identification, topic and sentence segmentation, and text-to-speech applications. In this paper, we continue to specifically focus on the Arabic language, as other languages have received a lot of attention in this regard. Moreover, we aim to improve the performance of our previously-used techniques, of which the support vector machine (SVM) method was the best performing, by applying the two above-mentioned powerful classification approaches. The recorded continuous speech is first segmented into sentences using both energy and time duration parameters. The prosodic features are then extracted from each sentence and fed into each of the two proposed classifiers so as to classify each sentence as a Question or a Non-Question sentence. Our extensive simulation work, based on a moderately-sized database, showed the two proposed classifiers outperform SVM in all of the experiments carried out, with the type-2 FLS classifier consistently exhibiting the best performance, because of its ability to handle all forms of uncertainties.

基于特征融合和B-SVM的鸟鸣声识别算法

为了实现在野外通过低成本嵌入式系统识别鸟类,提出了基于特征融合和B-SVM的鸟鸣声识别方法。对鸟鸣声信号提取梅尔频率倒谱系数、翻转梅尔频率倒谱系数、短时能量和短时过零率组成特征参数,通过线性判别算法对特征参数进行特征融合。利用黑寡妇算法通过测试集对支持向量机模型的核参数和损失值进行优化得到B-SVM模型。利用Xeno-canto鸟鸣声数据集对本文算法进行了测试,结果表明该方法的识别准确率为93.23%。算法维度参数的大小和融合特征维度的高低是影响算法识别效果的重要因素。在相同条件下,文中所提的基于特征融合和B-SVM模型的鸟鸣声识别算法相较于其他特征参数和模型,识别的准确率更高,为野外鸟类识别提供了参考。

微胶囊相变材料改良粉砂土的导热系数及预测模型

【目的】针对季节冻土地区渠道冻融破坏,分析微胶囊相变材料(microencapsulated phase change materials,mPCM)改良粉砂土层渠基的温度场,对改良粉砂土的导热系数进行研究。【方法】以mPCM为改良剂,掺入渠基粉砂土形成mPCM改良粉砂土;对mPCM改良粉砂土进行导热系数实验和内部结构表征;采用多元线性回归和支持向量机(support vector machine,SVM)方法分别建立mPCM改良粉砂土的导热系数预测模型。【结果】mPCM改良粉砂土导热系数与含水率、干密度、mPCM掺量有关,且受冰水相对含量、冰水相变潜热、mPCM相变潜热和mPCM填充密实作用的影响,具有明显的温度效应;mPCM改良粉砂土导热系数的变化与实验温度和mPCM相变温度有关,可分为快速降低、缓慢降低和逐步上升3个阶段;多元线性回归和SVM模型均能较好地拟合预测mPCM改良粉砂土的导热系数,但SVM模型更适用于表征mPCM改良粉砂土导热系数各影响因素间的非线性关系。【结论】mPCM改良粉砂土的导热系数提高能够有效调控渠基土温度场,减轻渠道冻害,且SVM模型能更加准确地进行导热系数预测。

选煤厂大煤块智能图像识别方法研究

针对现有选煤厂大煤块识别受到大煤块与矸石线性不可分影响,导致识别效率低、识别精度差的问题,设计了一种选煤厂大煤块智能图像识别方法。首先,应用图像采集设备获取煤块混合图像,并对图像进行增强、去噪等操作,以完成图像预处理。然后,计算煤块的面积和厚度,以获取煤块与矸石的物质特征。最后,基于煤块与矸石的X射线衰减曲线完成识别阈值的设定,并结合最小二乘支持向量机解决煤块与矸石的线性不可分问题,从而完成大煤块图像的智能识别。对比试验结果表明,所提方法应用效果较好、识别率及识别精度较高、识别速度较快,总体性能优于对比方法。该方法可大幅提升大煤块图像识别效果,有较高的应用价值。

基于近红外光谱技术的六大茶类快速识别

为构建高质量的六大茶类识别模型,本研究中收集了370份样品,通过采集其近红外光谱(near-infrared spectroscopy,NIRS),结合光谱预处理、特征提取以及数据挖掘分类器算法,建立六大茶类快速识别模型。结果表明:1)支持向量机(support vector machine,SVM)与随机森林(random forest,RF)分类器皆适于六大茶类快速识别模型的构建;2)SVM分类器更适于结合原始光谱(original spectrum,OS)建模,预处理易使基于该分类器建立的模型鉴别性能减弱;3)随机森林(RF)分类器更适用于预处理后光谱建模,所得模型较OS模型在识别正确率(recognition accuracy,RA)及受试者工作特征曲线下面积(area under the curve,AUC)均得到明显提升;4)特征提取中线性判别分析(linear discriminant analysis,LDA)算法表现最好,所得模型的RA较OS模型明显提升,其中最佳模型OS-LDA-SVM的RA为100.00%,AUC为1.00,识别正确率高、泛化能力强、模型性能优异,可产业化应用。综上所述,近红外光谱结合预处理、特征提取算法及分类器建立模型,进行六大茶类识别的可行性强,模型的识别正确率高、性能优异,可为茶叶贸易的茶类快速识别提供科学、准确、高效的技术支撑,为国际茶类识别模型的产业化应用奠定基础。