Learning Vector Quantization Neural Network Method for Network Intrusion Detection

A new intrusion detection method based on learning vector quantization （LVQ） with low overhead and high efficiency is presented. The computer vision system employs LVQ neural networks as classifier to recognize intrusion. The recognition process includes three stages： （1） feature selection and data normalization processing;（2） learning the training data selected from the feature data set; （3） identifying the intrusion and generating the result report of machine condition classification. Experimental results show that the proposed method is promising in terms of detection accuracy, computational expense and implementation for intrusion detection.

Reinforcement Learning Based Quantization Strategy Optimal Assignment Algorithm for Mixed Precision

The quantization algorithm compresses the original network by reducing the numerical bit width of the model,which improves the computation speed. Because different layers have different redundancy and sensitivity to databit width. Reducing the data bit width will result in a loss of accuracy. Therefore, it is difficult to determinethe optimal bit width for different parts of the network with guaranteed accuracy. Mixed precision quantizationcan effectively reduce the amount of computation while keeping the model accuracy basically unchanged. In thispaper, a hardware-aware mixed precision quantization strategy optimal assignment algorithm adapted to low bitwidth is proposed, and reinforcement learning is used to automatically predict the mixed precision that meets theconstraints of hardware resources. In the state-space design, the standard deviation of weights is used to measurethe distribution difference of data, the execution speed feedback of simulated neural network accelerator inferenceis used as the environment to limit the action space of the agent, and the accuracy of the quantization model afterretraining is used as the reward function to guide the agent to carry out deep reinforcement learning training. Theexperimental results show that the proposed method obtains a suitable model layer-by-layer quantization strategyunder the condition that the computational resources are satisfied, and themodel accuracy is effectively improved.The proposed method has strong intelligence and certain universality and has strong application potential in thefield of mixed precision quantization and embedded neural network model deployment.

Machine Learning and Artificial Neural Network for Predicting Heart Failure Risk

Heart failure is now widely spread throughout the world.Heart disease affects approximately 48%of the population.It is too expensive and also difficult to cure the disease.This research paper represents machine learning models to predict heart failure.The fundamental concept is to compare the correctness of various Machine Learning(ML)algorithms and boost algorithms to improve models’accuracy for prediction.Some supervised algorithms like K-Nearest Neighbor(KNN),Support Vector Machine(SVM),Decision Trees(DT),Random Forest(RF),Logistic Regression(LR)are considered to achieve the best results.Some boosting algorithms like Extreme Gradient Boosting(XGBoost)and Cat-Boost are also used to improve the prediction using Artificial Neural Networks(ANN).This research also focuses on data visualization to identify patterns,trends,and outliers in a massive data set.Python and Scikit-learns are used for ML.Tensor Flow and Keras,along with Python,are used for ANN model train-ing.The DT and RF algorithms achieved the highest accuracy of 95%among the classifiers.Meanwhile,KNN obtained a second height accuracy of 93.33%.XGBoost had a gratified accuracy of 91.67%,SVM,CATBoost,and ANN had an accuracy of 90%,and LR had 88.33%accuracy.

Identification of dynamic systems using support vector regression neural networks

A novel adaptive support vector regression neural network （SVR-NN） is proposed, which combines respectively merits of support vector machines and a neural network. First, a support vector regression approach is applied to determine the initial structure and initial weights of the SVR-NN so that the network architecture is easily determined and the hidden nodes can adaptively be constructed based on support vectors. Furthermore, an annealing robust learning algorithm is presented to adjust these hidden node parameters as well as the weights of the SVR-NN. To test the validity of the proposed method, it is demonstrated that the adaptive SVR-NN can be used effectively for the identification of nonlinear dynamic systems. Simulation results show that the identification schemes based on the SVR-NN give considerably better performance and show faster learning in comparison to the previous neural network method.

基于LVQ神经网络的水果图像分割研究

由于传统边沿检测算子在水果颜色多样、亮度不均匀时,难以分割得到完整、无噪声的二值图像且依赖优化的阈值,本研究提出了一种基于LVQ神经网络的水果图像分割方案。首先将彩色图像转变为灰度图像;然后对Canny算子获得的边沿图像随机选取一些像素作为网络的学习监督信号,仅以灰度图像中相同位置像素3×3邻域的Kirsch算子梯度值作为输入,训练权值;最后重新将原灰度图像的Kirsch算子梯度值输入到训练好的网络中,获得封闭的边沿并填充得到二值图像。考察了14幅像素为640×480的水果图像,结果表明:网络在很宽广的阈值范围内(0.001~0.99)分割得到完整、一致的二值图像;面积误差最小为0.9%,最大为8.83%,不依赖于优化的阈值,不需要对原始图像滤波预处理。与没有阈值及滤波的算法相比,本方案的误差和时间复杂度均更低;与设置了阈值和/或滤波的算法相比,本方案与之相当,甚至效果更优。

A Comparative Study of Support Vector Machine and Artificial Neural Network for Option Price Prediction

Option pricing has become one of the quite important parts of the financial market. As the market is always dynamic, it is really difficult to predict the option price accurately. For this reason, various machine learning techniques have been designed and developed to deal with the problem of predicting the future trend of option price. In this paper, we compare the effectiveness of Support Vector Machine (SVM) and Artificial Neural Network (ANN) models for the prediction of option price. Both models are tested with a benchmark publicly available dataset namely SPY option price-2015 in both testing and training phases. The converted data through Principal Component Analysis (PCA) is used in both models to achieve better prediction accuracy. On the other hand, the entire dataset is partitioned into two groups of training (70%) and test sets (30%) to avoid overfitting problem. The outcomes of the SVM model are compared with those of the ANN model based on the root mean square errors (RMSE). It is demonstrated by the experimental results that the ANN model performs better than the SVM model, and the predicted option prices are in good agreement with the corresponding actual option prices.

A STUDY OF METHODS FOR IMPROVING LEARNING VECTOR QUANTIZATION

Learning Vector Quantization(LVQ)originally proposed by Kohonen(1989)is aneurally-inspired classifier which pays attention to approximating the optimal Bayes decisionboundaries associated with a classification task.With respect to several defects of LVQ2 algorithmstudied in this paper,some‘soft’competition schemes such as‘majority voting’scheme andcredibility calculation are proposed for improving the ability of classification as well as the learningspeed.Meanwhile,the probabilities of winning are introduced into the corrections for referencevectors in the‘soft’competition.In contrast with the conventional sequential learning technique,a novel parallel learning technique is developed to perform LVQ2 procedure.Experimental resultsof speech recognition show that these new approaches can lead to better performance as comparedwith the conventional

基于相空间重构与GSA-LVQ的有载调压变压器分接开关机械故障诊断

针对有载调压变压器分接开关机械故障诊断准确率不高以及潜在机械故障不能及时被发现的问题,提出了一种基于互补集合经验模态分解(CEEMD)、相空间重构结合万有引力搜索法(GSA)改进学习矢量量化神经网络(LVQ)的有载分接开关机械故障诊断新方法。采用CEEMD对振动信号进行时频域分解,然后通过C-C算法确定延迟时间和嵌入维数,对反映不同频率特征的固有模态函数(IMF)进行相空间重构,并提取反映混沌特征的两个特征量李雅普诺夫指数和关联维数构成特征向量。利用GSA优化LVQ,解决网络对初始连接权值敏感的问题,增强网络对有载分接开关机械故障分类识别性能。通过对有载分接开关机械状态的实验分析,证明了相空间重构结合GSA-LVQ算法的可行性和有效性。

Using Neural Networks to Predict Secondary Structure for Protein Folding

Protein Secondary Structure Prediction (PSSP) is considered as one of the major challenging tasks in bioinformatics, so many solutions have been proposed to solve that problem via trying to achieve more accurate prediction results. The goal of this paper is to develop and implement an intelligent based system to predict secondary structure of a protein from its primary amino acid sequence by using five models of Neural Network (NN). These models are Feed Forward Neural Network (FNN), Learning Vector Quantization (LVQ), Probabilistic Neural Network (PNN), Convolutional Neural Network (CNN), and CNN Fine Tuning for PSSP. To evaluate our approaches two datasets have been used. The first one contains 114 protein samples, and the second one contains 1845 protein samples.

Application of Feature Extraction through Convolution Neural Networks and SVM Classifier for Robust Grading of Apples

This paper proposes a novel grading method of apples,in an automated grading device that uses convolutional neural networks to extract the size,color,texture,and roundness of an apple.The developed machine learning method uses the ability of learning representative features by means of a convolutional neural network(CNN),to determine suitable features of apples for the grading process.This information is fed into a one-to-one classifier that uses a support vector machine(SVM),instead of the softmax output layer of the CNN.In this manner,Yantai apples with similar shapes and low discrimination are graded using four different approaches.The fusion model using both CNN and SVM classifiers is much more accurate than the simple k-nearest neighbor(KNN),SVM,and CNN model when used separately for grading,and the learning ability and the generalization ability of the model is correspondingly increased by the combined method.Grading tests are carried out using the automated grading device that is developed in the present work.It is verified that the actual effect of apple grading using the combined CNN-SVM model is fast and accurate,which greatly reduces the manpower and labor costs of manual grading,and has important commercial prospects.

State of the art in applications of machine learning in steelmaking process modeling

With the development of automation and informatization in the steelmaking industry,the human brain gradually fails to cope with an increasing amount of data generated during the steelmaking process.Machine learning technology provides a new method other than production experience and metallurgical principles in dealing with large amounts of data.The application of machine learning in the steelmaking process has become a research hotspot in recent years.This paper provides an overview of the applications of machine learning in the steelmaking process modeling involving hot metal pretreatment,primary steelmaking,secondary refining,and some other aspects.The three most frequently used machine learning algorithms in steelmaking process modeling are the artificial neural network,support vector machine,and case-based reasoning,demonstrating proportions of 56%,14%,and 10%,respectively.Collected data in the steelmaking plants are frequently faulty.Thus,data processing,especially data cleaning,is crucially important to the performance of machine learning models.The detection of variable importance can be used to optimize the process parameters and guide production.Machine learning is used in hot metal pretreatment modeling mainly for endpoint S content prediction.The predictions of the endpoints of element compositions and the process parameters are widely investigated in primary steelmaking.Machine learning is used in secondary refining modeling mainly for ladle furnaces,Ruhrstahl–Heraeus,vacuum degassing,argon oxygen decarburization,and vacuum oxygen decarburization processes.Further development of machine learning in the steelmaking process modeling can be realized through additional efforts in the construction of the data platform,the industrial transformation of the research achievements to the practical steelmaking process,and the improvement of the universality of the machine learning models.

Predicting Stock Movement Using Sentiment Analysis of Twitter Feed with Neural Networks

External factors, such as social media and financial news, can have wide-spread effects on stock price movement. For this reason, social media is considered a useful resource for precise market predictions. In this paper, we show the effectiveness of using Twitter posts to predict stock prices. We start by training various models on the Sentiment 140 Twitter data. We found that Support Vector Machines (SVM) performed best (0.83 accuracy) in the sentimental analysis, so we used it to predict the average sentiment of tweets for each day that the market was open. Next, we use the sentimental analysis of one year’s data of tweets that contain the “stock market”, “stocktwits”, “AAPL” keywords, with the goal of predicting the corresponding stock prices of Apple Inc. (AAPL) and the US’s Dow Jones Industrial Average (DJIA) index prices. Two models, Boosted Regression Trees and Multilayer Perceptron Neural Networks were used to predict the closing price difference of AAPL and DJIA prices. We show that neural networks perform substantially better than traditional models for stocks’ price prediction.

Total organic carbon content logging prediction based on machine learning:A brief review

The total organic carbon content usually determines the hydrocarbon generation potential of a formation.A higher total organic carbon content often corresponds to a greater possibility of generating large amounts of oil or gas.Hence,accurately calculating the total organic carbon content in a formation is very important.Present research is focused on precisely calculating the total organic carbon content based on machine learning.At present,many machine learning methods,including backpropagation neural networks,support vector regression,random forests,extreme learning machines,and deep learning,are employed to evaluate the total organic carbon content.However,the principles and perspectives of various machine learning algorithms are quite different.This paper reviews the application of various machine learning algorithms to deal with total organic carbon content evaluation problems.Of various machine learning algorithms used for TOC content predication,two algorithms,the backpropagation neural network and support vector regression are the most commonly used,and the backpropagation neural network is sometimes combined with many other algorithms to achieve better results.Additionally,combining multiple algorithms or using deep learning to increase the number of network layers can further improve the total organic carbon content prediction.The prediction by backpropagation neural network may be better than that by support vector regression;nevertheless,using any type of machine learning algorithm improves the total organic carbon content prediction in a given research block.According to some published literature,the determination coefficient(R^(2))can be increased by up to 0.46 after using machine learning.Deep learning algorithms may be the next breakthrough direction that can significantly improve the prediction of the total organic carbon content.Evaluating the total organic carbon content based on machine learning is of great significance.

Machine-Learning-Enabled Obesity Level Prediction Through Electronic Health Records

Obesity is a critical health condition that severely affects an individual’s quality of life andwell-being.The occurrence of obesity is strongly associated with extreme health conditions,such as cardiac diseases,diabetes,hypertension,and some types of cancer.Therefore,it is vital to avoid obesity and or reverse its occurrence.Incorporating healthy food habits and an active lifestyle can help to prevent obesity.In this regard,artificial intelligence(AI)can play an important role in estimating health conditions and detecting obesity and its types.This study aims to see obesity levels in adults by implementing AIenabled machine learning on a real-life dataset.This dataset is in the form of electronic health records(EHR)containing data on several aspects of daily living,such as dietary habits,physical conditions,and lifestyle variables for various participants with different health conditions(underweight,normal,overweight,and obesity type I,II and III),expressed in terms of a variety of features or parameters,such as physical condition,food intake,lifestyle and mode of transportation.Three classifiers,i.e.,eXtreme gradient boosting classifier(XGB),support vector machine(SVM),and artificial neural network(ANN),are implemented to detect the status of several conditions,including obesity types.The findings indicate that the proposed XGB-based system outperforms the existing obesity level estimation methods,achieving overall performance rates of 98.5%and 99.6%in the scenarios explored.

基于机器学习的“一带一路”投资国别风险预测研究

“一带一路”倡议提出十年间,中国对沿线国家的投资规模持续扩大。然而,企业在抓住机遇,进行“一带一路”沿线国家投资的同时,也需要重点关注“一带一路”投资国别风险。本文从政治、经济、社会和对华关系4个维度构建“一带一路”投资国别风险预测指标体系;运用灰色关联分析计算样本国家的综合风险评价值;基于2012~2022年间“一带一路”沿线国家的数据,利用机器学习构建GA-BP神经网络、支持向量回归和随机森林3种预测模型;通过对比预测精度,确定最佳预测模型,利用2021年的指标数据,对2022年的投资国别风险进行预测。研究结果表明:(1)在“一带一路”投资国别风险的研究背景下,支持向量回归模型预测效果最优,证明机器学习模型能够有效应用于风险管理领域;(2)“一带一路”投资国别风险存在明显的地区差异,中东欧地区和东南亚地区投资国别风险普遍较低,而南亚地区投资国别风险普遍较高,但都存在特例。本文研究结果可为“走出去”企业在“一带一路”沿线国家的投资决策提供参考。

神经网络压缩联合优化方法的研究综述

随着人工智能应用的实时性、隐私性和安全性需求增大,在边缘计算平台上部署高性能的神经网络成为研究热点。由于常见的边缘计算平台在存储、算力、功耗上均存在限制,因此深度神经网络的端侧部署仍然是一个巨大的挑战。目前,克服上述挑战的一个思路是对现有的神经网络压缩以适配设备部署条件。现阶段常用的模型压缩算法有剪枝、量化、知识蒸馏,多种方法优势互补同时联合压缩可实现更好的压缩加速效果,正成为研究的热点。本文首先对常用的模型压缩算法进行简要概述,然后总结了“知识蒸馏+剪枝”、“知识蒸馏+量化”和“剪枝+量化”3种常见的联合压缩算法,重点分析论述了联合压缩的基本思想和方法,最后提出了神经网络压缩联合优化方法未来的重点发展方向。

基于CNN-SVM和集成学习的固井质量评价方法

为解决固井质量评价问题,提出一种基于CNN-SVM和集成学习的固井质量评价方法.首先,针对DenseNet模型采取缩减网络层数、增加多尺度卷积层、嵌入卷积注意力模块等改进措施,以提高模型的训练速度和评价准确率;其次,利用InceptionV1模块和扩张卷积构建一个模型复杂度相对较小且评价准确率相对较高的Inception-DCNN模型;再次,优选3个经典的卷积神经网络模型(ResNet50,MobileNetV3-Small, GhostNet),利用卷积神经网络强大的特征提取能力及支持向量机的结构风险最小化能力,将上述模型分别与支持向量机组合成新的CNN-SVM模型,以提升模型的泛化能力;最后,采用Bagging方式将5个新的CNN-SVM模型集成为一个强学习器,从而提升评价结果的准确度,增强模型的抗干扰能力.实验结果表明,该方法对测试集中的3类评价样本的准确率为97.69%,与单个模型和其他方法相比提升了1~9个百分点,验证了采用基于CNN-SVM和集成学习的方法进行固井质量评价是切实可行的.

基于字符和词特征融合的恶意域名检测

针对现有恶意域名检测方法对域名生成算法(domain generation algorithm, DGA)随机产生的恶意域名检测性能不高,且对由随机单词组成的恶意域名检测效果较差的问题,提出一种基于字符和词特征融合的恶意域名检测算法(cha-racter and word network, CWNet)。利用并行卷积神经网络分别提取域名中字符和词的特征;将两种特征进行拼接,构造成融合特征;利用Softmax函数实现合法域名与恶意域名的检测。实验结果表明,该算法可以提升对恶意域名的检测能力,对更具挑战性的恶意域名家族的检测准确率提升效果更为明显。

基于改进两步法采样策略和卷积神经网络的崩塌易发性评价

机器学习在崩塌滑坡泥石流地质灾害易发性分析评价领域已得到广泛的研究性应用,非灾害样本的选取是易发性建模过程中的关键问题,传统随机抽样和手工标注方法可能存在随机性和主观性。将土质崩塌易发性评价视为正例无标记(positive and unlabeled,简称PU)学习,提出了一种结合信息量(information value,简称IV)和间谍技术(Spy)的两步卷积神经网络(convolutional neural networks,简称CNN)框架(ISpy-CNN)。以广州市黄埔区崩塌编录和15类基础环境因子,通过信息量模型筛选出部分低信息量样本;采用间谍技术训练CNN模型,从低信息量样本中识别出具有高置信度的可靠负例划分为非崩塌样本;分别基于该学习框架、传统间谍技术和随机抽样,使用支持向量机(support vector machine,简称SVM)和随机森林(random forest,简称RF)对比验证。结果表明,ISpy-CNN框架在验证集上的准确率、F1值、敏感度和特异度较随机采样分别提升了6.82%,6.82%,6.82%,8.23%,较传统Spy技术分别提升了2.86%,2.89%,2.86%,2.31%;PU学习中第2步采用CNN模型的预测精度高于RF和SVM模型;与传统Spy技术相比,增加相同数量训练样本,ISpy-CNN框架筛选的样本集表现出较高的稳定性、预测精度和增长率。本研究提出的ISpy-CNN框架能更好地辅助选取高质量非灾害样本,且崩塌易发性分区结果更符合实际的崩塌空间分布。

结合遗传算法的LVQ神经网络在声学底质分类中的应用

学习向量量化(Learning Vector Quantization,LVQ)神经网络在声学底质分类中具有广泛应用.常用的LVQ神经网络存在神经元未被充分利用以及算法对初值敏感的问题,影响底质分类精度.本文提出采用遗传算法(Genetic Algorithms,GA)优化神经网络的初始值,将GA与LVQ神经网络结合起来,迅速得到最佳的神经网络初始权值向量,实现对海底基岩、砾石、砂、细砂以及泥等底质类型的快速、准确识别.将其应用于青岛胶州湾海区底质分类识别研究中,通过与标准的LVQ神经网络的分类结果进行比较表明,该方法在分类速度以及精度上都有了较大提高.