New Spam Filtering Method with Hadoop Tuning-Based MapReduce Naïve Bayes

As the importance of email increases,the amount of malicious email is also increasing,so the need for malicious email filtering is growing.Since it is more economical to combine commodity hardware consisting of a medium server or PC with a virtual environment to use as a single server resource and filter malicious email using machine learning techniques,we used a Hadoop MapReduce framework and Naïve Bayes among machine learning methods for malicious email filtering.Naïve Bayes was selected because it is one of the top machine learning methods(Support Vector Machine(SVM),Naïve Bayes,K-Nearest Neighbor(KNN),and Decision Tree)in terms of execution time and accuracy.Malicious email was filtered with MapReduce programming using the Naïve Bayes technique,which is a supervised machine learning method,in a Hadoop framework with optimized performance and also with the Python program technique with the Naïve Bayes technique applied in a bare metal server environment with the Hadoop environment not applied.According to the results of a comparison of the accuracy and predictive error rates of the two methods,the Hadoop MapReduce Naïve Bayes method improved the accuracy of spam and ham email identification 1.11 times and the prediction error rate 14.13 times compared to the non-Hadoop Python Naïve Bayes method.

Attribute Weighted Naïve Bayes Classifier

The naïve Bayes classifier is one of the commonly used data mining methods for classification.Despite its simplicity,naïve Bayes is effective and computationally efficient.Although the strong attribute independence assumption in the naïve Bayes classifier makes it a tractable method for learning,this assumption may not hold in real-world applications.Many enhancements to the basic algorithm have been proposed in order to alleviate the violation of attribute independence assumption.While these methods improve the classification performance,they do not necessarily retain the mathematical structure of the naïve Bayes model and some at the expense of computational time.One approach to reduce the naïvetéof the classifier is to incorporate attribute weights in the conditional probability.In this paper,we proposed a method to incorporate attribute weights to naïve Bayes.To evaluate the performance of our method,we used the public benchmark datasets.We compared our method with the standard naïve Bayes and baseline attribute weighting methods.Experimental results show that our method to incorporate attribute weights improves the classification performance compared to both standard naïve Bayes and baseline attribute weighting methods in terms of classification accuracy and F1,especially when the independence assumption is strongly violated,which was validated using the Chi-square test of independence.

Integration of Expectation Maximization using Gaussian Mixture Models and Naïve Bayes for Intrusion Detection

Intrusion detection is the investigation process of information about the system activities or its data to detect any malicious behavior or unauthorized activity.Most of the IDS implement K-means clustering technique due to its linear complexity and fast computing ability.Nonetheless,it is Naïve use of the mean data value for the cluster core that presents a major drawback.The chances of two circular clusters having different radius and centering at the same mean will occur.This condition cannot be addressed by the K-means algorithm because the mean value of the various clusters is very similar together.However,if the clusters are not spherical,it fails.To overcome this issue,a new integrated hybrid model by integrating expectation maximizing(EM)clustering using a Gaussian mixture model(GMM)and naïve Bays classifier have been proposed.In this model,GMM give more flexibility than K-Means in terms of cluster covariance.Also,they use probabilities function and soft clustering,that’s why they can have multiple cluster for a single data.In GMM,we can define the cluster form in GMM by two parameters:the mean and the standard deviation.This means that by using these two parameters,the cluster can take any kind of elliptical shape.EM-GMM will be used to cluster data based on data activity into the corresponding category.

Naïve Bayes Algorithm for Large Scale Text Classification

This paper proposed an improved Naïve Bayes Classifier for sentimental analysis from a large-scale dataset such as in YouTube.YouTube contains large unstructured and unorganized comments and reactions,which carry important information.Organizing large amounts of data and extracting useful information is a challenging task.The extracted information can be considered as new knowledge and can be used for deci sion-making.We extract comments from YouTube on videos and categorized them in domain-specific,and then apply the Naïve Bayes classifier with improved techniques.Our method provided a decent 80%accuracy in classifying those comments.This experiment shows that the proposed method provides excellent adaptability for large-scale text classification.

Nave Bayes分类器制导的专业网页爬取算法

从Web中快速、准确地检索出所需信息的迫切需求催生了专业搜索引擎技术。在专业搜索引擎中,网络爬虫(Crawler)负责在Web上搜集特定专业领域的信息,是专业搜索引擎的重要核心部件。该文对中文专业网页的爬取问题进行了研究,基于KL距离验证了网页内容与链接前后文在分布上的差异,在此基础上提出了以链接锚文本及其前后文为特征、Nave Bayes分类器制导的中文专业网页爬取算法,设计了自动获取带链接类标的训练数据的算法。以金融专业网页的爬取为例,分别对所提出的算法进行了离线和在线测试,结果表明,Nave Bayes分类器制导的网络爬虫可以达到近90%的专业网页收割率。

结合特征和非特征信息改进Nave Bayes及其应用

朴素贝叶斯算法是一种常见的基于内容的垃圾邮件过滤算法,但是,传统朴素贝叶斯过滤存在判断内容的不确定性和邮件表示不完整性等问题。分析邮件信头各域在正常邮件和垃圾邮件中表现出的不同属性,提取非特征信息,结合特征信息和非特征信息改进朴素贝叶斯算法。实验结果表明,改进的朴素贝叶斯分类方法与单纯使用特征信息的方法相比,垃圾邮件的召回率和准确率更高,凸显了该方法涵盖邮件信息、克服内容判断缺陷的优势。

基于改进的Nave Bayes和BP神经网络的垃圾邮件过滤

不同用户对垃圾邮件的判定有所差别,考虑到同一用户的自认垃圾邮件相似度较大,提出对特定用户进行针对性的垃圾邮件过滤方法.系统除重点利用邮件正文信息外,还尝试加入发件人、群发信息和主题相关度信息,改进朴素贝叶斯公式用于邮件正文的概率计算,基于BP神经网络构造垃圾邮件判别系统.实验表明,改进的朴素贝叶斯公式用于本文的系统是可行的,基于BP神经网络的垃圾邮件过滤系统能有效综合以上四项数值进行全局判别,进而对特定用户的邮件产生不错的过滤效果.

Improved Bearing Fault Diagnosis by Feature Extraction Based on GLCM, Fusion of Selection Methods, and Multiclass-Naïve Bayes Classification

The presence of bearing faults reduces the efficiency of rotating machines and thus increases energy consumption or even the total stoppage of the machine. It becomes essential to correctly diagnose the fault caused by the bearing. Hence the importance of determining an effective features extraction method that best describes the fault. The vision of this paper is to merge the features selection methods in order to define the most relevant featuresin the texture of the vibration signal images. In this study, the Gray Level Co-occurrence Matrix (GLCM) in texture analysis is applied on the vibration signal represented in images. Features selection based on the merge of PCA (Principal component Analysis) method and SFE (Sequential Features Extraction) method is done to obtain the most relevant features. The multiclass-Na?ve Bayesclassifier is used to test the proposed approach. The success rate of this classification is 98.27%. The relevant features obtained give promising results and are more efficient than the methods observed in the literature.

Predicting crash injury severity at unsignalized intersections using support vector machines and naïve Bayes classifiers

The Washington,DC crash statistic report for the period from 2013 to 2015 shows that the city recorded about 41789 crashes at unsignalized intersections,which resulted in 14168 injuries and 51 fatalities.The economic cost of these fatalities has been estimated to be in the millions of dollars.It is therefore necessary to investigate the predictability of the occurrence of theses crashes,based on pertinent factors,in order to provide mitigating measures.This research focused on the development of models to predict the injury severity of crashes using support vector machines(SVMs)and Gaussian naïve Bayes classifiers(GNBCs).The models were developed based on 3307 crashes that occurred from 2008 to 2015.Eight SVM models and a GNBC model were developed.The most accurate model was the SVM with a radial basis kernel function.This model predicted the severity of an injury sustained in a crash with an accuracy of approximately 83.2%.The GNBC produced the worst-performing model with an accuracy of 48.5%.These models will enable transport officials to identify crash-prone unsignalized intersections to provide the necessary countermeasures beforehand.

Identification of maize(Zea mays L.)progeny genotypes based on two probabilistic approaches:Logistic regression and naïve Bayes

Weused two probabilisticmethods,Gaussian Naïve Bayes and Logistic Regression to predict the genotypes of the offspring of two maize strains,the BLC and the JNE genotypes,based on the phenotypic traits of the parents.We determined the prediction performance of the two models with the overall accuracy and the area under the receiver operating curve(AUC).The overall accuracy for both models ranged between 82%and 87%.The values of the area under the receiver operating curvewere 0.90 or higher for Logistic Regression models,and 0.85 or higher for Gaussian Naïve Bayesmodels.These statistics indicated that the two models were very effective in predicting the genotypes of the offspring.Furthermore,bothmodels predicted the BLC genotype with higher accuracy than they did the JNE genotype.The BLC genotype appeared more homogeneous and more predictable.A Chi-square test for the homogeneity of the confusionmatrices showed that in all cases the twomodels produced similar prediction results.That finding was in line with the assertion by Mitchell(2010)who theoretically showed that the twomodels are essentially the same.With logistic regression,each subset of the original data or its corresponding principal components produced exactly the same prediction results.The AUC value may be viewed as a criterion for parent-offspring resemblance for each set of phenotypic traits considered in the analysis.

单边截断分布族参数的经验Bayes检验:NA样本情形

本文运用同分布 NA样本密度函数的核估计 ,构造一类单边截断型分布族参数的经验 Bayes检验 ,讨论它的渐近最优性 ,建立其收敛速度 .在适当的条件下 ,证明了该收敛速度可以任意接近于 1 ,最后给出适合定理条件的一个例子 .

连续型单参指数族参数的经验Bayes检验问题:NA样本情形

本文对连续型单参指数族单边和双边假设检验问题导出了Bayes检验函数 ,利用同分布NA样本构造了经验Bayes(EB)检验函数 ,在适当条件下证明了EB检验函数的渐近最优性并获得了其收敛速度 。

连续型单参数指数族参数的经验Bayes估计问题:NA样本情形

对连续型单参数指数族在平方损失下导出了参数的Bayes估计,利用同分布负相协(NA)样本构造了经验Bayes(EB)估计量,并在适当条件下获得了EB估计的收敛速度.文末给出一个满足定理条件的例子.

NA样本下两参数Lomax分布形状参数的经验Bayes检验

文章在加权线性损失函数下,讨论了NA样本情形下两参数Lomax分布参数θ的经验Bayes单侧检验问题:H0:θ≤θ0圮H1:θ>θ0,利用概率密度函数的核估计构造了参数的经验Bayes单侧检验函数,并获得了它的渐近最优(a.o)性,并在适当的条件下证明了所提出的经验Bayes检验函数的收敛速度可任意接近O(n-12)。

Multitude Classifier Using Rough Set Jelinek Mercer Naïve Bayes for Disease Diagnosis

Classification model has received great attention in any domain of research and also a reliable tool for medical disease diagnosis. The domain of classification model is used in disease diagnosis, disease prediction, bio informatics, crime prediction and so on. However, an efficient disease diagnosis model was compromised the disease prediction. In this paper, a Rough Set Rule-based Multitude Classifier (RS-RMC) is developed to improve the disease prediction rate and enhance the class accuracy of disease being diagnosed. The RS-RMC involves two steps. Initially, a Rough Set model is used for Feature Selection aiming at minimizing the execution time for obtaining the disease feature set. A Multitude Classifier model is presented in second step for detection of heart disease and for efficient classification. The Na?ve Bayes Classifier algorithm is designed for efficient identification of classes to measure the relationship between disease features and improving disease prediction rate. Experimental analysis shows that RS-RMC is used to reduce the execution time for extracting the disease feature with minimum false positive rate compared to the state-of-the-art works.

Pareto分布参数的经验Bayes检验的收敛速度:NA样本情形

本文讨论了NA(negativeassociation)样本情形Pareto分布参数的经验Bayes(EB)单侧和双侧检验问题.利用概率密度函数的核估计构造了参数的经验Bayes检验函数,在适当的条件下证明了所提出的经验Bayes检验函数的渐近最优(a.o.)性并获得了其收敛速度.

Bayes-Q-Learning Algorithm in Edge Computing for Waste Tracking

The major environmental hazard in this pandemic is the unhygienic dis-posal of medical waste.Medical wastage is not properly managed it will become a hazard to the environment and humans.Managing medical wastage is a major issue in the city,municipalities in the aspects of the environment,and logistics.An efficient supply chain with edge computing technology is used in managing medical waste.The supply chain operations include processing of waste collec-tion,transportation,and disposal of waste.Many research works have been applied to improve the management of wastage.The main issues in the existing techniques are ineffective and expensive and centralized edge computing which leads to failure in providing security,trustworthiness,and transparency.To over-come these issues,in this paper we implement an efficient Naive Bayes classifier algorithm and Q-Learning algorithm in decentralized edge computing technology with a binary bat optimization algorithm(NBQ-BBOA).This proposed work is used to track,detect,and manage medical waste.To minimize the transferring cost of medical wastage from various nodes,the Q-Learning algorithm is used.The accuracy obtained for the Naïve Bayes algorithm is 88%,the Q-Learning algo-rithm is 82%and NBQ-BBOA is 98%.The error rate of Root Mean Square Error(RMSE)and Mean Error(MAE)for the proposed work NBQ-BBOA are 0.012 and 0.045.

NA样本下单边截断型分布族位置参数的经验Bayes估计

该文在充分运用同分布 N A样本密度函数的核估计方法的情况下 ,构造出了一类单边截断型分布族位置参数θ的经验 Bayce(EB)估计 ;由分析可知 ,在适当的条件下 ,证明了位置参数θ的 EB估计的收敛速度 O(n- q) ,其中 q =λα(δ -2 ) /(2α + 4)δ,α >0 ,1>λ >0 ,δ >2。

NA样本下Rayleigh分布参数的经验Bayes检验

文章在线性损失函数下,讨论了NA样本情形下Rayleigh分布参数θ的经验Bayes单侧检验问题:H0:θ≤θ0圳H1:θ>θ0,利用概率密度函数的核估计构造了参数的经验Bayes单侧检验函数,并获得了它的渐近最优(a.o)性;在适当的条件下证明了所提出的经验Bayes检验函数的收敛速度可任意接近O(n-1/2)。

Rayleigh分布参数的经验Bayes检验:NA样本情形

目的研究负相依样本情形下Rayleigh分布参数的经验Bayes检验问题。方法利用概率密度函数核估计方法获得密度函数及其导数的非参数估计。结果获得了经验Bayes检验函数,证明了检验函数的渐近最优性,得到其收敛速度。结论利用单调经验Bayes方法证明该检验函数可以达到最优。