Nave Bayes分类器制导的专业网页爬取算法

从Web中快速、准确地检索出所需信息的迫切需求催生了专业搜索引擎技术。在专业搜索引擎中,网络爬虫(Crawler)负责在Web上搜集特定专业领域的信息,是专业搜索引擎的重要核心部件。该文对中文专业网页的爬取问题进行了研究,基于KL距离验证了网页内容与链接前后文在分布上的差异,在此基础上提出了以链接锚文本及其前后文为特征、Nave Bayes分类器制导的中文专业网页爬取算法,设计了自动获取带链接类标的训练数据的算法。以金融专业网页的爬取为例,分别对所提出的算法进行了离线和在线测试,结果表明,Nave Bayes分类器制导的网络爬虫可以达到近90%的专业网页收割率。

结合特征和非特征信息改进Nave Bayes及其应用

朴素贝叶斯算法是一种常见的基于内容的垃圾邮件过滤算法,但是,传统朴素贝叶斯过滤存在判断内容的不确定性和邮件表示不完整性等问题。分析邮件信头各域在正常邮件和垃圾邮件中表现出的不同属性,提取非特征信息,结合特征信息和非特征信息改进朴素贝叶斯算法。实验结果表明,改进的朴素贝叶斯分类方法与单纯使用特征信息的方法相比,垃圾邮件的召回率和准确率更高,凸显了该方法涵盖邮件信息、克服内容判断缺陷的优势。

基于改进的Nave Bayes和BP神经网络的垃圾邮件过滤

不同用户对垃圾邮件的判定有所差别,考虑到同一用户的自认垃圾邮件相似度较大,提出对特定用户进行针对性的垃圾邮件过滤方法.系统除重点利用邮件正文信息外,还尝试加入发件人、群发信息和主题相关度信息,改进朴素贝叶斯公式用于邮件正文的概率计算,基于BP神经网络构造垃圾邮件判别系统.实验表明,改进的朴素贝叶斯公式用于本文的系统是可行的,基于BP神经网络的垃圾邮件过滤系统能有效综合以上四项数值进行全局判别,进而对特定用户的邮件产生不错的过滤效果.

New Spam Filtering Method with Hadoop Tuning-Based MapReduce Naïve Bayes

As the importance of email increases,the amount of malicious email is also increasing,so the need for malicious email filtering is growing.Since it is more economical to combine commodity hardware consisting of a medium server or PC with a virtual environment to use as a single server resource and filter malicious email using machine learning techniques,we used a Hadoop MapReduce framework and Naïve Bayes among machine learning methods for malicious email filtering.Naïve Bayes was selected because it is one of the top machine learning methods(Support Vector Machine(SVM),Naïve Bayes,K-Nearest Neighbor(KNN),and Decision Tree)in terms of execution time and accuracy.Malicious email was filtered with MapReduce programming using the Naïve Bayes technique,which is a supervised machine learning method,in a Hadoop framework with optimized performance and also with the Python program technique with the Naïve Bayes technique applied in a bare metal server environment with the Hadoop environment not applied.According to the results of a comparison of the accuracy and predictive error rates of the two methods,the Hadoop MapReduce Naïve Bayes method improved the accuracy of spam and ham email identification 1.11 times and the prediction error rate 14.13 times compared to the non-Hadoop Python Naïve Bayes method.

Attribute Weighted Naïve Bayes Classifier

The naïve Bayes classifier is one of the commonly used data mining methods for classification.Despite its simplicity,naïve Bayes is effective and computationally efficient.Although the strong attribute independence assumption in the naïve Bayes classifier makes it a tractable method for learning,this assumption may not hold in real-world applications.Many enhancements to the basic algorithm have been proposed in order to alleviate the violation of attribute independence assumption.While these methods improve the classification performance,they do not necessarily retain the mathematical structure of the naïve Bayes model and some at the expense of computational time.One approach to reduce the naïvetéof the classifier is to incorporate attribute weights in the conditional probability.In this paper,we proposed a method to incorporate attribute weights to naïve Bayes.To evaluate the performance of our method,we used the public benchmark datasets.We compared our method with the standard naïve Bayes and baseline attribute weighting methods.Experimental results show that our method to incorporate attribute weights improves the classification performance compared to both standard naïve Bayes and baseline attribute weighting methods in terms of classification accuracy and F1,especially when the independence assumption is strongly violated,which was validated using the Chi-square test of independence.

Improved Bearing Fault Diagnosis by Feature Extraction Based on GLCM, Fusion of Selection Methods, and Multiclass-Naïve Bayes Classification

The presence of bearing faults reduces the efficiency of rotating machines and thus increases energy consumption or even the total stoppage of the machine. It becomes essential to correctly diagnose the fault caused by the bearing. Hence the importance of determining an effective features extraction method that best describes the fault. The vision of this paper is to merge the features selection methods in order to define the most relevant featuresin the texture of the vibration signal images. In this study, the Gray Level Co-occurrence Matrix (GLCM) in texture analysis is applied on the vibration signal represented in images. Features selection based on the merge of PCA (Principal component Analysis) method and SFE (Sequential Features Extraction) method is done to obtain the most relevant features. The multiclass-Na?ve Bayesclassifier is used to test the proposed approach. The success rate of this classification is 98.27%. The relevant features obtained give promising results and are more efficient than the methods observed in the literature.

Integration of Expectation Maximization using Gaussian Mixture Models and Naïve Bayes for Intrusion Detection

Intrusion detection is the investigation process of information about the system activities or its data to detect any malicious behavior or unauthorized activity.Most of the IDS implement K-means clustering technique due to its linear complexity and fast computing ability.Nonetheless,it is Naïve use of the mean data value for the cluster core that presents a major drawback.The chances of two circular clusters having different radius and centering at the same mean will occur.This condition cannot be addressed by the K-means algorithm because the mean value of the various clusters is very similar together.However,if the clusters are not spherical,it fails.To overcome this issue,a new integrated hybrid model by integrating expectation maximizing(EM)clustering using a Gaussian mixture model(GMM)and naïve Bays classifier have been proposed.In this model,GMM give more flexibility than K-Means in terms of cluster covariance.Also,they use probabilities function and soft clustering,that’s why they can have multiple cluster for a single data.In GMM,we can define the cluster form in GMM by two parameters:the mean and the standard deviation.This means that by using these two parameters,the cluster can take any kind of elliptical shape.EM-GMM will be used to cluster data based on data activity into the corresponding category.

单边截断分布族参数的经验Bayes检验:NA样本情形

本文运用同分布 NA样本密度函数的核估计 ,构造一类单边截断型分布族参数的经验 Bayes检验 ,讨论它的渐近最优性 ,建立其收敛速度 .在适当的条件下 ,证明了该收敛速度可以任意接近于 1 ,最后给出适合定理条件的一个例子 .

Naïve Bayes Algorithm for Large Scale Text Classification

This paper proposed an improved Naïve Bayes Classifier for sentimental analysis from a large-scale dataset such as in YouTube.YouTube contains large unstructured and unorganized comments and reactions,which carry important information.Organizing large amounts of data and extracting useful information is a challenging task.The extracted information can be considered as new knowledge and can be used for deci sion-making.We extract comments from YouTube on videos and categorized them in domain-specific,and then apply the Naïve Bayes classifier with improved techniques.Our method provided a decent 80%accuracy in classifying those comments.This experiment shows that the proposed method provides excellent adaptability for large-scale text classification.

连续型单参指数族参数的经验Bayes检验问题:NA样本情形

本文对连续型单参指数族单边和双边假设检验问题导出了Bayes检验函数 ,利用同分布NA样本构造了经验Bayes(EB)检验函数 ,在适当条件下证明了EB检验函数的渐近最优性并获得了其收敛速度 。

连续型单参数指数族参数的经验Bayes估计问题:NA样本情形

对连续型单参数指数族在平方损失下导出了参数的Bayes估计,利用同分布负相协(NA)样本构造了经验Bayes(EB)估计量,并在适当条件下获得了EB估计的收敛速度.文末给出一个满足定理条件的例子.

NA样本下两参数Lomax分布形状参数的经验Bayes检验

文章在加权线性损失函数下,讨论了NA样本情形下两参数Lomax分布参数θ的经验Bayes单侧检验问题:H0:θ≤θ0圮H1:θ>θ0,利用概率密度函数的核估计构造了参数的经验Bayes单侧检验函数,并获得了它的渐近最优(a.o)性,并在适当的条件下证明了所提出的经验Bayes检验函数的收敛速度可任意接近O(n-12)。

Multitude Classifier Using Rough Set Jelinek Mercer Naïve Bayes for Disease Diagnosis

Classification model has received great attention in any domain of research and also a reliable tool for medical disease diagnosis. The domain of classification model is used in disease diagnosis, disease prediction, bio informatics, crime prediction and so on. However, an efficient disease diagnosis model was compromised the disease prediction. In this paper, a Rough Set Rule-based Multitude Classifier (RS-RMC) is developed to improve the disease prediction rate and enhance the class accuracy of disease being diagnosed. The RS-RMC involves two steps. Initially, a Rough Set model is used for Feature Selection aiming at minimizing the execution time for obtaining the disease feature set. A Multitude Classifier model is presented in second step for detection of heart disease and for efficient classification. The Na?ve Bayes Classifier algorithm is designed for efficient identification of classes to measure the relationship between disease features and improving disease prediction rate. Experimental analysis shows that RS-RMC is used to reduce the execution time for extracting the disease feature with minimum false positive rate compared to the state-of-the-art works.

Pareto分布参数的经验Bayes检验的收敛速度:NA样本情形

本文讨论了NA(negativeassociation)样本情形Pareto分布参数的经验Bayes(EB)单侧和双侧检验问题.利用概率密度函数的核估计构造了参数的经验Bayes检验函数,在适当的条件下证明了所提出的经验Bayes检验函数的渐近最优(a.o.)性并获得了其收敛速度.

NA样本下单边截断型分布族位置参数的经验Bayes估计

该文在充分运用同分布 N A样本密度函数的核估计方法的情况下 ,构造出了一类单边截断型分布族位置参数θ的经验 Bayce(EB)估计 ;由分析可知 ,在适当的条件下 ,证明了位置参数θ的 EB估计的收敛速度 O(n- q) ,其中 q =λα(δ -2 ) /(2α + 4)δ,α >0 ,1>λ >0 ,δ >2。

NA样本下Rayleigh分布参数的经验Bayes检验

文章在线性损失函数下,讨论了NA样本情形下Rayleigh分布参数θ的经验Bayes单侧检验问题:H0:θ≤θ0圳H1:θ>θ0,利用概率密度函数的核估计构造了参数的经验Bayes单侧检验函数,并获得了它的渐近最优(a.o)性;在适当的条件下证明了所提出的经验Bayes检验函数的收敛速度可任意接近O(n-1/2)。

Rayleigh分布参数的经验Bayes检验:NA样本情形

目的研究负相依样本情形下Rayleigh分布参数的经验Bayes检验问题。方法利用概率密度函数核估计方法获得密度函数及其导数的非参数估计。结果获得了经验Bayes检验函数,证明了检验函数的渐近最优性,得到其收敛速度。结论利用单调经验Bayes方法证明该检验函数可以达到最优。

伽玛分布族参数的经验Bayes双边检验的收敛速度:NA样本情形

本文研究了NA样本情形下,伽玛分布族形状参数的经验Bayes(EB)双边检验问题.利用概率密度函数的核估计,构造了参数的经验Bayes检验函数,并在适当的条件下,证明了所提出的经验Bayes检验函数的渐近取优(a.o.)性,获得了其收敛速度.

一类特殊的指数分布族参数的经验Bayes检验:NA样本

在线性损失函数下,对NA样本下一类指数分布族参数θ的经验Bayes单侧检验问题进行了研究.通过构造参数的经验Bayes单侧检验函数,获得了它的渐近最优(a.o)性,在适当条件下得出了所提出的经验Bayes检验函数的收敛速度可以任意接近O(n-1/2).

NA样本情形线性指数分布参数的经验Bayes估计

利用同分布负相协(NA)样本构造密度函数及其导数的核估计,得到线性指数分布参数的经验Bayes(EB)估计,并给出EB估计在适当条件下的收敛速度.