基于改进的Porter Stemmer词干提取与核方法的垃圾邮件过滤算法

统计学习方法现已大量应用于垃圾邮件识别,其中表现突出的包括贝叶斯过滤器、支持向量机等。近年来,为应对日益严重的垃圾邮件问题,提出诸多改进算法或创新思路。通过改进Porter Stemmer并使之适用于垃圾邮件过滤,从而充分提取文本的有效特征,摒弃冗余信息,加强了过滤效果;将改进方法的Porter Stemmer与原方法分别应用于线性核、高斯核、多项式核支持向量机以及贝叶斯过滤器,对比实验结果可知,错误率分别下降了63.7%,63.1%,61.3%和11.4%,证明了改进方法的显著效果;另外,实验结果证明SVM过滤器显著优于贝叶斯过滤器,且能更大程度体现改进方法的优势;最后,给出多种定量评价和语义角度的分析,启发采用用户个性化定制的过滤器。

A New Enhanced Arabic Light Stemmer for IR in Medical Documents

This paper introduces a new enhanced Arabic stemming algorithm for solving the information retrieval problem,especially in medical documents.Our proposed algorithm is a light stemming algorithm for extracting stems and roots from the input data.One of the main challenges facing the light stemming algorithm is cutting off the input word,to extract the initial segments.When initiating the light stemmer with strong initial segments,the final extracting stems and roots will be more accurate.Therefore,a new enhanced segmentation based on deploying the Direct Acyclic Graph(DAG)model is utilized.In addition to extracting the powerful initial segments,the main two procedures(i.e.,stems and roots extraction),should be also reinforced with more efficient operators to improve the final outputs.To validate the proposed enhanced stemmer,four data sets are used.The achieved stems and roots resulted from our proposed light stemmer are compared with the results obtained from five other well-known Arabic light stemmers using the same data sets.This evaluation process proved that the proposed enhanced stemmer outperformed other comparative stemmers.

Developing Two Different Novel Techniques for Arabic Text Stemming

Stemming is used to produce stem or root of words. The process is vital to different research fields such as text mining, sentiment analysis, and text categorization, etc. Several techniques have been proposed to stemming Arabic text and among them, Khoja and light-10 stemmers are the most widely used. In this paper, we propose and evaluate two different stemming techniques to Arabic that are based on light stemming techniques. The new stemmers are compared to best reported light stemmer, which is light-10. Results and experiments, which were conducted using standard collections, reveal that The proposed stemmers yield 5.13% and 13.1% improvement in retrieval performance over light 10 with 0.369 average precision and 0.397, respectively and the improvement is statistically significant.

归纳所达不到的跳跃!──浅析施泰默为归纳推理有效性的辩护

休谟问题引发了为归纳推理有效性的辩护。迄今为止,凡是站在先验哲学立场上为归纳推理和归纳法进行辩护,最终都归于失败;直到施泰默应用大量科学事实为归纳辩护的出现,才为归纳问题的解决开辟了一条阳光之道。本文认为,为归纳推理有效性进行辩护的基础并非静态的事实,而是从假设出发归于事实的动态过程;为归纳推理有效性进行辩护的基础也并非唯一,不同的辩护者有不同的辩护过程,对辩护的基础,他们也会有不同的要求,这些不同的要求将决定归纳推理的命运。

The Enhancement of Arabic Stemming by Using Light Stemming and Dictionary-Based Stemming

Word stemming is one of the most important factors that affect the performance of many natural language processing applications such as part of speech tagging, syntactic parsing, machine translation system and information retrieval systems. Computational stemming is an urgent problem for Arabic Natural Language Processing, because Arabic is a highly inflected language. The existing stemmers have ignored the handling of multi-word expressions and identification of Arabic names. We used the enhanced stemming for extracting the stem of Arabic words that is based on light stemming and dictionary-based stemming approach. The enhanced stemmer includes the handling of multiword expressions and the named entity recognition. We have used Arabic corpus that consists of ten documents in order to evaluate the enhanced stemmer. We reported the accuracy values for the enhanced stemmer, light stemmer, and dictionary-based stemmer in each document. The results obtain shows that the average of accuracy in enhanced stemmer on the corpus is 96.29%. The experimental results showed that the enhanced stemmer is better than the light stemmer and dictionary-based stemmer that achieved highest accuracy values.

一款家用葡萄除梗机的研发

对一款电动葡萄除梗机的研发背景、设计原则和实施方案、机器组成、结构设计、工作原理、技术参数及其试验效果逐一作介绍,指出无论是外形体积,还是使用效果,它都不失为一款真正意义上的家用葡萄除梗机。

词干提取方法及工具的对比分析研究

首先,论述词干提取和语言形态学的关系。其次,对词干提取的主要算法和相关技术进行详细分类。而后,对词干提取方法的主要特点进行分析并对各类方法的优点、缺点和特点进行对比,并归纳实现词干提取功能的工具及其特点。最后,对词干提取研究的现有问题和未来发展趋势进行探讨。

提高维吾尔文搜索质量的一些关键技术研究

维吾尔语和土耳其语、乌兹别克语、哈萨克语等同属于阿尔泰语系突厥语分支,是一种粘着型语言,具有丰富的构词合构形附加成分.主要分布于中国新疆维吾尔自治区,以及中西亚地区的维吾尔族使用.目前使用基于阿拉伯文字母的维吾尔文.当前用google,bing等搜索引擎搜索维吾尔文时搜索结果不包括该词语的构词合构形附加成分等其它形式.严重影响了搜索结果的准确性.通过词干提取和词缀附加生成词语组合的方法以解决提高维吾尔文搜索质量的问题.此问题解决,将在大幅度提高维吾尔文搜索的准确性,并针对提高其它突厥语族语言(土耳其语、乌兹别克语、哈萨克语等)的搜索质量、数据挖掘、数据分析、数据安全、自然语言理解等研究领域有一定的参考价值.