摘要
信息技术的飞速发展造成了大量的文本数据累积,其中很大一部分是短文本数据。文本分类技术对于从这些海量短文中自动获取知识具有重要意义。但是对于关键词出现次数少的短文,现有的一般文本挖掘算法很难得到可接受的准确度。一些基于语义的分类方法获得了较好的准确度但又由于其低效性而无法适用于海量数据。针对这个问题提出了一个新颖的基于频繁词集聚类的短文分类算法。该算法使用频繁词集聚类来压缩数据,并使用语义信息进行分类。实验表明该算法在对海量短文进行分类时,其准确度和性能超过其它的算法。
With the rapid development of information technology, huge data is accumulated. A vast amount of such data appears as short documents. It is very useful to classify such short documents to get knowledge automatically form the data. But most of the current classification algorithms can not get acceptable accuracy since key words appear few times in short documents. Some classification algorithms based on semantic information are fnore accurate but they are inefficient to be used to process very large document sets. A novel classification method based on frequent term set clustering is proposed. This method uses frequent term set clustering to compress massive data and uses semantic information to improve accuracy. Experimental study shows that this method is more accurate and efficient than other classification algorithms when classifying massive short documents.
出处
《计算机工程与设计》
CSCD
北大核心
2007年第8期1744-1746,1780,共4页
Computer Engineering and Design
基金
国家863高技术研究发展计划基金项目(2004AA112020
2003AA115210
2003AA111020)
关键词
文本挖掘
分类
海量
短文
频繁词集
text mining
classification
massive
short document
frequent term set