摘要
本文综合分析了网页内容块各方面的特征,提出了一个联合使用布局特征和语言特征的网页主要内容块发现方法,有效地解决了以往模型中通用性与高准确率不能共存的缺点。该方法使用网页视觉块树表示网页,对网页内容块的布局特征和语言特征分别建立了独立的分类器,然后组合这两个分类器来进行网页内容块分类。实验结果表明,在保持非噪音块召回率在90%以上的同时,组合分类器的准确率达到85%,比只使用布局特征的分类器提高5个百分点,比只使用语言特征的分类器提高15个百分点;在5个站点上的分类结果表明组合分类器在不同站点上性能稳定,具有良好的通用性。
This paper analyzed the different feature types of webpage blocks, and presented a webpage content block detection rnethod based on layout features and language features, which effectively resolved the seesaw problern be-tween detection accuracy and model generality across different types of webpages. The method used the vision-block tree to represent webpage, built two individual classifiers respectively for webpage's layout features and language features, and used different strategies to combine these two classifiers. The experimental results show that, with holding the content block detection recall higher than 90%, the combined classifiers' accuracy can reach 85 per-cents, 5 percents higher than the classifier using only the layout features, and 15 percents higher than the classifier using only the language features; and the experimental results also show that the combined classifiers obtained good detection performance over five selected websites which means that it have good generality.
出处
《中文信息学报》
CSCD
北大核心
2008年第1期15-21,共7页
Journal of Chinese Information Processing
基金
国家自然科学基金资助项目(60673042)
北京市自然科学基金资助项目(4052027
4073043)
关键词
计算机应用
中文信息处理
网页清理
主要内容块发现
网页切分
布局特征
语言特征
computer application
Chinese information processing
webpage cleaning
content block detection
webpage segment
layout feature
language feature