摘要
在深入研究网络信息采集技术的基础上,提出一个基于Web结构的新闻采集模型。该模型加载采集入口地址后,通过信息采集和过滤算法确定新闻列表页,结合正则表达式技术自动识别新闻内容页的链接地址,访问目标新闻内容页,使用采集算法自动提取新闻信息数据。同时,它可以过滤在此页面中嵌入的广告等信息。实践结果表明,该模型工作良好,可以自动化、高效率地采集新闻信息。
On the basis of depth studying the technology of web information gathering, a web structure-based news gathering model is proposed. It load the gathering entry address, find the news list page with the information gathering and filter algorithm, identify and improve the news content page link address according to the rules set by acquisition and the regular expression technology automatically. Furthermore, it load the target page--news content page, gather the news information with the algorithm automatically. At the same time, it can filter any information that is set in this page such as embedded advertising messages. Practical results show that the proposed model works well and gathers news information efficiently and automatically.
出处
《井冈山大学学报（自然科学版）》
2012年第2期54-57,共4页
Journal of Jinggangshan University (Natural Science)
关键词
信息采集
WEB结构
正则表达式
数据挖掘
新闻采集
information gathering
Web structure
regular expressions
data mining
news gathering
