基于Web的智能信息采集及处理系统设计与实现

Design and Implementation of Intelligent Information Collection and Processing System Based on Web

互联网信息日益扩展的同时,如何采集和利用Web信息越来越备受关注。该文设计和实现的基于Web的智能信息采集及处理系统,采用高效的URL去重和基于模版的下载机制,提高了采集Web资源的性能;应用自然语言处理技术,对采集信息做智能分类和摘要,在发布上突出个性化的信息服务。与同类系统相比,智能性、实用性都显示出了明显的优势。

With the rapid development of Internet, collecting and exploiting Web information is extensively addressed. This paper designs and realizes one intelligent system on Web information collection and processing. On one hand, thanks to unrepreated URLs and template-based downloading, the collection performance is improved greatly. On the other hand, mature and advanced natural language processing techniques are used for classifying and abstracting the collected information. Thus, the personalization is highlighted. Experimental results show that the proposed system outperforms related work greatly.