基于网络爬虫机制的文献资源可用性监测技术

Availability Monitoring Technology of Literature Resources Based on Web Crawler

在信息化的时代背景下,对电子文献资源库的可用性进行监测,是科研及教学的一项重要保障。该文对基于网络爬虫机制的文献资源监测技术进行了研究,重点探讨了技术选型、HTML文档抓取以及DOM树解析等内容,并对文献资源特征和系统架构展开了分析。该技术以Web服务器访问端口为数据输入源,因而具有较强的平台无关性,能适应网络多样化的特点。

In the era of information technology,availability monitoring of electronic literature resources is important for scientific research and teaching.In this paper,we study the monitoring technology based on web crawler mechanism.We focus on technology selection,HTML document capturing,DOM parsing and analyzing the system architecture.This technology takes web services as data source,so it has strong platform independence and is adaptable for complex networks.