Web信息增量采集与保存管理平台的设计与实现

Design and implementation of Web information incremental collection and storage management platform

随着网络的普及和发展,互联网资源已经成为人类文明记忆的载体,及时并完整的保存网络信息资源变得尤为重要。文章结合国家图书馆网络信息资源采集与保存工作的发展历程、技术发展和互联网信息更新快等特点,提出基于Heritrix开源架构的网络信息增量采集与保存管理平台的设计和架构,并详细介绍平台在网络资源增量采集和增量回显一站式服务的实现技术,提升国家图书馆网络资源保存工作效率和服务质量的同时,以期为业界提供有益的参考和借鉴。

With the popularization and development of the network,Internet resources have become the carrier of human civilization memory,timely and complete preservation of network information resources has become particularly important. Article combined with the development of the national library network information resources collection and preservation work development,technology development and Internet information update fast,based on the Heritrix open source architecture design and preservation management platform and architecture,and introduces the platform in the network resources incremental acquisition and incremental display one-stop service technology,improve the national library network resources preservation work efficiency and service quality at the same time,in order to provide useful reference and reference for the industry.