万维网动态文档的爬取分析

Analysis of Crawling World Wide Web Dynamic Document

万维网(下文简写WWW)动态文档是浏览器访问万维网服务器时由应用程序临时生成的文档,是万维网资源中最有价值的部分。为了实现对动态文档中数据的爬取,研究动态文档的生成逻辑分析。在给出万维网原理和分析工具的基础上,提出对动态文档请求和动态文档源码进行分析的一般方法。实践证明,该方法可有效地指导动态文档爬虫程序的设计。

World Wide Web(Abbreviation:WWW)dynamic document is the document generated by the application temporarily when the browser accesses the WWW server.It is the most valuable part of the WWW resources.To crawl the data in the dynamic document,you need to understand and simulate how the dynamic document is generated.Based on the principle of WWW and analysis tools,a general method for analyzing dynamic document request and dynamic document source code is proposed.A lot of practice has proved that this method can effectively guide the design of dynamic document crawler.