期刊文献+

针对Web论坛的一种结构化数据自动抽取方法 被引量:1

Automatic structured data extraction from Web forums
原文传递
导出
摘要 由于网页布局设计的复杂性和用户发表帖子的灵活性,从论坛网页中抽取结构化的数据是一项未能很好解决并非常具有挑战性的任务。本文提出了一种从任意的论坛站点中自动抽取结构化数据的通用解决方案,通过分析网页结构发现列表页和帖子页中的数据记录,并利用一组产生式规则从发现的数据记录中抽取结构化的数据。实验结果表明该方法在抽取论坛数据记录方面明显优于已有的方法,对论坛帖子的标题、作者、发表时间和内容文本块等元数据的抽取达到了较高的准确率。 Because of both complex page layout designs and unrestricted user created posts,extracting structured data from Web forum pages is a very challenging task and not easily solved.A general solution to automatically extract structured data from any forum site was proposed.By analyzing page structure,a group of data records were found from both list page and post page,and then a set of production rules was used to extract structured data from these data records.Experimental results showed that the proposed approach significantly outperformed some existing methods in extracting data records and achieved high accuracy in extracting some metadata of Web forums such as title,author,time and content.
作者 关冕 马军
出处 《山东大学学报(理学版)》 CAS CSCD 北大核心 2010年第5期42-47,共6页 Journal of Shandong University(Natural Science)
基金 国家自然科学基金资助项目(60970047) 山东省自然科学基金资助项目(Y2008G19) 山东省科技攻关资助项目(2008GG10001026 2007GG10001002)
关键词 论坛 结构化数据 信息抽取 WEB挖掘 Web forums structured data information extraction Web mining
  • 引文网络
  • 相关文献

参考文献16

  • 1CONG G, WANG L, LIN CY, et al. Finding question - answer pairs from online forums [ C ]//Proceedings of the 31 st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM Press, 2008: 467-474.
  • 2GLANCE N, HURST M, NIGAM K, et al. Deriving marketing intelligence from online discussion [ C ]//Proceedings of the 11th Annual International ACM SIGKDD Conference on Knowledge Discovery and Data Mining. New York, USA: ACM Press, 2005: 419-428.
  • 3ZHANG J, ACKERMAN MS, ADAMIC L. Expertise networks in online communities: structure and algorithms [ C ]//Proceedings of the 16th Intemational Conference on World Wide Web. New York, USA: ACM Press, 2007: 221-230.
  • 4KUSHMERICK N. Wrapper induction: efficiency and expressiveness [ J ]. Artificial Intelligence, 2000, 118 : 15- 68.
  • 5LERMAN K, MINTON S, KNOBLOCK C. Wrapper maintenance: a machine learning approach[ J]. Journal of Artificial Intelligence Research, 2003, 18: 149-181.
  • 6ZHAI Y, LIU B. Web data extraction based on partial tree alignment [ C ]//Proceedings of the 14th International Conference on World Wide Web. New York, USA: ACM Press, 2005: 76-85.
  • 7ZHENG S, WU D, SONG R, WEN J R. Joint optimization of wrapper generation and template detection [ C ]// Proceedings of the 13th Annual International ACM SIGKDD Conference on Knowledge Discovery and Data Mining. New York, USA: ACM Press, 2007: 894-902.
  • 8杨少华,林海略,韩燕波.针对模板生成网页的一种数据自动抽取方法(英文)[J].软件学报,2008,19(2):209-223. 被引量:45
  • 9BUNESCU R, MOONEY R J. Collective information extraction with relational Markov networks [ C ]//Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics. San Francisco, USA: Morgan Kaufmann Publishers, 2004: 439-446.
  • 10PINTO D, MCCALLUM A, WEI X, et al. Table extraction using conditional random fields [ C ]//Proceedings of the 26th Annual Intemational ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM Press, 2003: 235-242.

二级参考文献18

  • 1Chang Chia-Hui, Kayed M, Girgis M R. A Survey of Web Information Extraction Systems[J]. IEEE Transaction on Know-ledge and Data Engineering, 2006, 18( 10): 1411 - 1428.
  • 2Crescenzi V, Mecca G, Merialdo R Road-runner: Towards Automatic Data Extraction from Large Web Sites[C]//Proc. of the 26th Int'l Conf. on Very Large Database Systems. Roma, Italy: [s. n.], 2001: 109-118.
  • 3Chang Chia-Hui, Lui C. IEPAD: Information Extraction Based on Pattern Discovery[C]//Proceedings of the 10th International Conference on World Wide Web. Hong Kong, China: [s. n.], 2001: 681-688.
  • 4Liu Bing, Grossman R, Zhai Yanhong. Mining Data Records in Web Pages[C]//Proceedings of KDD'03. Washington D. C., USA: [s. n.], 2003: 601-606.
  • 5Phong L Vuong B Gao Xiaoying, et al. Data Extraction from Semi-structured Web Pages by Clustering[C]//Proceedings of WI'06. Hong Kong, China: [s. n.], 2006: 374-377.
  • 6Wu Yang. Identifying Syntactic Differences Between Two Programs[J]. Software-practice and Experience, 1991, 21(7): 739-755.
  • 7Chang CH, Kayed M, Girgis MR, Shaalan K. A survey of Web information extraction systems. IEEE Trans. on Knowledge and Data Engineering, 2006,18(10): 1411-1428.
  • 8Gold ME. Language identification in the limit. Information and Control, 1967,10(5):447-474.
  • 9Laender AHF, Ribeiro-Neto BA, da Silva AD, Teixeira JS. A brief survey of Web data extraction tools. SIGMOD Record, 2002,31 (2):84-93.
  • 10Arasu A, Hector GM. Extracting structured data from Web pages. In: Proc. of the ACM SIGMOD Int'l Conf. on Management of Data. San Diego: ACM Press, 2003. 337-348.

共引文献51

同被引文献5

  • 1王涛.Web页面中结构化数据抽取的实现与应用[D].天津:天津大学,2008.
  • 2Liu Bing. Web Data Mining[M].俞勇,薛贵荣,韩定一,译.北京:清华大学出版社,2009:265-269.
  • 3Liu Bing, Zhai Yanhong. NET-A system for extracting Web data from flat and nested data records[M]//Web Information Systems Engineering. New York: Springer Verlag, 2005: 487-495.
  • 4师雪霖,程文涛.Web信息抽取与语义检索框架[J].郑州大学学报(理学版),2010,42(1):29-32. 被引量:4
  • 5李晶,陈恩红.Web信息抽取[J].计算机科学,2003,30(6):78-81. 被引量:17

引证文献1

;
使用帮助 返回顶部