基于网络资源与用户行为信息的领域术语提取

Domain-Specific Terms Extraction Based on Web Resource and User Behavior

领域术语是反映领域特征的词语.领域术语自动抽取是自然语言处理中的一项重要任务,可以应用在领域本体抽取、专业搜索、文本分类、类语言建模等诸多研究领域,利用互联网上大规模的特定领域语料来构建领域词典成为一项既有挑战性又有实际价值的工作.当前,领域术语提取工作所利用的网络语料主要是网页对应的正文,但是由于网页正文信息抽取所面临的难题会影响领域术语抽取的效果,那么利用网页的锚文本和查询文本替代网页正文进行领域术语抽取,则可以避免网页正文信息抽取所面临的难题.针对锚文本和查询文本所存在的文本长度过短、语义信息不足等缺点,提出一种适用于各种类型网络数据及网络用户行为数据的领域数据提取方法,并使用该方法基于提取到的网页正文数据、网页锚文本数据、用户查询信息数据、用户浏览信息数据等开展了领域术语提取工作,重点考察不同类型网络资源和用户行为信息对领域术语提取工作的效果差异.在海量规模真实网络数据上的实验结果表明,基于用户查询信息和用户浏览过的锚文本信息比基于网页正文提取技术得到的正文取得了更好的领域术语提取效果.

The automatic domain-specific term extraction is an important task in natural language processing, which can be adoptedin domain-specific ontology construction, vertical search, text classification, class-based language model etc. A Web page contains lots of noises and irrelevantcontents, therefore, extracting domain-specific terms from original pages becomes a challenging task. Different from previous works, which rely on the original text of Web pages, this study focuses on anchor text and query log history of pages. Thisstrategy would avoid the trouble of information extraction from the original Web pageand therefore improves the term extraction performance. In this paper, a novel term extraction algorithmis based onanalysis into Web resource and user behaviors. Different Web resources including body of the page data, anchor text of the page and the information of user query data were employed to extract the domain-specific terms and their performances were compared.The result based on scale of the network datademonstratesthe resources of anchor text and the way user query data can obtain a better effect.