开源威胁情报生产与应用综述

A Survey of Open-Source Threat Intelligence Production and Application

网络威胁情报是通过及时收集与组织网络安全相关的内部及外部威胁信息而综合分析出的可指导组织应对当前网络威胁的知识,可极大提升组织的网络安全防御效率。其中一种威胁情报是通过收集互联网上的多源威胁信息后综合分析生产出来的,即开源威胁情报,其可以识别和分析潜在的网络威胁、恶意活动和攻击趋势等,具有极高的应用价值。然而,在开源威胁情报生产过程中,需要克服开源情报信息非结构化表达、多源情报间表达异构和内容冲突等困难,这吸引了学术界和产业界的众多关注。鉴于此,文章首先深入研究近年来网络威胁情报的行业报告、白皮书以及学术成果,归纳出开源威胁情报生产及应用框架。其中,开源威胁情报生产过程中首先对情报可靠性进行评估,还负责实现非结构化威胁信息中的情报抽取以及多源情报间存在的表达结构及内容冲突处理,情报应用则覆盖威胁狩猎、应急响应以及威胁归因的全防御生命周期。因此,文章从威胁情报抽取、情报冲突处理和情报应用研究三个方面整理已有研究成果并进行总结。具体地,现有研究首先从定性和定量两个方向对情报质量进行评估,再通过各种技术从多个信息来源中抽取出多种类型的情报,但抽取类型及情报来源多是定制化的、片面的。关于异构情报消冗的研究成果较少,情报内容的不一致性检测则受到越来越多的关注,但大多集中于如漏洞影响产品、情报披露时间等非语义信息情报的不一致性检测上。研究人员还专注于将生产的威胁情报进行关联应用,但未考虑生产出的威胁情报的完整性。最后,文章指出开源威胁情报生产与应用的未来研究趋势,即自动化威胁信息全面抽取、语义威胁情报的对齐与不一致性研究、基于已有知识的情报完整性提升研究以及情报应用自动化技术研究等方面。文章期望通过梳理和分析已有的开源威胁情报生产和应用研究概况,推进我国开源威胁情报生产和应用工作的发展,实现网络安全整体防御能力的提升。

Cyber threat intelligence is the knowledge that can guide organizations to deal with current cyber threats through the timely collection of internal and external threat information related to the organizational cyber security and comprehensive analysis,which can greatly improve the efficiency of the organizational cyber security defense.One type of threat intelligence is produced by collecting the multiple threat information on the Internet and then analyzing it comprehensively,i.e.,open source threat intelligence,which can identify and analyze potential cyber threats,malicious activities,and attack trends,etc.,and it has extremely high application value.However,in the production process of open source threat intelligence,it is necessary to overcome the difficulties of unstructured expression of intelligence,heterogeneity of expression among multiple sources of intelligence,and conflict of intelligence content,which attracts the attention of academia and industry.In view of this,recent cyber threat intelligence industry reports,white papers and academic results are first deeply studied,summarizing the open source threat intelligence production and application framework.Wherein,In the open source threat intelligence production process,the reliability of the intelligence is first assessed,which is also responsible for extracting intelligence from unstructured threat information,expression structure and content conflicts that exist between multiple sources of intelligence,and the intelligence application covers the entire defense life cycle of threat hunting,emergency response,and threat attribution.Hence,existing research results are organized and summarized from the aspects of threat intelligence extraction,intelligence conflict processing and intelligence application.Specifically,the existing research results first evaluate the quality of intelligence from both qualitative and quantitative perspectives,and multiple types of intelligence from multiple information sources through various techniques are extracted,but the extraction types and intelligence sources are mostly customized and one-sided.There are fewer research results on heterogeneous intelligence redundancy,while the intelligence inconsistency has received more and more attention,but most of them focus on the inconsistency detection of non-semantic intelligence,such as vulnerability affected product and intelligence disclosure time.Researchers have also focused on the related application of produced threat intelligence,but have not considered the integrity of the produced threat intelligence.Finally,the future research trends of open source threat intelligence production and application are given in this paper,including to the automated threat information comprehensive extraction,semantic threat information alignment and inconsistency research,intelligence integrity enhancement research based on existing knowledge,and research on intelligence application automation technology.By sorting out and analyzing the existing research overview of open source threat intelligence production and application,the development of China s open source threat intelligence production and application is promoted,and the improvement of the overall defense capability of network security is realized.