大规模异构的政府统计报表信息抽取与集成融合研究

Information Extraction and Integration of Large-scale Heterogeneous Socio-economic Statistical Statements

政府统计数据作为国家的"战略金矿",充分挖掘其内在价值,使之更好地服务于政府及公众,已成为当前智慧政务和新型智库发展中大数据系统建设的必然要求。但政府统计报表的半结构化和大规模异构特点,使得统计数据之间无法直接关联及聚合,影响了统计数据资源的深度挖掘与开发。鉴于此,本文针对已有研究的不足,在分析政府统计报表语义构成要素的基础上,结合其信息抽取与集成融合的应用目标,将处理任务分解为表格语义结构解析、表头语义关系识别、数值信息抽取表示、指标术语消冗转换及不一致统计数据消歧等五个逻辑过程,并定义了各过程的作用与主要任务,且研究构建了面向该任务的总体技术框架及其处理流程。大规模真实数据集上的应用结果表明,本研究方法能够较为有效地实现异构型政府统计报表的抽取与集成融合,具备较好的实际价值,同时也为其他基于半结构化表格的大数据建设与应用研究提供参考借鉴。

To better serve the government and the public,full mining of government statistics such as the National Strategic Gold Mine has become an inevitable requirement for the development of big data systems in current smart e-government and new think tanks.However,it is impossible to directly correlate and aggregate statistics due to the semi-structured and large-scale heterogeneous characteristics of statistical statements,which causes significant difficulties in terms of standardized management,deep mining,and extensive utilization of statistical resources.In view of the deficiencies in existing research,this study defines the processing tasks based on the analysis of the semantic elements of government statistical statements and the application objectives of information extraction and integration.The processing tasks are divided into five logical processes:table semantic structure analysis,header semantic relationship recognition,numerical information extraction and representation,index terminology redundancy conversion,and inconsistent statistical data disambiguation loading and the roles and main tasks of each process are described.Finally,this study investigates and constructs the overall technical framework and processing flow.The processing and application results for large-scale real data sets reveal that this method can effectively solve the research question,and has a certain practical value.At the same time,it can also pro‐vide reference for other big data construction and application research based on semi-structured tables.