多源异构数据融合技术的研究

Research on Multi-source Heterogeneous Data Fusion Technology

多源数据融合是时代发展的必然要求。多源异构数据融合技术涉及数据采集、数据清洗、数据融合分析三个阶段,本文通过对多源异构数据融合技术的研究,将其典型应用于晋中市科技计划管理信息平台,实现对关系型数据库中数据、上传的PDF数据、图片数据、日志数据、网页爬虫数据等进行融合分析,消除数据之间的不精确、不一致,提高数据可靠性,从多维度、全方位为决策提供支持;运用到项目查重模块,实现项目在进行查重时,从原有基于项目名称、项目负责人等结构化查重的基础上,增加了基于项目全文、网络爬虫数据的非结构化查重分析,更精确、更科学地对申报项目作出客观评价。

Multi-source data fusion is an inevitable requirement of the development of the times.Multi-source heterogeneous data fusion technology involves three stages:data collection,data cleaning and data fusion analysis.This article uses the research on multi-source heterogeneous data fusion technology and applies it to the Jinzhong Science and Technology Plan Management Information Platform to make the data in relational databases,uploaded PDF data,image data,log data,web crawler data,etc fused and analyzed so as to eliminate inaccuracies and inconsistencies between data;improve data reliability,and provide support for decision-making in a multi-dimensional and comprehensive manner.The project duplicate check module is used to realize the unstructured duplicate check based on the full text of the project and web crawler data from the original structured duplicate check based on the project name and project leader when the project is checked.Analyze,make an objective evaluation of the declared project more accurately and scientifically.