基于大数据的医疗数据采集交换共享系统的研究与实现

Research and Implementation of Medical Data Acquisition, Exchange and Sharing System Based on Big Data

本文基于大数据及数据交换技术,构建了一种基于大数据技术的医疗数据采集与交换共享系统,其实现功能基于spark sql工作流程计算引擎实现数据分析、统计、清洗操作;配置mapred-site.xml文件里的参数,设置map reduce执行引擎,配置spark-defaults.conf文件、hive参数,搭建基层hadoop环境,编写动态解析配置文件方法,基于mule进行集成调度kettle,传参分装到配置文件。本文构建的系统能够实现医疗数据批量自动化处理,数据交换共享速率高;采集过程采用压缩技术,能够降低网络宽带的压力;全流程自动监控处理过程,能够降低数据维护量。

Based on big data and data exchange technology,this paper constructs a medical data acquisition and exchange sharing system based on big data technology and its implementation method.Its implementation function is based on spark sql computing engine to realize data analysis,statistics and cleaning operation.Configure the parameters in the mapred-site.xml file,set the map reduce execution engine,configure the spark-defaults.conf file and hive parameters,build the basic hadoop environment,and write the dynamic analysis configuration file method.Perform an integrated scheduling kettle based on mule,transfer parameters and load them to the configuration file.The system constructed in this paper can realize batch automatic processing of medical data,and the data exchange and sharing rate are high.The acquisition process adopts compression technology,which can reduce the pressure of network broadband;Automatic monitoring of the whole process can reduce the amount of data maintenance.