In order to address the problems of the single encryption algorithm,such as low encryption efficiency and unreliable metadata for static data storage of big data platforms in the cloud computing environment,we propose...In order to address the problems of the single encryption algorithm,such as low encryption efficiency and unreliable metadata for static data storage of big data platforms in the cloud computing environment,we propose a Hadoop based big data secure storage scheme.Firstly,in order to disperse the NameNode service from a single server to multiple servers,we combine HDFS federation and HDFS high-availability mechanisms,and use the Zookeeper distributed coordination mechanism to coordinate each node to achieve dual-channel storage.Then,we improve the ECC encryption algorithm for the encryption of ordinary data,and adopt a homomorphic encryption algorithm to encrypt data that needs to be calculated.To accelerate the encryption,we adopt the dualthread encryption mode.Finally,the HDFS control module is designed to combine the encryption algorithm with the storage model.Experimental results show that the proposed solution solves the problem of a single point of failure of metadata,performs well in terms of metadata reliability,and can realize the fault tolerance of the server.The improved encryption algorithm integrates the dual-channel storage mode,and the encryption storage efficiency improves by 27.6% on average.展开更多
针对现有客票交易中间件缺乏有效机制管理配置文件的问题,在研究Z o o k e e p e r的基础上,提出了一种基于Zookeeper的统一配置管理方案,将配置文件内容存储在Zookeeper的节点中,利用Zookeeper的发布/订阅机制,实现了配置文件在管理中...针对现有客票交易中间件缺乏有效机制管理配置文件的问题,在研究Z o o k e e p e r的基础上,提出了一种基于Zookeeper的统一配置管理方案,将配置文件内容存储在Zookeeper的节点中,利用Zookeeper的发布/订阅机制,实现了配置文件在管理中心与客票交易中间件同步,结合日常维护需求,提供了配置文件的版本管理与B/S管理功能。测试数据表明该方案简化了配置文件的管理过程,提高了升级维护的效率。展开更多
基于业务发展需求,市国土资源局需要集成下属县(市)区国土资源局的档案数据,并提供全大市统一档案查询视图,故设计和实现了一种基于Zookeeper服务注册中心机制,采用REST服务风格的分布式档案数据查询框架,通过RESTful Web Service解决...基于业务发展需求,市国土资源局需要集成下属县(市)区国土资源局的档案数据,并提供全大市统一档案查询视图,故设计和实现了一种基于Zookeeper服务注册中心机制,采用REST服务风格的分布式档案数据查询框架,通过RESTful Web Service解决了异地分布数据查询的问题,同时结合Lucene搜索引擎提供的全文索引功能,提高了数据查询速度和查准率,实际系统投入使用效果良好。展开更多
文摘In order to address the problems of the single encryption algorithm,such as low encryption efficiency and unreliable metadata for static data storage of big data platforms in the cloud computing environment,we propose a Hadoop based big data secure storage scheme.Firstly,in order to disperse the NameNode service from a single server to multiple servers,we combine HDFS federation and HDFS high-availability mechanisms,and use the Zookeeper distributed coordination mechanism to coordinate each node to achieve dual-channel storage.Then,we improve the ECC encryption algorithm for the encryption of ordinary data,and adopt a homomorphic encryption algorithm to encrypt data that needs to be calculated.To accelerate the encryption,we adopt the dualthread encryption mode.Finally,the HDFS control module is designed to combine the encryption algorithm with the storage model.Experimental results show that the proposed solution solves the problem of a single point of failure of metadata,performs well in terms of metadata reliability,and can realize the fault tolerance of the server.The improved encryption algorithm integrates the dual-channel storage mode,and the encryption storage efficiency improves by 27.6% on average.
文摘针对现有客票交易中间件缺乏有效机制管理配置文件的问题,在研究Z o o k e e p e r的基础上,提出了一种基于Zookeeper的统一配置管理方案,将配置文件内容存储在Zookeeper的节点中,利用Zookeeper的发布/订阅机制,实现了配置文件在管理中心与客票交易中间件同步,结合日常维护需求,提供了配置文件的版本管理与B/S管理功能。测试数据表明该方案简化了配置文件的管理过程,提高了升级维护的效率。
文摘基于业务发展需求,市国土资源局需要集成下属县(市)区国土资源局的档案数据,并提供全大市统一档案查询视图,故设计和实现了一种基于Zookeeper服务注册中心机制,采用REST服务风格的分布式档案数据查询框架,通过RESTful Web Service解决了异地分布数据查询的问题,同时结合Lucene搜索引擎提供的全文索引功能,提高了数据查询速度和查准率,实际系统投入使用效果良好。
