一种跨域数据计算共享系统设计方法

Method of Cross-domain Data and Computing Resources Sharing System

针对数字化应用实践中需要在不同的物理域和信息域中进行数据的访问交换以及共享计算等需求,本文分析了在数据平台、数据集成系统以及信息交换系统中存在的问题。在基于联邦学习的基础上,提出一种跨域数据计算共享系统,能够同时共享数据和计算资源,并支持在线计算。在架构设计上设计了基础资源层、共享资源层及应用层,突出共享资源层中针对共享数据资源及共享计算资源的设计逻辑。模块设计中重点解决跨域数据集搜索问题以及算力共享激励问题,首先通过容器虚拟化屏蔽计算资源的异构性,设置下载控制量实现一定程度的隐私保护;然后使用基于数据连接的跨域数据集搜索,使得搜索能够从数据集简介层面上深入到数据集表,提升跨域数据集搜索的准确度,为了扩充和平稳系统算力,分别设置了用户奖励与惩罚策略,通过资源异构性屏蔽、搜索策略改进以及激励策略的综合使用,提升了跨域数据计算共享系统的易用性、稳定性;最后通过本文方法与两种数据共享系统的方法进行了验证对比,证明本方法对跨域数据计算共享效率的提升。

For the needs of accessing and exchanging data and sharing computation in different physical and information domains in digital application practice,this paper analyzes the problems in data platforms,data integration systems and information exchange systems.Based on federal learning,a cross-domain data computing sharing system is proposed,which can share data and computing resources simultaneously and support online computing.The architecture is designed with a basic resource layer,a shared resource layer and an application layer,highlighting the design logic of the shared resource layer for shared data resources and shared computing resources.The system firstly shields the heterogeneity of computing resources through container virtualization and sets the download control amount to achieve a certain degree of privacy protection;Then uses the cross-domain dataset search based on data connection,which enables the search to go deep into the dataset table from the dataset profile level and improves the accuracy of cross-domain dataset search;In order to expand and smoothing the system arithmetic,user reward and penalty strategies are set up respectively.Through the combination of resource heterogeneity shielding,search strategy improvement and incentive strategy,the ease of use and stability of the cross-domain data computation sharing system are improved.Finally,a validation comparison between the method of this paper and two data sharing systems is conducted to demonstrate the improvement of the efficiency of this method for cross-domain data computation sharing.