利用联邦学习技术促进大数据中心的集成应用

Integrated Application of Big Data Center Using the Technology of Federated Learning

基于开源联邦学习框架,构建支持横向联邦学习、纵向联邦学习以及联邦迁移学习的联邦学习平台,提供从数据接入到模型部署的一站式开发能力,简化联邦学习模型的开发和上线工作,降低使用的门槛。联邦学习平台能够有机地融入现有大数据中心,结合现有的数据资源分布特点,灵活运用平台的角色分配机制,创建有效的联邦学习模型。联邦学习平台及其部署方案能够打破现在大数据中心独立建设带来的“数据孤岛”局面,全方位提升大数据平台的使用效率。

Based on the open-source federated learning framework,the federated learning platform is built to support horizontal federated learning,vertical federated learning and federated transfer learning,providing one-step development capabilities from data access to model deployment,simplifying the development and deployment of federated learning models and reducing the threshold for use.The federated learning platform can be organically integrated into the existing big data center.With the distribution characteristics of the existing data resources combined,the role allocation mechanism is flexibly used,and the effective federated learning model is created.The federated learning platform and the deployment solution are able to break the“data island”situation brought by the independent construction of the big data center,comprehensively improving the efficiency of the big data platform.