Federated Learning on Internet of Things:Extensive and Systematic Review

The proliferation of IoT devices requires innovative approaches to gaining insights while preserving privacy and resources amid unprecedented data generation.However,FL development for IoT is still in its infancy and needs to be explored in various areas to understand the key challenges for deployment in real-world scenarios.The paper systematically reviewed the available literature using the PRISMA guiding principle.The study aims to provide a detailed overview of the increasing use of FL in IoT networks,including the architecture and challenges.A systematic review approach is used to collect,categorize and analyze FL-IoT-based articles.Asearch was performed in the IEEE,Elsevier,Arxiv,ACM,and WOS databases and 92 articles were finally examined.Inclusion measures were published in English and with the keywords“FL”and“IoT”.The methodology begins with an overview of recent advances in FL and the IoT,followed by a discussion of how these two technologies can be integrated.To be more specific,we examine and evaluate the capabilities of FL by talking about communication protocols,frameworks and architecture.We then present a comprehensive analysis of the use of FL in a number of key IoT applications,including smart healthcare,smart transportation,smart cities,smart industry,smart finance,and smart agriculture.The key findings from this analysis of FL IoT services and applications are also presented.Finally,we performed a comparative analysis with FL IID(independent and identical data)and non-ID,traditional centralized deep learning(DL)approaches.We concluded that FL has better performance,especially in terms of privacy protection and resource utilization.FL is excellent for preserving privacy becausemodel training takes place on individual devices or edge nodes,eliminating the need for centralized data aggregation,which poses significant privacy risks.To facilitate development in this rapidly evolving field,the insights presented are intended to help practitioners and researchers navigate the complex terrain of FL and IoT.

An Incentive Mechanism for Federated Learning:A Continuous Zero-Determinant Strategy Approach

As a representative emerging machine learning technique, federated learning(FL) has gained considerable popularity for its special feature of “making data available but not visible”. However, potential problems remain, including privacy breaches, imbalances in payment, and inequitable distribution.These shortcomings let devices reluctantly contribute relevant data to, or even refuse to participate in FL. Therefore, in the application of FL, an important but also challenging issue is to motivate as many participants as possible to provide high-quality data to FL. In this paper, we propose an incentive mechanism for FL based on the continuous zero-determinant(CZD) strategies from the perspective of game theory. We first model the interaction between the server and the devices during the FL process as a continuous iterative game. We then apply the CZD strategies for two players and then multiple players to optimize the social welfare of FL, for which we prove that the server can keep social welfare at a high and stable level. Subsequently, we design an incentive mechanism based on the CZD strategies to attract devices to contribute all of their high-accuracy data to FL.Finally, we perform simulations to demonstrate that our proposed CZD-based incentive mechanism can indeed generate high and stable social welfare in FL.

A Privacy-Preserving Federated Learning Algorithm for Intelligent Inspection in Pumped Storage Power Station

As a distributed machine learning architecture,Federated Learning(FL)can train a global model by exchanging users’model parameters without their local data.However,with the evolution of eavesdropping techniques,attackers can infer information related to users’local data with the intercepted model parameters,resulting in privacy leakage and hindering the application of FL in smart factories.To meet the privacy protection needs of the intelligent inspection task in pumped storage power stations,in this paper we propose a novel privacy-preserving FL algorithm based on multi-key Fully Homomorphic Encryption(FHE),called MFHE-PPFL.Specifically,to reduce communication costs caused by deploying the FHE algorithm,we propose a self-adaptive threshold-based model parameter compression(SATMPC)method.It can reduce the amount of encrypted data with an adaptive thresholds-enabled user selection mechanism that only enables eligible devices to communicate with the FL server.Moreover,to protect model parameter privacy during transmission,we develop a secret sharing-based multi-key RNS-CKKS(SSMR)method that encrypts the device’s uploaded parameter increments and supports decryption in device dropout scenarios.Security analyses and simulation results show that our algorithm can prevent four typical threat models and outperforms the state-of-the-art in communication costs with guaranteed accuracy.

Smart and collaborative industrial IoT: A federated learning and data space approach

Industry 4.0 has become a reality by fusing the Industrial Internet of Things(IIoT)and Artificial Intelligence(AI),providing huge opportunities in the way manufacturing companies operate.However,the adoption of this paradigm shift,particularly in the field of smart factories and production,is still in its infancy,suffering from various issues,such as the lack of high-quality data,data with high-class imbalance,or poor diversity leading to inaccurate AI models.However,data is severely fragmented across different silos owned by several parties for a range of reasons,such as compliance and legal concerns,preventing discovery and insight-driven IIoT innovation.Notably,valuable and even vital information often remains unutilized as the rise and adoption of AI and IoT in parallel with the concerns and challenges associated with privacy and security.This adversely influences interand intra-organization collaborative use of IIoT data.To tackle these challenges,this article leverages emerging multi-party technologies,privacy-enhancing techniques(e.g.,Federated Learning),and AI approaches to present a holistic,decentralized architecture to form a foundation and cradle for a cross-company collaboration platform and a federated data space to tackle the creeping fragmented data landscape.Moreover,to evaluate the efficiency of the proposed reference model,a collaborative predictive diagnostics and maintenance case study is mapped to an edge-enabled IIoT architecture.Experimental results show the potential advantages of using the proposed approach for multi-party applications accelerating sovereign data sharing through Findable,Accessible,Interoperable,and Reusable(FAIR)principles.

RIS-Assisted Federated Learning in Multi-Cell Wireless Networks

Over-the-air computation(AirComp)based federated learning(FL)has been a promising technique for distilling artificial intelligence(AI)at the network edge.However,the performance of AirComp-based FL is decided by the device with the lowest channel gain due to the signal alignment property.More importantly,most existing work focuses on a single-cell scenario,where inter-cell interference is ignored.To overcome these shortages,a reconfigurable intelligent surface(RIS)-assisted AirComp-based FL system is proposed for multi-cell networks,where a RIS is used for enhancing the poor user signal caused by channel fading,especially for the device at the cell edge,and reducing inter-cell interference.The convergence of FL in the proposed system is first analyzed and the optimality gap for FL is derived.To minimize the optimality gap,we formulate a joint uplink and downlink optimization problem.The formulated problem is then divided into two separable nonconvex subproblems.Following the successive convex approximation(SCA)method,we first approximate the nonconvex term to a linear form,and then alternately optimize the beamforming vector and phase-shift matrix for each cell.Simulation results demonstrate the advantages of deploying a RIS in multi-cell networks and our proposed system significantly improves the performance of FL.

Secure Federated Learning over Wireless Communication Networks with Model Compression

Although federated learning(FL)has become very popular recently,it is vulnerable to gradient leakage attacks.Recent studies have shown that attackers can reconstruct clients’private data from shared models or gradients.Many existing works focus on adding privacy protection mechanisms to prevent user privacy leakages,such as differential privacy(DP)and homomorphic encryption.These defenses may cause an increase in computation and communication costs or degrade the performance of FL.Besides,they do not consider the impact of wireless network resources on the FL training process.Herein,we propose weight compression,a defense method to prevent gradient leakage attacks for FL over wireless networks.The gradient compression matrix is determined by the user’s location and channel conditions.We also add Gaussian noise to the compressed gradients to strengthen the defense.This joint learning of wireless resource allocation and weight compression matrix is formulated as an optimization problem with the objective of minimizing the FL loss function.To find the solution,we first analyze the convergence rate of FL and quantify the effect of the weight matrix on FL convergence.Then,we seek the optimal resource block(RB)allocation by exhaustive search or ant colony optimization(ACO)and then use the CVX toolbox to obtain the optimal weight matrix to minimize the optimization function.The simulation results show that the optimized RB can accelerate the convergence of FL.

Implicit Learning and Explicit Learning in Second Language Vocabulary Acquisition

Implicit and explicit learning strategies of SL vocabulary acquisition are summarized based on precious studies and experiments. It is concluded that implicit learning strategies dolittlehelpto SL vocabulary acquisition, but explicit learning strategies play a very important part in SL vocabulary acquisition. Besides, an assumption is proposed: the more obvious explicit learning is in vocabulary acquisition, the more words learners can acquire. It is hoped that this research has certain implications for SL learners and teaching.

基于FL-XGBoost算法的砂泥岩识别方法——以胜利油田牛庄地区为例

砂泥岩识别任务通常基于测井曲线,依据经验公式、实地岩心取样、交会图和聚类分析等传统方法实现,但这些方法难以充分利用测井曲线所包含的砂泥岩特征,且精度低、效率低,人为影响因素大。为此,以测井和录井资料为基础,综合砂泥岩识别的关键技术难点,对测井参数进行敏感性分析,以选取适当的影响因素,通过多项预处理操作构建完整的训练数据集,并根据测井标签稀疏性的特点,引入Focal Loss函数,提出FL-XGBoost模型,进而开展胜利油田牛庄地区砂泥岩识别。研究结果表明,采用FL-XGBoost算法的砂泥岩识别模型对研究区砂泥岩识别的准确率达到了0.827。通过5种公开分类数据集设计对比实验,证明FL-XGBoost算法在识别分类领域上具有强泛化能力。

Blockchain-Based Architectural Framework for Vertical Federated Learning

The introduction of blockchain to federated learning(FL)is a promising solution to enable anonymous clients to collaboratively learn a shared prediction model using local data while avoiding the risk caused by the central server.However,the current researches only apply a shallow convergence between the two technologies.The aroused problems,such as the unsuitable consensus,the lack of incentive mechanism,and the incompetence of handling vertically partitioned data,make the blockchain-based FL exist in name only.This paper puts forward a novel blockchain-based framework for vertical FL with a specified consensus and incentive.Moreover,a real-world example is demonstrated to prove the practicability of our work.

Stereotypes in a Foreign Language Classroom --Modifying Negative Attitudes to Enhance Foreign Language Learning

The aim of the paper is to present various aspects of the phenomenon of stereotyping in the context of FL （foreign language） learning and teaching and to discuss practical solutions to be used in a FL classroom to teach the worm about the worm by questioning the stereotypes learners have of other nations and languages. This paper is an attempt to present some ideas of FL teachers＇ role in developing students＇ socio-cultural competence with the aim of raising their cross-cultural awareness and questioning the stereotypes students bring into a FL classroom. The methodology used was an analysis of fragment of tape scripts from listening comprehension activities from a course book preparing Polish secondary students for the school leaving exam. The topics discussed concern opinions about attitudes towards and judgments of various cultural aspects, be it drinking tea or discussing the weather, impressions people have about other nations, or languages people speak.

不平衡数据分类问题的FL逻辑回归算法

针对不平衡数据的分类问题,文章利用焦点损失函数可以挖掘困难样本的特性,提出了一种新的逻辑回归算法。首先,定义逻辑回归模型新的损失函数;其次,基于牛顿迭代法,设计FL逻辑回归算法;最后,在比较实验中,运用随机森林进行特征选择,以阈值优化逻辑回归模型为分类模型进行实验。实验结果表明,与传统逻辑回归算法相比,改进后的算法提高了少数类样本的分类精度,增强了模型的整体分类性能。

A Situated Learning Practice for Language Teaching Classes： Teaching Spoken English With Authentic Sketches

SL （situated learning） is a term first proposed by Lave and Wenger （1991） as a model of learning in a community of practice. According to Lave and Wenger （1991）, learning should not be viewed as simply the transmission of abstract and decontextualised knowledge from one individual to another, but a social process whereby knowledge is co-constructed. The exposure to spoken language and cultural elements of foreign language is the best way of teaching the language itself rather than grammatical patterns and rules of the language. In this study, we aim to review ＂situational learning approach＂ in context with its role and efficiency of teaching spoken language. An experimental study was conducted on the university students in the preparatory classes at the School of Tourism of Erzincan University. Twelve male and 11 female students in the control group and 14 male and 10 female students in the experimental group took part in the research. The language levels of the students were determined by a language proficiency test which is used as pre-test of the study. Language proficiency test composed of mainly dialogues including spoken language patterns. After eight weeks of lectures with authentic sketches which were used as reading materials in experimental group and classical reading materials in control group, the students were given the same language proficiency test as post-test. When pre- and post-test results were evaluated, significant difference was found between the pre- and post-test results of the subjects on behalf of the students in the experimental group. It is concluded that spoken language can be achieved by authentic sketches which are designed to serve as a situated learning setting.

The Role of Lexis in Developing EFL Learners＇ Speaking Skill

EFL （English as a Foreign Language） speaking is a very demanding skill that requires learners＇ socio-pragmatic as well as strategic competence in any interactional situation, and lexis proves to play a crucial role in this process. However, few studies have investigated how both EFL teachers and learners view and analyze situations in which learners are not producing enough spoken language in class, and the reasons behind them. The present study will pinpoint the significant role of lexis in Moroccan learners＇ speaking production. To this end, 40 EFL teachers and 200 Moroccan high school students are surveyed and interviewed to reveal their perceptions of the speaking skill and the corresponding high significance of lexis in this instance. Results show that both teachers and learners identify vocabulary deficiency as the main factor behind students＇ inability to speak English. In the present paper, among the many suggestions that could be proposed to deal with this situation, it is argued that one efficient way would be to assist the students during the process of L2 （second language） vocabulary learning through vocabulary learning strategy instruction. Pedagogical and research implication will be given in response to the difficulties encountered in this area as have been identified by the EFL teachers and learners surveyed.

基于相关性的Swarm联邦降维方法

联邦学习(Federated learning,FL)在解决人工智能(Artificial intelligence,AI)面临的隐私泄露和数据孤岛问题方面具有显著优势.针对联邦学习的已有研究未考虑联邦数据之间的关联性和高维性问题,提出一种基于联邦数据相关性的去中心化联邦降维方法.该方法基于Swarm学习(Swarm learning,SL)思想,通过分离耦合特征,构建典型相关分析(Canonical correlation analysis,CCA)的Swarm联邦框架,以提取Swarm节点的低维关联特征.为保护协作参数的隐私安全,还构建一种随机扰乱策略来隐藏Swarm特征隐私.在真实数据集上的实验验证了所提方法的有效性.

预算约束下多任务联邦学习激励机制

联邦学习是一种实现数据隐私保护的分布式机器学习范式,性能取决于数据源的质量和数据规模。客户端是理性个体,参与联邦学习将耗费计算、通信和隐私等成本,需要通过激励提高客户端的参与意愿。因此联邦学习能成功应用的关键之一是尽可能多地激励高质量数据客户端参与训练。多任务联邦学习环境下客户端拥有面向不同任务且质量不同的数据,并具有执行能力的约束。为提高多个学习任务的整体性能,在预算受限的条件下设计一种面向任务的客户选择和报酬机制。通过分析影响模型精度的重要因素,提出一种基于客户端数据样本分布特征的质量评估标准,并结合客户端成本信息,设计一种逆向拍卖的激励机制(EMD-MQMFL),实现客户端的任务指派和支付策略。从理论上分析和证明了该机制具有诚实性、个人理性以及预算可行性,并通过大量实验验证了该方法在联邦学习性能上的有效性。在MNIST、Fashion-MNIST、Cifar-10数据集上的实验结果表明,EMD-MQMFL在数据不平衡的情况下,平均模型精度比已有的机制至少提高5.6个百分点。

双因子更新的车联网双层异步联邦学习研究

针对车联网(IoV)中节点资源异构、拓扑结构动态变化等特点,该文建立了一个双因子更新的双层异步联邦学习(TTAFL)框架。考虑到模型版本差和车辆参与联邦学习(FL)次数对局部模型更新的影响,提出基于陈旧因子和贡献因子的模型更新方案。同时,为了避免训练过程中,车辆移动带来路侧单元切换的问题,给出考虑驻留时间的节点选择方案。最后,为了减少精度损失与系统能耗,利用强化学习方法优化联邦学习的本地迭代次数与路侧单元局部模型更新次数。仿真结果表明,所提算法有效提高了联邦学习的训练效率和训练精度,降低了系统能耗。

非独立同分布下联邦半监督学习的数据分享研究

联邦学习作为一种保护本地数据隐私安全的分布式机器学习方法,联合分散的设备共同训练共享模型。通常联邦学习在数据均有标签情况下进行训练,然而现实中无法保证标签数据完全存在,提出联邦半监督学习。在联邦半监督学习中,如何利用无标签数据提升系统性能和如何缓解数据异质性带来的负面影响是两大挑战。针对标签数据仅在服务器场景,基于分享的思想,设计一种可应用在联邦半监督学习系统上的方法Share&Mark,该方法将客户端的分享数据由专家标记后参与联邦训练。同时,为充分利用分享的数据,根据各客户端模型在服务器数据集上的损失值动态调整各客户端模型在联邦聚合时的占比,即ServerLoss聚合算法。综合考虑隐私牺牲、通信开销以及人工标注成本3个方面的因素,对不同分享率下的实验结果进行分析,结果表明,约3%的数据分享比例能平衡各方面因素。此时,采用Share&Mark方法的联邦半监督学习系统FedMatch在CIFAR-10和Fashion-MNIST数据集上训练的模型准确率均可提升8%以上,并具有较优的鲁棒性。

车联网中基于有向无环图区块链的个性化联邦互蒸馏学习方法

联邦学习(FL)作为一种分布式训练方法,在车联网(IoV)中得到了广泛应用。区别于传统机器学习,FL允许智能网联车辆(CAVs)通过共享模型而非原始数据来协同训练全局模型,从而保护CAV隐私和数据安全。为了提升联邦学习模型精度,降低通信开销,该文首先提出一种基于有向无环图(DAG)区块链和CAVs的IoV架构,分别负责全局模型共享和本地模型训练。其次,设计了一种基于DAG区块链的异步联邦互蒸馏学习(DAFML)算法在本地同时训练教师和学生模型,教师模型的专业级网络结构可取得更高精度,学生模型的轻量级网络结构可降低通信开销,并采用互蒸馏学习使教师模型和学生模型从互相转移的软标签中学习知识以更新模型。此外,为了进一步提高模型精度,基于全局训练轮次和模型精度设定个性化权值来调节互蒸馏占比。仿真结果表明,DAFML算法在模型精度和蒸馏比率方面优于其他比较算法。

面向非独立同分布数据的车联网多阶段联邦学习机制

车联网在智慧城市建设中扮演着不可或缺的角色,汽车不仅仅是交通工具,更是大数据时代信息采集和传输的重要载体.随着车辆采集的数据量飞速增长和人们隐私保护意识的增强,如何在车联网环境中确保用户数据安全,防止数据泄露,成为亟待解决的难题.联邦学习采用“数据不动模型动”的方式,为保护用户隐私和实现良好性能提供了可行方案.然而,受限于采集设备、地域环境、个人习惯的差异,多台车辆采集的数据通常表现为非独立同分布(non-independent and identically distributed,non-IID)数据,而传统的联邦学习算法在non-IID数据环境中,其模型收敛速度较慢.针对这一挑战,提出了一种面向non-IID数据的车联网多阶段联邦学习机制,称为FedWO.第1阶段采用联邦平均算法,使得全局模型快速达到一个基本的模型准确度;第2阶段采用联邦加权多方计算,依据各车辆的数据特性计算其在全局模型中的权重,聚合后得到性能更优的全局模型,同时采用传输控制策略,减少模型传输带来的通信开销;第3阶段为个性化计算阶段,车辆利用各自的数据进行个性化学习,微调本地模型获得与本地数据更匹配的模型.实验采用了驾驶行为数据集进行实验评估,结果表明相较于传统方法,在non-IID数据场景下,FedWO机制保护了数据隐私,同时提高了算法的准确度.

双端聚类的自动调整聚类联邦学习

联邦学习(FL)是一种分布式机器学习方法,旨在共同训练全局模型,然而全局模型难以胜任多数据分布情况。为应对多分布挑战,引入聚类联邦学习,以客户端分组方式优化共享多模型。其中,服务器端聚类难以修正分类错误,而客户端聚类则对初始模型的选择至关重要。为解决这些问题,提出自动调整聚类联邦学习(AACFL)框架,所提框架采用双端聚类整合服务器端和客户端聚类。首先用双端聚类将客户端分为可调整集群,其次自动调整局部客户端身份,最后获取正确的客户集群。在非独立同分布下,在3个经典联邦数据集上的评估实验结果表明,AACFL能够在双端聚类结果存在错误的情况下通过调整获得正确集群,当簇数为4,客户端数为100时,与联邦平均(FedAvg)算法、聚类联邦学习(CFL)和IFCA(Iterative Federated Clustering Algorithm)等方法相比,有效地提高模型收敛速度和获得正确聚类结果的速度,准确率平均提升0.20~23.16个百分点。验证了所提框架能够高效聚类,并提高模型收敛速度和准确率。