Deep learning and transfer learning for device-free human activity recognition:A survey

Device-free activity recognition plays a crucial role in smart building,security,and human–computer interaction,which shows its strength in its convenience and cost-efficiency.Traditional machine learning has made significant progress by heuristic hand-crafted features and statistical models,but it suffers from the limitation of manual feature design.Deep learning overcomes such issues by automatic high-level feature extraction,but its performance degrades due to the requirement of massive annotated data and cross-site issues.To deal with these problems,transfer learning helps to transfer knowledge from existing datasets while dealing with the negative effect of background dynamics.This paper surveys the recent progress of deep learning and transfer learning for device-free activity recognition.We begin with the motivation of deep learning and transfer learning,and then introduce the major sensor modalities.Then the deep and transfer learning techniques for device-free human activity recognition are introduced.Eventually,insights on existing works and grand challenges are summarized and presented to promote future research.

Estimating the State of Health for Lithium-ion Batteries:A Particle Swarm Optimization-Assisted Deep Domain Adaptation Approach

The state of health(SOH)is a critical factor in evaluating the performance of the lithium-ion batteries(LIBs).Due to various end-user behaviors,the LIBs exhibit different degradation modes,which makes it challenging to estimate the SOHs in a personalized way.In this article,we present a novel particle swarm optimization-assisted deep domain adaptation(PSO-DDA)method to estimate the SOH of LIBs in a personalized manner,where a new domain adaptation strategy is put forward to reduce cross-domain distribution discrepancy.The standard PSO algorithm is exploited to automatically adjust the chosen hyperparameters of developed DDA-based method.The proposed PSODDA method is validated by extensive experiments on two LIB datasets with different battery chemistry materials,ambient temperatures and charge-discharge configurations.Experimental results indicate that the proposed PSO-DDA method surpasses the convolutional neural network-based method and the standard DDA-based method.The Py Torch implementation of the proposed PSO-DDA method is available at https://github.com/mxt0607/PSO-DDA.

Label Recovery and Trajectory Designable Network for Transfer Fault Diagnosis of Machines With Incorrect Annotation

The success of deep transfer learning in fault diagnosis is attributed to the collection of high-quality labeled data from the source domain.However,in engineering scenarios,achieving such high-quality label annotation is difficult and expensive.The incorrect label annotation produces two negative effects:1)the complex decision boundary of diagnosis models lowers the generalization performance on the target domain,and2)the distribution of target domain samples becomes misaligned with the false-labeled samples.To overcome these negative effects,this article proposes a solution called the label recovery and trajectory designable network(LRTDN).LRTDN consists of three parts.First,a residual network with dual classifiers is to learn features from cross-domain samples.Second,an annotation check module is constructed to generate a label anomaly indicator that could modify the abnormal labels of false-labeled samples in the source domain.With the training of relabeled samples,the complexity of diagnosis model is reduced via semi-supervised learning.Third,the adaptation trajectories are designed for sample distributions across domains.This ensures that the target domain samples are only adapted with the pure-labeled samples.The LRTDN is verified by two case studies,in which the diagnosis knowledge of bearings is transferred across different working conditions as well as different yet related machines.The results show that LRTDN offers a high diagnosis accuracy even in the presence of incorrect annotation.

Enhancing source domain availability through data and feature transfer learning for building power load forecasting

During the initial phases of operation following the construction or renovation of existing buildings,the availability of historical power usage data is limited,which leads to lower accuracy in load forecasting and hinders normal usage.Fortunately,by transferring load data from similar buildings,it is possible to enhance forecasting accuracy.However,indiscriminately expanding all source domain data to the target domain is highly likely to result in negative transfer learning.This study explores the feasibility of utilizing similar buildings(source domains)in transfer learning by implementing and comparing two distinct forms of multi-source transfer learning.Firstly,this study focuses on the Higashita area in Kitakyushu City,Japan,as the research object.Four buildings that exhibit the highest similarity to the target buildings within this area were selected for analysis.Next,the two-stage TrAdaBoost.R^(2) algorithm is used for multi-source transfer learning,and its transfer effect is analyzed.Finally,the application effects of instance-based(IBMTL)and feature-based(FBMTL)multi-source transfer learning are compared,which explained the effect of the source domain data on the forecasting accuracy in different transfer modes.The results show that combining the two-stage TrAdaBoost.R^(2) algorithm with multi-source data can reduce the CV-RMSE by 7.23%compared to a single-source domain,and the accuracy improvement is significant.At the same time,multi-source transfer learning,which is based on instance,can better supplement the integrity of the target domain data and has a higher forecasting accuracy.Overall,IBMTL tends to retain effective data associations and FBMTL shows higher forecasting stability.The findings of this study,which include the verification of real-life algorithm application and source domain availability,can serve as a theoretical reference for implementing transfer learning in load forecasting.

Deep learning for broadleaf weed seedlings classification incorporating data variability and model flexibility across two contrasting environments

The increasing deployment of deep learning models for distinguishing weeds and crops has witnessed notable strides in agricultural scenarios.However,a conspicuous gap endures in the literature concerning the training and testing of models across disparate environmental conditions.Predominant methodologies either delineate a single dataset distribution into training,validation,and testing subsets or merge datasets from diverse condi-tions or distributions before their division into the subsets.Our study aims to ameliorate this gap by extending to several broadleaf weed categories across varied distributions,evaluating the impact of training convolutional neural networks on datasets specific to particular conditions or distributions,and assessing their performance in entirely distinct settings through three experiments.By evaluating diverse network architectures and training approaches(finetuning versus feature extraction),testing various architectures,employing different training strategies,and amalgamating data,we devised straightforward guidelines to ensure the model's deployability in contrasting environments with sustained precision and accuracy.In Experiment 1,conducted in a uniform environment,accuracy ranged from 80%to 100%across all models and training strategies,with finetune mode achieving a superior performance of 94%to 99.9%compared to the feature extraction mode at 80%to 92.96%.Experiment 2 underscored a significant performance decline,with accuracy fig-ures between 25%and 60%,primarily at 40%,when the origin of the test data deviated from the train and valida-tion sets.Experiment 3,spotlighting dataset and distribution amalgamation,yielded promising accuracy metrics,notably a peak of 99.6%for ResNet in finetuning mode to a low of 69.9%for InceptionV3 in feature extraction mode.These pivotal findings emphasize that merging data from diverse distributions,coupled with finetuned training on advanced architectures like ResNet and MobileNet,markedly enhances performance,contrasting with the rel-atively lower performance exhibited by simpler networks like AlexNet.Our results suggest that embracing data diversity and flexible training methodologies are crucial for optimizing weed classification models when dispa-rate data distributions are available.This study gives a practical alternative for treating diverse datasets with real-world agricultural variances.

深度多模态迁移学习在轴承故障诊断中的研究

在工程实际中,设备数据样本往往以正常数据居多。故障样本稀缺且模态单一使得可用于训练的故障信息特征提取不足,同时训练和测试数据分布往往存在差异,从而导致模型迁移诊断能力较弱。针对该问题,提出将深度学习模型CNN与多模态融合迁移学习技术相结合(Deep Multimodal Fusion Transfer Learning,DMFTL)应用于轴承的故障诊断中。首先以CNN为基本学习框架,将原始一维振动信号的时域和频域进行多模态信息融合对模型预训练;然后以最大均值差异(MMD)为度量准则,通过域自适应来最小化源域和目标域的差异;最后引入构造的正则项到模型中,以完成跨域诊断。通过对CWRU轴承数据集的迁移诊断试验及对比分析,验证了该方法的有效性和优越性。

面向手术导航3D/2D配准的无监督跨域迁移网络

3D/2D配准是手术导航的关键技术,现有基于深度学习的配准方法通过网络提取图像特征,继而回归出相应的姿态变换参数。此类方法依赖于真实的样本以及对应的3D标签用于训练,然而这部分专家标注的医疗数据十分稀缺。替代的方案用数字重建放射影像(DRR)图像训练网络,由于图像特征跨域的差异,在X射线图像上难以保持原有的配准精度。针对上述问题,设计基于自注意力的无监督跨域迁移网络(UCDTN),无须依赖X射线图像与其对应的3D空间标签作为训练样本,将源域所捕获的图像特征与空间变换间的对应关系迁移到目标域,借助公共特征减小域间特征的差距、降低跨域所带来的负面影响。实验结果表明,UCDTN预测结果的平均配准误差(mTRE)为2.66 mm;与未经跨域迁移训练的模型相比,mTRE指标降低了70.61%,验证了UCDTN在跨域配准任务上的有效性。

开放环境下的跨域物体检测综述

传统的物体检测模型假设模型的训练和测试数据来自相同或相似的场景,然而该假设在实际运用中难以满足,即检测模型被要求在不同的环境或场景下进行工作,使得传统模型不可避免地受到影响,导致检测精度明显下降.为了解决这个问题,近年来跨域物体检测问题受到了广泛关注.文中介绍了近几年跨域物体检测问题的发展历程和相关方法,将跨域物体检测方法归纳为基于迁移学习的、自学习的和图像生成的3大类.其中,基于迁移学习的方法结合域适应和物体检测方法,提升模型对不同环境的适应能力;基于自学习的方法利用伪标签提升模型在目标域上的迁移能力;基于图像生成的方法利用生成式对抗网络生成相关的图像辅助模型训练,提升模型在目标域的效果.同时,介绍了用于跨域物体检测的相关数据集和代表性方法的性能.最后总结跨域物体检测现阶段的分类以及存在的不足,并指出对未知域泛化性能的探索、数据隐私问题的解决,以及视觉提示技术的应用等新发展方向.

脉冲噪声下基于域自适应的调制识别

针对基于深度神经网络进行调制识别会存在模型失配导致识别率显著下降的问题,本文提出了一种基于域自适应的调制识别算法,在测试样本与训练样本存在较大分布差异时,仍具有很高的识别率。该算法对脉冲噪声下的调制信号的星座图进行处理,恢复出几何特征,强化特征并生成图片,将生成的图片输入到深度域自适应网络中完成调制识别。该算法实现了兼具大动态范围的广义信噪比和大动态范围的脉冲强度条件下的调制识别。仿真实验表明:该算法可以有效解决模型失配问题,具有良好的识别性能。

基于注意力掩码的领域泛化研究

深度学习在区分特征方面表现较好,但将其应用于未知领域时,已训练好的模型往往会因领域移位而导致模型性能下降。针对该情况,领域泛化(DG)从多个源领域学习可迁移特征,将其泛化到未知的目标领域。由于不同领域训练的模型更偏向其中最显著的特征,往往会忽略与任务相关的一般性特征,而可迁移特征通常并不是该领域最显著的特征。因此,从这个角度提出一种基于注意力掩码来屏蔽特征的正则化方法,通过注意力掩码模块生成注意力掩码,对权重高的特征进行屏蔽,以提升模型泛化性能。实验表明,在3个基准数据集上测试的精度相较于基线模型分别提升2.6%、2.0%、4.2%,证明该方法既能提升模型在未知领域上的性能,也体现了其在领域泛化数据集上的普适性。

Transformer在域适应中的应用研究综述

作为迁移学习的重要分支,域适应旨在解决传统机器学习算法在训练样本和测试样本服从不同数据分布时性能急剧下降的问题。Transformer是基于自注意力机制的深度学习框架,具有强大的全局特征提取能力和建模能力,近年来Transformer与域适应相结合也成为研究的热点。虽然已有大量相关方法问世,但Transformer应用在域适应的综述却未见报道。为了填补这个领域的空白,为相关研究提供借鉴和参考,对近年来出现的一些基于Transformer的典型域适应方法进行归纳总结与分析,概述域适应的相关概念与Transformer的基本结构,从图像分类、图像语义分割、目标检测、医学图像分析四个应用梳理了各种基于Transformer的域适应方法,对图像分类下的域适应方法进行比较,总结当前域适应Transformer模型存在的挑战并探讨未来可行的研究方向。

多源域适应方法综述

域适应是解决源域样本和目标域样本不满足独立同分布问题的迁移学习范式,是当下研究的重点方法。然而实际情况下获取源域样本的渠道和方法并不唯一,这会导致源域中存在多种不同分布的样本。多源域适应方法是解决源域样本分布多样性问题的有效途径,其主要研究各源域分布间的关系和与目标域分布对齐的策略,进一步减轻各域之间的域偏移,具有实用意义和挑战价值。随着深度学习技术的不断进步,多源域适应方法主要使用深度神经网络提取各域的域不变特征作为分布对齐的依据,结合使用度量准则衡量分布差异或者利用对抗思想对齐域间分布。经过理论证明和实验验证,多源域适应方法训练的模型比单源域方法训练的模型具有更好的泛化性能,更符合现实需求。通过介绍多源域适应的研究现状和相关概念,对现有算法进行总结和综述,按照迁移方式不同对多源域适应方法进行分类,进一步分析多源域适应方法性能的实验结果,阐述其存在的不足和缺点,并对多源域适应领域的发展和趋势进行预测。

基于无监督域适应的室外点云语义分割

为处理室外大规模场景中语义分割网络训练需求数据量过大的问题,提出一种基于无监督域自适应的点云语义分割方法。该方法使用改进的RandLA-Net以SPTLS3D真实世界数据集的少量点云作为目标对象进行语义分割。模型在SensatUrban数据集上完成分割网络的预训练,通过缩小源域和目标域之间的域差距来完成模型的迁移。RandLA-Net编码过程会缺失原始点云全局特征,因此本文提出一种额外获取全局信息加入网络解码的方法。此外,为增强差异化信息的获取,RandLA-Net的局部注意力模块权值改为根据各点的特征和其邻域的平均特征的差值。实验显示,该网络在Se manticKITTI数据集上的平均交并比精度达到54.3%,在Semantic3D上的平均交并比精度达到了71.91%。预训练好的模型经过微调后平均交并比精度达到了80.05%,比直接训练的效果好8.83个百分点。

基于深度域适应CNN决策树的跨语料库情感识别

在跨语料库语音情感识别中,由于目标域和源域样本不匹配,导致情感识别性能很差。为了提高跨语料库语音情感识别性能,本文提出一种基于深度域适应和卷积神经网络(Convolutional neural network,CNN)决策树模型的跨语料库语音情感识别方法。首先构建基于联合约束深度域适应的局部特征迁移学习网络,通过最小化目标域和源域在特征空间和希尔伯特空间的联合差异,挖掘两个语料库之间的相关性,学习从目标域到源域的可迁移不变特征。然后,为了降低跨语料库背景下多种情感间的易混淆情感的分类误差,依据情感混淆度构建CNN决策树多级分类模型,对多种情感先粗分类再细分类。使用CASIA,EMO-DB和RAVDESS三个语料库进行验证。实验结果表明,本文的跨语料库语音情感识别方法比CNN基线方法平均识别率高19.32%~31.08%,系统性能得到很大提升。

基于深度迁移学习的天气图像识别

当对天气图像等场景复杂和特征不明显的图像进行识别时,往往存在识别率不高和特征冗余等问题。基于此,本文提出了一种基于深度迁移学习的图像分类算法。该算法利用ImageNet数据集的模型参数构建ResNeXt、Xception以及SENet 3种网络模型提取图像特征,采用领域自适应的判别联合分布自适应算法来相似化特征向量,完成高质量的特征表示,并以其结果为准则融合模型特征,将融合特征经过多层感知机训练以实现高准确率识别的图像分类。实验结果表明,该算法的性能优于传统的单一网络模型,进一步提升了图像分类准确率的上限。

具有类间差异约束的多对抗深度域适应模型

为实现目标域样本能够与源域中同类样本准确对齐,并在保证样本准确识别率的条件下进一步提高不同类别样本特征间的可区分性,提出了一种带有类间差异约束的域适应模型。首先,该模型采用深度卷积神经网络对源域样本进行了有监督学习,并在训练过程中基于提出的类间差异测量函数对源域样本特征加以类间差异性约束;其次,该模型采用了多对抗域鉴别网络结构,其中提出了一种目标域样本伪标签计算方法,从而将无标签的样本指定到合理的域鉴别网络进行训练;最后,通过最小化分类损失与最大化域鉴别损失,获得最优特征提取器与特征分类器。实验结果表明,对于4种数据集,提出的模型在目标域上平均识别准确率可以达到0.860,同类间的平均距离、不同类间的平均距离、目标域中样本错误识别率相对于改进前分别降低0.003,提升0.065,降低0.025,从而验证了提出模型的性能得到了明显提升。

Rotating machinery fault detection and diagnosis based on deep domain adaptation:A survey

In practical mechanical fault detection and diagnosis,it is difficult and expensive to collect enough large-scale supervised data to train deep networks.Transfer learning can reuse the knowledge obtained from the source task to improve the performance of the target task,which performs well on small data and reduces the demand for high computation power.However,the detection performance is significantly reduced by the direct transfer due to the domain difference.Domain adaptation(DA)can transfer the distribution information from the source domain to the target domain and solve a series of problems caused by the distribution difference of data.In this survey,we review various current DA strategies combined with deep learning(DL)and analyze the principles,advantages,and disadvantages of each method.We also summarize the application of DA combined with DL in the field of fault diagnosis.This paper provides a summary of the research results and proposes future work based on analysis of the key technologies.

基于迁移学习的表面肌电信号手势识别

为了解决基于深度学习方法的表面肌电(sEMG)信号手势识别模型训练中需要大量数据的问题,该文将迁移学习方法与卷积神经网络(CNN)相结合,设计了一种微调模型参数的训练方法。设计实验对16名被试者采集了8种手势的sEMG信号。利用已有被试者的sEMG信号作为源域数据对CNN进行训练,模型在源域数据上的分类准确率为85.3%~98.11%。取其他被试者的sEMG信号作为目标域数据,数据量大小为源域数据的10%。在已训练好的模型基础上固定前2层卷积层,微调其余层级的参数,完成模型在不同个体间的迁移。迁移后的模型在目标域上的分类准确率为84.14%~97.4%。

基于联合对抗训练的鲁棒度量迁移

迁移度量学习旨在从强大且可靠的距离度量中迁移知识来改善目标度量的效果,这些度量往往来自于学习目标相关的任务.现有的迁移度量学习算法仅关注于如何迁移知识,而这些知识容易过拟合到源域中.首先研究如何在源域中训练一个适合于迁移的源域度量,然后设计了一种通用的深度异质迁移算法来进行高效的迁移学习.值得注意的是,将源域度量以联合对抗学习的方式进行训练,再以深度神经网络的方式将其参数化表示并对其进行迁移.迁移中通过表征模仿的方式来学习源域度量中的知识,这种方式允许源域和目标域中的知识来自于异质域.此外,严格限制目标度量网络的大小,使得目标网络更够进行高效的推理计算.在人脸识别数据集上的实验展现了本方法的有效性.

基于域特征融合网络的跨工况下多组件设备寿命预测方法研究

针对不同工况下多组件设备退化数据分布存在差异导致设备的寿命预测模型精度下降的问题,本文提出一种能适应于不同工况的域特征融合网络(DFF-Net)。首先,把不同工况的退化数据输入到特征提取网络以获取跨工况特征,然后利用域特征融合网络(DFF-Net)对跨工况特征进行域适应调整,最后把调整后的数据输入寿命预测模型,输出不同工况下设备的寿命预测结果。通过在公开数据集上的试验表明,相比于没有增加域特征融合网络的寿命预测模型,本文模型在测试集上预测结果的MAE和RMSE分别降低了6.5%和7.4%,说明本文模型能有效地提高跨工况设备寿命预测的准确率。