针对BERT预训练与下游任务微调阶段存在不匹配差异,以及人工对文本数据进行情感倾向性标注可能存在误差的问题,提出一种基于MacBERT和标签平滑的网络模型(MacLMC).首先,在BERT的基础上引入MLM as correction策略,利用近义词替换被掩码词...针对BERT预训练与下游任务微调阶段存在不匹配差异,以及人工对文本数据进行情感倾向性标注可能存在误差的问题,提出一种基于MacBERT和标签平滑的网络模型(MacLMC).首先,在BERT的基础上引入MLM as correction策略,利用近义词替换被掩码词,通过MacBERT预训练模型获取词向量;其次,经过双层LSTM学习长距离依赖;再次,采用双通道多卷积核的卷积操作,分别提取信息的最大特征和均值特征;最后,利用标签平滑策略降低模型预测类别的概率,提升模型对于标签的容错能力,提高模型泛化性.实验结果表明:与现有主流模型相比,本文模型在多种数据集上性能表现更佳,能够更好地用于新冠疫情公众情感分析任务.展开更多
Existing unsupervised person re-identification approaches fail to fully capture thefine-grained features of local regions,which can result in people with similar appearances and different identities being assigned the...Existing unsupervised person re-identification approaches fail to fully capture thefine-grained features of local regions,which can result in people with similar appearances and different identities being assigned the same label after clustering.The identity-independent information contained in different local regions leads to different levels of local noise.To address these challenges,joint training with local soft attention and dual cross-neighbor label smoothing(DCLS)is proposed in this study.First,the joint training is divided into global and local parts,whereby a soft attention mechanism is proposed for the local branch to accurately capture the subtle differences in local regions,which improves the ability of the re-identification model in identifying a person’s local significant features.Second,DCLS is designed to progressively mitigate label noise in different local regions.The DCLS uses global and local similarity metrics to semantically align the global and local regions of the person and further determines the proximity association between local regions through the cross information of neighboring regions,thereby achieving label smoothing of the global and local regions throughout the training process.In extensive experiments,the proposed method outperformed existing methods under unsupervised settings on several standard person re-identification datasets.展开更多
文摘针对BERT预训练与下游任务微调阶段存在不匹配差异,以及人工对文本数据进行情感倾向性标注可能存在误差的问题,提出一种基于MacBERT和标签平滑的网络模型(MacLMC).首先,在BERT的基础上引入MLM as correction策略,利用近义词替换被掩码词,通过MacBERT预训练模型获取词向量;其次,经过双层LSTM学习长距离依赖;再次,采用双通道多卷积核的卷积操作,分别提取信息的最大特征和均值特征;最后,利用标签平滑策略降低模型预测类别的概率,提升模型对于标签的容错能力,提高模型泛化性.实验结果表明:与现有主流模型相比,本文模型在多种数据集上性能表现更佳,能够更好地用于新冠疫情公众情感分析任务.
基金supported by the National Natural Science Foundation of China under Grant Nos.62076117 and 62166026the Jiangxi Key Laboratory of Smart City under Grant No.20192BCD40002Jiangxi Provincial Natural Science Foundation under Grant No.20224BAB212011.
文摘Existing unsupervised person re-identification approaches fail to fully capture thefine-grained features of local regions,which can result in people with similar appearances and different identities being assigned the same label after clustering.The identity-independent information contained in different local regions leads to different levels of local noise.To address these challenges,joint training with local soft attention and dual cross-neighbor label smoothing(DCLS)is proposed in this study.First,the joint training is divided into global and local parts,whereby a soft attention mechanism is proposed for the local branch to accurately capture the subtle differences in local regions,which improves the ability of the re-identification model in identifying a person’s local significant features.Second,DCLS is designed to progressively mitigate label noise in different local regions.The DCLS uses global and local similarity metrics to semantically align the global and local regions of the person and further determines the proximity association between local regions through the cross information of neighboring regions,thereby achieving label smoothing of the global and local regions throughout the training process.In extensive experiments,the proposed method outperformed existing methods under unsupervised settings on several standard person re-identification datasets.
