Face forgery detection is drawing ever-increasing attention in the academic community owing to security concerns.Despite the considerable progress in existing methods,we note that:Previous works overlooked finegrain f...Face forgery detection is drawing ever-increasing attention in the academic community owing to security concerns.Despite the considerable progress in existing methods,we note that:Previous works overlooked finegrain forgery cues with high transferability.Such cues positively impact the model’s accuracy and generalizability.Moreover,single-modality often causes overfitting of the model,and Red-Green-Blue(RGB)modal-only is not conducive to extracting the more detailed forgery traces.We propose a novel framework for fine-grain forgery cues mining with fusion modality to cope with these issues.First,we propose two functional modules to reveal and locate the deeper forged features.Our method locates deeper forgery cues through a dual-modality progressive fusion module and a noise adaptive enhancement module,which can excavate the association between dualmodal space and channels and enhance the learning of subtle noise features.A sensitive patch branch is introduced on this foundation to enhance the mining of subtle forgery traces under fusion modality.The experimental results demonstrate that our proposed framework can desirably explore the differences between authentic and forged images with supervised learning.Comprehensive evaluations of several mainstream datasets show that our method outperforms the state-of-the-art detection methods with remarkable detection ability and generalizability.展开更多
针对“校园大数据”累积的海量数据呈现出离散性、稀疏性等问题,如何从基数大、活动广、个性强的校园学生群体中检测出潜在的、有异常行为的学生,已成为学生异常行为分析亟需解决的问题.本文提出了一种大数据环境下基于多模态融合的大...针对“校园大数据”累积的海量数据呈现出离散性、稀疏性等问题,如何从基数大、活动广、个性强的校园学生群体中检测出潜在的、有异常行为的学生,已成为学生异常行为分析亟需解决的问题.本文提出了一种大数据环境下基于多模态融合的大学生异常行为预警方法(early warning method for abnormal behavior of college students based on multi-modal fusion in big data environment,EWMAB).首先,针对学生行为画像的表征不够丰富,行为标签存在时效性、动态性等问题,建立一种基于多模态特征深度学习的跨模态学生行为画像模型;其次,针对学生异常行为预测、预警的时效性和后置性问题,在学生行为画像和学生行为分类预测基础上,提出了一种基于多模态融合的学生异常行为预警方法,通过长短期记忆神经网络(long and short term memory networks,LSTM),结合学生行为多指标数据和文本信息来解决学生异常行为预警问题;最后,本文通过应用实例验证模型以学生学习成绩异常预警为例,与其他预警算法相比,EWMAB方法可以提高预警的准确性,实现学生异常行为预警的时效性和前置性,从而使学生教育工作更具有针对性、个性化和预测性.展开更多
基金This study is supported by the Fundamental Research Funds for the Central Universities of PPSUC under Grant 2022JKF02009.
文摘Face forgery detection is drawing ever-increasing attention in the academic community owing to security concerns.Despite the considerable progress in existing methods,we note that:Previous works overlooked finegrain forgery cues with high transferability.Such cues positively impact the model’s accuracy and generalizability.Moreover,single-modality often causes overfitting of the model,and Red-Green-Blue(RGB)modal-only is not conducive to extracting the more detailed forgery traces.We propose a novel framework for fine-grain forgery cues mining with fusion modality to cope with these issues.First,we propose two functional modules to reveal and locate the deeper forged features.Our method locates deeper forgery cues through a dual-modality progressive fusion module and a noise adaptive enhancement module,which can excavate the association between dualmodal space and channels and enhance the learning of subtle noise features.A sensitive patch branch is introduced on this foundation to enhance the mining of subtle forgery traces under fusion modality.The experimental results demonstrate that our proposed framework can desirably explore the differences between authentic and forged images with supervised learning.Comprehensive evaluations of several mainstream datasets show that our method outperforms the state-of-the-art detection methods with remarkable detection ability and generalizability.
文摘针对“校园大数据”累积的海量数据呈现出离散性、稀疏性等问题,如何从基数大、活动广、个性强的校园学生群体中检测出潜在的、有异常行为的学生,已成为学生异常行为分析亟需解决的问题.本文提出了一种大数据环境下基于多模态融合的大学生异常行为预警方法(early warning method for abnormal behavior of college students based on multi-modal fusion in big data environment,EWMAB).首先,针对学生行为画像的表征不够丰富,行为标签存在时效性、动态性等问题,建立一种基于多模态特征深度学习的跨模态学生行为画像模型;其次,针对学生异常行为预测、预警的时效性和后置性问题,在学生行为画像和学生行为分类预测基础上,提出了一种基于多模态融合的学生异常行为预警方法,通过长短期记忆神经网络(long and short term memory networks,LSTM),结合学生行为多指标数据和文本信息来解决学生异常行为预警问题;最后,本文通过应用实例验证模型以学生学习成绩异常预警为例,与其他预警算法相比,EWMAB方法可以提高预警的准确性,实现学生异常行为预警的时效性和前置性,从而使学生教育工作更具有针对性、个性化和预测性.
