针对图表示学习模型依赖具体任务进行特征保留以及节点表示的泛化性有限等问题,本文提出一种基于自监督信息增强的图表示学习模型(Self-Variational Graph Auto Encoder,Self-VGAE)。Self-VGAE首先使用图卷积编码器和节点表示内积解码...针对图表示学习模型依赖具体任务进行特征保留以及节点表示的泛化性有限等问题,本文提出一种基于自监督信息增强的图表示学习模型(Self-Variational Graph Auto Encoder,Self-VGAE)。Self-VGAE首先使用图卷积编码器和节点表示内积解码器构建变分图自编码器(Variational Graph Auto Encoder,VGAE),并对原始图进行特征提取和编码;然后,使用拓扑结构和节点属性生成自监督信息,在模型训练过程中约束节点表示的生成。在多个图分析任务中,Self-VGAE的实验表现均优于当前较为先进的基线模型,表明引入自监督信息能够增强对节点特征相似性和差异性的保留能力以及对拓扑结构的保持、推断能力,并且Self-VGAE具有较强的泛化能力。展开更多
As image manipulation technology advances rapidly,the malicious use of image tampering has alarmingly escalated,posing a significant threat to social stability.In the realm of image tampering localization,accurately l...As image manipulation technology advances rapidly,the malicious use of image tampering has alarmingly escalated,posing a significant threat to social stability.In the realm of image tampering localization,accurately localizing limited samples,multiple types,and various sizes of regions remains a multitude of challenges.These issues impede the model’s universality and generalization capability and detrimentally affect its performance.To tackle these issues,we propose FL-MobileViT-an improved MobileViT model devised for image tampering localization.Our proposed model utilizes a dual-stream architecture that independently processes the RGB and noise domain,and captures richer traces of tampering through dual-stream integration.Meanwhile,the model incorporating the Focused Linear Attention mechanism within the lightweight network(MobileViT).This substitution significantly diminishes computational complexity and resolves homogeneity problems associated with traditional Transformer attention mechanisms,enhancing feature extraction diversity and improving the model’s localization performance.To comprehensively fuse the generated results from both feature extractors,we introduce the ASPP architecture for multi-scale feature fusion.This facilitates a more precise localization of tampered regions of various sizes.Furthermore,to bolster the model’s generalization ability,we adopt a contrastive learning method and devise a joint optimization training strategy that leverages fused features and captures the disparities in feature distribution in tampered images.This strategy enables the learning of contrastive loss at various stages of the feature extractor and employs it as an additional constraint condition in conjunction with cross-entropy loss.As a result,overfitting issues are effectively alleviated,and the differentiation between tampered and untampered regions is enhanced.Experimental evaluations on five benchmark datasets(IMD-20,CASIA,NIST-16,Columbia and Coverage)validate the effectiveness of our proposed model.The meticulously calibrated FL-MobileViT model consistently outperforms numerous existing general models regarding localization accuracy across diverse datasets,demonstrating superior adaptability.展开更多
文摘针对图表示学习模型依赖具体任务进行特征保留以及节点表示的泛化性有限等问题,本文提出一种基于自监督信息增强的图表示学习模型(Self-Variational Graph Auto Encoder,Self-VGAE)。Self-VGAE首先使用图卷积编码器和节点表示内积解码器构建变分图自编码器(Variational Graph Auto Encoder,VGAE),并对原始图进行特征提取和编码;然后,使用拓扑结构和节点属性生成自监督信息,在模型训练过程中约束节点表示的生成。在多个图分析任务中,Self-VGAE的实验表现均优于当前较为先进的基线模型,表明引入自监督信息能够增强对节点特征相似性和差异性的保留能力以及对拓扑结构的保持、推断能力,并且Self-VGAE具有较强的泛化能力。
基金This study was funded by the Science and Technology Project in Xi’an(No.22GXFW0123)this work was supported by the Special Fund Construction Project of Key Disciplines in Ordinary Colleges and Universities in Shaanxi Province,the authors would like to thank the anonymous reviewers for their helpful comments and suggestions.
文摘As image manipulation technology advances rapidly,the malicious use of image tampering has alarmingly escalated,posing a significant threat to social stability.In the realm of image tampering localization,accurately localizing limited samples,multiple types,and various sizes of regions remains a multitude of challenges.These issues impede the model’s universality and generalization capability and detrimentally affect its performance.To tackle these issues,we propose FL-MobileViT-an improved MobileViT model devised for image tampering localization.Our proposed model utilizes a dual-stream architecture that independently processes the RGB and noise domain,and captures richer traces of tampering through dual-stream integration.Meanwhile,the model incorporating the Focused Linear Attention mechanism within the lightweight network(MobileViT).This substitution significantly diminishes computational complexity and resolves homogeneity problems associated with traditional Transformer attention mechanisms,enhancing feature extraction diversity and improving the model’s localization performance.To comprehensively fuse the generated results from both feature extractors,we introduce the ASPP architecture for multi-scale feature fusion.This facilitates a more precise localization of tampered regions of various sizes.Furthermore,to bolster the model’s generalization ability,we adopt a contrastive learning method and devise a joint optimization training strategy that leverages fused features and captures the disparities in feature distribution in tampered images.This strategy enables the learning of contrastive loss at various stages of the feature extractor and employs it as an additional constraint condition in conjunction with cross-entropy loss.As a result,overfitting issues are effectively alleviated,and the differentiation between tampered and untampered regions is enhanced.Experimental evaluations on five benchmark datasets(IMD-20,CASIA,NIST-16,Columbia and Coverage)validate the effectiveness of our proposed model.The meticulously calibrated FL-MobileViT model consistently outperforms numerous existing general models regarding localization accuracy across diverse datasets,demonstrating superior adaptability.
