With the remarkable success of change detection(CD)in remote sensing images in the context of deep learning,many convolutional neural network(CNN)based methods have been proposed.In the current research,to obtain a be...With the remarkable success of change detection(CD)in remote sensing images in the context of deep learning,many convolutional neural network(CNN)based methods have been proposed.In the current research,to obtain a better context modeling method for remote sensing images and to capture more spatiotemporal characteristics,several attention-based methods and transformer(TR)-based methods have been proposed.Recent research has also continued to innovate on TR-based methods,and many new methods have been proposed.Most of them require a huge number of calculation to achieve good results.Therefore,using the TR-based mehtod while maintaining the overhead low is a problem to be solved.Here,we propose a GNN-based multi-scale transformer siamese network for remote sensing image change detection(GMTS)that maintains a low network overhead while effectively modeling context in the spatiotemporal domain.We also design a novel hybrid backbone to extract features.Compared with the current CNN backbone,our backbone network has a lower overhead and achieves better results.Further,we use high/low frequency(HiLo)attention to extract more detailed local features and the multi-scale pooling pyramid transformer(MPPT)module to focus on more global features respectively.Finally,we leverage the context modeling capabilities of TR in the spatiotemporal domain to optimize the extracted features.We have a relatively low number of parameters compared to that required by current TR-based methods and achieve a good effect improvement,which provides a good balance between efficiency and performance.展开更多
针对无人机场景下行人重识别所呈现的多视角多尺度特点,以及传统的基于卷积神经网络的行人重识别算法受限于局部感受野结构和下采样操作,很难对行人图像的全局特征进行提取且图像空间特征分辨率不高。提出一种无人机场景下基于Transfor...针对无人机场景下行人重识别所呈现的多视角多尺度特点,以及传统的基于卷积神经网络的行人重识别算法受限于局部感受野结构和下采样操作,很难对行人图像的全局特征进行提取且图像空间特征分辨率不高。提出一种无人机场景下基于Transformer的轻量化行人重识别(Lightweight Transformer-based Person Re-Identification,LTReID)算法,利用多头多注意力机制从全局角度提取人体不同部分特征,使用Circle损失和边界样本挖掘损失,以提高图像特征提取和细粒度图像检索性能,并利用快速掩码搜索剪枝算法对Transformer模型进行训练后轻量化,以提高模型的无人机平台部署能力。更进一步,提出一种可学习的面向无人机场景的空间信息嵌入,在训练过程中通过学习获得优化的非视觉信息,以提取无人机多视角下行人的不变特征,提升行人特征识别的鲁棒性。最后,在实际的无人机行人重识别数据库中,讨论了在不同量级主干网和不同剪枝率情况下所提LTReID算法的行人重识别性能,并与多种行人重识别算法进行了性能对比,结果表明了所提算法的有效性和优越性。展开更多
基金The authors acknowledge the National Natural Science Foundation of China(Grant nos.61772319,62002200,62202268 and 62272281)Shandong Natural Science Foundation of China(Grant no.ZR2021QF134 and ZR2021MF107)Yantai Science And Technology Innovation Development Plan(2022JCYJ031).
文摘With the remarkable success of change detection(CD)in remote sensing images in the context of deep learning,many convolutional neural network(CNN)based methods have been proposed.In the current research,to obtain a better context modeling method for remote sensing images and to capture more spatiotemporal characteristics,several attention-based methods and transformer(TR)-based methods have been proposed.Recent research has also continued to innovate on TR-based methods,and many new methods have been proposed.Most of them require a huge number of calculation to achieve good results.Therefore,using the TR-based mehtod while maintaining the overhead low is a problem to be solved.Here,we propose a GNN-based multi-scale transformer siamese network for remote sensing image change detection(GMTS)that maintains a low network overhead while effectively modeling context in the spatiotemporal domain.We also design a novel hybrid backbone to extract features.Compared with the current CNN backbone,our backbone network has a lower overhead and achieves better results.Further,we use high/low frequency(HiLo)attention to extract more detailed local features and the multi-scale pooling pyramid transformer(MPPT)module to focus on more global features respectively.Finally,we leverage the context modeling capabilities of TR in the spatiotemporal domain to optimize the extracted features.We have a relatively low number of parameters compared to that required by current TR-based methods and achieve a good effect improvement,which provides a good balance between efficiency and performance.
文摘针对无人机场景下行人重识别所呈现的多视角多尺度特点,以及传统的基于卷积神经网络的行人重识别算法受限于局部感受野结构和下采样操作,很难对行人图像的全局特征进行提取且图像空间特征分辨率不高。提出一种无人机场景下基于Transformer的轻量化行人重识别(Lightweight Transformer-based Person Re-Identification,LTReID)算法,利用多头多注意力机制从全局角度提取人体不同部分特征,使用Circle损失和边界样本挖掘损失,以提高图像特征提取和细粒度图像检索性能,并利用快速掩码搜索剪枝算法对Transformer模型进行训练后轻量化,以提高模型的无人机平台部署能力。更进一步,提出一种可学习的面向无人机场景的空间信息嵌入,在训练过程中通过学习获得优化的非视觉信息,以提取无人机多视角下行人的不变特征,提升行人特征识别的鲁棒性。最后,在实际的无人机行人重识别数据库中,讨论了在不同量级主干网和不同剪枝率情况下所提LTReID算法的行人重识别性能,并与多种行人重识别算法进行了性能对比,结果表明了所提算法的有效性和优越性。