Semantic segmentation is for pixel-level classification tasks,and contextual information has an important impact on the performance of segmentation.In order to capture richer contextual information,we adopt ResNet as ...Semantic segmentation is for pixel-level classification tasks,and contextual information has an important impact on the performance of segmentation.In order to capture richer contextual information,we adopt ResNet as the backbone network and designs an encoder-decoder architecture based on multidimensional attention(MDA)module and multiscale upsampling(MSU)module.The MDA module calculates the attention matrices of the three dimensions to capture the dependency of each position,and adaptively captures the image features.The MSU module adopts parallel branches to capture the multiscale features of the images,and multiscale feature aggregation can enhance contextual information.A series of experiments demonstrate the validity of the model on Cityscapes and Camvid datasets.展开更多
Scene segmentation is widely used in autonomous driving for environmental perception.Semantic scene segmentation has gained considerable attention owing to its rich semantic information.It assigns labels to the pixels...Scene segmentation is widely used in autonomous driving for environmental perception.Semantic scene segmentation has gained considerable attention owing to its rich semantic information.It assigns labels to the pixels in an image,thereby enabling automatic image labeling.Current approaches are based mainly on convolutional neural networks(CNN),however,they rely on numerous labels.Therefore,the use of a small amount of labeled data to achieve semantic segmentation has become increasingly important.In this study,we developed a domain adaptation framework based on optimal transport(OT)and an attention mechanism to address this issue.Specifically,we first generated the output space via a CNN owing to its superior of feature representation.Second,we utilized OT to achieve a more robust alignment of the source and target domains in the output space,where the OT plan defined a well attention mechanism to improve the adaptation of the model.In particular,the OT reduced the number of network parameters and made the network more interpretable.Third,to better describe the multiscale properties of the features,we constructed a multiscale segmentation network to perform domain adaptation.Finally,to verify the performance of the proposed method,we conducted an experiment to compare the proposed method with three benchmark and four SOTA methods using three scene datasets.The mean intersection-over-union(mIOU)was significantly improved,and visualization results under multiple domain adaptation scenarios also show that the proposed method performed better than semantic segmentation methods.展开更多
目的目标语义特征提取效果直接影响图像语义分割的精度,传统的单尺度特征提取方法对目标的语义分割精度较低,为此,提出一种基于多尺度特征融合的工件目标语义分割方法,利用卷积神经网络提取目标的多尺度局部特征语义信息,并将不同尺度...目的目标语义特征提取效果直接影响图像语义分割的精度,传统的单尺度特征提取方法对目标的语义分割精度较低,为此,提出一种基于多尺度特征融合的工件目标语义分割方法,利用卷积神经网络提取目标的多尺度局部特征语义信息,并将不同尺度的语义信息进行像素融合,使神经网络充分捕获图像中的上下文信息,获得更好的特征表示,有效实现工件目标的语义分割。方法使用常用的多类工件图像定义视觉任务,利用残差网络模块获得目标的单尺度语义特征图,再结合本文提出的多尺度特征提取方式获得不同尺度的局部特征语义信息,通过信息融合获得目标分割图。使用上述方法经多次迭代训练后得到与视觉任务相关的工件目标分割模型,并对训练权重与超参数进行保存。结果将本文方法和传统的单尺度特征提取方法做定性和定量的测试实验,结果表明,获得的分割网络模型对测试集中的目标都具有较精确的分割能力,与单尺度特征提取方法相比,本文方法的平均交并比m IOU(mean intersection over union)指标在验证集上训练精度提高了4.52%,在测试集上分割精度提高了4.84%。当测试样本中包含的目标种类较少且目标边缘清晰时,本文方法能够得到更精准的分割结果。结论本文提出的语义分割方法,通过多尺度特征融合的方式增强了神经网络模型对目标特征的提取能力,使训练得到的分割网络模型比传统的单尺度特征提取方式在测试集上具有更优秀的性能,从而验证了所提出方法的有效性。展开更多
目的遥感图像语义分割是根据土地覆盖类型对图像中每个像素进行分类,是遥感图像处理领域的一个重要研究方向。由于遥感图像包含的地物尺度差别大、地物边界复杂等原因,准确提取遥感图像特征具有一定难度,使得精确分割遥感图像比较困难...目的遥感图像语义分割是根据土地覆盖类型对图像中每个像素进行分类,是遥感图像处理领域的一个重要研究方向。由于遥感图像包含的地物尺度差别大、地物边界复杂等原因,准确提取遥感图像特征具有一定难度,使得精确分割遥感图像比较困难。卷积神经网络因其自主分层提取图像特征的特点逐步成为图像处理领域的主流算法,本文将基于残差密集空间金字塔的卷积神经网络应用于城市地区遥感图像分割,以提升高分辨率城市地区遥感影像语义分割的精度。方法模型将带孔卷积引入残差网络,代替网络中的下采样操作,在扩大特征图感受野的同时能够保持特征图尺寸不变;模型基于密集连接机制级联空间金字塔结构各分支,每个分支的输出都有更加密集的感受野信息;模型利用跳线连接跨层融合网络特征,结合网络中的高层语义特征和低层纹理特征恢复空间信息。结果基于ISPRS(International Society for Photogrammetry and Remote Sensing)Vaihingen地区遥感数据集展开充分的实验研究,实验结果表明,本文模型在6种不同的地物分类上的平均交并比和平均F1值分别达到69.88%和81.39%,性能在数学指标和视觉效果上均优于SegNet、pix2pix、Res-shuffling-Net以及SDFCN(symmetrical dense-shortcut fully convolutional network)算法。结论将密集连接改进空间金字塔池化网络应用于高分辨率遥感图像语义分割,该模型利用了遥感图像不同尺度下的特征、高层语义信息和低层纹理信息,有效提升了城市地区遥感图像分割精度。展开更多
基金Fundamental Research Fund in Heilongjiang Provincial Universities(Nos.135409602,135409102)。
文摘Semantic segmentation is for pixel-level classification tasks,and contextual information has an important impact on the performance of segmentation.In order to capture richer contextual information,we adopt ResNet as the backbone network and designs an encoder-decoder architecture based on multidimensional attention(MDA)module and multiscale upsampling(MSU)module.The MDA module calculates the attention matrices of the three dimensions to capture the dependency of each position,and adaptively captures the image features.The MSU module adopts parallel branches to capture the multiscale features of the images,and multiscale feature aggregation can enhance contextual information.A series of experiments demonstrate the validity of the model on Cityscapes and Camvid datasets.
基金supported by the National Natural Science Foundation of China(11971296)National Key R&D Program of China(2021YFA1003004).
文摘Scene segmentation is widely used in autonomous driving for environmental perception.Semantic scene segmentation has gained considerable attention owing to its rich semantic information.It assigns labels to the pixels in an image,thereby enabling automatic image labeling.Current approaches are based mainly on convolutional neural networks(CNN),however,they rely on numerous labels.Therefore,the use of a small amount of labeled data to achieve semantic segmentation has become increasingly important.In this study,we developed a domain adaptation framework based on optimal transport(OT)and an attention mechanism to address this issue.Specifically,we first generated the output space via a CNN owing to its superior of feature representation.Second,we utilized OT to achieve a more robust alignment of the source and target domains in the output space,where the OT plan defined a well attention mechanism to improve the adaptation of the model.In particular,the OT reduced the number of network parameters and made the network more interpretable.Third,to better describe the multiscale properties of the features,we constructed a multiscale segmentation network to perform domain adaptation.Finally,to verify the performance of the proposed method,we conducted an experiment to compare the proposed method with three benchmark and four SOTA methods using three scene datasets.The mean intersection-over-union(mIOU)was significantly improved,and visualization results under multiple domain adaptation scenarios also show that the proposed method performed better than semantic segmentation methods.
文摘目的目标语义特征提取效果直接影响图像语义分割的精度,传统的单尺度特征提取方法对目标的语义分割精度较低,为此,提出一种基于多尺度特征融合的工件目标语义分割方法,利用卷积神经网络提取目标的多尺度局部特征语义信息,并将不同尺度的语义信息进行像素融合,使神经网络充分捕获图像中的上下文信息,获得更好的特征表示,有效实现工件目标的语义分割。方法使用常用的多类工件图像定义视觉任务,利用残差网络模块获得目标的单尺度语义特征图,再结合本文提出的多尺度特征提取方式获得不同尺度的局部特征语义信息,通过信息融合获得目标分割图。使用上述方法经多次迭代训练后得到与视觉任务相关的工件目标分割模型,并对训练权重与超参数进行保存。结果将本文方法和传统的单尺度特征提取方法做定性和定量的测试实验,结果表明,获得的分割网络模型对测试集中的目标都具有较精确的分割能力,与单尺度特征提取方法相比,本文方法的平均交并比m IOU(mean intersection over union)指标在验证集上训练精度提高了4.52%,在测试集上分割精度提高了4.84%。当测试样本中包含的目标种类较少且目标边缘清晰时,本文方法能够得到更精准的分割结果。结论本文提出的语义分割方法,通过多尺度特征融合的方式增强了神经网络模型对目标特征的提取能力,使训练得到的分割网络模型比传统的单尺度特征提取方式在测试集上具有更优秀的性能,从而验证了所提出方法的有效性。
文摘目的遥感图像语义分割是根据土地覆盖类型对图像中每个像素进行分类,是遥感图像处理领域的一个重要研究方向。由于遥感图像包含的地物尺度差别大、地物边界复杂等原因,准确提取遥感图像特征具有一定难度,使得精确分割遥感图像比较困难。卷积神经网络因其自主分层提取图像特征的特点逐步成为图像处理领域的主流算法,本文将基于残差密集空间金字塔的卷积神经网络应用于城市地区遥感图像分割,以提升高分辨率城市地区遥感影像语义分割的精度。方法模型将带孔卷积引入残差网络,代替网络中的下采样操作,在扩大特征图感受野的同时能够保持特征图尺寸不变;模型基于密集连接机制级联空间金字塔结构各分支,每个分支的输出都有更加密集的感受野信息;模型利用跳线连接跨层融合网络特征,结合网络中的高层语义特征和低层纹理特征恢复空间信息。结果基于ISPRS(International Society for Photogrammetry and Remote Sensing)Vaihingen地区遥感数据集展开充分的实验研究,实验结果表明,本文模型在6种不同的地物分类上的平均交并比和平均F1值分别达到69.88%和81.39%,性能在数学指标和视觉效果上均优于SegNet、pix2pix、Res-shuffling-Net以及SDFCN(symmetrical dense-shortcut fully convolutional network)算法。结论将密集连接改进空间金字塔池化网络应用于高分辨率遥感图像语义分割,该模型利用了遥感图像不同尺度下的特征、高层语义信息和低层纹理信息,有效提升了城市地区遥感图像分割精度。
