基于对比学习及背景挖掘的少样本语义分割

Few-shot Semantic Segmentation Based on Contrastive Learning and Background Mining

少样本语义分割是在具有少量标注样本的查询图像的条件下,对潜在对象类别进行分割的计算机视觉任务.然而,现有方法仍然存在两个问题,这对它们构成了挑战.首先是原型偏差问题,这导致原型具有较少的前景目标信息,难以模拟真实的类别统计信息.另一个是特征破坏问题,这意味着模型只关注当前类别而不关注潜在类别.本文提出了一个基于对比原型以及背景挖掘的新网络.该网络主要思想是使模型学习更具代表性的原型,并从背景中识别潜在类别.具体而言,特定类学习分支构建了一个大且一致的原型字典,然后使用InfoNCE损失使原型更具区分性.另一方面,背景挖掘分支初始化背景原型,并使用构建的背景原型与字典之间的注意力机制来挖掘潜在类别.在PASCAL-5i和COCO-20i数据集上的实验证明模型有优秀的性能.在使用ResNet-50网络的1-shot设置下,达到了64.9%和44.2%,相较于基准模型分别提升了4.0%和1.9%.

Few-shot semantic segmentation is a computer vision task that involves segmenting potential object categories in query images with a small number of annotated samples.However,existing methods still face two challenges.Firstly,there is a prototype bias problem,resulting in prototypes having less foreground object information and making it difficult to simulate real category statistics.The other issue is feature degradation,which means that the model only focuses on the current category rather than potential categories.This study proposes a new network based on contrastive prototypes and background mining.The main idea of the network is to enable the model to learn more representative prototypes and identify potential categories from the background.Specifically,a specific class learning branch constructs a large and consistent prototype dictionary and then uses InfoNCE loss to make the prototypes more discriminative.On the other hand,the background mining branch initializes background prototypes and uses an attention mechanism between the constructed background prototypes and the dictionary to mine potential categories.Experimental results on the PASCAL-5i and COCO-20i datasets demonstrate excellent performance of the model.Under the 1-shot setting using the ResNet-50 network,64.9% and 44.2% are achieved,an improvement of 4.0% and 1.9%,respectively,compared to the baseline model.