基于Polygon-RefineNet的违禁品X线图像自动标注方法

Automatic Annotation Approach for Prohibited Item in X-Ray Image Based on Polygon-RefineNet

近年来,随着深度学习的快速发展,其在智慧安检领域的应用已经成为了当下的研究热点.众所周知,深度学习方法是以海量训练数据为基础的,然而手工标注真值(ground truth)是一项十分繁琐的工作.为此,本文提出一种基于Polygon-RefineNet的违禁品X线图像自动标注方法,该方法在用户设定的包含感兴趣区域的初始边框(bounding box)内自动预测出目标的多边形轮廓,旨在生成可用真值的情况下最大限度地减少标注时间.由于违禁品X线图像存在大量的重叠现象,导致图像背景十分杂乱、违禁品轮廓模糊不清,因此本文首先引入多路径优化机制,通过有效利用深度网络下采样过程中提取的底层空间信息和高层语义信息来优化多边形轮廓的边缘细节,从而提高标注精度;其次,本文设计一种混合损失函数用于优化多边形轮廓的整体形状和位置,并同时消除真值本身存在的主观性误差使模型具有强大的泛化能力.最后,为了验证所提出方法的有效性,本文建立了一个违禁品X线数据集,该数据集包含2623张经过手工标注的X线图像,共10类7257个违禁品带有像素级真值.实验表明,本文提出的方法在标注违禁品时达到了93.1%的准确率,且速度约是手工标注的3.7倍.本文进一步证明了该方法在Cityscapes数据集、MS COCO数据集等其它域外数据集上的有效性.

In recent years,with the rapid development of deep learning,its application in the field of smart security inspection has become a research hotspot.However,unfortunately,there is no X-ray segmentation dataset so far for research in relevant fields such as prohibited item instance segmentation and Threat Image Projection(TIP).At the meantime,it is universally acknowledged that deep learning approaches are data hungry and their performance is closely related to the amount of training data,though,but manually annotating ground truth instance masks is an extremely time-consuming task,especially in X-ray images.For these reasons,this paper proposes an efficient approach based on Polygon-RefineNet(PRN) for automatic annotation of prohibited items in X-ray images,aiming at minimizing the annotation time and yield high quality annotations.In particular,we construct a "fully convolutional" network to adapt prohibited items in different scales and styles,which takes as input an arbitrary-size initial bounding box given by a user containing region of interest and automatically produces a series of correspondinglysized continuous vertices of the polygon outlining the prohibited item by efficient learning and inference.And our model allows the user to correct vertex at the end to produce as accurate annotation results as desired.Because of a large amount of overlapping phenomenon in X-ray images,the background of the images is complex and the boundary of the prohibited item is blurred.To solve this problem,we first introduce a multi-path refinement mechanism to refine the edge details of the polygon by making full use of the low-level spatial information and high-level semantic information that are extracted during the down-sampling process.Then,we design a mixed loss function by introducing the evaluation metric into the loss function for modifying the overall shape and position of the polygon and meanwhile eliminating the subjectivity error on account of the ground truth itself,which makes the network have strong generalization ability.Finally,to evaluate the proposed approach,we present a high-quality X-ray segmentation dataset named Prohibited Item X-ray(PIXray).The dataset consists of 2623 X-ray images by manually labeling,in which 10 classes of 7257 prohibited items have pixel-level ground truth.For a fair comparison,we validate our approach on our PIXray dataset and the public Cityscapes instance segmentation dataset,respectively.Experimental results demonstrate that our approach accelerates the annotation process by a factor of 3.7 in all classes of PIXray,while achieving93.1% agreement after manual fine-tuning in IoU(Intersection over Union) with original ground truth,matching the typical agreement between human annotators.Besides,we outperform the baselines in 8 out of 10 categories,particularly well in the wrench,pliers,bat and razor.Our approach also obtains an IoU of 67.93% on Cityscapes dataset without using any fine-tuning.We further prove the effectiveness of our approach on Aerial Rooftop dataset,MS COCO dataset and PASCAL VOC dataset.The results show that the users can use our approach to achieve a relatively high reduction in time for annotating a new segmentation dataset.