基于视觉注意机制和条件随机场的图像标注

Image annotation method based on visual attention mechanism and conditional random field

传统的图像标注方法对图像各个区域同等标注,忽视了人们对图像的理解方式。为此提出了基于视觉注意机制和条件随机场的图像标注方法。首先,由于人们在对图像认识的过程中,对显著区域会有较多的关注,因此通过视觉注意机制来取得图像的显著区域,用支持向量机对显著区域赋予语义标签;再利用k-NN聚类算法对非显著区域进行标注;最后,又由于显著区域的标注词与非显著区域的标注词在逻辑上存在一定的关联性,因此条件随机场模型可以根据标注词的关联性校正并确定图像的最终标注向量。在Corel5k、IAPR TC-12和ESP Game图像库上进行实验并且和其他方法进行比较,从平均查准率、平均查全率和F1的实验结果验证了本文方法的有效性。

Traditional image annotation methods interpret all image regions equally, neglecting any understanding olthe image. Therefore, an image annotation method based on the visual attention mechanism and conditional randomfield, called VAMCRF, is proposed. Firstly, people pay more attention to image salient regions during the processol image recognition ; this can be achieved through the visual attention mechanism and the support vector machine isthen used to assign semantic labels. It then labels the non-salient regions using a k-NN clustering algorithm. Finally, as the annotations of salient and non-salient regions are logically related, the ultimate label vector of the imagecan be corrected and determined by a conditional random field （CRF） model and inter-word correlation. From thevalues of average precision, average recall, and F1, the experimental results on Corel5k , IAPR TC-12, and ESPGame confirm that the proposed method is efficient compared with traditional annotation methods.