基于多任务学习的蜡染纹样图案检索方法

Batik pattern retrieval method based on multi-task learning

为解决数字文化遗产保护中民族图案的解读与重用中存在的问题,以贵州苗族蜡染纹样为例,提出一种基于多任务学习的跨模态检索方法,实现图-图与文-图的检索模式。该方法采用BERT预训练模型析出文本特征,以ResNet50为基础,通过网络结构的改进对图片特征进行提取,并定义两个损失函数实现多个预测任务的图案检索。在Batik Dataset上验证了该方法的有效性,实验结果表明,图-图检索任务平均准确率及平均查询时间均优于单任务模型。

To solve the problems existing in the interpretation and reuse of ethnic patterns in the protection of digital cultural heritage,taking the batik pattern of Miao nationality in Guizhou province as an example,a cross-modal retrieval method based on multi-task learning was proposed to realize the graph to graph and text to graph retrieval modes.This method uses BERT pre-training model to separate text features.Based on ResNet50,the image features were extracted through the improvement of network structure,and two loss functions were defined to realize the pattern retrieval of multiple prediction tasks.The effectiveness of this method was verified in Batik Dataset.The final experimental results showed that the average accuracy rate and average query time of graph-graph retrieval tasks are better than that of the single-task model.