摘要
为解决数字文化遗产保护中民族图案的解读与重用中存在的问题,以贵州苗族蜡染纹样为例,提出一种基于多任务学习的跨模态检索方法,实现图-图与文-图的检索模式。该方法采用BERT预训练模型析出文本特征,以ResNet50为基础,通过网络结构的改进对图片特征进行提取,并定义两个损失函数实现多个预测任务的图案检索。在Batik Dataset上验证了该方法的有效性,实验结果表明,图-图检索任务平均准确率及平均查询时间均优于单任务模型。
To solve the problems existing in the interpretation and reuse of ethnic patterns in the protection of digital cultural heritage,taking the batik pattern of Miao nationality in Guizhou province as an example,a cross-modal retrieval method based on multi-task learning was proposed to realize the graph to graph and text to graph retrieval modes.This method uses BERT pre-training model to separate text features.Based on ResNet50,the image features were extracted through the improvement of network structure,and two loss functions were defined to realize the pattern retrieval of multiple prediction tasks.The effectiveness of this method was verified in Batik Dataset.The final experimental results showed that the average accuracy rate and average query time of graph-graph retrieval tasks are better than that of the single-task model.
作者
邹悦
潘伟杰
吕健
方年丽
岳迪
朱姝蔓
ZOU Yue;PAN Wei-jie;LYU Jian;FANG Nian-li;YUE Di;ZHU Shu-man(Key Laboratory of Advanced Manufacturing Technology of the Ministry of Education,Guizhou University,Guiyang 550025,China)
出处
《计算机工程与设计》
北大核心
2022年第4期1052-1058,共7页
Computer Engineering and Design
基金
国家自然科学基金项目(52065010)
贵州省科技支撑计划基金项目([2019]2010、黔科合基础[2018]1049、YJSCXJH(2018)088、[2017]1046、[2017]2016)。
关键词
多任务学习
深度学习
蜡染
语义分析
残差网络
multi-task learning
deep learning
batik
semantic analysis
residual network