小型文物摄影测量三维建模主体对象识别

Recognition method of the main object of three-dimensional photogrammetric modeling of cultural relics

通过拍照进行三维建模的摄影测量技术是文物数字化的重要手段,然而,在摄影测量文物三维重建过程中,文物所在的场景背景也同时参与计算和建模,需要设法删除,以获得文物单体模型。针对上述问题,本文提出一种小型文物三维模型主体对象识别方法,能够自动删除文物摄影测量三维模型的背景数据。该方法分别利用深度学习网络Mask R-CNN和改进为自动获得初值的One Cut进行文物图像前景内容识别,再融合两种方法的识别结果进行文物图像主体对象分割,最后以二维图像分割结果为依据,逐三角形识别三维模型主体对象,并运用多视角约束法优化识别精度。实验表明,本文方法能够获得小型文物单体化三维模型,无需人工干预且精度较高。

Photogrammetry technology helps us reconstruct three dimensional models of cultural relics just by taking photos.However,the background where the cultural relics are located also participates in modeling simultaneously,which wastes storage space and computing resources.Meanwhile,the independence and aesthetics of the three dimensional models are destroyed.Additionally,pure models of cultural relics are obtained by manually deleting the background in three dimensional scenes,which is time consuming and cannot satisfy the practical needs of the flourishing development of digital cultural heritage.This research aims to obtain the three dimensional pure cultural relic models by deleting the redundant background of the photogrammetric model on the basis of object recognition without manual interaction.This paper proposed a method to delete the background of the three dimensional photogrammetric model of cultural relics by objects recognition.First,we recognized the foreground of the cultural relic image by using the deep learning network Mask R-CNN and One Cut,respectively.Second,we extracted the masks of cultural relics by combining the results of Mask R-CNN and One Cut.Last,we applied the masks of cultural relics to delete the background of three dimensional cultural relic models on the basis of the mapping relationships between images and three dimensional models.Moreover,we used the multi-view constraints to optimize the three dimensional recognition accuracy.Additionally,we improved the One Cut method by automatically setting the initial value.In the processing of three dimensional projecting to two dimensional,regarding the cases where triangles overlap,we applied the depth information to distinguish the triangles of foreground and background in three dimensional models.To evaluate proposed method,two cultural relics were selected for the experiments,including Buddha statues in the Beilin Museum in Shaanxi and Mayan masks in the Mexican Museum.We took photos of them and obtained three dimensional models via GET3D(get3d.cn).Our method performs effectively for the Buddha model and the Mayan masks model.Apparently,most of the background of the models is eliminated,and the main bodies of the models are completely preserved.Compared with the artificially labeled ground truth,it can be found that 1)our method preserved three dimensional models complete with a satisfactory recall of 99.23%and 99.20%for the Buddha model and the Mayan masks model,respectively;2)the algorithm erased the triangles of background with a simplification rate of 85.34%and 86.44%for the Buddha model and the Mayan masks model,respectively;3)with the advantage of the multi-view constraints,the recognition accuracy of the three dimensional model is higher than two dimensional image.The method proposed in this paper can automatically delete the background of the three dimensional photogrammetric model without manual intervention and preserve the integrity of the object well.The experimental results demonstrate the proposed method is feasible and effective.However,when applied to large three dimensional models,our method is limited to efficiency,given that we distinguished the overlapped triangles successively.Moreover,our pipeline provides a reference for recognizing three dimensional objects in various three dimensional scenes.