基于空谱联合特征的壁画稀疏多光谱图像颜料分类方法

Pigment Classification Method of Mural Sparse Multi-spectral Image Based on Space Spectrum Joint Feature

由于受到现场条件和保护要求限制,对壁画进行光谱成像数据采集时需要快速完成,利用稀疏通道成像能够提高数据采集的效率,但其像元颜料光谱反射率曲线呈现非线性,影响壁画多光谱图像颜料分类精度。针对该问题,提出了基于空谱联合特征的壁画稀疏多光谱图像颜料分类方法,采用长短期记忆神经网络中的双曲正切激活函数提取非线性光谱特征,减小对分类精度的影响;针对多光谱成像空间分辨率较高导致相邻像元相关性较强的问题,利用卷积神经网络中线性整流函数把特征图映射到非线性空间,提高模型非线性特征的表达能力;最后使用多尺度融合策略将空间特征和光谱特征相加,消除光谱非线性和空间相关性的问题对分类结果的影响。实验结果表明,OA和Kappa系数分别达到了97%和0.97以上,有效提高了壁画稀疏多光谱图像的颜料分类精度。

Murals are treasures in the long history of Chinese culture. It has high research value in history,science and art. Pigment is the material carrier of the main form of mural expression. Simultaneously,it is also an important part of murals. After a long period of disease and corrosion,the surface of the mural may be damaged to varying degrees,making it difficult for researchers to distinguish the type of pigments in the murals. The accurate identification of pigments is the premise of conservation and restoration of cultural relics. The traditional method needs to take samples from the murals,which will cause irreversible damage to the murals. In this paper,multispectral imaging technology and deep learning related classification algorithm are used to analyze and identify the pigments in mural multi-spectral images to assist researchers in mural identification and cultural relic restoration. Rich spatial and spectral information is included in mural multispectral images. In traditional algorithms,spatial or spectral information is used as a feature of mural multispectral image classification. This method leads to low classification accuracy of mural multispectral image. In order to improve the classification accuracy of mural multispectral images,the deep learning algorithm is used in this paper,which can make full use of the spatial and spectral information of multi-spectral images. In the actual shooting,due to the limitations of site conditions and protection requirements,mural spectral imaging data need to be collected quickly. The efficiency of data acquisition can be improved by using sparse channel imaging methods. However,this method can make the spectral reflectance curve of pigments appear nonlinear,which can affect the classification accuracy of mural multispectral image. In order to solve this problem,a pigment classification method for mural sparse multispectral images based on spatial spectral combination features is proposed in this paper. The nonlinear spectral features are extracted by using the hyperbolic tangent activation function in the Long Short-term Memory(LSTM). Firstly,the spectral reconstruction of the mural multi-spectral image is carried out.Then the one-dimensional spectral vector is input into LSTM,which can actively learn under unsupervised conditions to reduce the influence of spectral curve nonlinearity on the classification accuracy. In order to solve the problems of high spatial resolution and strong correlation between adjacent pixels in multispectral imaging,the linear correction function in Convolution Neural Network(CNN)is used to map the feature map to nonlinear space. The activation function is added after the convolution operation of each layer to improve the nonlinear expression ability of the mural multi-spectral image. The spatial spectral unity of mural multi-spectral images can not be fully utilized,if only spatial or spectral features are used. For this reason,the multi-scale fusion strategy combining spectral and spatial features is used to eliminate the influence of spectral nonlinearity and spatial correlation on the classification results. Firstly,a Spatial Spectral Joint Feature Network Model(SSJF) is established to train pigment samples. Then,the loss function is designed by cross entropy,and the gradient is updated by back propagation algorithm. Finally,the softmax classifier is used to output the probability of each pigment. The experimental results show that the pigments in the paint board and self-made murals can be correctly classified through SSJF. The OA and Kappa coefficients reached 97% and 0.97, respectively, which effectively improved the pigment classification accuracy of the sparse multi-spectral image of the mural.