基于多路径特征提取的实时语义分割方法

Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction

深度学习在图像语义分割领域的应用极大地提升了分割精确度,但由于深度学习网络在速度、内存等方面的限制,其并不能直接应用于嵌入式设备进行实时分割。针对语义分割模型存在的的网络结构复杂和计算开销巨大的问题,提出了结合边缘检测算法的多路径特征提取的实时语义分割算法。模型通过Sobel算子、Scharr算子和Laplacian算子对图像的轮廓信息进行提取。算法设计了空间路径提取图像的空间位置信息、语义路径提取图像高级语义信息,以及通过边缘检测路径提取图像中具有代表性的纹理特征,并采用Ghost轻量化模块来减少模型参数量,提高算法的分割速度。在480像素×360像素的CamVid数据集上的实验结果表明,在3种边缘检测算子上,模型的分割准确率均能得到有效提升,尤其是在加入3×3大小的Sobel算子下算法的性能提升最为明显,在CamVid测试集图像处理速度为349 frames/s的基础上,分割精度达到了42.9%。所提算法在分割精度和分割速度上均取得了较好的效果,在实时性和准确性之间达到了很好的平衡。

The application of deep learning in the field of image semantic segmentation has greatly improved the accuracy of segmentation,but due to the limitations of speed and memory,these models can not be directly applied to embedded devices for real-time segmentation.Aiming at the problems of complex network structure and huge computation cost of semantic segmentation model,a real-time semantic segmentation algorithm based on multi-path feature extraction combined with edge detection algorithm is proposed.The model uses Sobel operator,Scharr operator and Laplacian operator to extract the contour information of the image.The algorithm designs the spatial path to extract the spatial position information of the image,designs the semantic path to extract the advanced semantic information of the image,and uses the edge detection path to extract the representative texture features of the image.The ghost lightweight module is used to reduce the amount of model parameters and improve the segmentation speed of the algorithm.Experimental results on 480 pixel and 360 pixel CamVid dataset show that the segmentation accuracy of the model can be improved on the three edge detection operators,especially when the Sobel operator with the size of 3×3 is added,the performance of the algorithm is improved most obviously,and the segmentation accuracy reaches 42.9%on the basis of the image processing speed of 349 frames/s on CamVid test set.Both the segmentation accuracy and segmentation speed achieve good results,and achieve a good balance between real-time and accuracy.