一种基于残差MLP的多模态点云分类网络

A residual MLP-based multi-modal point cloud classification network

针对PCT等先进点云算法存在模态单一、特征提取器复杂、参数量大、计算效率低等问题,提出一种精简快速的多模态点云分类网络Res-CLIP。将ResMLP-PC与CLIP结合,通过学习多模态信息提高主干网络性能和迁移学习能力,使用残差MLP提高算法效率;将仿射变换模块融入主干网络提高算法精度。排水管道缺陷数据集实验结果表明:与PCT等算法相比,ResMLP-PC算法的精确率、召回率均有提升,且参数量减少近50%,检测速度提升23%。Zero Shot实验结果表明:与现有多模态点云网络相比,Res-CLIP算法在2类公开数据集上的Zero Shot精度均较优,比ULIP相比分别提升4.6%、0.5%。

Currently,the advanced point cloud algorithms such as PCT suffer from issues like single modality,complex feature extractors,high parameter count and low computational efficiency.To address these problems,this paper proposes a streamlined and fast multi-modal point cloud classification network called Res-CLIP.The network combines ResMLP-PC with CLIP to leverage multi-modal information and improve the performance and transfer learning capabilities of the backbone network.The residual MLP is employed to enhance algorithm efficiency.The affine transformation module is integrated into the backbone network to improve algorithm accuracy.Our experimental results on the drainage pipeline defect dataset show ResMLP-PC exhibits improved precision and recall rates compared to PCT algorithm while it reduces the parameter count by almost half,thus improving the detection speed by 23%.Our Zero-Shot experiments demonstrate Res-CLIP achieves superior zero-shot accuracy on two publicly available datasets,surpassing ULIP by 4.6%and 0.5%respectively compared to existing multimodal point cloud networks.