基于视觉大模型的激光粉末床熔融铺粉缺陷检测

Powder⁃Spreading Defect Detection in Laser Powder Bed Fusion Based on Large Vision Model

激光粉末床熔融增材制造面临质量稳定一致性的挑战,铺粉质量是影响成形件质量的重要因素。近年来,计算机视觉在铺粉缺陷监测中的应用表现突出,但其性能却受到标注数据数量不足的限制。针对这一问题,笔者设计了基于视觉大模型分割一切模型(SAM)的铺粉缺陷分割模型(PSAM)。针对SAM预训练参数的知识迁移问题,引入Adapter模块实现参数微调;针对铺粉分割任务中类别信息的需求,改进了SAM中的掩码解码器;针对工业场景中人工提示难的问题,提出了自动提示生成器,实现了视觉提示的自动生成。在训练样本数量仅为50的情况下,PSAM表现出了良好的分割性能,平均交并比(mIoU)可达到65.02%,相较于Deeplab v3和U-Net分别提升了8.52个百分点和5.31个百分点。本研究展示了视觉大模型在增材过程监控中的应用价值和应用潜力。

Objective To date,laser powder bed fusion(LPBF)is considered as the most advanced metal additive manufacturing technology.It has been widely adopted for the production of critical metal components in aerospace and healthcare industries.However,realizing quality stability and consistency is challenging because of the coupled effects of various factors during LPBF.The powderspreading quality is a crucial characteristic of LPBF process monitoring.Defects during powder spreading can introduce defects into the formed components.In recent years,the application of computer vision in powderspreading defect detection has shown promising results.However,the limited availability of annotated data constrains its performance.Large vision models,such as the segment anything model(SAM),exhibit remarkable generalization capabilities owing to pretraining on an extremely large dataset.This allows its transfer to various downstream tasks with minimal training data.However,owing to the lack of defect knowledge,absence of category information,and dependence on manual prompts,SAM cannot be directly applied to powderspreading defect segmentation.This study addresses the requirements for powderspreading defect segmentation by improving SAM,achieving excellent defect segmentation performance with minimal training samples,and exploring the potential application of large vision models in monitoring the additive manufacturing process.Methods In this study,the powderspreading defect segment anything model(PSAM),based on SAM,was introduced.The overall structure of PSAM was similar to that of SAM,which consisted of an image encoder,an autoprompt generator,and a mask decoder.Compared to the original SAM,PSAM incorporated the following improvements:To address the issue of knowledge transfer concerning SAM's pretrained parameters,four Adapter modules were introduced into the SAM image encoder structure.These Adapter modules enabled efficient adjustment of image feature encoding.They were inserted behind the multihead attention layer in the transformer module and comprised linear and convolutional layers.To satisfy the requirements for category information in the powderspreading segmentation task,PSAM utilized an improved mask decoder.This decoder outputted segmentation masks that were equal to the number of categories in a single pass.Each output corresponded to a specific categorysegmentation result.These outputs were then integrated to obtain a classification output.To overcome the challenges of manual prompting in industrial settings,an autoprompt generator was designed.This generator is a fully convolutional neural network with residual connections that extracts features from input images and generates prompt embeddings that can be used by a mask decoder.A combination of crossentropy loss,focal loss,and Dice loss was used as the final loss function.Specifically,mean intersection over union(mIoU)was employed as the evaluation metric.Results and Discussions This study utilizes an offaxis industrial camera to acquire powderspreading images during the formation of several components.A subset of these images is selected for pixellevel annotation and is categorized into six classes:background,superelevation,incomplete,hopping,streaking,and lattice.The images and their corresponding labels are organized into a dataset and are divided according to certain proportions.The model is trained using the training set,and even with only 50 training images,PSAM exhibits excellent segmentation performance.The evaluation of PSAM using the test set yields an mIoU of 65.02%,representing an improvement of 8.51 percentage points over Deeplab v3 and 5.31 percentage points over UNet(Table 1).The limited amount of data restricts the ability of the model to perform deeper feature learning,thus hindering its capacity for richer feature representation.However,the pretrained SAM possesses strong imagefeature extraction capabilities.Therefore,excellent defect segmentation performance can be realized without extensive data training.Ablation experiments are conducted to evaluate the proposed improvements.The results indicate that the introduction of the Adapter module successfully transfers feature representation capabilities,and the automatic prompting effectively guides the mask decoder's output.Compared with the original SAM,PSAM realizes an mIoU improvement of 11.33 percentage points(Table 2).Conclusions The design of PSAM based on SAM realizes excellent segmentation performance even with a small amount of training data.Firstly,the model achieves transferring image encoder feature extraction capabilities from natural images to powder-spreading images by introducing the Adapter modules.Second,the mask decoder is modified to output the category masks.Finally,by incorporating an autoprompt generator to encode the input images,automatic generation of visual prompt embeddings is achieved.Large artificial intelligence(AI)models are rapidly advancing.However,owing to the unique characteristics and complex operating conditions of industrial settings,these models still face challenges in practical applications in industrial scenarios.This study provides a preliminary exploration of the application of large vision models for powderspreading defect detection in additive manufacturing.However,the full potential of large vision models has yet to be fully realized.The outstanding feature extraction,zeroshot generalization,and multimodal knowledge fusion capabilities of these large models can provide new solutions and approaches for additive manufacturing process monitoring,which are worth further exploration in future research.