多光谱图像融合的IC器件表面缺陷检测

Multispectral image fusion method for surface defect detection of IC devices

针对单可见光或单红外条件下的IC器件表面缺陷对比度不足,缺陷检测精度低的问题,提出多光谱图像融合的IC器件表面缺陷检测方法。针对IC器件可见光与红外图像配准中存在尺度不一致和对比度反转问题,引入拉普拉斯金字塔和特征描述符重组策略改进ORB(Oriented FAST and Rotated BRIEF)图像配准算法。在图像配准的基础上,提出NSST_VP图像融合方法,以非下采样剪切波变换(Non-Subsample Shearlet Transform, NSST)得到红外图像和已配准可见光图像的低频和高频子带,对低频子带采用视觉显著图(Visual Significance Map, VSM)加权融合规则,高频子带则采用自适应脉冲耦合神经网络(PA-Pulse Coupled Neural Network, PA-PCNN)决策融合规则,进而通过NSST逆变换得到高质量多光谱融合图像。最后,将融合图像输入YOLOv8s模型进行检测。实验结果表明,改进ORB的图像配准平均精度为87.8%,比ORB图像配准精度提高了62%,NSST_VP图像融合算法在主观视觉效果和客观评价指标上均有所提高。在缺陷检测实验中,NSST_VP融合方法的均值平均精度(mean Average Precision, mAP)达到83.15%,比单可见光、单红外缺陷图像检测的mAP分别提高了22.97%,28.31%,比双树复小波变换融合、曲线变换融合、非下采样轮廓波变换融合方法的mAP分别提高了13.14%,15.01%,20.35%。

To address the issue of low defect detection accuracy in IC devices due to insufficient contrast under either visible light or infrared conditions alone,this paper introduces a multi-spectral fusion approach.Initially,to overcome scale inconsistency and contrast inversion challenges during IC device image registration,we enhance the ORB(Oriented FAST and Rotated BRIEF)algorithm with a Laplacian pyramid and feature descriptor recombination strategy.Following image registration,we propose the NSST_VP image fusion method,which processes the infrared and visible images'low and high frequency subbands through Non-Subsample Shearlet Transform(NSST).For fusion,the low frequency subband uses a visual significance map(VSM)weighted rule,and the high frequency subband employs a PA-Pulse Coupled Neural Network (PA-PCNN) decision rule, with the final image produced by reversing the NSST. The fused image is then analyzed using the YOLOv8s model. Experimental findings reveal an 87.8% average accuracy with the improved ORB registration, marking a 62% enhancement over the stan-dard ORB. The NSST_VP fusion algorithm significantly boosts both subjective and objective metrics, achieving an mAP of 83.15%-surpassing single light mode detections by 22.97% and 28.31%, and outper-forming Dual-Tree Complex Wavelet, Non-Subsampled Contourlet, and Curvelet Transform fusion meth-ods by 13.14%, 15.01%, and 20.35%, respectively.