水下结构状态观测中的悬浮杂质遮挡消除方法

Method for Eliminating Visual Occlusion from Suspended Impurity in Underwater Structural State Observation

水下结构状态视觉检测过程中,观测相机镜头易受到水体中枯叶、藻类等悬浮杂质遮挡,导致水下结构表观影像信息丢失,影响检测效果。针对该问题,利用水下视频序列中的帧内空间信息与帧间运动信息,提出一种悬浮杂质遮挡消除方法。根据相邻帧间的光流场分布信息,提出位移补偿策略,消除相机移动导致的帧间背景偏移;结合悬浮杂质成像特性,建立动态视觉感知模型,在对齐后的相邻帧基础上,实现不同形态悬浮杂质的准确检测;构建一种混合引导修复模型,确立帧间最优互补信息,还原悬浮杂质遮挡区域。在构建的真实与合成数据集上的测试结果表明,本文方法能够准确检测并消除悬浮杂质的遮挡,处理后的图像质量在多项指标上均得到明显提升。

Objective During the visual inspection of underwater structures,the camera lens used for observation is often obscured by suspended impurities such as dead leaves and algae in the water.This obstruction leads to a loss of clear image information,affecting the effectiveness of the inspection.Currently,there is limited research focused on removing these suspended impurities underwater.To address this issue,we propose a method for eliminating visual occlusion from impurities using the intra-frame spatial information and inter-frame motion information from underwater video sequences.Our approach thoroughly analyzes the imaging characteristics of these impurities in water.Then we provide prior information according to analysis and leverage the dynamic perception and information complementarity between adjacent frames to detect and repair occluded regions affected by underwater suspended impurities.We aim to enhance the quality of underwater images and enable underwater robots to more effectively detect the condition of underwater structures.Methods In this study,we propose an underwater suspended impurity occlusion elimination method by combining the characteristics of suspended impurity imaging,the motion information and complementary information between adjacent consecutive frames.Firstly,we take the current frame and its two adjacent frames as a set of input data and estimate the optical flows of the three input frames.Based on the distribution of optical flow between adjacent frames,a displacement compensation strategy is proposed to eliminate the background shift caused by camera movement.Secondly,considering the imaging characteristics of suspended impurities,we build a dynamic visual perception model by extracting the motion and color information,which aims to accurately detect impurities with different shapes in aligned neighboring frames.Finally,a hybrid guided restoration model is constructed to determine the optimal complementary information between frames and restore the areas obstructed by suspended impurities.In detail,there are three steps.According to the built hybrid guided repair model,we match the complementary information between the aligned adjacent frames to initially repair the occluded regions,maximizing the retention of the real scene information.Then we adopt the STTN algorithm to carry out the secondary repair of the occluded regions that cannot be complemented by adjacent frames.Next,the two repair results are merged to obtain the final image repair frame.After the current input data processing is completed,the next set of adjacent frames are updated and the above processes are conducted until all the video frames are repaired.Results and Discussions Test results on both real and synthetic datasets demonstrate that the proposed method can accurately detect and eliminate obstructive impurities,resulting in significantly improved image quality across multiple metrics.Specifically,the adjacent frame alignment by background displacement compensation effectively eliminates the interference of background movement on the detection of suspended impurity regions.The detection of suspended impurities by dynamic visual perception is not affected by their color,size and morphology and can accurately detect and segment various types of suspended impurity regions in the map(Fig.6).The estimated hybrid guided repair model map can accurately judge the best matching region in adjacent frames(Fig.7),effectively guiding the effective repair of the occluded region.The proposed method enables good smoothness and natural transition between the repaired region and the surrounding pixels,and the original detail information of the crack and other regions is better preserved(Fig.8),which improves the clarity and quality of the image(Fig.9).This method takes approximately 8–9 s to process each frame segment,and the processing speed needs to be further improved.The selection method of parameters also needs further optimization.Conclusions In this paper,we propose a suspended impurity occlusion elimination method in underwater structure apparent state detection video for information loss caused by suspended impurities occluding the lens in the actual water body environment.Experiments on the real and synthetic underwater suspended impurity video datasets verify the validity of the proposed method.This method can accurately detect underwater suspended impurity regions of different sizes,colors and densities and can effectively restore the original information of the occluded regions.In conclusion,this study is of great significance for improving the quality of underwater images and helping underwater robots to better detect the state of underwater structures. However, this method still has some limitations that need to be further overcome. Since the proposed method mainly relies on motion and color information to detect underwater suspended impurities, the detection and removal of suspended impurities with unclear motion characteristics due to slow movement or convergence with the camera′s motion direction and speed are not effective. Therefore, in future research work, depth information perception will be combined to further improve the detection and removal effect of underwater suspended impurities. In addition, a dataset containing more scenes of underwater suspended impurities will be further established, and an end-to-end underwater suspended impurity detection and repair model will be established. Optimization terms for various parameters will be designed to decrease specific parameters, reduce the complexity of the model, and improve video processing speed to meet practical application needs.