YOLOv5-Based Seabed Sediment Recognition Method for Side-Scan Sonar Imagery

Seabed sediment recognition is vital for the exploitation of marine resources.Side-scan sonar(SSS)is an excellent tool for acquiring the imagery of seafloor topography.Combined with ocean surface sampling,it provides detailed and accurate images of marine substrate features.Most of the processing of SSS imagery works around limited sampling stations and requires manual interpretation to complete the classification of seabed sediment imagery.In complex sea areas,with manual interpretation,small targets are often lost due to a large amount of information.To date,studies related to the automatic recognition of seabed sediments are still few.This paper proposes a seabed sediment recognition method based on You Only Look Once version 5 and SSS imagery to perform real-time sedi-ment classification and localization for accuracy,particularly on small targets and faster speeds.We used methods such as changing the dataset size,epoch,and optimizer and adding multiscale training to overcome the challenges of having a small sample and a low accuracy.With these methods,we improved the results on mean average precision by 8.98%and F1 score by 11.12%compared with the original method.In addition,the detection speed was approximately 100 frames per second,which is faster than that of previous methods.This speed enabled us to achieve real-time seabed sediment recognition from SSS imagery.

DcNet: Dilated Convolutional Neural Networks for Side-Scan Sonar Image Semantic Segmentation

In ocean explorations,side-scan sonar(SSS)plays a very important role and can quickly depict seabed topography.As-sembling the SSS to an autonomous underwater vehicle(AUV)and performing semantic segmentation of an SSS image in real time can realize online submarine geomorphology or target recognition,which is conducive to submarine detection.However,because of the complexity of the marine environment,various noises in the ocean pollute the sonar image,which also encounters the intensity inhomogeneity problem.In this paper,we propose a novel neural network architecture named dilated convolutional neural network(DcNet)that can run in real time while addressing the above-mentioned issues and providing accurate semantic segmentation.The proposed architecture presents an encoder-decoder network to gradually reduce the spatial dimension of the input image and recover the details of the target,respectively.The core of our network is a novel block connection named DCblock,which mainly uses dilated convolution and depthwise separable convolution between the encoder and decoder to attain more context while still retaining high accuracy.Furthermore,our proposed method performs a super-resolution reconstruction to enlarge the dataset with high-quality im-ages.We compared our network to other common semantic segmentation networks performed on an NVIDIA Jetson TX2 using our sonar image datasets.Experimental results show that while the inference speed of the proposed network significantly outperforms state-of-the-art architectures,the accuracy of our method is still comparable,which indicates its potential applications not only in AUVs equipped with SSS but also in marine exploration.

Multi-beam Sonar and Side-scan Sonar Image Co-registering and Fusing

Multi-beam Sonar and Side-scan Sonar compensate each other. In order to fully utilize all information, it is necessary to fuse two kinds of image and data. And the image co-registration is an important and complicated job before fusion. This paper suggests combining bathymetric data with intensity image, obtaining the characteristic points through the minimal angles of lines, and then deciding the corresponding image points by the maximal correlate coefficient in searching space. Finally, the second order polynomial is applied to the deformation model. After the images have been co-registered, Wavelet is used to fuse the images. It is shown that this algorithm can be used in the flat seafloor or the isotropic seabed. Verification is made in the paper with the observed data.

RepDNet:A re-parameterization despeckling network for autonomous underwater side-scan sonar imaging with prior-knowledge customized convolution

Side-scan sonar(SSS)is now a prevalent instrument for large-scale seafloor topography measurements,deployable on an autonomous underwater vehicle(AUV)to execute fully automated underwater acoustic scanning imaging along a predetermined trajectory.However,SSS images often suffer from speckle noise caused by mutual interference between echoes,and limited AUV computational resources further hinder noise suppression.Existing approaches for SSS image processing and speckle noise reduction rely heavily on complex network structures and fail to combine the benefits of deep learning and domain knowledge.To address the problem,Rep DNet,a novel and effective despeckling convolutional neural network is proposed.Rep DNet introduces two re-parameterized blocks:the Pixel Smoothing Block(PSB)and Edge Enhancement Block(EEB),preserving edge information while attenuating speckle noise.During training,PSB and EEB manifest as double-layered multi-branch structures,integrating first-order and secondorder derivatives and smoothing functions.During inference,the branches are re-parameterized into a 3×3 convolution,enabling efficient inference without sacrificing accuracy.Rep DNet comprises three computational operations:3×3 convolution,element-wise summation and Rectified Linear Unit activation.Evaluations on benchmark datasets,a real SSS dataset and Data collected at Lake Mulan aestablish Rep DNet as a well-balanced network,meeting the AUV computational constraints in terms of performance and latency.

A semantic segmentation-based underwater acoustic image transmission framework for cooperative SLAM

With the development of underwater sonar detection technology,simultaneous localization and mapping(SLAM)approach has attracted much attention in underwater navigation field in recent years.But the weak detection ability of a single vehicle limits the SLAM performance in wide areas.Thereby,cooperative SLAM using multiple vehicles has become an important research direction.The key factor of cooperative SLAM is timely and efficient sonar image transmission among underwater vehicles.However,the limited bandwidth of underwater acoustic channels contradicts a large amount of sonar image data.It is essential to compress the images before transmission.Recently,deep neural networks have great value in image compression by virtue of the powerful learning ability of neural networks,but the existing sonar image compression methods based on neural network usually focus on the pixel-level information without the semantic-level information.In this paper,we propose a novel underwater acoustic transmission scheme called UAT-SSIC that includes semantic segmentation-based sonar image compression(SSIC)framework and the joint source-channel codec,to improve the accuracy of the semantic information of the reconstructed sonar image at the receiver.The SSIC framework consists of Auto-Encoder structure-based sonar image compression network,which is measured by a semantic segmentation network's residual.Considering that sonar images have the characteristics of blurred target edges,the semantic segmentation network used a special dilated convolution neural network(DiCNN)to enhance segmentation accuracy by expanding the range of receptive fields.The joint source-channel codec with unequal error protection is proposed that adjusts the power level of the transmitted data,which deal with sonar image transmission error caused by the serious underwater acoustic channel.Experiment results demonstrate that our method preserves more semantic information,with advantages over existing methods at the same compression ratio.It also improves the error tolerance and packet loss resistance of transmission.

Fast Segmentation Method of Sonar Images for Jacket Installation Environment

It has remained a hard nut for years to segment sonar images of jacket installation environment,most of which are noisy images with inevitable blur after noise reduction.For the purpose of solutions to this problem,a fast segmen-tation algorithm is proposed on the basis of the gray value characteristics of sonar images.This algorithm is endowed with the advantage in no need of segmentation thresholds.To realize this goal,we follow the undermentioned steps:first,calcu-late the gray matrix of the fuzzy image background.After adjusting the gray value,the image is divided into three regions:background region,buffer region and target regions.Afterfiltering,we reset the pixels with gray value lower than 255 to binarize images and eliminate most artifacts.Finally,the remaining noise is removed by morphological processing.The simulation results of several sonar images show that the algorithm can segment the fuzzy sonar images quickly and effectively.Thus,the stable and feasible method is testified.

Improving Yolo5 for Real-Time Detection of Small Targets in Side Scan Sonar Images

Side scan sonar(SSS)is an important means to detect and locate seafloor targets.Autonomous underwater vehicles(AUVs)carrying SSS stay near the seafloor to obtain high-resolution images and provide the outline of the target for observers.The target feature information of an SSS image is similar to the background information,and a small target has less pixel information;therefore,accu-rately identifying and locating small targets in SSS images is challenging.We collect the SSS images of iron metal balls(with a diameter of 1m)and rocks to solve the problem of target misclassification.Thus,the dataset contains two types of targets,namely,‘ball’and‘rock’.With the aim to enable AUVs to accurately and automatically identify small underwater targets in SSS images,this study designs a multisize parallel convolution module embedded in state-of-the-art Yolo5.An attention mechanism transformer and a convolutional block attention module are also introduced to compare their contributions to small target detection accuracy.The performance of the proposed method is further evaluated by taking the lightweight networks Mobilenet3 and Shufflenet2 as the backbone network of Yolo5.This study focuses on the performance of convolutional neural networks for the detection of small targets in SSS images,while another comparison experiment is carried out using traditional HOG+SVM to highlight the neural network’s ability.This study aims to improve the detection accuracy while ensuring the model efficiency to meet the real-time working requirements of AUV target detection.

Sedimentary processes in Zenisu deep-sea channel revealed by side-scan imagery

Side-scan sonar data collected by Cruises 99-09 Leg 2 and 00-06 Leg l of R/V Yokosuka were used to reveal the sedimentary processes in Zenisu deep-sea channel. The middle and lower segments of the channel are rich in turbidite and other debrite deposits. By high-resolution imaging, three sedimentary processes were distinguished with distinct acoustic features. 1. Slumps and slides occur with contrasting backscatter, rough surface textures, blockings, and acoustic shadows at headwalls. They are very extensive and often in lobate form downslope. 2. Debris flow has uniform, general medium backscatter, sometimes showing marbling/lineation in lobate form. 3. Turbidity current is characterized by low backscatter confined to the channel as acoustic signal is attenuated. Regional tectonics must be the dominating factor that controls deposition pattern in this area.

Underwater Terrain-Aided Navigation Based on Multibeam Bathymetric Sonar Images

Underwater terrain-aided navigation is used to complement the traditional inertial navigation employed by autonomous underwater vehicles during lengthy missions. It can provide fixed estimations by matching real-time depth data with a digital terrain map, This study presents the concept of using image processing techniques in the underwater terrain matching process. A traditional gray-scale histogram of an image is enriched by incorporation with spatial information in pixels. Edge comer pixels are then defined and used to construct an edge comer histogram, which employs as a template to scan the digital terrain map and estimate the fixes of the vehicle by searching the correlation peak. Simulations are performed to investigate the robustness of the proposed method, particularly in relation to its sensitivity to background noise, the scale of real-time images, and the travel direction of the vehicle. At an image resolution of 1 m2/pixel, the accuracy of localization is more than 10 meters.

Sonar Image Detection Algorithm Based on Two-Phase Manifold Partner Clustering

According to the characteristics of sonar image data with manifold feature,the sonar image detection method based on two-phase manifold partner clustering algorithm is proposed. Firstly,K-means block clustering based on euclidean distance is proposed to reduce the data set. Mean value,standard deviation,and gray minimum value are considered as three features based on the relatinship between clustering model and data structure. Then K-means clustering algorithm based on manifold distance is utilized clustering again on the reduced data set to improve the detection efficiency. In K-means clustering algorithm based on manifold distance,line segment length on the manifold is analyzed,and a new power function line segment length is proposed to decrease the computational complexity. In order to quickly calculate the manifold distance,new allsource shortest path as the pretreatment of efficient algorithm is proposed. Based on this,the spatial feature of the image block is added in the three features to get the final precise partner clustering algorithm. The comparison with the other typical clustering algorithms demonstrates that the proposed algorithm gets good detection result. And it has better adaptability by experiments of the different real sonar images.

IMPROVED SYNTHETIC APERTURE SONAR MOTION COMPENSATION COMBINED DPCA WITH SUB-APERTURE IMAGE CORRELATION

Estimation precision of Displaced Phase Center Algorithm(DPCA) is affected by the number of displaced phase center pairs,the bandwidth of transmitting signal and many other factors.Detailed analysis is made on DPCA's estimation precision.Analysis results show that the directional vector estimation precision of DPCA is low,which will produce accumulating errors when phase cen-ters' track is estimated.Because of this reason,DPCA suffers from accumulating errors seriously.To overcome this problem,a method combining DPCA with Sub Aperture Image Correlation(SAIC) is presented.Large synthetic aperture is divided into sub-apertures.Micro errors in sub-aperture are estimated by DPCA and compensated to raw echo data.Bulk errors between sub-apertures are esti-mated by SAIC and compensated directly to sub-aperture images.After that,sub-aperture images are directly used to generate ultimate SAS image.The method is applied to the lake-trial dataset of a 20 kHz SAS prototype system.Results show the method can successfully remove the accumulating error and produce a better SAS image.

Landslide data mosaicking based on an airborne laser point cloud and multi-beam sonar images

Landslides are one of the most disastrous geological hazards in southwestern China.Once a landslide becomes unstable,it threatens the lives and safety of local residents.However,empirical studies on landslides have predominantly focused on landslides that occur on land.To this end,we aim to investigate ashore and underwater landslide data synchronously.This study proposes an optimized mosaicking method for ashore and underwater landslide data.This method fuses an airborne laser point cloud with multi-beam depth sounder images.Owing to their relatively high efficiency and large coverage area,airborne laser measurement systems are suitable for emergency investigations of landslides.Based on the airborne laser point cloud,the traversal of the point with the lowest elevation value in the point set can be used to perform rapid extraction of the crude channel boundaries.Further meticulous extraction of the channel boundaries is then implemented using the probability mean value optimization method.In addition,synthesis of the integrated ashore and underwater landslide data angle is realized using the spatial guide line between the channel boundaries and the underwater multibeam sonar images.A landslide located on the right bank of the middle reaches of the Yalong River is selected as a case study to demonstrate that the proposed method has higher precision thantraditional methods.The experimental results show that the mosaicking method in this study can meet the basic needs of landslide modeling and provide a basis for qualitative and quantitative analysis and stability prediction of landslides.

Sonar Image Registration and Mosaic Based on Line Detection and Triangle Matching

Image registration is an important research topic in the field of computer vision,in which the registration and mosaic of side-scan sonar images is the keypoints of underwater navigation.However,the image registration method of keypoints is not suitable for sonar images which do not have obvious feature points.Therefore,a method of sonar-image registration and mosaic based on line segment extraction and triangle matching is proposed in this paper.Firstly,in order to extract features from sonar image,the LSD method is introduced to detect line feature from images,and line segments are filtered by the principle of attention;after that,triangles are formed from line segments,an image transformation matrix can be calculated through the heuristic greedy algorithm from these triangles;finally,images are merged based on the transformation information.On the basis of practical tests,it is found that,the feature extraction method used in this paper can better describe the outline of underwater terrain,and there is no obvious stitching gap between the result of sonar images stitched.Experimental results show that the proposed method is effective than the keypoints method of the registration and mosaic of sonar images.

Segmentation of complex objects’ sonar images using parameter-fixed MRF model

The effective method of the recognition of underwater complex objects in sonar image is to segment sonar image into target, shadow and sea-bottom reverberation regions and then extract the edge of the object. Because of the time-varying and space-varying characters of underwater acoustics environment, the sonar images have poor quality and serious speckle noise, so traditional image segmentation is unable to achieve precise segmentation. In the paper, the image segmentation process based on MRF (Markov random field) model is studied, and a practical method of estimating model parameters is proposed. Through analyzing the impact of chosen model parameters, a sonar imagery segmentation algorithm based on fixed parameters’ MRF model is proposed. Both of the segmentation effect and the low computing load are gained. By applying the algorithm to the synthesized texture image and actual side-scan sonar image, the algorithm can be achieved with precise segmentation result.

Sonar Image Processing System for an Autonomous Underwater Vehicle(AUV)

Sonar image processing system is an important intelligent system of Autonomous Un-derwater Vehicle.Based on TMS320C30 high speed DSP,it is used to realize sonar imagecompression and underwater object detections including obstacle recognition in real time.Inthis paper,the software and hardware designs of this system are introduced and the experi-mental results are given.

Underwater sonar image recognition based on gray-spatial histograms

A new gray-spatial histogram is proposed, which incorporates spatial informatio n with gray compositions without sacrificing the robustness of traditional gray histograms. The purpose is to consider the representation role of gray compositi ons and spatial information simultaneously. Each entry in the gray-spatial hist ogram is the gray frequency and corresponding position information of images. In the experiments of sonar image recognition, the results show that the gray-spa tial histogram is effective in practical use.

An Automated Approach to Passive Sonar Classification Using Binary Image Features

This paper proposes a new method for ship recognition and classification using sound produced and radiated underwater. To do so, a three-step procedure is proposed. First, the preprocessing operations are utilized to reduce noise effects and provide signal for feature extraction. Second, a binary image, made from frequency spectrum of signal segmentation, is formed to extract effective features. Third, a neural classifier is designed to classify the signals. Two approaches, the proposed method and the fractal-based method are compared and tested on real data. The comparative results indicated better recognition ability and more robust performance of the proposed method than the fractal-based method. Therefore, the proposed method could improve the recognition accuracy of underwater acoustic targets.

Review of research on sonar imaging technology in China

Over the past 20 years,sonar imaging technology particularly for the high-technology sector has been a focus of research,in which many developed countries,especially those with coast lines,have been competing with each other.It has seen a rapid development with increasing widespread applications that has played an important and irreplaceable role in underwater exploration with great prospects for social,economic,scientific,and military benefits.The fundamental techniques underlying sonar imaging,including multi-beamforming,synthetic-aperture and inverse synthetic-aperture sonar,acoustic lensing,and acoustical holography,are described in this paper.This is followed by a comprehensive and systematic review on the advantages and disadvantages of these imaging techniques,applicability conditions,development trends,new ideas,new methods,and improvements in old methods over recent years with an emphasis on the situation in China,along with a bold and constructive prediction to some development characteristics of sonar imaging technology in the near future in China.The perspectives presented in this paper are offered with the idea of providing some degree of guidance and promotion of research on sonar imaging technology.

Texture image classification using multi fractal dimension

This paper presents a supervised classification method of sonar image, which takes advantages of both multi-fractal theory and wavelet analysis. In the process of feature extraction, image transformation and wavelet decomposition are combined and a feature set based on multi-fractal dimension is obtained. In the part of classifier construction, the Learning Vector Quantization (LVQ) network is adopted as a classifier. Experiments of sonar image classification were carried out with satisfactory results, which verify the effectiveness of this method.

Compensating for intensity loss in a large-aperture MIMO sonar imaging system

To reduce the computation burden of a large-aperture multiple-input multiple-output（MIMO） sonar imaging system,the phase-shift beamformer（PSBF） is used at the cost of bringing the intensity loss（IL）.The cause of the IL is analyzed in detail and a variable termed as IL factor is defined to quantify the loss amount.To compensate for the IL,two methods termed as intensity compensation for the PSBF（IC-PSBF） and the hybrid beamforming（HBF）,respectively,are proposed.The IC-PSBF uses previously estimated IL factors to compensate for output intensities of all PSBFs;and the HBF applies the IC-PSBF to the center beam region and the shifted-sideband beamformer（SSBF） to the side beam region,respectively.Numerical simulations demonstrate the effectiveness of the two proposed methods.