Hierarchical Visual Attention Model for Saliency Detection Inspired by Avian Visual Pathways

Visual attention is a mechanism that enables the visual system to detect potentially important objects in complex environment. Most computational visual attention models are designed with inspirations from mammalian visual systems.However, electrophysiological and behavioral evidences indicate that avian species are animals with high visual capability that can process complex information accurately in real time. Therefore,the visual system of the avian species, especially the nuclei related to the visual attention mechanism, are investigated in this paper. Afterwards, a hierarchical visual attention model is proposed for saliency detection. The optic tectum neuron responses are computed and the self-information is used to compute primary saliency maps in the first hierarchy. The "winner-takeall" network in the tecto-isthmal projection is simulated and final saliency maps are estimated with the regularized random walks ranking in the second hierarchy. Comparison results verify that the proposed model, which can define the focus of attention accurately, outperforms several state-of-the-art models.This study provides insights into the relationship between the visual attention mechanism and the avian visual pathways. The computational visual attention model may reveal the underlying neural mechanism of the nuclei for biological visual attention.

Saliency detection based on superpixels clustering and stereo disparity

Reliable saliency detection can be used to quickly and effectively locate objects in images. In this paper, a novel algorithm for saliency detection based on superpixels clustering and stereo disparity （SDC） is proposed. Firstly, we use an improved superpixels clustering method to decompose the given image. Then, the disparity of each superpixel is computed by a modified stereo correspondence algorithm. Finally, a new measure which combines stereo disparity with color contrast and spatial coherence is defined to evaluate the saliency of each superpixel. From the experiments we can see that regions with high disparity can get higher saliency value, and the saliency maps have the same resolution with the source images, objects in the map have clear boundaries. Due to the use of superpixel and stereo disparity information, the proposed method is computationally efficient and outperforms some state-of-the-art color- based saliency detection methods.

Saliency detection of infrared image based on region covariance and global feature

In order to better represent infrared target features under different environments, a saliency detection method based on region covariance and global feature is proposed. Firstly, the region covariance features on different scale spaces and different image regions are extracted and transformed into sigma features,then combined with central position feature, the local salient map is generated. Next, a global salient map is generated by gray contrast and density estimation. Finally, the saliency detection result of infrared images is obtained by fusing the local and global salient maps. The experimental results show that the salient map of the proposed method has complete target features and obvious edges,and the proposed method is better than the state of art method both qualitatively and quantitatively.

Maritime Target Saliency Detection for UAV Based on the Stimulation Competition Selection Mechanism of Raptor Vision

A maritime target saliency detection method inspired by the stimulation competition and selection mechanism of raptor vision is presented for the airborne vision system of unmanned aerial vehicle(UAV)in an unknown maritime environment.The stimulation competition and selection mechanism in the visual pathway of raptor vision based on the phenomenon of raptor capturing prey in complex scenes are studied.Then,the mathematical model of the stimulation competition and selection mechanism of raptor vision is established and employed for the salient object detection.Popular image datasets and practical scene datasets are applied to verify the effectiveness of the presented method.Results show that the detection performance of the proposed method is better than that of other comparison methods.The proposed algorithm provides an idea for maritime target salient detection and cross-domain joint mission for UAV or other unmanned equipment.

Co-salient object detection with iterative purification and predictive optimization

Background Co-salient object detection(Co-SOD)aims to identify and segment commonly salient objects in a set of related images.However,most current Co-SOD methods encounter issues with the inclusion of irrelevant information in the co-representation.These issues hamper their ability to locate co-salient objects and significantly restrict the accuracy of detection.Methods To address this issue,this study introduces a novel Co-SOD method with iterative purification and predictive optimization(IPPO)comprising a common salient purification module(CSPM),predictive optimizing module(POM),and diminishing mixed enhancement block(DMEB).Results These components are designed to explore noise-free joint representations,assist the model in enhancing the quality of the final prediction results,and significantly improve the performance of the Co-SOD algorithm.Furthermore,through a comprehensive evaluation of IPPO and state-of-the-art algorithms focusing on the roles of CSPM,POM,and DMEB,our experiments confirmed that these components are pivotal in enhancing the performance of the model,substantiating the significant advancements of our method over existing benchmarks.Experiments on several challenging benchmark co-saliency datasets demonstrate that the proposed IPPO achieves state-of-the-art performance.

Seismic Attribute Analysis with Saliency Detection in Fractional Fourier Transform Domain

Most image saliency detection models are dependent on prior knowledge and demand high computational cost. However, spectral residual（SR） and phase spectrum of the Fourier transform（PFT） models are simple and fast saliency detection approaches based on two-dimensional Fourier transform without the prior knowledge. For seismic data, the geological structure of the underground rock formation changes more obviously in the time direction. Therefore, one-dimensional Fourier transform is more suitable for seismic saliency detection. Fractional Fourier transform（FrFT） is an improved algorithm for Fourier transform, therefore we propose the seismic SR and PFT models in one-dimensional FrF T domain to obtain more detailed saliency maps. These two models use the amplitude and phase information in FrFT domain to construct the corresponding saliency maps in spatial domain. By means of these two models, several saliency maps at different fractional orders can be obtained for seismic attribute analysis. These saliency maps can characterize the detailed features and highlight the object areas, which is more conducive to determine the location of reservoirs. The performance of the proposed method is assessed on both simulated and real seismic data. The results indicate that our method is effective and convenient for seismic attribute extraction with good noise immunity.

Saliency Detection via Manifold Ranking Based on Robust Foreground

The graph-based manifold ranking saliency detection only relies on the boundary background to extract foreground seeds,resulting in a poor saliency detection result,so a method that obtains robust foreground for manifold ranking is proposed in this paper.First,boundary connectivity is used to select the boundary background for manifold ranking to get a preliminary saliency map,and a foreground region is acquired by a binary segmentation of the map.Second,the feature points of the original image and the filtered image are obtained by using color boosting Harris corners to generate two different convex hulls.Calculating the intersection of these two convex hulls,a final convex hull is found.Finally,the foreground region and the final convex hull are combined to extract robust foreground seeds for manifold ranking and getting final saliency map.Experimental results on two public image datasets show that the proposed method gains improved performance compared with some other classic methods in three evaluation indicators:precision-recall curve,F-measure and mean absolute error.

A Detection Method of Bolts on Axlebox Cover Based on Cascade Deep Convolutional Neural Network

This paper proposes a cascade deep convolutional neural network to address the loosening detection problem of bolts on axlebox covers.Firstly,an SSD network based on ResNet50 and CBAM module by improving bolt image features is proposed for locating bolts on axlebox covers.And then,theA2-PFN is proposed according to the slender features of the marker lines for extracting more accurate marker lines regions of the bolts.Finally,a rectangular approximationmethod is proposed to regularize themarker line regions asaway tocalculate the angle of themarker line and plot all the angle values into an angle table,according to which the criteria of the angle table can determine whether the bolt with the marker line is in danger of loosening.Meanwhile,our improved algorithm is compared with the pre-improved algorithmin the object localization stage.The results show that our proposed method has a significant improvement in both detection accuracy and detection speed,where ourmAP(IoU=0.75)reaches 0.77 and fps reaches 16.6.And in the saliency detection stage,after qualitative comparison and quantitative comparison,our method significantly outperforms other state-of-the-art methods,where our MAE reaches 0.092,F-measure reaches 0.948 and AUC reaches 0.943.Ultimately,according to the angle table,out of 676 bolt samples,a total of 60 bolts are loose,69 bolts are at risk of loosening,and 547 bolts are tightened.

Vehicle Detection Based on Visual Saliency and Deep Sparse Convolution Hierarchical Model

Traditional vehicle detection algorithms use traverse search based vehicle candidate generation and hand crafted based classifier training for vehicle candidate verification.These types of methods generally have high processing times and low vehicle detection performance.To address this issue,a visual saliency and deep sparse convolution hierarchical model based vehicle detection algorithm is proposed.A visual saliency calculation is firstly used to generate a small vehicle candidate area.The vehicle candidate sub images are then loaded into a sparse deep convolution hierarchical model with an SVM-based classifier to perform the final detection.The experimental results demonstrate that the proposed method is with 94.81% correct rate and 0.78% false detection rate on the existing datasets and the real road pictures captured by our group,which outperforms the existing state-of-the-art algorithms.More importantly,high discriminative multi-scale features are generated by deep sparse convolution network which has broad application prospects in target recognition in the field of intelligent vehicle.

Drogue detection for autonomous aerial refueling via hybrid pigeon-inspired optimized color opponent and saliency aggregation

Drogue detection is one of the challenging tasks in autonomous aerial refueling due to the requirement for accuracy and rapidity.Saliency detection based on image intrinsic cues can achieve fast detection,but with poor accuracy.Recent studies reveal that optimization-based methods provide accurate and quick solutions for saliency detection.This paper presents a hybrid pigeon-inspired optimization method,the optimized color opponent,that aims to adjust the weight of color opponent channels to detect the drogue region.It can optimize the weights in the selected aerial refueling scene offline,and the results are applied for drogue detection in the scene.A novel algorithm aggregated by the optimized color opponent and robust background detection is presented to provide better precision and robustness.Experimental results on benchmark datasets and aerial refueling images show that the proposed method successfully extracts the saliency region or drogue and exhibits superior performance against the other saliency detection methods with intrinsic cues.The algorithm designed in this paper is competent for the drogue detection task of autonomous aerial refueling.

Robust detection algorithm with triple constraints for cooperative target based on spectral residual

The accurate detection of cooperative targets plays a key and foundational role in unmanned aerial vehicle (UAV) landing autonomously. The standard method based on fixed threshold is too susceptible to both illumination variations and interference. To overcome issues above, a robust detection algorithm with triple constraints for cooperative targets based on spectral residual (TCSR) is proposed. Firstly, by designing an asymmetric cooperative target, which comprises red background, green H and triangle target, the captured original image is converted into a Lab color space, whose saliency map is yielded by constructing the spectral residual. Then, the triple constraints are developed according to the prior knowledge of the cooperative target. Finally, the salient region in saliency map is considered as the cooperative target, and it meets the triple constraints. Experimental results in complex environments show that the proposed TCSR outperforms the standard methods in higher detection accuracy and lower false alarm rate.

A Saliency Based Image Fusion Framework for Skin Lesion Segmentation and Classification

Melanoma,due to its higher mortality rate,is considered as one of the most pernicious types of skin cancers,mostly affecting the white populations.It has been reported a number of times and is now widely accepted,that early detection of melanoma increases the chances of the subject’s survival.Computer-aided diagnostic systems help the experts in diagnosing the skin lesion at earlier stages using machine learning techniques.In thiswork,we propose a framework that accurately segments,and later classifies,the lesion using improved image segmentation and fusion methods.The proposed technique takes an image and passes it through two methods simultaneously;one is the weighted visual saliency-based method,and the second is improved HDCT based saliency estimation.The resultant image maps are later fused using the proposed image fusion technique to generate a localized lesion region.The resultant binary image is later mapped back to the RGB image and fed into the Inception-ResNet-V2 pre-trained model-trained by applying transfer learning.The simulation results show improved performance compared to several existing methods.

Cluster-Based Saliency-Guided Content-Aware Image Retargeting

Straightforward image resizing operators without considering image contents （e.g., uniform scaling） cannot usually produce satisfactory results, while content-aware image retargeting aims to arbitrarily change image size while preserving visually prominent features. In this paper, a cluster-based saliency-guided seam carving algorithm for content- aware image retargeting is proposed. To cope with the main drawback of the original seam carving algorithm relying on only gradient-based image importance map, we integrate a gradient-based map and a cluster-based saliency map to generate a more reliable importance map, resulting in better single image retargeting results. Experimental results have demonstrated the efficacy of the proposed algorithm.

Refined Sparse Representation Based Similar Category Image Retrieval

Given one specific image,it would be quite significant if humanity could simply retrieve all those pictures that fall into a similar category of images.However,traditional methods are inclined to achieve high-quality retrieval by utilizing adequate learning instances,ignoring the extraction of the image’s essential information which leads to difficulty in the retrieval of similar category images just using one reference image.Aiming to solve this problem above,we proposed in this paper one refined sparse representation based similar category image retrieval model.On the one hand,saliency detection and multi-level decomposition could contribute to taking salient and spatial information into consideration more fully in the future.On the other hand,the cross mutual sparse coding model aims to extract the image’s essential feature to the maximumextent possible.At last,we set up a database concluding a large number of multi-source images.Adequate groups of comparative experiments show that our method could contribute to retrieving similar category images effectively.Moreover,adequate groups of ablation experiments show that nearly all procedures play their roles,respectively.

Biological Eagle-eye Inspired Target Detection for Unmanned Aerial Vehicles Equipped with a Manipulator

Inspired by eagle eye mechanisms,the structure and information processing characteristics of the eagle′s visual system are used for the target capture task of an unmanned aerial vehicle(UAV)with a mechanical arm.In this paper,a novel eagle-eye inspired multi-camera sensor and a saliency detection method are proposed.A combined camera system is built by simulating the double fovea structure on the eagle retina.A saliency target detection method based on the eagle midbrain inhibition mechanism is proposed by measuring the static saliency information and dynamic features.Thus,salient targets can be accurately detected through the collaborative work between different cameras of the proposed multi-camera sensor.Experimental results show that the eagle-eye inspired visual system is able to continuously detect targets in outdoor scenes and that the proposed algorithm has a strong inhibitory effect on moving background interference.

Visual tracking based on transfer learning of deep salience information

In this paper,we propose a new visual tracking method in light of salience information and deep learning.Salience detection is used to exploit features with salient information of the image.Complicated representations of image features can be gained by the function of every layer in convolution neural network(CNN).The characteristic of biology vision in attention-based salience is similar to the neuroscience features of convolution neural network.This motivates us to improve the representation ability of CNN with functions of salience detection.We adopt the fully-convolution networks(FCNs)to perform salience detection.We take parts of the network structure to perform salience extraction,which promotes the classification ability of the model.The network we propose shows great performance in tracking with the salient information.Compared with other excellent algorithms,our algorithm can track the target better in the open tracking datasets.We realize the 0.5592 accuracy on visual object tracking 2015(VOT15)dataset.For unmanned aerial vehicle 123(UAV123)dataset,the precision and success rate of our tracker is 0.710 and 0.429.

Phase spectrum based automatic ship detection in synthetic aperture radar images

This paper proposes an automatic ship detection approach in Synthetic Aperture Radar(SAR)Images using phase spectrum.The proposed method mainly contains two stages:Firstly,sea-land segmentation of SAR Images is one of the key stages for SAR image application such as sea-targets detection and recognition,which are easily detected only in sea regions.In order to eliminate the influence of land regions in SAR images,a novel land removing method is explored.The removing method employs a Harris corner detector to obtain some image patches belonging to land,and the probability density function(PDF)of land area can be estimated by these patches.Thus,an appropriate land segmentation threshold is accordingly obtained.Secondly,an automatic ship detector based on phase spectrum is proposed.The proposed detector is free from various idealized assumptions and can accurately detect ships in SAR images.Experimental results demonstrate the efficiency of the proposed ship detection algorithm in diversified SAR images.

Automatic Video Segmentation Based on Information Centroid and Optimized SaliencyCut

We propose an automatic video segmentation method based on an optimized SaliencyCut equipped with information centroid(IC)detection according to level balance principle in physical theory.Unlike the existing methods,the image information of another dimension is provided by the IC to enhance the video segmentation accuracy.Specifically,our IC is implemented based on the information-level balance principle in the image,and denoted as the information pivot by aggregating all the image information to a point.To effectively enhance the saliency value of the target object and suppress the background area,we also combine the color and the coordinate information of the image in calculating the local IC and the global IC in the image.Then saliency maps for all frames in the video are calculated based on the detected IC.By applying IC smoothing to enhance the optimized saliency detection,we can further correct the unsatisfied saliency maps,where sharp variations of colors or motions may exist in complex videos.Finally,we obtain the segmentation results based on IC-based saliency maps and optimized SaliencyCut.Our method is evaluated on the DAVIS dataset,consisting of different kinds of challenging videos.Comparisons with the state-of-the-art methods are also conducted to evaluate our method.Convincing visual results and statistical comparisons demonstrate its advantages and robustness for automatic video segmentation.

Saliency guided self-attention network for pedestrian attribute recognition in surveillance scenarios

Pedestrian attribute recognition is often considered as a multi-label image classification task. In order to make full use of attribute-related location information, a saliency guided self-attention network(SGSA-Net) was proposed to weakly supervise attribute localization, without annotations of attribute-related regions. Saliency priors were integrated into the spatial attention module(SAM). Meanwhile, channel-wise attention and spatial attention were introduced into the network. Moreover, a weighted binary cross-entropy loss(WCEL) function was employed to handle the imbalance of training data. Extensive experiments on richly annotated pedestrian(RAP) and pedestrian attribute(PETA) datasets demonstrated that SGSA-Net outperformed other state-of-the-art methods.

3D reconstruction of movable cultural relics based on salient region optimization

How to protect cultural retics is of great significance to the transmission and dissemination of history and culture.Digital 3-dimensional(3D)modeling of cultural relics is an effective way to preserve them.The efficiency and complexity of cultural relic model reconstruction algorithms are significant challenges due to redundant data.To tackle the above issue,a 3D reconstruction algorithm,named COLMAP+LSH,was proposed for movable cultural relics based on salient region optimization.COLMAP+LSH algorithm introduces saliency region detection and locality-sensetive Hashing(LSH)to achieve efficient,accurate,and robust digital 3D modeling of cultural relics.Specifically,400 cultural model data were collected through offline and online collection.COLMAP+LSH algorithm detects the salient region interactively and reduces the number of images in the salient region by feature diffusion.Additionally,COLMAP+LSH algorithm utilizes LSH to calculate the image selection scores and employs the image selection scores to reduce the redundant image.The experiments on the self-constructed cultural relics dataset show that COLMAP+LSH algorithm can efficiently achieve image feature diffusion and ensure the quality of artifact reconstruction while selecting most of the redundant image data.