A Concise and Varied Visual Features-Based Image Captioning Model with Visual Selection

Image captioning has gained increasing attention in recent years.Visual characteristics found in input images play a crucial role in generating high-quality captions.Prior studies have used visual attention mechanisms to dynamically focus on localized regions of the input image,improving the effectiveness of identifying relevant image regions at each step of caption generation.However,providing image captioning models with the capability of selecting the most relevant visual features from the input image and attending to them can significantly improve the utilization of these features.Consequently,this leads to enhanced captioning network performance.In light of this,we present an image captioning framework that efficiently exploits the extracted representations of the image.Our framework comprises three key components:the Visual Feature Detector module(VFD),the Visual Feature Visual Attention module(VFVA),and the language model.The VFD module is responsible for detecting a subset of the most pertinent features from the local visual features,creating an updated visual features matrix.Subsequently,the VFVA directs its attention to the visual features matrix generated by the VFD,resulting in an updated context vector employed by the language model to generate an informative description.Integrating the VFD and VFVA modules introduces an additional layer of processing for the visual features,thereby contributing to enhancing the image captioning model’s performance.Using the MS-COCO dataset,our experiments show that the proposed framework competes well with state-of-the-art methods,effectively leveraging visual representations to improve performance.The implementation code can be found here:https://github.com/althobhani/VFDICM(accessed on 30 July 2024).

Discerning Weld Seam Pro les from Strong Arc Background for the Robotic Automated Welding Process via Visual Attention Features

In the robotic welding process with thick steel plates,laser vision sensors are widely used to profile the weld seam to implement automatic seam tracking.The weld seam profile extraction(WSPE)result is a crucial step for identifying the feature points of the extracted profile to guide the welding torch in real time.The visual information processing system may collapse when interference data points in the image survive during the phase of feature point identification,which results in low tracking accuracy and poor welding quality.This paper presents a visual attention featurebased method to extract the weld seam profile(WSP)from the strong arc background using clustering results.First,a binary image is obtained through the preprocessing stage.Second,all data points with a gray value 255 are clustered with the nearest neighborhood clustering algorithm.Third,a strategy is developed to discern one cluster belonging to the WSP from the appointed candidate clusters in each loop,and a scheme is proposed to extract the entire WSP using visual continuity.Compared with the previous methods the proposed method in this paper can extract more useful details of the WSP and has better stability in terms of removing the interference data.Considerable WSPE tests with butt joints and T-joints show the anti-interference ability of the proposed method,which contributes to smoothing the welding process and shows its practical value in robotic automated welding with thick steel plates.

Human Visual Attention Mechanism-Inspired Point-and-Line Stereo Visual Odometry for Environments with Uneven Distributed Features

Visual odometry is critical in visual simultaneous localization and mapping for robot navigation.However,the pose estimation performance of most current visual odometry algorithms degrades in scenes with unevenly distributed features because dense features occupy excessive weight.Herein,a new human visual attention mechanism for point-and-line stereo visual odometry,which is called point-line-weight-mechanism visual odometry(PLWM-VO),is proposed to describe scene features in a global and balanced manner.A weight-adaptive model based on region partition and region growth is generated for the human visual attention mechanism,where sufficient attention is assigned to position-distinctive objects(sparse features in the environment).Furthermore,the sum of absolute differences algorithm is used to improve the accuracy of initialization for line features.Compared with the state-of-the-art method(ORB-VO),PLWM-VO show a 36.79%reduction in the absolute trajectory error on the Kitti and Euroc datasets.Although the time consumption of PLWM-VO is higher than that of ORB-VO,online test results indicate that PLWM-VO satisfies the real-time demand.The proposed algorithm not only significantly promotes the environmental adaptability of visual odometry,but also quantitatively demonstrates the superiority of the human visual attention mechanism.

Integrating Audio-Visual Features and Text Information for Story Segmentation of News Video

Video data are composed of multimodal information streams including visual, auditory and textual streams, so an approach of story segmentation for news video using multimodal analysis is described in this paper. The proposed approach detects the topic-caption frames, and integrates them with silence clips detection results, as well as shot segmentation results to locate the news story boundaries. The integration of audio-visual features and text information overcomes the weakness of the approach using only image analysis techniques. On test data with 135 400 frames, when the boundaries between news stories are detected, the accuracy rate 85.8% and the recall rate 97.5% are obtained. The experimental results show the approach is valid and robust.

Loop Closure Detection of Visual SLAM Based on Point and Line Features

For traditional loop closure detection algorithm,only using the vectorization of point features to build visual dictionary is likely to cause perceptual ambiguity.In addition,when scene lacks texture information,the number of point features extracted from it will be small and cannot describe the image effectively.Therefore,this paper proposes a loop closure detection algorithm which combines point and line features.To better recognize scenes with hybrid features,the building process of traditional dictionary tree is improved in the paper.The features with different flag bits were clustered separately to construct a mixed dictionary tree and word vectors that can represent the hybrid features,which can better describe structure and texture information of scene.To ensure that the similarity score between images is more reasonable,different similarity coefficients were set in different scenes,and the candidate frame with the highest similarity score was selected as the candidate closed loop.Experiments show that the point line comprehensive feature was superior to the single feature in the structured scene and the strong texture scene,the recall rate of the proposed algorithm was higher than the state of the art methods when the accuracy is 100%,and the algorithm can be applied to more diverse environments.

Robust Visual Tracking with Hierarchical Deep Features Weighted Fusion

To solve the problem of low robustness of trackers under significant appearance changes in complex background,a novel moving target tracking method based on hierarchical deep features weighted fusion and correlation filter is proposed.Firstly,multi-layer features are extracted by a deep model pre-trained on massive object recognition datasets.The linearly separable features of Relu3-1,Relu4-1 and Relu5-4 layers from VGG-Net-19 are especially suitable for target tracking.Then,correlation filters over hierarchical convolutional features are learned to generate their correlation response maps.Finally,a novel approach of weight adjustment is presented to fuse response maps.The maximum value of the final response map is just the location of the target.Extensive experiments on the object tracking benchmark datasets demonstrate the high robustness and recognition precision compared with several state-of-the-art trackers under the different conditions.

Real-Time Visual Tracking with Compact Shape and Color Feature

The colour feature is often used in the object tracking.The tracking methods extract the colour features of the object and the background,and distinguish them by a classifier.However,these existing methods simply use the colour information of the target pixels and do not consider the shape feature of the target,so that the description capability of the feature is weak.Moreover,incorporating shape information often leads to large feature dimension,which is not conducive to real-time object tracking.Recently,the emergence of visual tracking methods based on deep learning has also greatly increased the demand for computing resources of the algorithm.In this paper,we propose a real-time visual tracking method with compact shape and colour feature,which forms low dimensional compact shape and colour feature by fusing the shape and colour characteristics of the candidate object region,and reduces the dimensionality of the combined feature through the Hash function.The structural classification function is trained and updated online with dynamic data flow for adapting to the new frames.Further,the classification and prediction of the object are carried out with structured classification function.The experimental results demonstrate that the proposed tracker performs superiorly against several state-of-the-art algorithms on the challenging benchmark dataset OTB-100 and OTB-13.

Filter and Embedded Feature Selection Methods to Meet Big Data Visualization Challenges

This study focuses on meeting the challenges of big data visualization by using of data reduction methods based the feature selection methods.To reduce the volume of big data and minimize model training time(Tt)while maintaining data quality.We contributed to meeting the challenges of big data visualization using the embedded method based“Select from model(SFM)”method by using“Random forest Importance algorithm(RFI)”and comparing it with the filter method by using“Select percentile(SP)”method based chi square“Chi2”tool for selecting the most important features,which are then fed into a classification process using the logistic regression(LR)algorithm and the k-nearest neighbor(KNN)algorithm.Thus,the classification accuracy(AC)performance of LRis also compared to theKNN approach in python on eight data sets to see which method produces the best rating when feature selection methods are applied.Consequently,the study concluded that the feature selection methods have a significant impact on the analysis and visualization of the data after removing the repetitive data and the data that do not affect the goal.After making several comparisons,the study suggests(SFMLR)using SFM based on RFI algorithm for feature selection,with LR algorithm for data classify.The proposal proved its efficacy by comparing its results with recent literature.

Video Concept Detection Based on Multiple Features and Classifiers Fusion

The rapid growth of multimedia content necessitates powerful technologies to filter, classify, index and retrieve video documents more efficiently. However, the essential bottleneck of image and video analysis is the problem of semantic gap that low level features extracted by computers always fail to coincide with high-level concepts interpreted by humans. In this paper, we present a generic scheme for the detection video semantic concepts based on multiple visual features machine learning. Various global and local low-level visual features are systelrtically investigated, and kernelbased learning method equips the concept detection system to explore the potential of these features. Then we combine the different features and sub-systen on both classifier-level and kernel-level fusion that contribute to a more robust system Our proposed system is tested on the TRECVID dataset. The resulted Mean Average Precision （MAP） score is rmch better than the benchmark perforrmnce, which proves that our concepts detection engine develops a generic model and perforrrs well on both object and scene type concepts.

Hard exudates referral system in eye fundus utilizing speeded up robust features

In the paper a referral system to assist the medical experts in the screening/referral of diabetic retinopathy is suggested. The system has been developed by a sequential use of different existing mathematical techniques. These techniques involve speeded up robust features（SURF）, K-means clustering and visual dictionaries（VD）. Three databases are mixed to test the working of the system when the sources are dissimilar. When experiments were performed an area under the curve（AUC） of 0.9343 was attained. The results acquired from the system are promising.

On Electromagnetic Scattering Features of Geological Radar Targets

The electromagnetic scattering principles of geological radar targets and various influential factors were discussed, and the importance of researching into the electromagnetic scattering features of the targets to the actual prospecting task was pointed out.

An Adaptive Padding Correlation Filter With Group Feature Fusion for Robust Visual Tracking

In recent visual tracking research,correlation filter(CF)based trackers become popular because of their high speed and considerable accuracy.Previous methods mainly work on the extension of features and the solution of the boundary effect to learn a better correlation filter.However,the related studies are insufficient.By exploring the potential of trackers in these two aspects,a novel adaptive padding correlation filter(APCF)with feature group fusion is proposed for robust visual tracking in this paper based on the popular context-aware tracking framework.In the tracker,three feature groups are fused by use of the weighted sum of the normalized response maps,to alleviate the risk of drift caused by the extreme change of single feature.Moreover,to improve the adaptive ability of padding for the filter training of different object shapes,the best padding is selected from the preset pool according to tracking precision over the whole video,where tracking precision is predicted according to the prediction model trained by use of the sequence features of the first several frames.The sequence features include three traditional features and eight newly constructed features.Extensive experiments demonstrate that the proposed tracker is superior to most state-of-the-art correlation filter based trackers and has a stable improvement compared to the basic trackers.

Autonomous map query:robust visual localization in urban environments using Multilayer Feature Graph

When a vehicle travels in urban areas, onboard global positioning system （GPS） signals may be obstructed by high-rise buildings and thereby cannot provide accurate positions. It is proposed to perform localization by registering ground images to a 2D building boundary map which is generated from aerial images. Multilayer feature graphs （MFG） is employed to model building facades from the ground images. MFG was reported in the previous work to facilitate the robot scene understand- ing in urhan areas. By constructing MFG, the 2D/3D positions of features can be obtained, inclu- cling line segments, ideal lines, and all primary vertical planes. Finally, a voting-based feature weighted localization method is developed based on MFGs and the 2D building boundary map. The proposed method has been implemented and validated in physical experiments. In the proposed ex- periments, the algorithm has achieved an overall localization accuracy of 2.2m, which is better than commercial GPS working in open environments.

Motion estimation based feature selection for visual SLAM

Feature selection is always an important issue in the visual SLAM （simultaneous location and mapping） literature. Considering that the location estimation can be improved by tracking features with larger value of visible time, a new feature selection method based on motion estimation is proposed. First, a k-step iteration algorithm is presented for visible time estimation using an affme motion model; then a delayed feature detection method is introduced for efficiently detecting features with the maximum visible time. As a means of validation for the proposed method, both simulation and real data experiments are carded out. Results show that the proposed method can improve both the estimation performance and the computational performance compared with the existing random feature selection method.

On Lemon Defect Recognition with Visual Feature Extraction and Transfers Learning

Applying machine learning to lemon defect recognition can improve the efficiency of lemon quality detection. This paper proposes a deep learning-based classification method with visual feature extraction and transfer learning to recognize defect lemons (i.e., green and mold defects). First, the data enhancement and brightness compensation techniques are used for data prepossessing. The visual feature extraction is used to quantify the defects and determine the feature variables as the bandit basis for classification. Then we construct a convolutional neural network with an embedded Visual Geometry Group 16 based (VGG16-based) network using transfer learning. The proposed model is compared with many benchmark models such as K-nearest Neighbor (KNN) and Support Vector Machine (SVM). Results show that the proposed model achieves the highest accuracy (95.44%) in the testing data set. The research provides a new solution for lemon defect recognition.

Video-Based Deception Detection with Non-Contact Heart Rate Monitoring and Multi-Modal Feature Selection

Deception detection plays a crucial role in criminal investigation.Videos contain a wealth of information regarding apparent and physiological changes in individuals,and thus can serve as an effective means of deception detection.In this paper,we investigate video-based deception detection considering both apparent visual features such as eye gaze,head pose and facial action unit(AU),and non-contact heart rate detected by remote photoplethysmography(rPPG)technique.Multiple wrapper-based feature selection methods combined with the K-nearest neighbor(KNN)and support vector machine(SVM)classifiers are employed to screen the most effective features for deception detection.We evaluate the performance of the proposed method on both a self-collected physiological-assisted visual deception detection(PV3D)dataset and a public bag-oflies(BOL)dataset.Experimental results demonstrate that the SVM classifier with symbiotic organisms search(SOS)feature selection yields the best overall performance,with an area under the curve(AUC)of 83.27%and accuracy(ACC)of 83.33%for PV3D,and an AUC of 71.18%and ACC of 70.33%for BOL.This demonstrates the stability and effectiveness of the proposed method in video-based deception detection tasks.

Improving VQA via Dual-Level Feature Embedding Network

Visual Question Answering(VQA)has sparked widespread interest as a crucial task in integrating vision and language.VQA primarily uses attention mechanisms to effectively answer questions to associate relevant visual regions with input questions.The detection-based features extracted by the object detection network aim to acquire the visual attention distribution on a predetermined detection frame and provide object-level insights to answer questions about foreground objects more effectively.However,it cannot answer the question about the background forms without detection boxes due to the lack of fine-grained details,which is the advantage of grid-based features.In this paper,we propose a Dual-Level Feature Embedding(DLFE)network,which effectively integrates grid-based and detection-based image features in a unified architecture to realize the complementary advantages of both features.Specifically,in DLFE,In DLFE,firstly,a novel Dual-Level Self-Attention(DLSA)modular is proposed to mine the intrinsic properties of the two features,where Positional Relation Attention(PRA)is designed to model the position information.Then,we propose a Feature Fusion Attention(FFA)to address the semantic noise caused by the fusion of two features and construct an alignment graph to enhance and align the grid and detection features.Finally,we use co-attention to learn the interactive features of the image and question and answer questions more accurately.Our method has significantly improved compared to the baseline,increasing accuracy from 66.01%to 70.63%on the test-std dataset of VQA 1.0 and from 66.24%to 70.91%for the test-std dataset of VQA 2.0.

A Visual Indoor Localization Method Based on Efficient Image Retrieval

The task of indoor visual localization, utilizing camera visual information for user pose calculation, was a core component of Augmented Reality (AR) and Simultaneous Localization and Mapping (SLAM). Existing indoor localization technologies generally used scene-specific 3D representations or were trained on specific datasets, making it challenging to balance accuracy and cost when applied to new scenes. Addressing this issue, this paper proposed a universal indoor visual localization method based on efficient image retrieval. Initially, a Multi-Layer Perceptron (MLP) was employed to aggregate features from intermediate layers of a convolutional neural network, obtaining a global representation of the image. This approach ensured accurate and rapid retrieval of reference images. Subsequently, a new mechanism using Random Sample Consensus (RANSAC) was designed to resolve relative pose ambiguity caused by the essential matrix decomposition based on the five-point method. Finally, the absolute pose of the queried user image was computed, thereby achieving indoor user pose estimation. The proposed indoor localization method was characterized by its simplicity, flexibility, and excellent cross-scene generalization. Experimental results demonstrated a positioning error of 0.09 m and 2.14° on the 7Scenes dataset, and 0.15 m and 6.37° on the 12Scenes dataset. These results convincingly illustrated the outstanding performance of the proposed indoor localization method.

基于注意力机制和能量函数的动作识别算法

针对零样本动作识别(ZSAR)算法的框架缺乏结构性指导的问题,以基于能量的模型(EBM)指导框架设计,提出基于注意力机制和能量函数的动作识别算法(ARAAE)。首先,为了得到EBM的输入,设计了光流加3D卷积(C3D)架构的组合以提取视觉特征,从而达到空间去冗余的效果;其次,将视觉Transformer(ViT)用于视觉特征的提取以减少时间冗余,同时利用ViT配合光流加C3D架构的组合以减少空间冗余,从而获得非冗余视觉空间;最后,为度量视觉空间和语义空间的相关性,实现能量评分评估机制,设计联合损失函数来进行优化实验。采用6个经典ZSAR算法及近年文献里的算法在两个数据集HMDB51和UCF101进行实验的结果表明:相较于CAGE(Coupling Adversarial Graph Embedding)、Bi-dir GAN(Bi-directional Generative Adversarial Network)和ETSAN(Energy-based Temporal Summarized Attentive Network)等算法,在平均分组的HMDB51数据集上,ARAAE平均识别准确率提升至(22.1±1.8)%,均明显优于对比算法;在平均分组的UCF101数据集上,ARAAE的平均识别准确率提升至(22.4±1.6)%,略优于对比算法;在以81/20为分割方式的UCF101数据集上,ARAAE的平均识别准确率提升至(40.2±2.6)%,均大于对比算法。可见,ARAAE在ZSAR中能有效提高识别性能。

食物饱腹感研究及其测试技术进展

饱腹感,即为进食后满足感和不再渴望进食的状态,在维护正常食欲、控制体质量、预防肥胖及改善代谢性疾病方面具有重要意义。文章系统分析影响饱腹感的多种因素,包括食物成分、感官特征、能量密度、心理因素及饮食行为等,并阐述这些因素对饱腹感的作用机制。同时,重点介绍模拟视觉量表、胃肠道激素检测及闪烁扫描等新兴技术在饱腹感测试中的应用,展示这些技术在评估饱腹感方面的科学性和有效性。此外,文章还讨论饱腹感及其测试技术的研究意义与未来展望,旨在为饱腹感相关机制的深入研究、饱腹感测试技术的创新以及高饱腹感食品的开发提供理论支持和实践参考。