SMSTracker:A Self-Calibration Multi-Head Self-Attention Transformer for Visual Object Tracking

Visual object tracking plays a crucial role in computer vision.In recent years,researchers have proposed various methods to achieve high-performance object tracking.Among these,methods based on Transformers have become a research hotspot due to their ability to globally model and contextualize information.However,current Transformer-based object tracking methods still face challenges such as low tracking accuracy and the presence of redundant feature information.In this paper,we introduce self-calibration multi-head self-attention Transformer(SMSTracker)as a solution to these challenges.It employs a hybrid tensor decomposition self-organizing multihead self-attention transformermechanism,which not only compresses and accelerates Transformer operations but also significantly reduces redundant data,thereby enhancing the accuracy and efficiency of tracking.Additionally,we introduce a self-calibration attention fusion block to resolve common issues of attention ambiguities and inconsistencies found in traditional trackingmethods,ensuring the stability and reliability of tracking performance across various scenarios.By integrating a hybrid tensor decomposition approach with a self-organizingmulti-head self-attentive transformer mechanism,SMSTracker enhances the efficiency and accuracy of the tracking process.Experimental results show that SMSTracker achieves competitive performance in visual object tracking,promising more robust and efficient tracking systems,demonstrating its potential to providemore robust and efficient tracking solutions in real-world applications.

Visual Quality Analysis of Urban Park Landscapes Based on Eye Tracking: A Case Study of Nanjing Xiaohong Stone Carving Park

The landscape quality of urban parks is an important aspect of tourists'landscape experience.A full understanding of their relationship is beneficial to the park design,construction and management.In this paper,Xiaohong Stone Carving Park in Nanjing was selected as a case study.Firstly,the subjective landscape visual quality evaluation of tourists was obtained through the scenic beauty evaluation,and the objective landscape visual preference of users was obtained through eye movement experiment and analysis.Secondly,the values of various park landscape elements were measured respectively.And then the correlation between the subjective and objective evaluation and the values of park landscape elements was analyzed.The results showed that:(1)The visual preference of plant cluster landscape with beautiful shape,distinct layers and significant color contrast is high;(2)The preference for waterfront scenes with significant terrain differences,rich background vegetation levels,and clear and vivid water side reflections is high;(3)The designed artificial structures often become the focus of attention in the whole scene,which can improve the beauty of the scene;(4)The overall landscape composition is very important for the beauty of the scene.Finally,through the comprehensive analysis with the design expectation,the optimization strategy was proposed.

Masked Autoencoders as Single Object Tracking Learners

Significant advancements have beenwitnessed in visual tracking applications leveragingViT in recent years,mainly due to the formidablemodeling capabilities of Vision Transformer(ViT).However,the strong performance of such trackers heavily relies on ViT models pretrained for long periods,limitingmore flexible model designs for tracking tasks.To address this issue,we propose an efficient unsupervised ViT pretraining method for the tracking task based on masked autoencoders,called TrackMAE.During pretraining,we employ two shared-parameter ViTs,serving as the appearance encoder and motion encoder,respectively.The appearance encoder encodes randomly masked image data,while the motion encoder encodes randomly masked pairs of video frames.Subsequently,an appearance decoder and a motion decoder separately reconstruct the original image data and video frame data at the pixel level.In this way,ViT learns to understand both the appearance of images and the motion between video frames simultaneously.Experimental results demonstrate that ViT-Base and ViT-Large models,pretrained with TrackMAE and combined with a simple tracking head,achieve state-of-the-art(SOTA)performance without additional design.Moreover,compared to the currently popular MAE pretraining methods,TrackMAE consumes only 1/5 of the training time,which will facilitate the customization of diverse models for tracking.For instance,we additionally customize a lightweight ViT-XS,which achieves SOTA efficient tracking performance.

Dynamic Visible Light Positioning Based on Enhanced Visual Target Tracking

In visible light positioning systems,some scholars have proposed target tracking algorithms to balance the relationship among positioning accuracy,real-time performance,and robustness.However,there are still two problems:(1)When the captured LED disappears and the uncertain LED reappears,existing tracking algorithms may recognize the landmark in error;(2)The receiver is not always able to achieve positioning under various moving statuses.In this paper,we propose an enhanced visual target tracking algorithm to solve the above problems.First,we design the lightweight recognition/demodulation mechanism,which combines Kalman filtering with simple image preprocessing to quickly track and accurately demodulate the landmark.Then,we use the Gaussian mixture model and the LED color feature to enable the system to achieve positioning,when the receiver is under various moving statuses.Experimental results show that our system can achieve high-precision dynamic positioning and improve the system’s comprehensive performance.

Long-term Visual Tracking: Review and Experimental Comparison

As a fundamental task in computer vision,visual object tracking has received much attention in recent years.Most studies focus on short-term visual tracking which addresses shorter videos and always-visible targets.However,long-term visual tracking is much closer to practical applications with more complicated challenges.There exists a longer duration such as minute-level or even hour-level in the long-term tracking task,and the task also needs to handle more frequent target disappearance and reappearance.In this paper,we provide a thorough review of long-term tracking,summarizing long-term tracking algorithms from two perspectives:framework architectures and utilization of intermediate tracking results.Then we provide a detailed description of existing benchmarks and corresponding evaluation protocols.Furthermore,we conduct extensive experiments and analyse the performance of trackers on six benchmarks:VOTLT2018,VOTLT2019(2020/2021),OxUvA,LaSOT,TLP and the long-term subset of VTUAV-V.Finally,we discuss the future prospects from multiple perspectives,including algorithm design and benchmark construction.To our knowledge,this is the first comprehensive survey for long-term visual object tracking.The relevant content is available at https://github.com/wangdongdut/Long-term-Visual-Tracking.

3-D visual tracking based on CMAC neural network and Kalman filter

In this paper, the Kalman filter is used to predict image feature positionaround which an image-processing window is then established to diminish feature-searching area andto heighten the image-processing speed. According to the fundamentals of image-based visual servoing(IBVS), the cerebellar model articulation controller (CMAC) neural network is inserted into thevisual servo control loop to implement the nonlinear mapping from the error signal in the imagespace to the control signal in the input space instead of the iterative adjustment and complicatedinverse solution of the image Jacobian. Simulation results show that the feature point can bepredicted efficiently using the Kalman filter and on-line supervised learning can be realized usingCMAC neural network; end-effector can track the target object very well.

OPTIMIZED MEANSHIFT TARGET REFERENCE MODEL BASED ON IMPROVED PIXEL WEIGHTING IN VISUAL TRACKING

The generic Meanshift is susceptible to interference of background pixels with the target pixels in the kernel of the reference model, which compromises the tracking performance. In this paper, we enhance the target color feature by attenuating the background color within the kernel through enlarging the pixel weightings which map to the pixels on the target. This way, the background pixel interference is largely suppressed in the color histogram in the course of constructing the target reference model. In addition, the proposed method also reduces the number of Meanshift iterations, which speeds up the algorithmic convergence. The two tests validate the proposed approach with improved tracking robustness on real-world video sequences.

Visual Object Tracking and Servoing Control of a Nano-Scale Quadrotor:System,Algorithms,and Experiments

There are two main trends in the development of unmanned aerial vehicle(UAV)technologies:miniaturization and intellectualization,in which realizing object tracking capabilities for a nano-scale UAV is one of the most challenging problems.In this paper,we present a visual object tracking and servoing control system utilizing a tailor-made 38 g nano-scale quadrotor.A lightweight visual module is integrated to enable object tracking capabilities,and a micro positioning deck is mounted to provide accurate pose estimation.In order to be robust against object appearance variations,a novel object tracking algorithm,denoted by RMCTer,is proposed,which integrates a powerful short-term tracking module and an efficient long-term processing module.In particular,the long-term processing module can provide additional object information and modify the short-term tracking model in a timely manner.Furthermore,a positionbased visual servoing control method is proposed for the quadrotor,where an adaptive tracking controller is designed by leveraging backstepping and adaptive techniques.Stable and accurate object tracking is achieved even under disturbances.Experimental results are presented to demonstrate the high accuracy and stability of the whole tracking system.

Seam Tracking and Visual Control for Robotic Arc Welding Based on Structured Light Stereovision

A real-time arc welding robot visual control system based on a local network with a multi-level hierarchy is developed in this paper. It consists of an intelligence and human-machine interface level, a motion planning level, a motion control level and a servo control level. The last three levels form a local real-time open robot controller, which realizes motion planning and motion control of a robot. A camera calibration method based on the relative movement of the end-effector connected to a robot is proposed and a method for tracking weld seam based on the structured light stereovision is provided. Combining the parameters of the cameras and laser plane, three groups of position values in Cartesian space are obtained for each feature point in a stripe projected on the weld seam. The accurate three-dimensional position of the edge points in the weld seam can be calculated from the obtained parameters with an information fusion algorithm. By calculating the weld seam parameter from position and image data, the movement parameters of the robot used for tracking can be determined. A swing welding experiment of type V groove weld is successfully conducted, the results of which show that the system has high resolution seam tracking in real-time, and works stably and efficiently.

A Visual Attention Model for Robot Object Tracking

Inspired by human behaviors, a robot object tracking model is proposed on the basis of visual attention mechanism, which is fit for the theory of topological perception. The model integrates the image-driven, bottom-up attention and the object-driven, top-down attention, whereas the previous attention model has mostly focused on either the bottom-up or top-down attention. By the bottom-up component, the whole scene is segmented into the ground region and the salient regions. Guided by top-down strategy which is achieved by a topological graph, the object regions are separated from the salient regions. The salient regions except the object regions are the barrier regions. In order to estimate the model, a mobile robot platform is developed, on which some experiments are implemented. The experimental results indicate that processing an image with a resolution of 752 × 480 pixels takes less than 200 ms and the object regions are unabridged. The analysis obtained by comparing the proposed model with the existing model demonstrates that the proposed model has some advantages in robot object tracking in terms of speed and efficiency.

Effective method for tracking multiple objects in real-time visual surveillance systems

An object model-based tracking method is useful for tracking multiple objects, but the main difficulties are modeling objects reliably and tracking objects via models in successive frames. An effective tracking method using the object models is proposed to track multiple objects in a real-time visual surveillance system. Firstly, for detecting objects, an adaptive kernel density estimation method is utilized, which uses an adaptive bandwidth and features combining colour and gradient. Secondly, some models of objects are built for describing motion, shape and colour features. Then, a matching matrix is formed to analyze tracking situations. If objects are tracked under occlusions, the optimal ＂visual＂ object is found to represent the occluded object, and the posterior probability of pixel is used to determine which pixel is utilized for updating object models. Extensive experiments show that this method improves the accuracy and validity of tracking objects even under occlusions and is used in real-time visual surveillance systems.

Sensor planning method for visual tracking in 3D camera networks

Most sensors or cameras discussed in the sensor network community are usually 3D homogeneous, even though their2 D coverage areas in the ground plane are heterogeneous. Meanwhile, observed objects of camera networks are usually simplified as 2D points in previous literature. However in actual application scenes, not only cameras are always heterogeneous with different height and action radiuses, but also the observed objects are with 3D features（i.e., height）. This paper presents a sensor planning formulation addressing the efficiency enhancement of visual tracking in 3D heterogeneous camera networks that track and detect people traversing a region. The problem of sensor planning consists of three issues：（i） how to model the 3D heterogeneous cameras;（ii） how to rank the visibility, which ensures that the object of interest is visible in a camera＇s field of view;（iii） how to reconfigure the 3D viewing orientations of the cameras. This paper studies the geometric properties of 3D heterogeneous camera networks and addresses an evaluation formulation to rank the visibility of observed objects. Then a sensor planning method is proposed to improve the efficiency of visual tracking. Finally, the numerical results show that the proposed method can improve the tracking performance of the system compared to the conventional strategies.

A creative design of robotic visual tracking system in tailed welded blanks based on TRIZ

According to the main tools of TRIZ, the theory of inventive problem solving, a new flowchart of the product conceptual design process to solve contradiction in TRIZ is proposed. In order to realize autonomous moving and automatic weld seam tracking for welding robot in Tailed Welded Blanks, a creative design of robotic visual tracking system bused on CMOS has been developed by using the flowchart. The new system is not only used to inspect the workpiece ahead of a welding torch and measure the joint orientation and lateral deviation caused by curvature or discontinuity in the joint part, but also to record and measure the image size of the weld pool. Moreover, the hardware and software components are discussed in brief.

Robust visual tracking algorithm based on Monte Carlo approach with integrated attributes

To improve the reliability and accuracy of visual tracker,a robust visual tracking algorithm based on multi-cues fusion under Bayesian framework is proposed.The weighed color and texture cues of the object are applied to describe the moving object.An adjustable observation model is incorporated into particle filtering,which utilizes the properties of particle filter for coping with non-linear,non-Gaussian assumption and the ability to predict the position of the moving object in a cluttered environment and two complementary attributes are employed to estimate the matching similarity dynamically in term of the likelihood ratio factors;furthermore tunes the weight values according to the confidence map of the color and texture feature on-line adaptively to reconfigure the optimal observation likelihood model,which ensured attaining the maximum likelihood ratio in the tracking scenario even if in the situations where the object is occluded or illumination,pose and scale are time-variant.The experimental result shows that the algorithm can track a moving object accurately while the reliability of tracking in a challenging case is validated in the experimentation.

Visual tracking based on transfer learning of deep salience information

In this paper,we propose a new visual tracking method in light of salience information and deep learning.Salience detection is used to exploit features with salient information of the image.Complicated representations of image features can be gained by the function of every layer in convolution neural network(CNN).The characteristic of biology vision in attention-based salience is similar to the neuroscience features of convolution neural network.This motivates us to improve the representation ability of CNN with functions of salience detection.We adopt the fully-convolution networks(FCNs)to perform salience detection.We take parts of the network structure to perform salience extraction,which promotes the classification ability of the model.The network we propose shows great performance in tracking with the salient information.Compared with other excellent algorithms,our algorithm can track the target better in the open tracking datasets.We realize the 0.5592 accuracy on visual object tracking 2015(VOT15)dataset.For unmanned aerial vehicle 123(UAV123)dataset,the precision and success rate of our tracker is 0.710 and 0.429.

MULTI-TARGET VISUAL TRACKING AND OCCLUSION DETECTION BY COMBINING BHATTACHARYYA COEFFICIENT AND KALMAN FILTER INNOVATION

This paper introduces an approach for visual tracking of multi-target with occlusion occurrence. Based on the author's previous work in which the Overlap Coefficient (OC) is used to detect the occlusion, in this paper a method of combining Bhattacharyya Coefficient (BC) and Kalman filter innovation term is proposed as the criteria for jointly detecting the occlusion occurrence. Fragmentation of target is introduced in order to closely monitor the occlusion development. In the course of occlusion, the Kalman predictor is applied to determine the location of the occluded target, and the criterion for checking the re-appearance of the occluded target is also presented. The proposed approach is put to test on a standard video sequence, suggesting the satisfactory performance in multi-target tracking.

Hierarchical Template Matching for Robust Visual Tracking with Severe Occlusions

To tackle the problem of severe occlusions in visual tracking, we propose a hierarchical template-matching method based on a layered appearance model. This model integrates holistic- and part-region matching in order to locate an object in a coarse-to-fine manner. Furthermore, in order to reduce ambiguity in object localization, only the discriminative parts of an object＇ s appearance template are chosen for similarity computing with respect to their cornerness measurements. The similarity between parts is computed in a layer-wise manner, and from this, occlusions can be evaluated. When the object is partly occluded, it can be located accurately by matching candidate regions with the appearance template. When it is completely occluded, its location can be predicted from its historical motion information using a Kalman filter. The proposed tracker is tested on several practical image sequences, and the experimental results show that it can consistently provide accurate object location for stable tracking, even for severe occlusions.

Real-Time Visual Tracking with Compact Shape and Color Feature

The colour feature is often used in the object tracking.The tracking methods extract the colour features of the object and the background,and distinguish them by a classifier.However,these existing methods simply use the colour information of the target pixels and do not consider the shape feature of the target,so that the description capability of the feature is weak.Moreover,incorporating shape information often leads to large feature dimension,which is not conducive to real-time object tracking.Recently,the emergence of visual tracking methods based on deep learning has also greatly increased the demand for computing resources of the algorithm.In this paper,we propose a real-time visual tracking method with compact shape and colour feature,which forms low dimensional compact shape and colour feature by fusing the shape and colour characteristics of the candidate object region,and reduces the dimensionality of the combined feature through the Hash function.The structural classification function is trained and updated online with dynamic data flow for adapting to the new frames.Further,the classification and prediction of the object are carried out with structured classification function.The experimental results demonstrate that the proposed tracker performs superiorly against several state-of-the-art algorithms on the challenging benchmark dataset OTB-100 and OTB-13.

In situ nanobubble sizing by visualization particle tracking and image-based dynamic light scattering

A novel method combining visualization particle tracking with image-based dynamic light scattering was developed to achieve the in situ and real-time size measurement of nanobubbles(NBs).First,the in situ size distribution of NBs was visualized by dark-field microscopy.Then,real-time size during the preparation was measured using image-based dynamic light scattering,and the longitudinal size distribution of NBs in the sample cell was obtained in a steady state.Results show that this strategy can provide a detailed and accurate size of bubbles in the whole sample compared with the commercial ZetaSizer Nano equipment.Therefore,the developed method is a real-time and simple technology with excellent accuracy,providing new insights into the accurate measurement of the size distribution of NBs or nanoparticles in solution.

Hybrid Efficient Convolution Operators for Visual Tracking

Visual tracking is a classical computer vision problem with many applications.Efficient convolution operators(ECO)is one of the most outstanding visual tracking algorithms in recent years,it has shown great performance using discriminative correlation filter(DCF)together with HOG,color maps and VGGNet features.Inspired by new deep learning models,this paper propose a hybrid efficient convolution operators integrating fully convolution network(FCN)and residual network(ResNet)for visual tracking,where FCN and ResNet are introduced in our proposed method to segment the objects from backgrounds and extract hierarchical feature maps of objects,respectively.Compared with the traditional VGGNet,our approach has higher accuracy for dealing with the issues of segmentation and image size.The experiments show that our approach would obtain better performance than ECO in terms of precision plot and success rate plot on OTB-2013 and UAV123 datasets.