Visual Object Tracking and Servoing Control of a Nano-Scale Quadrotor:System,Algorithms,and Experiments

There are two main trends in the development of unmanned aerial vehicle(UAV)technologies:miniaturization and intellectualization,in which realizing object tracking capabilities for a nano-scale UAV is one of the most challenging problems.In this paper,we present a visual object tracking and servoing control system utilizing a tailor-made 38 g nano-scale quadrotor.A lightweight visual module is integrated to enable object tracking capabilities,and a micro positioning deck is mounted to provide accurate pose estimation.In order to be robust against object appearance variations,a novel object tracking algorithm,denoted by RMCTer,is proposed,which integrates a powerful short-term tracking module and an efficient long-term processing module.In particular,the long-term processing module can provide additional object information and modify the short-term tracking model in a timely manner.Furthermore,a positionbased visual servoing control method is proposed for the quadrotor,where an adaptive tracking controller is designed by leveraging backstepping and adaptive techniques.Stable and accurate object tracking is achieved even under disturbances.Experimental results are presented to demonstrate the high accuracy and stability of the whole tracking system.

A Visual Attention Model for Robot Object Tracking

Inspired by human behaviors, a robot object tracking model is proposed on the basis of visual attention mechanism, which is fit for the theory of topological perception. The model integrates the image-driven, bottom-up attention and the object-driven, top-down attention, whereas the previous attention model has mostly focused on either the bottom-up or top-down attention. By the bottom-up component, the whole scene is segmented into the ground region and the salient regions. Guided by top-down strategy which is achieved by a topological graph, the object regions are separated from the salient regions. The salient regions except the object regions are the barrier regions. In order to estimate the model, a mobile robot platform is developed, on which some experiments are implemented. The experimental results indicate that processing an image with a resolution of 752 × 480 pixels takes less than 200 ms and the object regions are unabridged. The analysis obtained by comparing the proposed model with the existing model demonstrates that the proposed model has some advantages in robot object tracking in terms of speed and efficiency.

Effective method for tracking multiple objects in real-time visual surveillance systems

An object model-based tracking method is useful for tracking multiple objects, but the main difficulties are modeling objects reliably and tracking objects via models in successive frames. An effective tracking method using the object models is proposed to track multiple objects in a real-time visual surveillance system. Firstly, for detecting objects, an adaptive kernel density estimation method is utilized, which uses an adaptive bandwidth and features combining colour and gradient. Secondly, some models of objects are built for describing motion, shape and colour features. Then, a matching matrix is formed to analyze tracking situations. If objects are tracked under occlusions, the optimal ＂visual＂ object is found to represent the occluded object, and the posterior probability of pixel is used to determine which pixel is utilized for updating object models. Extensive experiments show that this method improves the accuracy and validity of tracking objects even under occlusions and is used in real-time visual surveillance systems.

Enhancing the Robustness of Visual Object Tracking via Style Transfer

The performance and accuracy of computer vision systems are affected by noise in different forms.Although numerous solutions and algorithms have been presented for dealing with every type of noise,a comprehensive technique that can cover all the diverse noises and mitigate their damaging effects on the performance and precision of various systems is still missing.In this paper,we have focused on the stability and robustness of one computer vision branch(i.e.,visual object tracking).We have demonstrated that,without imposing a heavy computational load on a model or changing its algorithms,the drop in the performance and accuracy of a system when it is exposed to an unseen noise-laden test dataset can be prevented by simply applying the style transfer technique on the train dataset and training the model with a combination of these and the original untrained data.To verify our proposed approach,it is applied on a generic object tracker by using regression networks.This method’s validity is confirmed by testing it on an exclusive benchmark comprising 50 image sequences,with each sequence containing 15 types of noise at five different intensity levels.The OPE curves obtained show a 40%increase in the robustness of the proposed object tracker against noise,compared to the other trackers considered.

3D Object Visual Tracking for the 220 kV/330 kV High-Voltage Live-Line Insulator Cleaning Robot

The 3D object visual tracking problem is studied for the robot vision system of the 220kV/330kV high-voltage live-line insulator cleaning robot. The SUSAN Edge based Scale Invariant Feature (SESIF) algorithm based 3D objects visual tracking is achieved in three stages: the first frame stage,tracking stage,and recovering stage. An SESIF based objects recognition algorithm is proposed to find initial location at both the first frame stage and recovering stage. An SESIF and Lie group based visual tracking algorithm is used to track 3D object. Experiments verify the algorithm's robustness. This algorithm will be used in the second generation of the 220kV/330kV high-voltage live-line insulator cleaning robot.

Depth-Aided Tracking Multiple Objects under Occlusion

In this paper, we have presented a novel tracking method aiming at detecting objects and maintaining their la-bel/identification over the time. The key factors of this method are to use depth information and different strategies to track objects under various occlusion scenarios. The foreground objects are detected and refined by background subtraction and shadow cancellation. The occlusion detection is based on information of foreground blobs in successive frames. The occlusion regions are projected to the projection plane XZ to analysis occlusion situation. According to the occlusion analysis results, different objects’ corresponding strategies are introduced to track objects under various occlusion scenarios including tracking occluded objects in similar depth layer and in different depth layers. The experimental results show that our proposed method can track the moving objects under the most typical and challenging occlusion scenarios.

SMSTracker:A Self-Calibration Multi-Head Self-Attention Transformer for Visual Object Tracking

Visual object tracking plays a crucial role in computer vision.In recent years,researchers have proposed various methods to achieve high-performance object tracking.Among these,methods based on Transformers have become a research hotspot due to their ability to globally model and contextualize information.However,current Transformer-based object tracking methods still face challenges such as low tracking accuracy and the presence of redundant feature information.In this paper,we introduce self-calibration multi-head self-attention Transformer(SMSTracker)as a solution to these challenges.It employs a hybrid tensor decomposition self-organizing multihead self-attention transformermechanism,which not only compresses and accelerates Transformer operations but also significantly reduces redundant data,thereby enhancing the accuracy and efficiency of tracking.Additionally,we introduce a self-calibration attention fusion block to resolve common issues of attention ambiguities and inconsistencies found in traditional trackingmethods,ensuring the stability and reliability of tracking performance across various scenarios.By integrating a hybrid tensor decomposition approach with a self-organizingmulti-head self-attentive transformer mechanism,SMSTracker enhances the efficiency and accuracy of the tracking process.Experimental results show that SMSTracker achieves competitive performance in visual object tracking,promising more robust and efficient tracking systems,demonstrating its potential to providemore robust and efficient tracking solutions in real-world applications.

动态场景下基于3D多目标追踪的实时视觉SLAM方法研究

近年来一些解决动态场景下的SLAM技术被提出,其中SLAM与MOT结合的技术路线不仅可解决动态场景问题,还可以提高系统对周围场景的理解,获得了更大关注。本文介绍了一种高效的实时在线视觉SLAMMOT融合系统,以双目视觉或RGBD作为输入,只须借助2D目标检测网络,便能高效、准确、鲁棒地跟踪相机以及动态目标的位姿,并生成稀疏点云地图。为提高多动态目标追踪的精度与准确度,引入了级联匹配与IOU匹配结合的策略;利用阿克曼转向模型来简化追踪目标的运动,减少求解动态目标位姿所需匹配点的数量;利用因子图将相机与动态目标的追踪结果进行联合优化,同时提高相机、追踪目标的位姿和地图点的精度。最后在KITTI跟踪数据集上与其他方法进行比较。结果表明,在满足实时性要求的前提下,该方法仍能准确地追踪相机以及动态目标位姿。

基于视觉跟踪实时引导的传送带跟随方法

针对滚筒输送线场景下的机器人随线质检问题,提出一种基于视觉跟踪实时引导的传送带跟随方法,即通过定位相机跟踪待测目标的运动,实时引导机器人调整动作跟随运动目标并检测。首先,通过顶部固定相机获取包含待测目标的实时视频,构建多目标跟踪(MOT)算法跟踪待测目标的位置与旋转角;然后,通过图像信息解算目标的6D位姿,进一步解算对应的机器人工作点坐标;最后,根据实时工作点坐标在线调整机器人动作,保持机器人末端与工作点重合,实现跟随检测。实验结果表明,所提方法可实现滚筒输送线场景下的机器人随线检测;相较于现有的基于目标检测的跟随方法,不存在跟随偏差累积问题。

基于计算机视觉的电力作业人员行为分析研究现状与展望

电力作业人员的有效监管是保障电力安全生产的基础。该文对电力视频中作业人员的行为识别研究进行了归类总结,涵盖静态行为分析(穿戴分析、动作分析和组合分析)和动态行为分析(复杂动作、时序行为和行为预测等);详细综述了电力作业行为分析中的核心算法模块,包括目标检测、姿态估计和视频跟踪等;论述了电力作业行为识别在算法高效性、鲁棒性、灵活性等方面所面临的应用难点和挑战,并展望了电力作业行为智能监控领域的未来发展方向,特别强调了在软硬件结合、通用大模型、生成式人工智能方面进行技术创新和改进所蕴含的潜在机会。

Multi-modal visual tracking:Review and experimental comparison

Visual object tracking has been drawing increasing attention in recent years,as a fundamental task in computer vision.To extend the range of tracking applications,researchers have been introducing information from multiple modalities to handle specific scenes,with promising research prospects for emerging methods and benchmarks.To provide a thorough review of multi-modal tracking,different aspects of multi-modal tracking algorithms are summarized under a unified taxonomy,with specific focus on visibledepth(RGB-D)and visible-thermal(RGB-T)tracking.Subsequently,a detailed description of the related benchmarks and challenges is provided.Extensive experiments were conducted to analyze the effectiveness of trackers on five datasets:PTB,VOT19-RGBD,GTOT,RGBT234,and VOT19-RGBT.Finally,various future directions,including model design and dataset construction,are discussed from different perspectives for further research.

Optimal Positioning Strategy for Multi-Camera Zooming Drones

In the context of multiple-target tracking and surveillance applications,this paper investigates the challenge of determining the optimal positioning of a single autonomous aerial vehicle or agent equipped with multiple independently-steerable zooming cameras to effectively monitor a set of targets of interest.Each camera is dedicated to tracking a specific target or cluster of targets.The key innovation of this study,in comparison to existing approaches,lies in incorporating the zooming factor for the onboard cameras into the optimization problem.This enhancement offers greater flexibility during mission execution by allowing the autonomous agent to adjust the focal lengths of the onboard cameras,in exchange for varying real-world distances to the corresponding targets,thereby providing additional degrees of freedom to the optimization problem.The proposed optimization framework aims to strike a balance among various factors,including distance to the targets,verticality of viewpoints,and the required focal length for each camera.The primary focus of this paper is to establish the theoretical groundwork for addressing the non-convex nature of the optimization problem arising from these considerations.To this end,we develop an original convex approximation strategy.The paper also includes simulations of diverse scenarios,featuring varying numbers of onboard tracking cameras and target motion profiles,to validate the effectiveness of the proposed approach.

全局搜索和多实例判别特征的长时跟踪方法

长时目标跟踪相对于短时目标跟踪仍然是一个巨大的挑战.然而现有的长时跟踪算法通常在面对目标频繁出现消失、目标外观发生剧变等挑战中表现不佳.本文提出了一种基于局部搜索模块和全局搜索跟踪模块的全新、鲁棒且实时的长时跟踪框架.局部搜索模块利用TransT短时跟踪器生成一系列候选框,并通过置信度评分确定最佳候选框.针对全局重新检测开发了一个新颖的全局搜索跟踪模块,以Faster R-CNN为基础模型,在RPN阶段与R-CNN阶段引入非局部操作和多级实例特征融合模块,以充分挖掘目标实例级特征.为了改进全局搜索跟踪模块的性能,设计了双模板更新策略来提升跟踪器的鲁棒能力.通过使用不同时间点上更新的模板能够更好地适应目标的变化.根据局部或全局置信度分数判断目标是否存在,并在下一帧中选择局部或全局搜索跟踪策略.同时能够为局部搜索模块估计目标的位置和大小.此外还为全局搜索跟踪器引入了排名损失函数,隐式学习了区域提议与原始查询目标的相似度.通过在多个跟踪数据集上进行大量实验对提出的跟踪框架进行了广泛评估.结果一致表明,本文提出的跟踪框架实现了令人满意的性能.

基于对抗生成网络与关键点视觉追踪模型的导线异物隐患可视化检出方法

导线关键点附着异物会造成短路或漏电,为了改善恶劣气象条件对图像特征维度对异物隐患可视化检出方法的影响,减少误检情况,提出基于对抗生成网络与关键点视觉追踪模型的导线异物隐患可视化检出方法。使用二维Otsu方法分割去雾导线图像,提取导线异物目标区域,依据所得分割图像,采用对抗生成网络实现导线异物隐患可视化检出。实验结果表明:该方法对导线各关键点的追踪位置与实际位置十分接近,最大偏差仅为0.03 m;经过去雾处理可以明显改善导线关键点图像的清晰度和颜色信息;能够准确、完整地提取导线异物目标区域,且边界信息处理较好。

Masked Autoencoders as Single Object Tracking Learners

Significant advancements have beenwitnessed in visual tracking applications leveragingViT in recent years,mainly due to the formidablemodeling capabilities of Vision Transformer(ViT).However,the strong performance of such trackers heavily relies on ViT models pretrained for long periods,limitingmore flexible model designs for tracking tasks.To address this issue,we propose an efficient unsupervised ViT pretraining method for the tracking task based on masked autoencoders,called TrackMAE.During pretraining,we employ two shared-parameter ViTs,serving as the appearance encoder and motion encoder,respectively.The appearance encoder encodes randomly masked image data,while the motion encoder encodes randomly masked pairs of video frames.Subsequently,an appearance decoder and a motion decoder separately reconstruct the original image data and video frame data at the pixel level.In this way,ViT learns to understand both the appearance of images and the motion between video frames simultaneously.Experimental results demonstrate that ViT-Base and ViT-Large models,pretrained with TrackMAE and combined with a simple tracking head,achieve state-of-the-art(SOTA)performance without additional design.Moreover,compared to the currently popular MAE pretraining methods,TrackMAE consumes only 1/5 of the training time,which will facilitate the customization of diverse models for tracking.For instance,we additionally customize a lightweight ViT-XS,which achieves SOTA efficient tracking performance.

深度长时目标跟踪算法综述

在视觉目标跟踪领域,长时跟踪因存在更为复杂的遮挡、相似物干扰和目标消失等具有现实意义的挑战场景,而越来越被研究者所重视。传统长时跟踪算法存在精度低和效率低等问题,已经无法满足如视频监控和自动驾驶等领域对跟踪器性能的应用需求。目前,大量的研究工作通过引入深度神经网络快速推动了长时跟踪技术的发展。为了深入分析深度长时跟踪算法的现状与未来发展,通过对比长短时跟踪数据集及评价指标,初步界定了长时跟踪任务范畴,归纳了长时跟踪任务的需求和难点,并介绍了长时跟踪数据集及评价体系的发展。基于深度长时目标跟踪算法的设计框架,详细描述了框架各组成部分的设计思路。以长时跟踪策略为切入点深入分析了现有研究工作,归纳了不同模型的优缺点及特性。依据对现有研究工作的整理和总结,讨论了该领域面临的挑战,并对未来的发展方向进行了展望。

基于重要特征的视觉目标跟踪可迁移黑盒攻击方法

视频目标跟踪的黑盒攻击方法受到越来越多的关注,目的是评估目标跟踪器的稳健性,进而提升跟踪器的安全性.目前大部分的研究都是基于查询的黑盒攻击,尽管取得较好的攻击效果,但在实际应用中往往不能获取大量的查询以进行攻击.本文提出一种基于迁移的黑盒攻击方法,通过对特征中与跟踪目标高度相关而不受源模型影响的重要特征进行攻击,将其重要程度降低,同时增强不重要的特征以实现具有可迁移性的攻击,即通过反向传播获得的所对应的梯度来体现其特征的重要程度,随后通过梯度得到的加权特征进行攻击.此外,本文使用视频相邻两帧之间相似这一时序信息,提出基于时序感知的特征相似性攻击方法,通过减小相邻帧之间的特征相似度以进行攻击.本文在目前主流的深度学习目标跟踪器上评估了提出的攻击方法,在多个数据集上的实验结果证明了本文方法的有效性及强可迁移性,在OTB数据集中,SiamRPN跟踪模型被攻击后跟踪成功率以及精确度分别下降了71.5%和79.9%.

时间压力下驾驶舱界面视觉搜索绩效研究

为探究不同时间压力下飞机驾驶舱界面搜索特征从而提升任务绩效,通过模拟飞机飞行状态下的任务搜索过程,采用眼动追踪、绩效评估及量表测量等方法来获取被试的视觉搜索轨迹与任务绩效数据并对所获取的多通道数据进行分析.结果表明:在不同时间压力(无、低、高)与不同界面复杂度(简单、复杂)下被试的视觉搜索特征存在显著差异;高时间压力与复杂界面显示都会给被试带来较高的心理认知负荷;时间压力在简单界面显示中存在倒“U”型关系.研究为飞机驾驶舱复杂人机界面设计提供了科学的理论依据.

无人机对地目标跟踪的快速初始化和自适应优化

目标跟踪算法的性能通常和初始跟踪框的质量有关。在无人机对地侦察任务中,由于反应时间有限,操作员通常难以选取精确的初始跟踪框,导致目标跟踪结果较差。针对这一问题,提出一种半自动的跟踪框快速初始化和自适应优化策略,并给出基于视觉显著性和显著图像分割的自适应优化算法样例,在性能提升和运行时间上均具有优势。与优化前相比,在2个数据集上的跟踪成功率最高提升0.262、跟踪精度最高提升0.177;在运行时间方面,处理200像素×200像素的图像切片时,理论并行速度可达10帧/s。提出的跟踪框初始化和优化策略,结合了人的主观选择和视觉认知,可以有效解决无人机对地侦察任务中目标难以锁定的问题,并具备在嵌入式设备中的可移植性。

基于动态物体跟踪的语义SLAM

针对传统视觉SLAM在动态场景下容易出现特征匹配错误从而导致定位精度下降的问题,提出了一种基于动态物体跟踪的语义SLAM算法。基于经典的视觉SLAM框架,提取动态物体进行帧间跟踪,并利用动态物体的位姿信息来辅助相机自身的定位。首先,算法在数据预处理中使用YOLACT、RAFT以及SC-Depth网络,分别提取图像中的语义掩膜、光流向量以及像素深度值。其次,视觉前端模块根据所提信息,通过语义分割掩膜、运动一致性检验以及遮挡点检验算法计算概率图以平滑区分场景中的动态特征与静态特征。然后,后端中的捆集调整模块融合了物体运动的多特征约束以提高算法在动态场景中的位姿估计性能。最后,在KITTI和OMD数据集的动态场景中进行对比验证。实验表明,所提算法能够准确地跟踪动态物体,在室内外动态场景中具备鲁棒、良好的定位性能。