Deep Learning Based Target Tracking and Classification for Infrared Videos Using Compressive Measurements

Although compressive measurements save data storage and bandwidth usage, they are difficult to be used directly for target tracking and classification without pixel reconstruction. This is because the Gaussian random matrix destroys the target location information in the original video frames. This paper summarizes our research effort on target tracking and classification directly in the compressive measurement domain. We focus on one particular type of compressive measurement using pixel subsampling. That is, original pixels in video frames are randomly subsampled. Even in such a special compressive sensing setting, conventional trackers do not work in a satisfactory manner. We propose a deep learning approach that integrates YOLO (You Only Look Once) and ResNet (residual network) for multiple target tracking and classification. YOLO is used for multiple target tracking and ResNet is for target classification. Extensive experiments using short wave infrared (SWIR), mid-wave infrared (MWIR), and long-wave infrared (LWIR) videos demonstrated the efficacy of the proposed approach even though the training data are very scarce.

Developing selective mining capability for longwall shearers using thermal infrared-based seam tracking

Longwall mining continues to remain the most efficient method for underground coal recovery. A key aspect in achieving safe and productive longwall mining is to ensure that the shearer is always correctly positioned within the coal seam. At present, this machine positioning task is the role of longwall personnel who must simultaneously monitor the longwall coal face and the shearer's cutting drum position to infer the geological trends of the coal seam. This is a labour intensive task which has negative impacts on the consistency and quality of coal production. As a solution to this problem, this paper presents a sensing method to automatically track geological coal seam features on the longwall face, known as marker bands, using thermal infrared imaging. These non-visible marker bands are geological features that link strongly to the horizontal trends present in layered coal seams. Tracking these line-like features allows the generation of a vertical datum that can be used to maintain the shearer in a position for optimal coal extraction. Details on the theory of thermal infrared imaging are given, as well as practical aspects associated with machine-based implementation underground. The feature detection and tracking tasks are given with real measurements to demonstrate the efficacy of the approach. The outcome is important as it represents a new selective mining capability to help address a long-standing limitation in longwall mining operations.

Target Tracking and Classification Using Compressive Measurements of MWIR and LWIR Coded Aperture Cameras

Pixel-wise Code Exposure (PCE) camera is one type of compressive sensing camera that has low power consumption and high compression ratio. Moreover, a PCE camera can control individual pixel exposure time that can enable high dynamic range. Conventional approaches of using PCE camera involve a time consuming and lossy process to reconstruct the original frames and then use those frames for target tracking and classification. In this paper, we present a deep learning approach that directly performs target tracking and classification in the compressive measurement domain without any frame reconstruction. Our approach has two parts: tracking and classification. The tracking has been done using YOLO (You Only Look Once) and the classification is achieved using Residual Network (ResNet). Extensive experiments using mid-wave infrared (MWIR) and long-wave infrared (LWIR) videos demonstrated the efficacy of our proposed approach.

基于改进ByteTrack算法的红外地面多目标跟踪方法

红外目标智能检测跟踪技术研究一直是同领域中的热点问题,尤其是在精确制导、海面监视和天空预警等方面。针对红外地面多目标跟踪场景中,由地面杂波干扰、多目标遮挡干扰、平台晃动等复杂场景造成的跟踪精度降低等问题,提出了一种基于改进ByteTrack算法的红外地面多目标跟踪方法。首先引用一种自适应调制噪声尺度的卡尔曼滤波器,缓解低质量检测对vanilla卡尔曼滤波器的影响;其次引入增强相关系数最大化算法对帧间图像进行配准,来补偿平台晃动产生的影响;然后增加了基于长短期记忆网络的运动模型,减小了卡尔曼滤波在非线性运动状态中产生的预测误差;最后引入连接模型和高斯平滑算法这两种轻量级离线算法来完善跟踪结果。在红外地面多目标数据集上进行了实验,结果表明,与Sort和Deepsort算法相比,改进算法的MOTA值分别提升了8.3%和10.2%,IDF1值分别提升了6.5%和5.6%。与同类算法相比,改进算法表现出了更好的有效性,在红外目标智能检测跟踪场景中会有较大应用。

A Novel Cuckoo Search Algorithm and Its Application

In this paper, the principle of Cuckoo algorithm is introduced, and the traditional Cuckoo algorithm is improved to establish a mathematical model of multi-objective optimization scheduling. Based on the improved algorithm, the model is optimized to a certain extent. Through analysis, it is proved that the improved algorithm has higher computational accuracy and can effectively improve the global convergence.

Hybrid tracking model and GSLM based neural network for crowd behavior recognition

Crowd behaviors analysis is the‘state of art’research topic in the field of computer vision which provides applications in video surveillance to crowd safety,event detection,security,etc.Literature presents some of the works related to crowd behavior detection and analysis.In crowd behavior detection,varying density of crowds and motion patterns appears to be complex occlusions for the researchers.This work presents a novel crowd behavior detection system to improve these restrictions.The proposed crowd behavior detection system is developed using hybrid tracking model and integrated features enabled neural network.The object movement and activity in the proposed crowded behavior detection system is assessed using proposed GSLM-based neural network.GSLM based neural network is developed by integrating the gravitational search algorithm with LM algorithm of the neural network to increase the learning process of the network.The performance of the proposed crowd behavior detection system is validated over five different videos and analyzed using accuracy.The experimentation results in the crowd behavior detection with a maximum accuracy of 93%which proves the efficacy of the proposed system in video surveillance with security concerns.

Research on the MPPT of Photovoltaic Power Generation Based on the CSA-INC Algorithm

The existing Maximum Power Point Tracking(MPPT)method has low tracking efficiency and poor stability.It is easy to fall into the Local Maximum Power Point(LMPP)in Partial Shading Condition(PSC),resulting in the degradation of output power quality and efficiency.It was found that various bio-inspired MPPT based optimization algorithms employ different mechanisms,and their performance in tracking the Global Maximum Power Point(GMPP)varies.Thus,a Cuckoo search algorithm(CSA)combined with the Incremental conductance Algorithm(INC)is proposed(CSA-INC)is put forward for the MPPT method of photovoltaic power generation.The method can improve the tracking speed by more than 52%compared with the traditional Cuckoo Search Algorithm(CSA),and the results of the study using this algorithm are compared with the popular Particle Swarm Optimization(PSO)and the Gravitational Search Algorithm(GSA).CSA-INC has an average tracking efficiency of 99.99%and an average tracking time of 0.19 s when tracking the GMPP,which improves PV power generation’s efficiency and power quality.

Human Body Tracking and Pose Estimation Using Modified Camshift Algorithm

In this paper, we propose multiple CAMShift Algorithm based on Kalman filter and weighted search windows that extracts skin color area and tracks several human body parts for real-time human tracking system. The CAMShift Algorithm we propose searches the skin color region by detecting the skin color area from background model. Kalman filter stabilizes the floated search area of CAMShift Algorithm. Each occlusion areas are avoided by using weighted window of non-search areas and main-search area. And shadows are eliminated from background model and intensity of shadow. The proposed modified Camshaft algorithm can estimate human pose in real-time and achieves 96.82% accuracy even in the case of occlusions.

基于SSA-ELM神经网络控制器的光伏MPPT方法

光伏电池板所处环境的非线性变化使得光伏电池的功率保持在最大功率点(maximum power point,MPP)非常困难。传统的最大功率点跟踪(maximum power point tracking,MPPT)方法普遍存在技术缺陷,无法满足当前需求。针对光伏发电MPPT问题,该文提出了一种基于麻雀搜索算法优化的极限学习机(sparrow search algorithm-extreme learning machine,SSA-ELM)神经网络控制器的MPPT方法。与传统技术相比,该MPPT方法在稳定性、速度、超调和MPP的振荡等方面的效果均较好。使用MATLAB/Simulink平台进行仿真实验,验证了所提控制策略及理论分析的正确性。

基于改进布谷鸟搜索算法的光伏MPPT控制

复杂阴影情况下,光伏阵列的P-V特性曲线会出现多个峰值,传统的MPPT算法因不能准确识别局部峰值和全局峰值,而无法进行复杂阴影情况下的最大功率点跟踪。针对传统布谷鸟搜索(Cuckoo Search,CS)算法因鸟窝之间缺乏交流能力导致可能陷入局部最优的问题,提出了一种多策略改进布谷鸟搜索(EGICS)算法。将传统CS算法中发现概率值的选择自适应变化,提高算法的搜索能力;将步长因子自适应化,提高算法的收敛速度;引入高斯扰动和精英反向学习策略,增加种群多样性,避免算法陷入局部最优。对EGICS算法在单峰和多峰函数中进行性能测试,并将其应用于光伏系统MPPT控制中进行仿真验证。仿真结果表明,EGICS算法在收敛速度、跟踪精度以及动态稳定三个方面有更好的效果。

红外地面目标智能抗遮挡跟踪算法研究

针对背景遮挡导致红外地面目标跟踪失败的问题,提出了一种基于改进轨迹预测网络的红外地面目标抗遮挡跟踪器。提出了遮挡判断准则,准确判断红外地面目标的遮挡情况;改进BiTrap轨迹预测网络,一方面通过孪生网络结构引入速度信息,采用单向预测的方法,提出了SiamTrap轨迹预测网络,提高了轨迹预测的精度;另一方面,通过改进训练方法和应用方法,可以更准确地预测地面目标的轨迹。对于短期遮挡,利用SiamTrap网络基于时间上下文信息预测目标遮挡位置。对于长期遮挡,提出了搜索扩展策略来处理真实目标信息缺乏导致的预测误差积累。提出了“二次判定”准则,实现了目标的精确捕获和正常跟踪。在含有遮挡的红外目标跟踪序列上对算法进行了对比测试。与基准跟踪器相比,文中所提算法在OPE评价指标下,成功率和准确率分别提高了5.2%和5.9%。这表明文中算法在应对红外地面目标被遮挡情况下具有良好的鲁棒性。

跨域无人集群协同反潜搜索方法研究

针对海上反潜搜索问题,提出一种将无人艇作为无人机的通信中继的跨域无人集群协同搜索方法,采用数字网格地图表征任务区域,构建了跨域平台的运动学模型;提出了跨域无人系统的协作方法,设计了无人系统之间的分布式信息融合机制;设计了异构平台的搜索目标函数,引导无人系统在搜索任务中实时决策。仿真结果表明:所提方法能够有效适用于文中反潜搜索任务想定,跨域平台之间能够通过优势互补提高搜索效能。

基于GLMB滤波的复杂场景下红外弱小目标自适应跟踪算法

针对红外弱小目标在复杂场景下受到漏检和杂波影响,导致跟踪不连续甚至失效的问题,本文提出一种红外弱小目标自适应跟踪算法。在预处理阶段,为了减少不必要的计算,首先定义一种衡量图像复杂度的算法。然后该算法通过计算红外图像多个特征得到场景复杂度来确认场景类型,再根据场景类型选取对应的检测算法提取目标候选位置、灰度以及局部直方图等特征建立对应的量测模型与似然函数。在目标跟踪阶段,为了自适应地匹配广义标签多伯努利(Generalized Labeled Multi-Bernoulli,GLMB)滤波器的滤波参数,在GLMB的基础上提出一种适应视频图像的新生算法进行航迹起始;针对红外图像序列目标检测概率未知的情况,将未知检测概率的基数化概率假设密度(Cardinality Probability Hypothesis Density,CPHD)滤波器集成到GLMB中实时估计目标检测概率以提升跟踪精度。仿真结果表明,所提出算法能有效地排除量测漏检和虚警的干扰,跟踪不同红外复杂场景下的弱小目标。

利用Transformer的多模态目标跟踪算法

目前目标跟踪方法大多通过融合不同模态信息进行定位决策,存在信息提取不充分、融合方法简单、弱光场景无法准确跟踪目标的问题。为此,提出一种基于Transformer的多模态目标跟踪算法(Trans-RGBT):利用伪孪生网络对可见光图像和红外图像分别进行特征提取,并在特征层面充分融合;将首帧目标信息调制到待跟踪帧的特征向量中,得到一个专用跟踪器;应用Transformer的方法对视野中的目标进行编解码,通过空间位置预测分支预测目标在视野中的空间位置,并结合历史信息滤除干扰目标,得到目标的准确位置;使用矩形框回归网络预测目标的外接矩形框,从而实现目标准确跟踪。在最新的大规模数据集VTUAV、RGBT234上进行了实验,与孪生网络(Siambased)、滤波(filter-based)算法相比,Trans-RGBT精度更高、鲁棒性更好、速度接近实时,达22 FPS。

基于改进涡流搜索算法的外骨骼迭代学习控制

为提升康复外骨骼机器人的步态跟踪性能,提出一种基于改进涡流搜索算法的迭代学习控制方法。首先针对传统迭代学习控制抗扰性差和控制信息缺失问题,引入PD控制器、自适应遗忘因子、误差过渡曲线和控制信息搜索等策略,改进迭代学习控制律;其次,基于多种策略对涡流搜索算法进行改进,提出了一种改进涡流搜索算法,改进后的算法可优化迭代学习控制的PD参数;最后进行行走实验,将提出的迭代学习控制方法与现有的同类算法进行仿真和数值比较,并测试了扰动情况下的跟踪性能。实验结果表明,所提方法的误差更小,跟踪性能更强。该算法改进了迭代学习控制的不足,具有较强的抗扰性能,保证了使用时的稳定性。

基于红外相机和毫米波雷达融合的烟雾遮挡无人驾驶车辆目标检测与跟踪

战场环境下无人驾驶车辆的感知系统易受烟雾、扬尘等天气的影响,对关键目标的检测与跟踪能力大大下降,造成目标漏检、目标误检、目标丢失等严重后果。针对该问题,开发毫米波雷达和红外相机融合系统,采用目标级融合方式建立简洁有效的融合规则,提炼和组合各传感器的优势信息,最终输出稳定的目标感知结果。对毫米波雷达的目标进行有效性检验和提取,并提出改进的基于密度的含噪声空间聚类应用算法,以减少毫米波雷达噪音干扰。以YOLOv4网络为基础,引入MobileNetv2主干网络,在网络训练过程中运用迁移学习方法,同时对红外数据样本进行扩充,解决了红外图像训练样本少的问题。试验结果表明,相较于仅基于红外相机的算法,融合检测算法在烟雾环境下的精度显著提升,且算法实时性高,实现了烟雾环境下毫米波雷达与红外相机融合的目标检测与跟踪,提高了无人驾驶车辆目标检测与跟踪系统的抗烟雾干扰能力。

考虑综合性能最优的非短视快速天基雷达多目标跟踪资源调度算法

合理有效的资源调度是天基雷达效能得以充分发挥的关键。针对天基雷达多目标跟踪资源调度问题,建立了综合考虑目标威胁度、跟踪精度与低截获概率(LPI)的代价函数;考虑目标的不确定、天基平台约束以及长远期期望代价,建立了多约束下的基于部分可观测的马尔可夫决策过程(POMDP)的资源调度模型;采用拉格朗日松弛法将多约束下的多目标跟踪资源调度问题转换分解为多个无约束的子问题;针对连续状态空间、连续动作空间及连续观测空间引起的维数灾难问题,采用基于蒙特卡罗树搜索(MCTS)的在线POMDP算法—POMCPOW算法进行求解,最终提出了一种综合多指标性能的非短视快速天基雷达多目标跟踪资源调度算法。仿真表明,与已有调度算法相比,所提算法资源分配更合理,系统性能更优。

不同复杂度干扰影响下疏散标志搜索行为研究

为分析不同复杂度干扰因素对疏散标志搜索的影响,招募33名志愿者进行疏散标志搜索眼动试验。组织被试者完成了3×3因素设计的疏散标志视觉搜索试验,并对眼动数据进行了非参数检验。结果显示:不同复杂度各类型干扰物对疏散标志搜索任务均具有显著影响。在低复杂度水平中,物品类材料标志搜索用时最长;当复杂度增加至中等水平时,文字类、物品类材料标志搜索用时更长;至高复杂度时,图像类材料标志搜索用时更长。眼动数据分析显示,不同复杂度水平下,各类型材料眼动数据表现出显著差异。在低复杂度水平下,物品类材料注视点数量、扫视长度数据水平更高;在中复杂度水平下,文字类材料注视点数量、扫视长度明显增加;在高复杂度水平下,图像类材料注视点数量、扫视长度大幅增加,各注视点注视时长显著低于文字类、物品类材料。研究分析了不同复杂度水平各类型干扰因素对疏散标志搜索效率影响,分解了疏散人员视觉搜索眼动行为,可为提高疏散引导系统效能提供实证依据。

基于改进YOLOv7-DeepSort的红外视频多目标跟踪

针对红外图像纹理弱及多目标遮挡导致跟踪精度低的问题,构建了基于改进YOLOv7模型和多目标跟踪算法DeepSort的融合红外目标跟踪模型MSB-YOLOv7-DeepSort。采用SE(squeeze and excitation)通道注意力机制和双向特征金字塔网络提高红外目标的特征提取质量;利用轻量化网络MobileNetV3替换YOLOv7骨干网络,提升融合模型的推理速度。实验结果表明,MSB-YOLOv7-DeepSort模型在跟踪准确度、跟踪精确度、正确目标跟踪比例和帧率等方面均具有较好的性能。

智能室内停车场定位导航系统设计

如今,各大商场的地下停车场日趋普遍化和复杂化。随着汽车持有量的急剧增长,车主在庞大的地下停车场中泊车寻车成为了难题。为此,提出了一种实现智能导航和停车管理的解决方案。依据室内定位及导航技术的应用需求,利用光敏模块、环形线圈检测及红外循迹技术,对停车场地形进行红外扫描,通过STM32单片机将数据传输到接收端。用户只需按照主控制器返回给终端的信息,即可快速实现点对点的定位服务。这一方案旨在提高人们出行的便捷性,并为加强城市停车规范管理提供可行性方案。