Robust Deep Transfer Learning Based Object Detection and Tracking Approach

At present days,object detection and tracking concepts have gained more importance among researchers and business people.Presently,deep learning(DL)approaches have been used for object tracking as it increases the perfor-mance and speed of the tracking process.This paper presents a novel robust DL based object detection and tracking algorithm using Automated Image Anno-tation with ResNet based Faster regional convolutional neural network(R-CNN)named(AIA-FRCNN)model.The AIA-RFRCNN method performs image anno-tation using a Discriminative Correlation Filter(DCF)with Channel and Spatial Reliability tracker(CSR)called DCF-CSRT model.The AIA-RFRCNN model makes use of Faster RCNN as an object detector and tracker,which involves region proposal network(RPN)and Fast R-CNN.The RPN is a full convolution network that concurrently predicts the bounding box and score of different objects.The RPN is a trained model used for the generation of the high-quality region proposals,which are utilized by Fast R-CNN for detection process.Besides,Residual Network(ResNet 101)model is used as a shared convolutional neural network(CNN)for the generation of feature maps.The performance of the ResNet 101 model is further improved by the use of Adam optimizer,which tunes the hyperparameters namely learning rate,batch size,momentum,and weight decay.Finally,softmax layer is applied to classify the images.The performance of the AIA-RFRCNN method has been assessed using a benchmark dataset and a detailed comparative analysis of the results takes place.The outcome of the experiments indicated the superior characteristics of the AIA-RFRCNN model under diverse aspects.

Towards Collaborative Robotics in Top View Surveillance:A Framework for Multiple Object Tracking by Detection Using Deep Learning

Collaborative Robotics is one of the high-interest research topics in the area of academia and industry.It has been progressively utilized in numerous applications,particularly in intelligent surveillance systems.It allows the deployment of smart cameras or optical sensors with computer vision techniques,which may serve in several object detection and tracking tasks.These tasks have been considered challenging and high-level perceptual problems,frequently dominated by relative information about the environment,where main concerns such as occlusion,illumination,background,object deformation,and object class variations are commonplace.In order to show the importance of top view surveillance,a collaborative robotics framework has been presented.It can assist in the detection and tracking of multiple objects in top view surveillance.The framework consists of a smart robotic camera embedded with the visual processing unit.The existing pre-trained deep learning models named SSD and YOLO has been adopted for object detection and localization.The detection models are further combined with different tracking algorithms,including GOTURN,MEDIANFLOW,TLD,KCF,MIL,and BOOSTING.These algorithms,along with detection models,help to track and predict the trajectories of detected objects.The pre-trained models are employed;therefore,the generalization performance is also investigated through testing the models on various sequences of top view data set.The detection models achieved maximum True Detection Rate 93%to 90%with a maximum 0.6%False Detection Rate.The tracking results of different algorithms are nearly identical,with tracking accuracy ranging from 90%to 94%.Furthermore,a discussion has been carried out on output results along with future guidelines.

Multiple Pedestrian Detection and Tracking in Night Vision Surveillance Systems

Pedestrian detection and tracking are vital elements of today’s surveillance systems,which make daily life safe for humans.Thus,human detection and visualization have become essential inventions in the field of computer vision.Hence,developing a surveillance system with multiple object recognition and tracking,especially in low light and night-time,is still challenging.Therefore,we propose a novel system based on machine learning and image processing to provide an efficient surveillance system for pedestrian detection and tracking at night.In particular,we propose a system that tackles a two-fold problem by detecting multiple pedestrians in infrared(IR)images using machine learning and tracking them using particle filters.Moreover,a random forest classifier is adopted for image segmentation to identify pedestrians in an image.The result of detection is investigated by particle filter to solve pedestrian tracking.Through the extensive experiment,our system shows 93%segmentation accuracy using a random forest algorithm that demonstrates high accuracy for background and roof classes.Moreover,the system achieved a detection accuracy of 90%usingmultiple templatematching techniques and 81%accuracy for pedestrian tracking.Furthermore,our system can identify that the detected object is a human.Hence,our system provided the best results compared to the state-ofart systems,which proves the effectiveness of the techniques used for image segmentation,classification,and tracking.The presented method is applicable for human detection/tracking,crowd analysis,and monitoring pedestrians in IR video surveillance.

Automated Video-Based Face Detection Using Harris Hawks Optimization with Deep Learning

Face recognition technology automatically identifies an individual from image or video sources.The detection process can be done by attaining facial characteristics from the image of a subject face.Recent developments in deep learning(DL)and computer vision(CV)techniques enable the design of automated face recognition and tracking methods.This study presents a novel Harris Hawks Optimization with deep learning-empowered automated face detection and tracking(HHODL-AFDT)method.The proposed HHODL-AFDT model involves a Faster region based convolution neural network(RCNN)-based face detection model and HHO-based hyperparameter opti-mization process.The presented optimal Faster RCNN model precisely rec-ognizes the face and is passed into the face-tracking model using a regression network(REGN).The face tracking using the REGN model uses the fea-tures from neighboring frames and foresees the location of the target face in succeeding frames.The application of the HHO algorithm for optimal hyperparameter selection shows the novelty of the work.The experimental validation of the presented HHODL-AFDT algorithm is conducted using two datasets and the experiment outcomes highlighted the superior performance of the HHODL-AFDT model over current methodologies with maximum accuracy of 90.60%and 88.08%under PICS and VTB datasets,respectively.

Segmentation Based Real Time Anomaly Detection and Tracking Model for Pedestrian Walkways

Presently,video surveillance is commonly employed to ensure security in public places such as traffic signals,malls,railway stations,etc.A major chal-lenge in video surveillance is the identification of anomalies that exist in it such as crimes,thefts,and so on.Besides,the anomaly detection in pedestrian walkways has gained significant attention among the computer vision communities to enhance pedestrian safety.The recent advances of Deep Learning(DL)models have received considerable attention in different processes such as object detec-tion,image classification,etc.In this aspect,this article designs a new Panoptic Feature Pyramid Network based Anomaly Detection and Tracking(PFPN-ADT)model for pedestrian walkways.The proposed model majorly aims to the recognition and classification of different anomalies present in the pedestrian walkway like vehicles,skaters,etc.The proposed model involves panoptic seg-mentation model,called Panoptic Feature Pyramid Network(PFPN)is employed for the object recognition process.For object classification,Compact Bat Algo-rithm(CBA)with Stacked Auto Encoder(SAE)is applied for the classification of recognized objects.For ensuring the enhanced results better anomaly detection performance of the PFPN-ADT technique,a comparison study is made using Uni-versity of California San Diego(UCSD)Anomaly data and other benchmark data-sets(such as Cityscapes,ADE20K,COCO),and the outcomes are compared with the Mask Recurrent Convolutional Neural Network(RCNN)and Faster Convolu-tional Neural Network(CNN)models.The simulation outcome demonstrated the enhanced performance of the PFPN-ADT technique over the other methods.

Simultaneous Multi-vehicle Detection and Tracking Framework with Pavement Constraints Based on Machine Learning and Particle Filter Algorithm

Due to the large variations of environment with ever-changing background and vehicles with different shapes, colors and appearances, to implement a real-time on-board vehicle recognition system with high adaptability, efficiency and robustness in complicated environments, remains challenging. This paper introduces a simultaneous detection and tracking framework for robust on-board vehicle recognition based on monocular vision technology. The framework utilizes a novel layered machine learning and particle filter to build a multi-vehicle detection and tracking system. In the vehicle detection stage, a layered machine learning method is presented, which combines coarse-search and fine-search to obtain the target using the Ada Boost-based training algorithm. The pavement segmentation method based on characteristic similarity is proposed to estimate the most likely pavement area. Efficiency and accuracy are enhanced by restricting vehicle detection within the downsized area of pavement. In vehicle tracking stage, a multi-objective tracking algorithm based on target state management and particle filter is proposed. The proposed system is evaluated by roadway video captured in a variety of traffics, illumination, and weather conditions. The evaluating results show that, under conditions of proper illumination and clear vehicle appearance, the proposed system achieves 91.2% detection rate and 2.6% false detection rate. Experiments compared to typical algorithms show that, the presented algorithm reduces the false detection rate nearly by half at the cost of decreasing 2.7%–8.6% detection rate. This paper proposes a multi-vehicle detection and tracking system, which is promising for implementation in an on-board vehicle recognition system with high precision, strong robustness and low computational cost.

Visual tracking based on transfer learning of deep salience information

In this paper,we propose a new visual tracking method in light of salience information and deep learning.Salience detection is used to exploit features with salient information of the image.Complicated representations of image features can be gained by the function of every layer in convolution neural network(CNN).The characteristic of biology vision in attention-based salience is similar to the neuroscience features of convolution neural network.This motivates us to improve the representation ability of CNN with functions of salience detection.We adopt the fully-convolution networks(FCNs)to perform salience detection.We take parts of the network structure to perform salience extraction,which promotes the classification ability of the model.The network we propose shows great performance in tracking with the salient information.Compared with other excellent algorithms,our algorithm can track the target better in the open tracking datasets.We realize the 0.5592 accuracy on visual object tracking 2015(VOT15)dataset.For unmanned aerial vehicle 123(UAV123)dataset,the precision and success rate of our tracker is 0.710 and 0.429.

A Novel Tracking-by-Detection Method with Local Binary Pattern and Kalman Filter

Tracking-Learning-Detection( TLD) is an adaptive tracking algorithm,which tracks by learning the appearance of the object as the video progresses and shows a good performance in long-term tracking task.But our experiments show that under some scenarios,such as non-uniform illumination changing,serious occlusion,or motion-blurred,it may fails to track the object. In this paper,to surmount some of these shortages,especially for the non-uniform illumination changing,and give full play to the performance of the tracking-learning-detection framework, we integrate the local binary pattern( LBP) with the cascade classifiers,and define a new classifier named ULBP( Uniform Local Binary Pattern) classifiers. When the object appearance has rich texture features,the ULBP classifier will work instead of the nearest neighbor classifier in TLD algorithm,and a recognition module is designed to choose the suitable classifier between the original nearest neighbor( NN) classifier and the ULBP classifier. To further decrease the computing load of the proposed tracking approach,Kalman filter is applied to predict the searching range of the tracking object.A comprehensive study has been conducted to confirm the effectiveness of the proposed algorithm (TLD _ULBP),and different multi-property datasets were used. The quantitative evaluations show a significant improvement over the original TLD,especially in various lighting case.

Embedded System Development for Detection of Railway Track Surface Deformation Using Contour Feature Algorithm

Derailment of trains is not unusual all around the world,especially in developing countries,due to unidentified track or rolling stock faults that cause massive casualties each year.For this purpose,a proper condition monitoring system is essential to avoid accidents and heavy losses.Generally,the detection and classification of railway track surface faults in real-time requires massive computational processing and memory resources and is prone to a noisy environment.Therefore,in this paper,we present the development of a novel embedded system prototype for condition monitoring of railway track.The proposed prototype system works in real-time by acquiring railway track surface images and performing two tasks a)detect deformation(i.e.,faults)like squats,shelling,and spalling using the contour feature algorithm and b)the vibration signature on that faulty spot by synchronizing acceleration and image data.A new illumination scheme is also proposed to avoid the sunlight reflection that badly affects the image acquisition process.The contour detection algorithm is applied here to detect the uneven shapes and discontinuities in the geometrical structure of the railway track surface,which ultimately detects unhealthy regions.It works by converting Red,Green,and Blue(RGB)images into binary images,which distinguishes the unhealthy regions by making them white color while the healthy regions in black color.We have used the multiprocessing technique to overcome the massive processing and memory issues.This embedded system is developed on Raspberry Pi by interfacing a vision camera,an accelerometer,a proximity sensor,and a Global Positioning System(GPS)sensors(i.e.,multi-sensors).The developed embedded system prototype is tested in real-time onsite by installing it on a Railway Inspection Trolley(RIT),which runs at an average speed of 15 km/h.The functional verification of the proposed system is done successfully by detecting and recording the various railway track surface faults.An unhealthy frame’s onsite detection processing time was recorded at approximately 25.6ms.The proposed system can synchronize the acceleration data on specific railway track deformation.The proposed novel embedded system may be beneficial for detecting faults to overcome the conventional manual railway track condition monitoring,which is still being practiced in various developing or underdeveloped countries.

Multiple-Object Tracking Using Histogram Stamp Extraction in CCTV Environments

Object tracking,an important technology in the field of image processing and computer vision,is used to continuously track a specific object or person in an image.This technology may be effective in identifying the same person within one image,but it has limitations in handling multiple images owing to the difficulty in identifying whether the object appearing in other images is the same.When tracking the same object using two or more images,there must be a way to determine that objects existing in different images are the same object.Therefore,this paper attempts to determine the same object present in different images using color information among the unique information of the object.Thus,this study proposes a multiple-object-tracking method using histogram stamp extraction in closed-circuit television applications.The proposed method determines the presence or absence of a target object in an image by comparing the similarity between the image containing the target object and other images.To this end,a unique color value of the target object is extracted based on its color distribution in the image using three methods:mean,mode,and interquartile range.The Top-N accuracy method is used to analyze the accuracy of each method,and the results show that the mean method had an accuracy of 93.5%(Top-2).Furthermore,the positive prediction value experimental results show that the accuracy of the mean method was 65.7%.As a result of the analysis,it is possible to detect and track the same object present in different images using the unique color of the object.Through the results,it is possible to track the same object that can minimize manpower without using personal information when detecting objects in different images.In the last response speed experiment,it was shown that when the mean was used,the color extraction of the object was possible in real time with 0.016954 s.Through this,it is possible to detect and track the same object in real time when using the proposed method.

Drone for Dynamic Monitoring and Tracking with Intelligent Image Analysis

Traditional monitoring systems that are used in shopping malls or com-munity management,mostly use a remote control to monitor and track specific objects;therefore,it is often impossible to effectively monitor the entire environ-ment.Whenfinding a suspicious person,the tracked object cannot be locked in time for tracking.This research replaces the traditionalfixed-point monitor with the intelligent drone and combines the image processing technology and automatic judgment for the movements of the monitored person.This intelligent system can effectively improve the shortcomings of low efficiency and high cost of the traditional monitor system.In this article,we proposed a TIMT(The Intel-ligent Monitoring and Tracking)algorithm which can make the drone have smart surveillance and tracking capabilities.It combined with Artificial Intelligent(AI)face recognition technology and the OpenPose which is able to monitor the phy-sical movements of multiple people in real time to analyze the meaning of human body movements and to track the monitored intelligently through the remote con-trol interface of the drone.This system is highly agile and could be adjusted immediately to any angle and screen that we monitor.Therefore,the system couldfind abnormal conditions immediately and track and monitor them automatically.That is the system can immediately detect when someone invades the home or community,and the drone can automatically track the intruder to achieve that the two significant shortcomings of the traditional monitor will be improved.Experimental results show that the intelligent monitoring and tracking drone sys-tem has an excellent performance,which not only dramatically reduces the num-ber of monitors and the required equipment but also achieves perfect monitoring and tracking.

基于深度学习和颜色特征的行人跟踪算法

针对行人跟踪算法中因行人遮挡而导致行人跟踪准确率低、跟踪速度慢的问题,论文提出了一种基于深度学习和颜色特征的行人跟踪算法。首先利用yolov5目标检测算法检测行人,得到带有行人框的视频帧,同时利用检测框坐标信息判断行人之间是否存在遮挡,若有遮挡,则把遮挡区域像素设为0,分割出非遮挡区域,将非遮挡区域转化为HSV颜色空间,量化HSV分量,构造颜色特征直方图,并表示为一维向量G。其次,以第一帧行人检测框坐标为基础构建行人跟踪模型,初始化跟踪对象,并根据行人质心变化预测行人位置。在公开数据集MOT-16数据集上测试,MOTA为49.78%,相比于Sort和DeepSort算法分别提高1.51%和0.33%,在IDF1分数上分别高于Sort和DeepSort算法7.07%和3.46%。跟踪速度比DeepSort提升24%。

基于关键特征点的改进TLD目标跟踪算法研究

TLD(Tracking-Learning-Detection)算法是一种新颖的单目标长时间视觉跟踪算法,在给定极少的先验知识的情况下,能够迅速地学习目标特征并进行有效的跟踪。TLD算法中跟踪器每次在跟踪目标上均匀地选取特征点进行跟踪,不能保证每个特征点都能够被可靠地跟踪。针对这个问题,提出一种基于关键特征点检测的改进TLD算法,保证所选特征点都能够被正确可靠地跟踪,防止跟踪结果发生漂移,提高了跟踪器的跟踪精度。另一方面,在TLD检测器中引入了基于轨迹连续性的在线位置预测,在保证正确跟踪的前提下,缩小了检测器的检测范围,提高了运算速度。实验结果表明,该算法有较高的跟踪精度和速度。

TLD框架下的内河船舶跟踪

闭路电视(closed circuit television,CCTV)系统是内河海事监管的重要手段.基于跟踪-学习-检测(tracking-learning-detection,TLD)框架研究并改进内河航道CCTV系统的船舶识别和跟踪.在TLD框架下提出特征值约束条件,可对像素的短期跟踪结果进行校验,不仅有效解决了像素对归一化相关系数值求解的繁琐问题,还很好地保留了图像中角点像素的跟踪结果,使船舶的短期跟踪足够可靠.用级联的目标检测器精确定位船舶时,在满足内河应用实时性前提下,提出通过对目标候选区域的模板匹配来保证算法准确性.实验结果表明,改进的算法在应用于内河CCTV系统的船舶识别与跟踪中保持了较高的实时性和鲁棒性,并提高了跟踪精度.

检测区域动态调整的TLD目标跟踪算法

针对经典跟踪-学习-检测（TLD）目标跟踪算法由于检测区域过大而导致的检测时间过长及对相似目标跟踪处理效果不理想的问题,提出一种检测区域可动态自适应调整的方法——TLD-DO。该方法利用两次Kalman滤波加速度矫正预测的检测区域优化算法DKF,通过缩小TLD检测器检测范围,以达到在跟踪精度略有提升的情况下提高跟踪速度的目的;同时此方法可排除画面内相似目标的干扰,提高在含有相似目标的复杂背景下目标跟踪的准确性。实验结果表明：TLD-DO算法在处理不同视频与跟踪目标时,检测速度有1.31-3.19倍提升;对含有相似目标干扰情况下,跟踪效果明显优于原TLD算法;对目标抖动及失真情况有较高的鲁棒性。

一种改进的TLD算法

针对跟踪算法目标遮挡后易出现跟踪漂移的问题,提出了一种跟踪学习检测(TLD)算法与Kalman滤波相结合的手势跟踪方法。在跟踪器跟踪成功后,加入识别窗的方法进行遮挡判定。产生遮挡后目标模型不再更新,学习器不再更新集合分类器。若是部分遮挡,则由TLD学习器处理;若是严重遮挡,则改由Kalman滤波算法预测目标的运动轨迹。该方法在保留TLD算法长期稳定跟踪、适应摄像机快速运动与复杂背景等优点的基础上,改善了目标遮挡后易出现跟踪漂移的问题。实验表明:提出的改进TLD算法比其他常见跟踪方法具有更加优异的性能。

基于改进TLD的自动目标跟踪方法

视觉跟踪一直是机器视觉研究热点,TLD(tracking-learning-detection)算法是近年来出现的一种高效的视觉跟踪算法,针对TLD算法中Lucas-Kanade(LK)光流法无法有效跟踪物体快速移动和尺度变化的问题,采用金字塔光流法对TLD算法进行改进。并将所跟踪物体形心作为图像定位参考点,提取物体定位信息,通过定位信息运用比例-积分-微分(proportion-integral-derivative,PID)控制算法控制摄像头舵机云台转向,使摄像头快速、灵活、精确地自动跟踪指定物体。通过系统测试,与传统TLD算法对比,采用金字塔光流法改进的TLD目标跟踪算法在跟踪目标发生光照变化、尺度变化等情况时,具有更加优良的跟踪性能,准确将跟踪目标形心位置提供给控制部分,控制算法高效灵活,在获取信息后精确、快速地控制摄像头方位,使其正对跟踪目标。该系统对目标跟踪技术、安防技术、自动瞄准系统具有重大意义。

基于TLD的增强现实跟踪注册方法

提出了一种新的增强现实跟踪注册方法。采用跟踪-学习-检测TLD算法,将目标跟踪任务分成跟踪器、学习器和检测器,通过跟踪器和检测器并行对视频流中的图像帧进行跟踪、检测操作,并使用P-N学习算法进行学习和更新,综合这三部分从而确定稳定的目标位置;在所跟踪到目标位置的基础上,使用基于局部SURF算法对目标的特征点进行检测与匹配操作;根据所获得的特征点计算出三维注册参数,完成增强现实系统的跟踪注册。实验结果表明,所提出的方法有效地解决了传统跟踪算法目标跟踪不稳定、易丢失目标和目标检测算法耗时严重、准确度不高等问题,使增强现实系统更稳定、更具鲁棒性。

检测区域自适应调整的TLD多目标跟踪算法

传统的TLD目标跟踪算法由于检测区域过大导致检测时间过长,并对相似目标跟踪效果不理想且只能对单个目标快速跟踪.针对这些问题,利用双Kalman滤波加速预测的DKF检测区域优化算法构造了一种检测区域可自适应调整的多目标跟踪算法——TLD-DOMO算法.TLD-DOMO算法的多目标检测器可对各目标的潜在运动范围进行预测,使其检测区域的大小及位置自适应地调整至最佳状态,以此提升对多目标跟踪的精度及效率.此外,该方法可有效地降低多目标间的相互干扰,支持对多相似目标的同时跟踪.实验结果表明:TLD-DOMO算法在对各测试视频的多目标跟踪中,跟踪速度均有提升,加速比为1.55~2.94倍;在多相似目标跟踪中,对各目标的检测与识别效果优于原TLD算法.

复杂道路监控场景下的车辆检测与跟踪数据集

针对现有大部分车辆检测与跟踪数据集通常存在的采集场景单一、数据集长尾分布以及图像采集环境简单等问题,本文构建一个车辆数据集VeDT-MSS,用于城市以及乡村监控场景下4种车辆类别(小汽车、卡车、公交车和摩托车)的检测以及跟踪研究。该数据集具有交通场景多样化、卡车的类内多样性大、摩托车标注实例占比高以及背景复杂程度高4个显著特性。为了验证该数据集的有效性,在目标检测以及多目标跟踪任务上进行了大量的基线实验。实验结果表明,VeDT-MSS数据集在评估现有算法的鲁棒性和泛化性方面具有实用性。该数据集的提出对促进车辆检测与跟踪研究具有相当的潜力,并为计算机视觉社区评估算法性能提供一个新的数据选择。