Lightweight Cross-Modal Multispectral Pedestrian Detection Based on Spatial Reweighted Attention Mechanism

Multispectral pedestrian detection technology leverages infrared images to provide reliable information for visible light images, demonstrating significant advantages in low-light conditions and background occlusion scenarios. However, while continuously improving cross-modal feature extraction and fusion, ensuring the model’s detection speed is also a challenging issue. We have devised a deep learning network model for cross-modal pedestrian detection based on Resnet50, aiming to focus on more reliable features and enhance the model’s detection efficiency. This model employs a spatial attention mechanism to reweight the input visible light and infrared image data, enhancing the model’s focus on different spatial positions and sharing the weighted feature data across different modalities, thereby reducing the interference of multi-modal features. Subsequently, lightweight modules with depthwise separable convolution are incorporated to reduce the model’s parameter count and computational load through channel-wise and point-wise convolutions. The network model algorithm proposed in this paper was experimentally validated on the publicly available KAIST dataset and compared with other existing methods. The experimental results demonstrate that our approach achieves favorable performance in various complex environments, affirming the effectiveness of the multispectral pedestrian detection technology proposed in this paper.

An Application of RGBD-Based Skeleton Reconstruction for Pedestrian Detection and Occlusion Handling

This study explores the challenges posed by pedestrian detection and occlusion in AR applications, employing a novel approach that utilizes RGB-D-based skeleton reconstruction to reduce the overhead of classical pedestrian detection algorithms during training. Furthermore, it is dedicated to addressing occlusion issues in pedestrian detection by using Azure Kinect for body tracking and integrating a robust occlusion management algorithm, significantly enhancing detection efficiency. In experiments, an average latency of 204 milliseconds was measured, and the detection accuracy reached an outstanding level of 97%. Additionally, this approach has been successfully applied in creating a simple yet captivating augmented reality game, demonstrating the practical application of the algorithm.

Multiple Pedestrian Detection and Tracking in Night Vision Surveillance Systems

Pedestrian detection and tracking are vital elements of today’s surveillance systems,which make daily life safe for humans.Thus,human detection and visualization have become essential inventions in the field of computer vision.Hence,developing a surveillance system with multiple object recognition and tracking,especially in low light and night-time,is still challenging.Therefore,we propose a novel system based on machine learning and image processing to provide an efficient surveillance system for pedestrian detection and tracking at night.In particular,we propose a system that tackles a two-fold problem by detecting multiple pedestrians in infrared(IR)images using machine learning and tracking them using particle filters.Moreover,a random forest classifier is adopted for image segmentation to identify pedestrians in an image.The result of detection is investigated by particle filter to solve pedestrian tracking.Through the extensive experiment,our system shows 93%segmentation accuracy using a random forest algorithm that demonstrates high accuracy for background and roof classes.Moreover,the system achieved a detection accuracy of 90%usingmultiple templatematching techniques and 81%accuracy for pedestrian tracking.Furthermore,our system can identify that the detected object is a human.Hence,our system provided the best results compared to the state-ofart systems,which proves the effectiveness of the techniques used for image segmentation,classification,and tracking.The presented method is applicable for human detection/tracking,crowd analysis,and monitoring pedestrians in IR video surveillance.

YOLOv2PD:An Efficient Pedestrian Detection Algorithm Using Improved YOLOv2 Model

Real-time pedestrian detection is an important task for unmanned driving systems and video surveillance.The existing pedestrian detection methods often work at low speed and also fail to detect smaller and densely distributed pedestrians by losing some of their detection accuracy in such cases.Therefore,the proposed algorithm YOLOv2(“YOU ONLY LOOK ONCE Version 2”)-based pedestrian detection(referred to as YOLOv2PD)would be more suitable for detecting smaller and densely distributed pedestrians in real-time complex road scenes.The proposed YOLOv2PD algorithm adopts a Multi-layer Feature Fusion(MLFF)strategy,which helps to improve the model’s feature extraction ability.In addition,one repeated convolution layer is removed from the final layer,which in turn reduces the computational complexity without losing any detection accuracy.The proposed algorithm applies the K-means clustering method on the Pascal Voc-2007+2012 pedestrian dataset before training to find the optimal anchor boxes.Both the proposed network structure and the loss function are improved to make the model more accurate and faster while detecting smaller pedestrians.Experimental results show that,at 544×544 image resolution,the proposed model achieves 80.7%average precision(AP),which is 2.1%higher than the YOLOv2 Model on the Pascal Voc-2007+2012 pedestrian dataset.Besides,based on the experimental results,the proposed model YOLOv2PD achieves a good trade-off balance between detection accuracy and real-time speed when evaluated on INRIA and Caltech test pedestrian datasets and achieves state-of-the-art detection results.

Improved pedestrian detection with peer AdaBoost cascade

Focusing on data imbalance and intraclass variation,an improved pedestrian detection with a cascade of complex peer AdaBoost classifiers is proposed.The series of the AdaBoost classifiers are learned greedily,along with negative example mining.The complexity of classifiers in the cascade is not limited,so more negative examples are used for training.Furthermore,the cascade becomes an ensemble of strong peer classifiers,which treats intraclass variation.To locally train the AdaBoost classifiers with a high detection rate,a refining strategy is used to discard the hardest negative training examples rather than decreasing their thresholds.Using the aggregate channel feature(ACF),the method achieves miss rates of 35%and 14%on the Caltech pedestrian benchmark and Inria pedestrian dataset,respectively,which are lower than that of increasingly complex AdaBoost classifiers,i.e.,44%and 17%,respectively.Using deep features extracted by the region proposal network(RPN),the method achieves a miss rate of 10.06%on the Caltech pedestrian benchmark,which is also lower than 10.53%from the increasingly complex cascade.This study shows that the proposed method can use more negative examples to train the pedestrian detector.It outperforms the existing cascade of increasingly complex classifiers.

A robust system for real-time pedestrian detection and tracking

A real-time pedestrian detection and tracking system using a single video camera was developed to monitor pedestrians. This system contained six modules: video flow capture, pre-processing, movement detection, shadow removal, tracking, and object classification. The Gaussian mixture model was utilized to extract the moving object from an image sequence segmented by the mean-shift technique in the pre-processing module. Shadow removal was used to alleviate the negative impact of the shadow to the detected objects. A model-free method was adopted to identify pedestrians. The maximum and minimum integration methods were developed to integrate multiple cues into the mean-shift algorithm and the initial tracking iteration with the competent integrated probability distribution map for object tracking. A simple but effective algorithm was proposed to handle full occlusion cases. The system was tested using real traffic videos from different sites. The results of the test confirm that the system is reliable and has an overall accuracy of over 85%.

Motion cue based pedestrian detection with two-frame-filtering

This study proposes a motion cue based pedestrian detection method with two-trame-filtering （Tff） for video surveillance. The novel motion cue is exploited by the gray value variation between two frames. Then Tff processing filters the gradient magnitude image by the variation map. Summa- tions of the Tff gradient magnitudes in cells are applied to train a pre-deteetor to exclude most of the background regions. Histogram of Tff oriented gradient （HTffOG） feature is proposed for pedestrian detection. Experimental results show that this method is effective and suitable for real-time surveil- lance applications.

Pedestrian detection in driver assistance using SSD and PS-GAN

Pedestrian detection is a critical challenge in the field of general object detection,the performance of object detection has advanced with the development of deep learning.However,considerable improvement is still required for pedestrian detection,considering the differences in pedestrian wears,action,and posture.In the driver assistance system,it is necessary to further improve the intelligent pedestrian detection ability.We present a method based on the combination of SSD and GAN to improve the performance of pedestrian detection.Firstly,we assess the impact of different kinds of methods which can detect pedestrians based on SSD and optimize the detection for pedestrian characteristics.Secondly,we propose a novel network architecture,namely data synthesis PS-GAN to generate diverse pedestrian data for verifying the effectiveness of massive training data to SSD detector.Experimental results show that the proposed manners can improve the performance of pedestrian detection to some extent.At last,we use the pedestrian detector to simulate a specific application of motor vehicle assisted driving which would make the detector focus on specific pedestrians according to the velocity of the vehicle.The results establish the validity of the approach.

Research on pedestrian detection based on multi-level fine-grained YOLOX algorithm

Purpose-The purpose of the study is to address the problems of low accuracy and missed detection of occluded pedestrians and small target pedestrians when using the YOLOX general object detection algorithm for pedestrian detection.This study proposes a multi-level fine-grained YOLOX pedestrian detection algorithm.Design/methodology/approach-First,to address the problem of the original YOLOX algorithm in obtaining a single perceptual field for the feature map before feature fusion,this study improves the PAFPN structure by adding the ResCoT module to increase the diversity of the perceptual field of the feature map and divides the pedestrian multi-scale features into finer granularity.Second,for the CSPLayer of the PAFPN,a weight gain-based normalization-based attention module(NAM)is proposed to make the model pay more attention to the context information when extracting pedestrian features and highlight the salient features of pedestrians.Finally,the authors experimentally determined the optimal values for the confidence loss function.Findings-The experimental results show that,compared with the original YOLOX algorithm,the AP of the improved algorithm increased by 2.90%,the Recall increased by 3.57%,and F1 increased by 2%on the pedestrian dataset.Research limitations/implications-The multi-level fine-grained YOLOX pedestrian detection algorithm can effectively improve the detection of occluded pedestrians and small target pedestrians.Originality/value-The authors introduce a multi-level fine-grained ResCoT module and a weight gain-based NAM attention module.

Replacing the human driver:An objective benchmark for occluded pedestrian detection

Early detection of vulnerable road users is a crucial requirement for autonomous vehicles to meet and exceed the object detection capabilities of human drivers.One of the most complex outstanding challenges is that of partial occlusion where a target object is only partially available to the sensor due to obstruction by another foreground object.A number of leading pedestrian detection benchmarks provide annotation for partial occlusion,however each benchmark varies greatly in their definition of the occurrence and severity of occlusion.Research demonstrates that a high degree of subjectivity is used to classify occlusion level in these cases and occlusion is typically categorized into 2–3 broad categories such as“partially”and“heavily”occluded.In addition,many pedestrian instances are impacted by multiple inhibiting factors which contribute to non-detection such as object scale,distance from camera,lighting variations and adverse weather.This can lead to inaccurate or inconsistent reporting of detection performance for partially occluded pedestrians depending on which benchmark is used.This research introduces a novel,objective benchmark for partially occluded pedestrian detection to facilitate the objective characterization of pedestrian detection models.Characterization is carried out on seven popular pedestrian detection models for a range of occlusion levels from 0%–99%to demonstrate the impact of progressive levels of partial occlusion on pedestrian detectability.Results show that the proposed benchmark provides more objective,fine grained analysis of pedestrian detection algorithms than the current state of the art.

Pedestrian and Vehicle Detection Based on Pruning YOLOv4 with Cloud-Edge Collaboration

Nowadays,the rapid development of edge computing has driven an increasing number of deep learning applications deployed at the edge of the network,such as pedestrian and vehicle detection,to provide efficient intelligent services to mobile users.However,as the accuracy requirements continue to increase,the components of deep learning models for pedestrian and vehicle detection,such as YOLOv4,become more sophisticated and the computing resources required for model training are increasing dramatically,which in turn leads to significant challenges in achieving effective deployment on resource-constrained edge devices while ensuring the high accuracy performance.For addressing this challenge,a cloud-edge collaboration-based pedestrian and vehicle detection framework is proposed in this paper,which enables sufficient training of models by utilizing the abundant computing resources in the cloud,and then deploying the well-trained models on edge devices,thus reducing the computing resource requirements for model training on edge devices.Furthermore,to reduce the size of the model deployed on edge devices,an automatic pruning method combines the convolution layer and BN layer is proposed to compress the pedestrian and vehicle detection model size.Experimental results show that the framework proposed in this paper is able to deploy the pruned model on a real edge device,Jetson TX2,with 6.72 times higher FPS.Meanwhile,the channel pruning reduces the volume and the number of parameters to 96.77%for the model,and the computing amount is reduced to 81.37%.

A Robust Pedestrian Detection Approach Based on Shapelet Feature and Haar Detector Ensembles

Detection of pedestrians in images and video sequences is important for many applications but is very challenging due to the various silhouettes of pedestrians and partial occlusions. This paper describes a two-stage robust pedestrian detection approach. The first stage uses a full body detector applied to a single image to generate pedestrian candidates. In the second stage, each pedestrian candidate is verified with a detector ensemble consisting of part detectors. The full body detector is trained based on improved shapelet features, while the part detectors make use of Haar-like wavelets as features. All the detectors are trained by a boosting method. The responses of the part detectors are then combined using a detector ensemble. The verification process is formulated as a combinatoria~ optimization problem with a genetic a^gorithm for optimization. Then, the detection results are regarded as equivalent classes so that multiple detections of the same pedestrian are quickly merged together. Tests show that this approach has a detection rate of over 95% for 0.1% FPPW on the INRIA dataset, which is significantly better than that of the original shapelet feature based approach and the existing detector ensemble approach. This approach can robustly detect pedestrians in different situations.

Pedestrian detection algorithm based on video sequences and laser point cloud

Pedestrian detection is a critical problem in the field of computer vision. Although most existing algorithms are able to detect pedestrians well in controlled environ- ments, it is often difficult to achieve accurate pedestrian de- tection from video sequences alone, especially in pedestrian- intensive scenes wherein pedestrians may cause mutual oc- clusion and thus incomplete detection. To surmount these dif- ficulties, this paper presents pedestrian detection algorithm based on video sequences and laser point cloud. First, laser point cloud is interpreted and classified to separate pedes- trian data and vehicle data. Then a fusion of video image data and laser point cloud data is achieved by calibration. The re- gion of interest after fusion is determined using feature in- formation contained in video image and three-dimensional information of laser point cloud to remove false detection of pedestrian and thus to achieve pedestrian detection in inten- sive scenes. Experimental verification and analysis in video sequences demonstrate that fusion of two data improves the performance of pedestrian detection and has better detection results.

Pedestrian Crossing Detection Based on HOG and SVM

In recent years,pedestrian detection is a hot research topic in the field of computer vision and artificial intelligence,it is widely used in the field of security and pedestrian analysis.However,due to a large amount of calculation in the traditional pedestrian detection technology,the speed of many systems for pedestrian recognition is very limited.But in some restricted areas,such as construction hazardous areas,real-time detection of pedestrians and cross-border behaviors is required.To more conveniently and efficiently detect whether there are pedestrians in the restricted area and cross-border behavior,this paper proposes a pedestrian cross-border detection method based on HOG(Histogram of Oriented Gradient)and SVM(Support Vector Machine).This method extracts the moving target through the GMM(Gaussian Mixture Model)background modeling and then extracts the characteristics of the moving target through gradient HOG.Finally,it uses SVM training to distinguish pedestrians from non-pedestrians,completes the detection of pedestrians,and labels the targets.The test results show that only the HOG feature extraction of the candidate area can greatly reduce the amount of calculation and reduce the time of feature extraction,eliminate background interference,thereby improving the efficiency of detection,and can be applied to occasions with real-time requirements.

Self-adaptive scale pedestrian detection algorithm based on deep residual network

Purpose–The conventional pedestrian detection algorithms lack in scale sensitivity.The purpose of this paper is to propose a novel algorithm of self-adaptive scale pedestrian detection,based on deep residual network(DRN),to address such lacks.Design/methodology/approach–First,the“Edge boxes”algorithm is introduced to extract region of interestsfrompedestrian images.Then,the extracted boundingboxesare incorporatedto differentDRNs,one is a large-scale DRN and the other one is the small-scale DRN.The height of the bounding boxes is used to classify the results of pedestrians and to regress the bounding boxes to the entity of the pedestrian.At last,a weighted self-adaptive scale function,which combines the large-scale results and small-scale results,is designed for the final pedestrian detection.Findings–Tovalidatetheeffectivenessandfeasibilityoftheproposedalgorithm,somecomparisonexperiments have been done on the common pedestrian detection data sets:Caltech,INRIA,ETH and KITTI.Experimental resultsshowthattheproposedalgorithmisadaptedforthevariousscalesofthepedestrians.Fortheharddetected small-scale pedestrians,the proposed algorithm has improved the accuracy and robustness of detections.Originality/value–By applying different models to deal with different scales of pedestrians,the proposed algorithm with the weighted calculation function has improved the accuracy and robustness for different scales of pedestrians.

A Real-Time Pedestrian Social Distancing Risk Alert System for COVID-19

The COVID-19 virus is usually spread by small droplets when talking,coughing and sneezing,so maintaining physical distance between people is necessary to slow the spread of the virus.The World Health Organization(WHO)recommends maintaining a social distance of at least six feet.In this paper,we developed a real-time pedestrian social distance risk alert system for COVID-19,whichmonitors the distance between people in real-time via video streaming and provides risk alerts to the person in charge,thus avoiding the problem of too close social distance between pedestrians in public places.We design a lightweight convolutional neural network architecture to detect the distance between people more accurately.In addition,due to the limitation of camera placement,the previous algorithm based on flat view is not applicable to the social distance calculation for cameras,so we designed and developed a perspective conversion module to reduce the image in the video to a bird’s eye view,which can avoid the error caused by the elevation view and thus provide accurate risk indication to the user.We selected images containing only person labels in theCOCO2017 dataset to train our networkmodel.The experimental results show that our network model achieves 82.3%detection accuracy and performs significantly better than other mainstream network architectures in the three metrics of Recall,Precision and mAP,proving the effectiveness of our system and the efficiency of our technology.

An Automated Player Detection and Tracking in Basketball Game

Vision-based player recognition is critical in sports applications.Accuracy,efficiency,and Low memory utilization is alluring for ongoing errands,for example,astute communicates and occasion classification.We developed an algorithm that tracks the movements of different players from a video of a basketball game.With their position tracked,we then proceed to map the position of these players onto an image of a basketball court.The purpose of tracking player is to provide the maximum amount of information to basketball coaches and organizations,so that they can better design mechanisms of defence and attack.Overall,our model has a high degree of identification and tracking of the players in the court.We directed investigations on soccer,basketball,ice hockey and pedestrian datasets.The trial comes about an exhibit that our technique can precisely recognize players under testing conditions.Contrasted and CNNs that are adjusted from general question identification systems,for example,Faster-RCNN,our approach accomplishes cutting edge exactness on three sorts of recreations(basketball,soccer and ice hockey)with 1000×fewer parameters.The all-inclusive statement of our technique is additionally shown on a standard passer-by recognition dataset in which our strategy accomplishes aggressive execution contrasted and cutting-edge methods.

CGTracker:Center Graph Network for One-Stage Multi-Pedestrian-Object Detection and Tracking

Most current online multi-object tracking(MOT)methods include two steps:object detection and data association,where the data association step relies on both object feature extraction and affinity computation.This often leads to additional computation cost,and degrades the efficiency of MOT methods.In this paper,we combine the object detection and data association module in a unified framework,while getting rid of the extra feature extraction process,to achieve a better speed-accuracy trade-off for MOT.Considering that a pedestrian is the most common object category in real-world scenes and has particularity characteristics in objects relationship and motion pattern,we present a novel yet efficient one-stage pedestrian detection and tracking method,named CGTracker.In particular,CGTracker detects the pedestrian target as the center point of the object,and directly extracts the object features from the feature representation of the object center point,which is used to predict the axis-aligned bounding box.Meanwhile,the detected pedestrians are constructed as an object graph to facilitate the multi-object association process,where the semantic features,displacement information and relative position relationship of the targets between two adjacent frames are used to perform the reliable online tracking.CGTracker achieves the multiple object tracking accuracy(MOTA)of 69.3%and 65.3%at 9 FPS on MOT17 and MOT20,respectively.Extensive experimental results under widely-used evaluation metrics demonstrate that our method is one of the best techniques on the leader board for the MOT17 and MOT20 challenges at the time of submission of this work.