A robust system for real-time pedestrian detection and tracking

A real-time pedestrian detection and tracking system using a single video camera was developed to monitor pedestrians. This system contained six modules: video flow capture, pre-processing, movement detection, shadow removal, tracking, and object classification. The Gaussian mixture model was utilized to extract the moving object from an image sequence segmented by the mean-shift technique in the pre-processing module. Shadow removal was used to alleviate the negative impact of the shadow to the detected objects. A model-free method was adopted to identify pedestrians. The maximum and minimum integration methods were developed to integrate multiple cues into the mean-shift algorithm and the initial tracking iteration with the competent integrated probability distribution map for object tracking. A simple but effective algorithm was proposed to handle full occlusion cases. The system was tested using real traffic videos from different sites. The results of the test confirm that the system is reliable and has an overall accuracy of over 85%.

Real-Time Detection and Instance Segmentation of Strawberry in Unstructured Environment

The real-time detection and instance segmentation of strawberries constitute fundamental components in the development of strawberry harvesting robots.Real-time identification of strawberries in an unstructured envi-ronment is a challenging task.Current instance segmentation algorithms for strawberries suffer from issues such as poor real-time performance and low accuracy.To this end,the present study proposes an Efficient YOLACT(E-YOLACT)algorithm for strawberry detection and segmentation based on the YOLACT framework.The key enhancements of the E-YOLACT encompass the development of a lightweight attention mechanism,pyramid squeeze shuffle attention(PSSA),for efficient feature extraction.Additionally,an attention-guided context-feature pyramid network(AC-FPN)is employed instead of FPN to optimize the architecture’s performance.Furthermore,a feature-enhanced model(FEM)is introduced to enhance the prediction head’s capabilities,while efficient fast non-maximum suppression(EF-NMS)is devised to improve non-maximum suppression.The experimental results demonstrate that the E-YOLACT achieves a Box-mAP and Mask-mAP of 77.9 and 76.6,respectively,on the custom dataset.Moreover,it exhibits an impressive category accuracy of 93.5%.Notably,the E-YOLACT also demonstrates a remarkable real-time detection capability with a speed of 34.8 FPS.The method proposed in this article presents an efficient approach for the vision system of a strawberry-picking robot.

Real-Time Object Detection and Face Recognition Application for the Visually Impaired

The advancement of navigation systems for the visually impaired has significantly enhanced their mobility by mitigating the risk of encountering obstacles and guiding them along safe,navigable routes.Traditional approaches primarily focus on broad applications such as wayfinding,obstacle detection,and fall prevention.However,there is a notable discrepancy in applying these technologies to more specific scenarios,like identifying distinct food crop types or recognizing faces.This study proposes a real-time application designed for visually impaired individuals,aiming to bridge this research-application gap.It introduces a system capable of detecting 20 different food crop types and recognizing faces with impressive accuracies of 83.27%and 95.64%,respectively.These results represent a significant contribution to the field of assistive technologies,providing visually impaired users with detailed and relevant information about their surroundings,thereby enhancing their mobility and ensuring their safety.Additionally,it addresses the vital aspects of social engagements,acknowledging the challenges faced by visually impaired individuals in recognizing acquaintances without auditory or tactile signals,and highlights recent developments in prototype systems aimed at assisting with face recognition tasks.This comprehensive approach not only promises enhanced navigational aids but also aims to enrich the social well-being and safety of visually impaired communities.

Analyzing the Impact of Scene Transitions on Indoor Camera Localization through Scene Change Detection in Real-Time

Real-time indoor camera localization is a significant problem in indoor robot navigation and surveillance systems.The scene can change during the image sequence and plays a vital role in the localization performance of robotic applications in terms of accuracy and speed.This research proposed a real-time indoor camera localization system based on a recurrent neural network that detects scene change during the image sequence.An annotated image dataset trains the proposed system and predicts the camera pose in real-time.The system mainly improved the localization performance of indoor cameras by more accurately predicting the camera pose.It also recognizes the scene changes during the sequence and evaluates the effects of these changes.This system achieved high accuracy and real-time performance.The scene change detection process was performed using visual rhythm and the proposed recurrent deep architecture,which performed camera pose prediction and scene change impact evaluation.Overall,this study proposed a novel real-time localization system for indoor cameras that detects scene changes and shows how they affect localization performance.

A YOLOv8-CE-based real-time traffic sign detection and identification method for autonomous vehicles

Traffic sign detection in real scenarios is challenging due to their complexity and small size,often preventing existing deep learning models from achieving both high accuracy and real-time performance.An improved YOLOv8 model for traffic sign detection is proposed.Firstly,by adding Coordinate Attention(CA)to the Backbone,the model gains location information,improving detection accuracy.Secondly,we also introduce EIoU to the localization function to address the ambiguity in aspect ratio descriptions by calculating the width-height difference based on CIoU.Additionally,Focal Loss is incorporated to balance sample difficulty,enhancing regression accuracy.Finally,the model,YOLOv8-CE(YOLOv8-Coordinate Attention-EIoU),is tested on the Jetson Nano,achieving real-time street scene detection and outperforming the Raspberry Pi 4B.Experimental results show that YOLOv8-CE excels in various complex scenarios,improving mAP by 2.8%over the original YOLOv8.The model size and computational effort remain similar,with the Jetson Nano achieving an inference time of 96 ms,significantly faster than the Raspberry Pi 4B.

A CNN-Based Single-Stage Occlusion Real-Time Target Detection Method

Aiming at the problem of low accuracy of traditional target detection methods for target detection in endoscopes in substation environments, a CNN-based real-time detection method for masked targets is proposed. The method adopts the overall design of backbone network, detection network and algorithmic parameter optimisation method, completes the model training on the self-constructed occlusion target dataset, and adopts the multi-scale perception method for target detection. The HNM algorithm is used to screen positive and negative samples during the training process, and the NMS algorithm is used to post-process the prediction results during the detection process to improve the detection efficiency. After experimental validation, the obtained model has the multi-class average predicted value (mAP) of the dataset. It has general advantages over traditional target detection methods. The detection time of a single target on FDDB dataset is 39 ms, which can meet the need of real-time target detection. In addition, the project team has successfully deployed the method into substations and put it into use in many places in Beijing, which is important for achieving the anomaly of occlusion target detection.

Lightweight Cross-Modal Multispectral Pedestrian Detection Based on Spatial Reweighted Attention Mechanism

Multispectral pedestrian detection technology leverages infrared images to provide reliable information for visible light images, demonstrating significant advantages in low-light conditions and background occlusion scenarios. However, while continuously improving cross-modal feature extraction and fusion, ensuring the model’s detection speed is also a challenging issue. We have devised a deep learning network model for cross-modal pedestrian detection based on Resnet50, aiming to focus on more reliable features and enhance the model’s detection efficiency. This model employs a spatial attention mechanism to reweight the input visible light and infrared image data, enhancing the model’s focus on different spatial positions and sharing the weighted feature data across different modalities, thereby reducing the interference of multi-modal features. Subsequently, lightweight modules with depthwise separable convolution are incorporated to reduce the model’s parameter count and computational load through channel-wise and point-wise convolutions. The network model algorithm proposed in this paper was experimentally validated on the publicly available KAIST dataset and compared with other existing methods. The experimental results demonstrate that our approach achieves favorable performance in various complex environments, affirming the effectiveness of the multispectral pedestrian detection technology proposed in this paper.

An Application of RGBD-Based Skeleton Reconstruction for Pedestrian Detection and Occlusion Handling

This study explores the challenges posed by pedestrian detection and occlusion in AR applications, employing a novel approach that utilizes RGB-D-based skeleton reconstruction to reduce the overhead of classical pedestrian detection algorithms during training. Furthermore, it is dedicated to addressing occlusion issues in pedestrian detection by using Azure Kinect for body tracking and integrating a robust occlusion management algorithm, significantly enhancing detection efficiency. In experiments, an average latency of 204 milliseconds was measured, and the detection accuracy reached an outstanding level of 97%. Additionally, this approach has been successfully applied in creating a simple yet captivating augmented reality game, demonstrating the practical application of the algorithm.

LDA-ID:An LDA-Based Framework for Real-Time Network Intrusion Detection

Network intrusion poses a severe threat to the Internet.However,existing intrusion detection models cannot effectively distinguish different intrusions with high-degree feature overlap.In addition,efficient real-time detection is an urgent problem.To address the two above problems,we propose a Latent Dirichlet Allocation topic model-based framework for real-time network Intrusion Detection(LDA-ID),consisting of static and online LDA-ID.The problem of feature overlap is transformed into static LDA-ID topic number optimization and topic selection.Thus,the detection is based on the latent topic features.To achieve efficient real-time detection,we design an online computing mode for static LDA-ID,in which a parameter iteration method based on momentum is proposed to balance the contribution of prior knowledge and new information.Furthermore,we design two matching mechanisms to accommodate the static and online LDA-ID,respectively.Experimental results on the public NSL-KDD and UNSW-NB15 datasets show that our framework gets higher accuracy than the others.

Portable FBAR based E-nose for cold chain real-time bananas shelf time detection

Being cheap,nondestructive,and easy to use,gas sensors play important roles in the food industry.However,most gas sensors are suitable more for laboratory-quality fast testing rather than for cold-chain continuous and cumulative testing.Also,an ideal electronic nose(E-nose)in a cold chain should be stable to its surroundings and remain highly accurate and portable.In this work,a portable film bulk acoustic resonator(FBAR)-based E-nose was built for real-time measurement of banana shelf time.The sensor chamber to contain the portable circuit of the E-nose is as small as a smartphone,and by introducing an air-tight FBAR as a reference,the E-nose can avoid most of the drift caused by surroundings.With the help of porous layer by layer(LBL)coating of the FBAR,the sensitivity of the E-nose is 5 ppm to ethylene and 0.5 ppm to isoamyl acetate and isoamyl butyrate,while the detection range is large enough to cover a relative humidity of 0.8.In this regard,the E-nose can easily discriminate between yellow bananas with green necks and entirely yellow bananas while allowing the bananas to maintain their biological activities in their normal storage state,thereby showing the possibility of real-time shelf time detection.This portable FBAR-based E-nose has a large testing scale,high sensitivity,good humidity tolerance,and low frequency drift to its surroundings,thereby meeting the needs of cold-chain usage.

Force Sensitive Resistors-Based Real-Time Posture Detection System Using Machine Learning Algorithms

To detect the improper sitting posture of a person sitting on a chair,a posture detection system using machine learning classification has been proposed in this work.The addressed problem correlates to the third Sustainable Development Goal(SDG),ensuring healthy lives and promoting well-being for all ages,as specified by the World Health Organization(WHO).An improper sitting position can be fatal if one sits for a long time in the wrong position,and it can be dangerous for ulcers and lower spine discomfort.This novel study includes a practical implementation of a cushion consisting of a grid of 3×3 force-sensitive resistors(FSR)embedded to read the pressure of the person sitting on it.Additionally,the Body Mass Index(BMI)has been included to increase the resilience of the system across individual physical variances and to identify the incorrect postures(backward,front,left,and right-leaning)based on the five machine learning algorithms:ensemble boosted trees,ensemble bagged trees,ensemble subspace K-Nearest Neighbors(KNN),ensemble subspace discriminant,and ensemble RUSBoosted trees.The proposed arrangement is novel as existing works have only provided simulations without practical implementation,whereas we have implemented the proposed design in Simulink.The results validate the proposed sensor placements,and the machine learning(ML)model reaches a maximum accuracy of 99.99%,which considerably outperforms the existing works.The proposed concept is valuable as it makes it easier for people in workplaces or even at individual household levels to work for long periods without suffering from severe harmful effects from poor posture.

Towards Cache-Assisted Hierarchical Detection for Real-Time Health Data Monitoring in IoHT

Real-time health data monitoring is pivotal for bolstering road services’safety,intelligence,and efficiency within the Internet of Health Things(IoHT)framework.Yet,delays in data retrieval can markedly hinder the efficacy of big data awareness detection systems.We advocate for a collaborative caching approach involving edge devices and cloud networks to combat this.This strategy is devised to streamline the data retrieval path,subsequently diminishing network strain.Crafting an adept cache processing scheme poses its own set of challenges,especially given the transient nature of monitoring data and the imperative for swift data transmission,intertwined with resource allocation tactics.This paper unveils a novel mobile healthcare solution that harnesses the power of our collaborative caching approach,facilitating nuanced health monitoring via edge devices.The system capitalizes on cloud computing for intricate health data analytics,especially in pinpointing health anomalies.Given the dynamic locational shifts and possible connection disruptions,we have architected a hierarchical detection system,particularly during crises.This system caches data efficiently and incorporates a detection utility to assess data freshness and potential lag in response times.Furthermore,we introduce the Cache-Assisted Real-Time Detection(CARD)model,crafted to optimize utility.Addressing the inherent complexity of the NP-hard CARD model,we have championed a greedy algorithm as a solution.Simulations reveal that our collaborative caching technique markedly elevates the Cache Hit Ratio(CHR)and data freshness,outshining its contemporaneous benchmark algorithms.The empirical results underscore the strength and efficiency of our innovative IoHT-based health monitoring solution.To encapsulate,this paper tackles the nuances of real-time health data monitoring in the IoHT landscape,presenting a joint edge-cloud caching strategy paired with a hierarchical detection system.Our methodology yields enhanced cache efficiency and data freshness.The corroborative numerical data accentuates the feasibility and relevance of our model,casting a beacon for the future trajectory of real-time health data monitoring systems.

Enhancing Fire Detection Performance Based on Fine-Tuned YOLOv10

In recent years,early detection and warning of fires have posed a significant challenge to environmental protection and human safety.Deep learning models such as Faster R-CNN(Faster Region based Convolutional Neural Network),YOLO(You Only Look Once),and their variants have demonstrated superiority in quickly detecting objects from images and videos,creating new opportunities to enhance automatic and efficient fire detection.The YOLO model,especially newer versions like YOLOv10,stands out for its fast processing capability,making it suitable for low-latency applications.However,when applied to real-world datasets,the accuracy of fire prediction is still not high.This study improves the accuracy of YOLOv10 for real-time applications through model fine-tuning techniques and data augmentation.The core work of the research involves creating a diverse fire image dataset specifically suited for fire detection applications in buildings and factories,freezing the initial layers of the model to retain general features learned from the dataset by applying the Squeeze and Excitation attention mechanism and employing the Stochastic Gradient Descent(SGD)with a momentum optimization algorithm to enhance accuracy while ensuring real-time fire detection.Experimental results demonstrate the effectiveness of the proposed fire prediction approach,where the YOLOv10 small model exhibits the best balance compared to other YOLO family models such as nano,medium,and balanced.Additionally,the study provides an experimental evaluation to highlight the effectiveness of model fine-tuning compared to the YOLOv10 baseline,YOLOv8 and Faster R-CNN based on two criteria:accuracy and prediction time.

NFA:A neural factorization autoencoder based online telephony fraud detection

The proliferation of internet communication channels has increased telecom fraud,causing billions of euros in losses for customers and the industry each year.Fraudsters constantly find new ways to engage in illegal activity on the network.To reduce these losses,a new fraud detection approach is required.Telecom fraud detection involves identifying a small number of fraudulent calls from a vast amount of call traffic.Developing an effective strategy to combat fraud has become challenging.Although much effort has been made to detect fraud,most existing methods are designed for batch processing,not real-time detection.To solve this problem,we propose an online fraud detection model using a Neural Factorization Autoencoder(NFA),which analyzes customer calling patterns to detect fraudulent calls.The model employs Neural Factorization Machines(NFM)and an Autoencoder(AE)to model calling patterns and a memory module to adapt to changing customer behaviour.We evaluate our approach on a large dataset of real-world call detail records and compare it with several state-of-the-art methods.Our results show that our approach outperforms the baselines,with an AUC of 91.06%,a TPR of 91.89%,an FPR of 14.76%,and an F1-score of 95.45%.These results demonstrate the effectiveness of our approach in detecting fraud in real-time and suggest that it can be a valuable tool for preventing fraud in telecommunications networks.

Real-Time Tunable Gas Sensing Platform Based on SnO_(2) Nanoparticles Activated by Blue Micro-Light-Emitting Diodes

Micro-light-emitting diodes(μLEDs)have gained significant interest as an activation source for gas sensors owing to their advantages,including room temperature operation and low power consumption.However,despite these benefits,challenges still exist such as a limited range of detectable gases and slow response.In this study,we present a blueμLED-integrated light-activated gas sensor array based on SnO_(2)nanoparticles(NPs)that exhibit excellent sensitivity,tunable selectivity,and rapid detection with micro-watt level power consumption.The optimal power forμLED is observed at the highest gas response,supported by finite-difference time-domain simulation.Additionally,we first report the visible light-activated selective detection of reducing gases using noble metal-decorated SnO_(2)NPs.The noble metals induce catalytic interaction with reducing gases,clearly distinguishing NH3,H2,and C2H5OH.Real-time gas monitoring based on a fully hardwareimplemented light-activated sensing array was demonstrated,opening up new avenues for advancements in light-activated electronic nose technologies.

Virtual Assembly Collision Detection Algorithm Using Backpropagation Neural Network

As computer graphics technology continues to advance,Collision Detection(CD)has emerged as a critical element in fields such as virtual reality,computer graphics,and interactive simulations.CD is indispensable for ensuring the fidelity of physical interactions and the realism of virtual environments,particularly within complex scenarios like virtual assembly,where both high precision and real-time responsiveness are imperative.Despite ongoing developments,current CD techniques often fall short in meeting these stringent requirements,resulting in inefficiencies and inaccuracies that impede the overall performance of virtual assembly systems.To address these limitations,this study introduces a novel algorithm that leverages the capabilities of a Backpropagation Neural Network(BPNN)to optimize the structural composition of the Hybrid Bounding Volume Tree(HBVT).Through this optimization,the research proposes a refined Hybrid Hierarchical Bounding Box(HHBB)framework,which is specifically designed to enhance the computational efficiency and precision of CD processes.The HHBB framework strategically reduces the complexity of collision detection computations,thereby enabling more rapid and accurate responses to collision events.Extensive experimental validation within virtual assembly environments reveals that the proposed algorithm markedly improves the performance of CD,particularly in handling complex models.The optimized HBVT architecture not only accelerates the speed of collision detection but also significantly diminishes error rates,presenting a robust and scalable solution for real-time applications in intricate virtual systems.These findings suggest that the proposed approach offers a substantial advancement in CD technology,with broad implications for its application in virtual reality,computer graphics,and related fields.

Multiple Pedestrian Detection and Tracking in Night Vision Surveillance Systems

Pedestrian detection and tracking are vital elements of today’s surveillance systems,which make daily life safe for humans.Thus,human detection and visualization have become essential inventions in the field of computer vision.Hence,developing a surveillance system with multiple object recognition and tracking,especially in low light and night-time,is still challenging.Therefore,we propose a novel system based on machine learning and image processing to provide an efficient surveillance system for pedestrian detection and tracking at night.In particular,we propose a system that tackles a two-fold problem by detecting multiple pedestrians in infrared(IR)images using machine learning and tracking them using particle filters.Moreover,a random forest classifier is adopted for image segmentation to identify pedestrians in an image.The result of detection is investigated by particle filter to solve pedestrian tracking.Through the extensive experiment,our system shows 93%segmentation accuracy using a random forest algorithm that demonstrates high accuracy for background and roof classes.Moreover,the system achieved a detection accuracy of 90%usingmultiple templatematching techniques and 81%accuracy for pedestrian tracking.Furthermore,our system can identify that the detected object is a human.Hence,our system provided the best results compared to the state-ofart systems,which proves the effectiveness of the techniques used for image segmentation,classification,and tracking.The presented method is applicable for human detection/tracking,crowd analysis,and monitoring pedestrians in IR video surveillance.

Pedestrian and Vehicle Detection Based on Pruning YOLOv4 with Cloud-Edge Collaboration

Nowadays,the rapid development of edge computing has driven an increasing number of deep learning applications deployed at the edge of the network,such as pedestrian and vehicle detection,to provide efficient intelligent services to mobile users.However,as the accuracy requirements continue to increase,the components of deep learning models for pedestrian and vehicle detection,such as YOLOv4,become more sophisticated and the computing resources required for model training are increasing dramatically,which in turn leads to significant challenges in achieving effective deployment on resource-constrained edge devices while ensuring the high accuracy performance.For addressing this challenge,a cloud-edge collaboration-based pedestrian and vehicle detection framework is proposed in this paper,which enables sufficient training of models by utilizing the abundant computing resources in the cloud,and then deploying the well-trained models on edge devices,thus reducing the computing resource requirements for model training on edge devices.Furthermore,to reduce the size of the model deployed on edge devices,an automatic pruning method combines the convolution layer and BN layer is proposed to compress the pedestrian and vehicle detection model size.Experimental results show that the framework proposed in this paper is able to deploy the pruned model on a real edge device,Jetson TX2,with 6.72 times higher FPS.Meanwhile,the channel pruning reduces the volume and the number of parameters to 96.77%for the model,and the computing amount is reduced to 81.37%.

Segmentation Based Real Time Anomaly Detection and Tracking Model for Pedestrian Walkways

Presently,video surveillance is commonly employed to ensure security in public places such as traffic signals,malls,railway stations,etc.A major chal-lenge in video surveillance is the identification of anomalies that exist in it such as crimes,thefts,and so on.Besides,the anomaly detection in pedestrian walkways has gained significant attention among the computer vision communities to enhance pedestrian safety.The recent advances of Deep Learning(DL)models have received considerable attention in different processes such as object detec-tion,image classification,etc.In this aspect,this article designs a new Panoptic Feature Pyramid Network based Anomaly Detection and Tracking(PFPN-ADT)model for pedestrian walkways.The proposed model majorly aims to the recognition and classification of different anomalies present in the pedestrian walkway like vehicles,skaters,etc.The proposed model involves panoptic seg-mentation model,called Panoptic Feature Pyramid Network(PFPN)is employed for the object recognition process.For object classification,Compact Bat Algo-rithm(CBA)with Stacked Auto Encoder(SAE)is applied for the classification of recognized objects.For ensuring the enhanced results better anomaly detection performance of the PFPN-ADT technique,a comparison study is made using Uni-versity of California San Diego(UCSD)Anomaly data and other benchmark data-sets(such as Cityscapes,ADE20K,COCO),and the outcomes are compared with the Mask Recurrent Convolutional Neural Network(RCNN)and Faster Convolu-tional Neural Network(CNN)models.The simulation outcome demonstrated the enhanced performance of the PFPN-ADT technique over the other methods.

A Real-Time Pedestrian Social Distancing Risk Alert System for COVID-19

The COVID-19 virus is usually spread by small droplets when talking,coughing and sneezing,so maintaining physical distance between people is necessary to slow the spread of the virus.The World Health Organization(WHO)recommends maintaining a social distance of at least six feet.In this paper,we developed a real-time pedestrian social distance risk alert system for COVID-19,whichmonitors the distance between people in real-time via video streaming and provides risk alerts to the person in charge,thus avoiding the problem of too close social distance between pedestrians in public places.We design a lightweight convolutional neural network architecture to detect the distance between people more accurately.In addition,due to the limitation of camera placement,the previous algorithm based on flat view is not applicable to the social distance calculation for cameras,so we designed and developed a perspective conversion module to reduce the image in the video to a bird’s eye view,which can avoid the error caused by the elevation view and thus provide accurate risk indication to the user.We selected images containing only person labels in theCOCO2017 dataset to train our networkmodel.The experimental results show that our network model achieves 82.3%detection accuracy and performs significantly better than other mainstream network architectures in the three metrics of Recall,Precision and mAP,proving the effectiveness of our system and the efficiency of our technology.