A Deep Learning-Based Crowd Counting Method and System Implementation on Neural Processing Unit Platform

In this paper, a deep learning-based method is proposed for crowdcountingproblems. Specifically, by utilizing the convolution kernel densitymap, the ground truth is generated dynamically to enhance the featureextractingability of the generator model. Meanwhile, the “cross stage partial”module is integrated into congested scene recognition network (CSRNet) toobtain a lightweight network model. In addition, to compensate for the accuracydrop owing to the lightweight model, we take advantage of “structuredknowledge transfer” to train the model in an end-to-end manner. It aimsto accelerate the fitting speed and enhance the learning ability of the studentmodel. The crowd-counting system solution for edge computing is alsoproposed and implemented on an embedded device equipped with a neuralprocessing unit. Simulations demonstrate the performance improvement ofthe proposed solution in terms of model size, processing speed and accuracy.The performance on the Venice dataset shows that the mean absolute error(MAE) and the root mean squared error (RMSE) of our model drop by32.63% and 39.18% compared with CSRNet. Meanwhile, the performance onthe ShanghaiTech PartB dataset reveals that the MAE and the RMSE of ourmodel are close to those of CSRNet. Therefore, we provide a novel embeddedplatform system scheme for public safety pre-warning applications.

Robust Counting in Overcrowded Scenes Using Batch-Free Normalized Deep ConvNet

The analysis of overcrowded areas is essential for flow monitoring,assembly control,and security.Crowd counting’s primary goal is to calculate the population in a given region,which requires real-time analysis of congested scenes for prompt reactionary actions.The crowd is always unexpected,and the benchmarked available datasets have a lot of variation,which limits the trained models’performance on unseen test data.In this paper,we proposed an end-to-end deep neural network that takes an input image and generates a density map of a crowd scene.The proposed model consists of encoder and decoder networks comprising batch-free normalization layers known as evolving normalization(EvoNorm).This allows our network to be generalized for unseen data because EvoNorm is not using statistics from the training samples.The decoder network uses dilated 2D convolutional layers to provide large receptive fields and fewer parameters,which enables real-time processing and solves the density drift problem due to its large receptive field.Five benchmark datasets are used in this study to assess the proposed model,resulting in the conclusion that it outperforms conventional models.

Multi-Scale Network with Integrated Attention Unit for Crowd Counting

Estimating the crowd count and density of highly dense scenes witnessed in Muslim gatherings at religious sites in Makkah and Madinah is critical for developing control strategies and organizing such a large gathering.Moreover,since the crowd images in this case can range from low density to high density,detection-based approaches are hard to apply for crowd counting.Recently,deep learning-based regression has become the prominent approach for crowd counting problems,where a density-map is estimated,and its integral is further computed to acquire the final count result.In this paper,we put forward a novel multi-scale network(named 2U-Net)for crowd counting in sparse and dense scenarios.The proposed framework,which employs the U-Net architecture,is straightforward to implement,computationally efficient,and has single-step training.Unpooling layers are used to retrieve the pooling layers’erased information and learn hierarchically pixelwise spatial representation.This helps in obtaining feature values,retaining spatial locations,and maximizing data integrity to avoid data loss.In addition,a modified attention unit is introduced and integrated into the proposed 2UNet model to focus on specific crowd areas.The proposed model concentrates on balancing the number of model parameters,model size,computational cost,and counting accuracy compared with other works,which may involve acquiring one criterion at the expense of other constraints.Experiments on five challenging datasets for density estimation and crowd counting have shown that the proposed model is very effective and outperforms comparable mainstream models.Moreover,it counts very well in both sparse and congested crowd scenes.The 2U-Net model has the lowest MAE in both parts(Part A and Part B)of the ShanghaiTech,UCSD,and Mall benchmarks,with 63.3,7.4,1.5,and 1.6,respectively.Furthermore,it obtains the lowest MSE in the ShanghaiTech-Part B,UCSD,and Mall benchmarks with 12.0,1.9,and 2.1,respectively.

A Deep Learning Approach for Crowd Counting in Highly Congested Scene

With the rapid progress of deep convolutional neural networks,several applications of crowd counting have been proposed and explored in the literature.In congested scene monitoring,a variety of crowd density estimating approaches has been developed.The understanding of highly congested scenes for crowd counting during Muslim gatherings of Hajj and Umrah is a challenging task,as a large number of individuals stand nearby and,it is hard for detection techniques to recognize them,as the crowd can vary from low density to high density.To deal with such highly congested scenes,we have proposed the Congested Scene Crowd Counting Network(CSCC-Net)using VGG-16 as a core network with its first ten layers due to its strong and robust transfer learning rate.A hole dilated convolutional neural network is used at the back end to widen the relevant field to extract a large range of information from the image without losing its original resolution.The dilated convolution neural network is mainly chosen to expand the kernel size without changing other parameters.Moreover,several loss functions have been applied to strengthen the evaluation accuracy of the model.Finally,the entire experiments have been evaluated using prominent data sets namely,ShanghaiTech parts A,B,UCF_CC_50,and UCF_QNRF.Our model has achieved remarkable results i.e.,68.0 and 9.0 MAE on ShanghaiTech parts A,B,199.1 MAE on UCF_CC_50,and 99.8 on UCF_QNRF data sets respectively.

Adaptive Scheme for Crowd Counting Using off-the-Shelf Wireless Routers

Since the outbreak of the world-wide novel coronavirus pandemic,crowd counting in public areas,such as in shopping centers and in commercial streets,has gained popularity among public health administrations for preventing the crowds from gathering.In this paper,we propose a novel adaptive method for crowd counting based on Wi-Fi channel state information(CSI)by using common commercial wireless routers.Compared with previous researches on device-free crowd counting,our proposed method is more adaptive to the change of environ-ment and can achieve high accuracy of crowd count estimation.Because the dis-tance between access point(AP)and monitor point(MP)is typically non-fixed in real-world applications,the strength of received signals varies and makes the tra-ditional amplitude-related models to perform poorly in different environments.In order to achieve adaptivity of the crowd count estimation model,we used convo-lutional neural network(ConvNet)to extract features from correlation coefficient matrix of subcarriers which are insensitive to the change of received signal strength.We conducted experiments in university classroom settings and our model achieved an overall accuracy of 97.79%in estimating a variable number of participants.

A Deep-CNN Crowd Counting Model for Enforcing Social Distancing during COVID19 Pandemic: Application to Saudi Arabia’s Public Places

With the emergence of the COVID19 virus in late 2019 and the declaration that the virus is a worldwide pandemic,health organizations and governments have begun to implement severe health precautions to reduce the spread of the virus and preserve human lives.The enforcement of social distancing at work environments and public areas is one of these obligatory precautions.Crowd management is one of the effective measures for social distancing.By reducing the social contacts of individuals,the spread of the disease will be immensely reduced.In this paper,a model for crowd counting in public places of high and low densities is proposed.The model works under various scene conditions and with no prior knowledge.A Deep CNN model(DCNN)is built based on convolutional neural network(CNN)structure with small kernel size and two fronts.To increase the efficiency of the model,a convolutional neural network(CNN)as the front-end and a multi-column layer with Dilated Convolution as the back-end were chosen.Also,the proposed method accepts images of arbitrary sizes/scales as inputs from different cameras.To evaluate the proposed model,a dataset was created from images of Saudi people with traditional and non-traditional Saudi outfits.The model was also trained and tested on some existing datasets.Compared to current counting methods,the results show that the proposed model has significantly improved efficiency and reduced the error rate.We achieve the lowest MAE by 67%,32%.and 15.63%and lowest MSE by around 47%,15%and 8.1%than M-CNN,Cascaded-MTL,and CSRNet respectively.

Crowd Counting for Real Monitoring Scene

Crowd counting is a challenging task in computer vision as realistic scenes are al?ways filled with unfavourable factors such as severe occlusions, perspective distortions and di?verse distributions. Recent state-of-the-art methods based on convolutional neural network (CNN) weaken these factors via multi-scale feature fusion or optimal feature selection through a front switch-net. L2 regression is used to regress the density map of the crowd, which is known to lead to an average and blurry result, and affects the accuracy of crowd count and po?sition distribution. To tackle these problems, we take full advantage of the application of gen?erative adversarial networks (GANs) in image generation and propose a novel crowd counting model based on conditional GANs to predict high-quality density maps from crowd images. Furthermore, we innovatively put forward a new regularizer so as to help boost the accuracy of processing extremely crowded scenes. Extensive experiments on four major crowd counting datasets are conducted to demonstrate the better performance of the proposed approach com?pared with recent state-of-the-art methods.

Crowd Counting Based on WiFi Channel State Information and Transfer Learning

With the popularity and development of indoor WiFi equipment, they have more sensing capability and can be used as a human monitoring device. We can collect the channel state information (CSI) from WiFi device and acquire the human state based on the measurements. These studies have attracted wide attention and become a hot research topic. This paper concentrated on the crowd counting based on CSI and transfer learning. We utilized the CSI signal fluctuations caused by human motion in WiFi coverage to identify the person count because different person counts would lead to unique signal propagation characteristics. First, this paper presented recent studies of crowd counting based on CSI. Then, we introduced the basic concept of CSI, and described the fundamental principle of CSI-based crowd counting. We also presented the system framework, experiment scenario, and neural network structure transferred from the ResNet. Next, we presented the experiment results and compared the accuracy using different neural network models. The system achieved recognition accuracy of this 100 percent for seven participants using the transfer learning technique. Finally, we concluded the paper by discussing the current problems and future work.

Lightweight Res-Connection Multi-Branch Network for Highly Accurate Crowd Counting and Localization

Crowd counting is a promising hotspot of computer vision involving crowd intelligence analysis,achieving tremendous success recently with the development of deep learning.However,there have been stillmany challenges including crowd multi-scale variations and high network complexity,etc.To tackle these issues,a lightweight Resconnection multi-branch network(LRMBNet)for highly accurate crowd counting and localization is proposed.Specifically,using improved ShuffleNet V2 as the backbone,a lightweight shallow extractor has been designed by employing the channel compression mechanism to reduce enormously the number of network parameters.A light multi-branch structure with different expansion rate convolutions is demonstrated to extract multi-scale features and enlarged receptive fields,where the information transmission and fusion of diverse scale features is enhanced via residual concatenation.In addition,a compound loss function is introduced for training themethod to improve global context information correlation.The proposed method is evaluated on the SHHA,SHHB,UCF-QNRF and UCF_CC_50 public datasets.The accuracy is better than those of many advanced approaches,while the number of parameters is smaller.The experimental results show that the proposed method achieves a good tradeoff between the complexity and accuracy of crowd counting,indicating a lightweight and high-precision method for crowd counting.

DTCC:Multi-level dilated convolution with transformer for weakly-supervised crowd counting

Crowd counting provides an important foundation for public security and urban management.Due to the existence of small targets and large density variations in crowd images,crowd counting is a challenging task.Mainstream methods usually apply convolution neural networks(CNNs)to regress a density map,which requires annotations of individual persons and counts.Weakly-supervised methods can avoid detailed labeling and only require counts as annotations of images,but existing methods fail to achieve satisfactory performance because a global perspective field and multi-level information are usually ignored.We propose a weakly-supervised method,DTCC,which effectively combines multi-level dilated convolution and transformer methods to realize end-to-end crowd counting.Its main components include a recursive swin transformer and a multi-level dilated convolution regression head.The recursive swin transformer combines a pyramid visual transformer with a fine-tuned recursive pyramid structure to capture deep multi-level crowd features,including global features.The multi-level dilated convolution regression head includes multi-level dilated convolution and a linear regression head for the feature extraction module.This module can capture both low-and high-level features simultaneously to enhance the receptive field.In addition,two regression head fusion mechanisms realize dynamic and mean fusion counting.Experiments on four well-known benchmark crowd counting datasets(UCF_CC_50,ShanghaiTech,UCF_QNRF,and JHU-Crowd++)show that DTCC achieves results superior to other weakly-supervised methods and comparable to fully-supervised methods.

Scene-adaptive crowd counting method based on meta learning with dual-input network DMNet

Crowd counting is recently becoming a hot research topic, which aims to count the number of the people in different crowded scenes. Existing methods are mainly based on training-testing pattern and rely on large data training, which fails to accurately count the crowd in real-world scenes because of the limitation of model’s generalization capability. To alleviate this issue, a scene-adaptive crowd counting method based on meta-learning with Dual-illumination Merging Network (DMNet) is proposed in this paper. The proposed method based on learning-to-learn and few-shot learning is able to adapt different scenes which only contain a few labeled images. To generate high quality density map and count the crowd in low-lighting scene, the DMNet is proposed, which contains Multi-scale Feature Extraction module and Element-wise Fusion Module. The Multi-scale Feature Extraction module is used to extract the image feature by multi-scale convolutions, which helps to improve network accuracy. The Element-wise Fusion module fuses the low-lighting feature and illumination-enhanced feature, which supplements the missing illumination in low-lighting environments. Experimental results on benchmarks, WorldExpo’10, DISCO, USCD, and Mall, show that the proposed method outperforms the existing state-of-the-art methods in accuracy and gets satisfied results.

Forget less,count better:a domain-incremental self-distillation learning benchmark for lifelong crowd counting

Crowd counting has important applications in public safety and pandemic control.A robust and practical crowd counting system has to be capable of continuously learning with the newly incoming domain data in real-world scenarios instead of fitting one domain only.Off-the-shelf methods have some drawbacks when handling multiple domains:(1)the models will achieve limited performance(even drop dramatically)among old domains after training images from new domains due to the discrepancies in intrinsic data distributions from various domains,which is called catastrophic forgetting;(2)the well-trained model in a specific domain achieves imperfect performance among other unseen domains because of domain shift;(3)it leads to linearly increasing storage overhead,either mixing all the data for training or simply training dozens of separate models for different domains when new ones are available.To overcome these issues,we investigate a new crowd counting task in incremental domain training setting called lifelong crowd counting.Its goal is to alleviate catastrophic forgetting and improve the generalization ability using a single model updated by the incremental domains.Specifically,we propose a self-distillation learning framework as a benchmark(forget less,count better,or FLCB)for lifelong crowd counting,which helps the model leverage previous meaningful knowledge in a sustainable manner for better crowd counting to mitigate the forgetting when new data arrive.A new quantitative metric,normalized Backward Transfer(nBwT),is developed to evaluate the forgetting degree of the model in the lifelong learning process.Extensive experimental results demonstrate the superiority of our proposed benchmark in achieving a low catastrophic forgetting degree and strong generalization ability.

A novel convolutional neural network method for crowd counting

Crowd density estimation,in general,is a challenging task due to the large variation of head sizes in the crowds.Existing methods always use a multi-column convolutional neural network(MCNN)to adapt to this variation,which results in an average effect in areas with different densities and brings a lot of noise to the density map.To address this problem,we propose a new method called the segmentation-aware prior network(SAPNet),which generates a high-quality density map without noise based on a coarse head-segmentation map.SAPNet is composed of two networks,i.e.,a foreground-segmentation convolutional neural network(FS-CNN)as the front end and a crowd-regression convolutional neural network(CR-CNN)as the back end.With only the single dot annotation,we generate the ground truth of segmentation masks in heads.Then,based on the ground truth,FS-CNN outputs a coarse head-segmentation map,which helps eliminate the noise in regions without people in the density map.By inputting the head-segmentation map generated by the front end,CR-CNN performs accurate crowd counting estimation and generates a high-quality density map.We demonstrate SAPNet on four datasets(i.e.,ShanghaiTech,UCF-CC-50,WorldExpo’10,and UCSD),and show the state-of-the-art performances on ShanghaiTech part B and UCF-CC-50 datasets.

DeepCount:Crowd Counting with Wi-Fi Using Deep Learning

The ubiquitous Wi-Fi devices and recent research efforts on wireless sensing have led to intelligent environments which can sense people’s locations and activities in a device-free manner.However,current works are mostly designed for single human environment owing to the complexity of multiple human environments,the limited bandwidth of Wi-Fi and in turn,greatly hinder this technology from the real implementation.To realize such device-free sensing in multiple human environments,the first step-stone is to estimate how many targets or in other words crowd counting in a closed environment,which is not only the basis for multiple human environmental sensing but also leads to many potential applications such as crowd control.To this end,we propose DeepCount—a solution using deep learning approach to infer the number of people in an indoor environment with Wi-Fi signals.Our scheme is based on the key intuition that,although with great complexity,the deep learning approaches can somehow be able to build a complex function to fit the correlation between the number of people and channel state information values.Furthermore,to alleviate the inadequate amount of data required and improve the adaptability of the deep learning approach,we add an online transfer learning approach,which utilizes the entering/leaving results to fine-tune the deep learning model.The prototype of Deep-Count is implemented and evaluated on the commercial Wi-Fi device.By the massive training samples,our deep learning model is able to estimate the number of crowd up to 5 with the mean accuracy of 82.3%by this end-to-end learning approach.Meanwhile,by using the amendment mechanism of the activity recognition model to judge door switch to get the variance of the crowd to amend deep learning predicted results,the accuracy is up to 87%in a rather effective manner.

Transferring priors from virtual data for crowd counting in real world

In recent years,crowd counting has increasingly drawn attention due to its widespread applications in the field of computer vision.Most of the existing methods rely on datasets with scarce labeled images to train networks.They are prone to suffer from the over-fitting problem.Further,these existing datasets usually just give manually labeled annotations related to the head center position.This kind of annotation provides limited information.In this paper,we propose to exploit virtual synthetic crowd scenes to improve the performance of the counting network in the real world.Since we can obtain people masks easily in a synthetic dataset,we first learn to distinguish people from the background via a segmentation network using the synthetic data.Then we transfer the learned segmentation priors from synthetic data to real-world data.Finally,we train a density estimation network on real-world data by utilizing the obtained people masks.Our experiments on two crowd counting datasets demonstrate the effectiveness of the proposed method.

Aggregated context network for crowd counting

Crowd counting has been applied to a variety of applications such as video surveillance,traffic monitoring,assembly control,and other public safety applications.Context information,such as perspective distortion and background interference,is a crucial factor in achieving high performance for crowd counting.While traditional methods focus merely on solving one specific factor,we aggregate sufficient context information into the crowd counting network to tackle these problems simultaneously in this study.We build a fully convolutional network with two tasks,i.e.,main density map estimation and auxiliary semantic segmentation.The main task is to extract the multi-scale and spatial context information to learn the density map.The auxiliary semantic segmentation task gives a comprehensive view of the background and foreground information,and the extracted information is finally incorporated into the main task by late fusion.We demonstrate that our network has better accuracy of estimation and higher robustness on three challenging datasets compared with state-of-the-art methods.