Intelligent Mobile Video Surveillance System with Multilevel Distillation

This paper proposes a mobile video surveillance system consisting of intelligent video analysis and mobile communication networking. This multilevel distillation approach helps mobile users monitor tremendous surveillance videos on demand through video streaming over mobile communication networks. The intelligent video analysis includes moving object detection/tracking and key frame selection which can browse useful video clips. The communication networking services, comprising video transcoding, multimedia messaging, and mobile video streaming, transmit surveillance information into mobile appliances. Moving object detection is achieved by background subtraction and particle filter tracking. Key frame selection, which aims to deliver an alarm to a mobile client using multimedia messaging service accompanied with an extracted clear frame, is reached by devising a weighted importance criterion considering object clarity and face appearance. Besides, a spatial- domain cascaded transcoder is developed to convert the filtered image sequence of detected objects into the mobile video streaming format. Experimental results show that the system can successfully detect all events of moving objects for a complex surveillance scene, choose very appropriate key frames for users, and transcode the images with a high power signal-to-noise ratio （PSNR）.

Semi-automatic Video Annotation Tool to Generate Ground Truth for Intelligent Video Surveillance Systems

Generating ground truth data for developing object detection algorithms of intelligent surveillance systems is a considerably important yet time-consuming task; therefore, a user-friendly tool to annotate videos efficiently and accurately is required. In this paper, the development of a semi-automatic video annotation tool is described. For efficiency, the developed tool can automatically generate the initial annotation data for the input videos utilizing automatic object detection modules, which are developed independently and registered in the tool. To guarantee the accuracy of the ground truth data, the system also has several user-friendly functions to help users check and edit the initial annotation data generated by the automatic object detection modules. According to the experiment＇s results, employing the developed annotation tool is considerably beneficial for reducing annotation time; when compared to manual annotation schemes, using the tool resulted in an annotation time reduction of up to 2.3 times.

Review for Anomaly Detection in Video Surveillance System Based on Deep Learning

In this paper,abnormal target detection and location in video surveillance system are studied.In recent years,with the rapid development of network information technology,video surveillance technology has been widely used,artificial anomaly detection methods have no way to meet the effective growth of video surveillance data,with 3D technology,face recognition technology,etc.,also promote the development of the field of computer vision,for the rapid analysis of a large number of video data to provide effective support.At present,abnormal target detection methods in video surveillance system mainly include the following two methods:One is to extract two-dimensional data features from video surveillance data,and effectively express video targets according to the extracted features.The information expressed mainly includes time information and spatial information.The second is to directly learn 3D space-time features for the module with motion information to detect the location of the abnormal target.Finally,the paper summarizes the full text and looks forward to the future development direction of video anomaly detection from three aspects:data set,method and evaluation index.

Application of robust face recognition in video surveillance systems

In this paper, we propose a video searching system that utilizes face recognition as searching indexing feature. As the applications of video cameras have great increase in recent years, face recognition makes a perfect fit for searching targeted individuals within the vast amount of video data. However, the performance of such searching depends on the quality of face images recorded in the video signals. Since the surveillance video cameras record videos without fixed postures for the object, face occlusion is very common in everyday video. The proposed system builds a model for occluded faces using fuzzy principal component analysis(FPCA), and reconstructs the human faces with the available information. Experimental results show that the system has very high efficiency in processing the real life videos, and it is very robust to various kinds of face occlusions. Hence it can relieve people reviewers from the front of the monitors and greatly enhances the efficiency as well. The proposed system has been installed and applied in various environments and has already demonstrated its power by helping solving real cases.

ISHD:Intelligent Standing Human Detection of Video Surveillance for the Smart Examination Environment

In the environment of smart examination rooms, it is important to quickly and accurately detect abnormal behavior(human standing) for the construction of a smart campus. Based on deep learning, we propose an intelligentstanding human detection (ISHD) method based on an improved single shot multibox detector to detect thetarget of standing human posture in the scene frame of exam room video surveillance at a specific examinationstage. ISHD combines the MobileNet network in a single shot multibox detector network, improves the posturefeature extractor of a standing person, merges prior knowledge, and introduces transfer learning in the trainingstrategy, which greatly reduces the computation amount, improves the detection accuracy, and reduces the trainingdifficulty. The experiment proves that the model proposed in this paper has a better detection ability for the smalland medium-sized standing human body posture in video test scenes on the EMV-2 dataset.

An Efficient Attention-Based Strategy for Anomaly Detection in Surveillance Video

In the present technological world,surveillance cameras generate an immense amount of video data from various sources,making its scrutiny tough for computer vision specialists.It is difficult to search for anomalous events manually in thesemassive video records since they happen infrequently and with a low probability in real-world monitoring systems.Therefore,intelligent surveillance is a requirement of the modern day,as it enables the automatic identification of normal and aberrant behavior using artificial intelligence and computer vision technologies.In this article,we introduce an efficient Attention-based deep-learning approach for anomaly detection in surveillance video(ADSV).At the input of the ADSV,a shots boundary detection technique is used to segment prominent frames.Next,The Lightweight ConvolutionNeuralNetwork(LWCNN)model receives the segmented frames to extract spatial and temporal information from the intermediate layer.Following that,spatial and temporal features are learned using Long Short-Term Memory(LSTM)cells and Attention Network from a series of frames for each anomalous activity in a sample.To detect motion and action,the LWCNN received chronologically sorted frames.Finally,the anomaly activity in the video is identified using the proposed trained ADSV model.Extensive experiments are conducted on complex and challenging benchmark datasets.In addition,the experimental results have been compared to state-ofthe-artmethodologies,and a significant improvement is attained,demonstrating the efficiency of our ADSV method.

Quantum Computing Based Neural Networks for Anomaly Classification in Real-Time Surveillance Videos

For intelligent surveillance videos,anomaly detection is extremely important.Deep learning algorithms have been popular for evaluating realtime surveillance recordings,like traffic accidents,and criminal or unlawful incidents such as suicide attempts.Nevertheless,Deep learning methods for classification,like convolutional neural networks,necessitate a lot of computing power.Quantum computing is a branch of technology that solves abnormal and complex problems using quantum mechanics.As a result,the focus of this research is on developing a hybrid quantum computing model which is based on deep learning.This research develops a Quantum Computing-based Convolutional Neural Network(QC-CNN)to extract features and classify anomalies from surveillance footage.A Quantum-based Circuit,such as the real amplitude circuit,is utilized to improve the performance of the model.As far as my research,this is the first work to employ quantum deep learning techniques to classify anomalous events in video surveillance applications.There are 13 anomalies classified from the UCF-crime dataset.Based on experimental results,the proposed model is capable of efficiently classifying data concerning confusion matrix,Receiver Operating Characteristic(ROC),accuracy,Area Under Curve(AUC),precision,recall as well as F1-score.The proposed QC-CNN has attained the best accuracy of 95.65 percent which is 5.37%greater when compared to other existing models.To measure the efficiency of the proposed work,QC-CNN is also evaluated with classical and quantum models.

A Personalized Video Synopsis Framework for Spherical Surveillance Video

Video synopsis is an effective way to easily summarize long-recorded surveillance videos.The omnidirectional view allows the observer to select the desired fields of view(FoV)from the different FoVavailable for spherical surveillance video.By choosing to watch one portion,the observer misses out on the events occurring somewhere else in the spherical scene.This causes the observer to experience fear of missing out(FOMO).Hence,a novel personalized video synopsis approach for the generation of non-spherical videos has been introduced to address this issue.It also includes an action recognition module that makes it easy to display necessary actions by prioritizing them.This work minimizes and maximizes multiple goals such as loss of activity,collision,temporal consistency,length,show,and important action cost respectively.The performance of the proposed framework is evaluated through extensive simulation and compared with the state-of-art video synopsis optimization algorithms.Experimental results suggest that some constraints are better optimized by using the latest metaheuristic optimization algorithms to generate compact personalized synopsis videos from spherical surveillance videos.

Weapons Detection for Security and Video Surveillance Using CNN and YOLO-V5s

In recent years,the number of Gun-related incidents has crossed over 250,000 per year and over 85%of the existing 1 billion firearms are in civilian hands,manual monitoring has not proven effective in detecting firearms.which is why an automated weapon detection system is needed.Various automated convolutional neural networks(CNN)weapon detection systems have been proposed in the past to generate good results.However,These techniques have high computation overhead and are slow to provide real-time detection which is essential for the weapon detection system.These models have a high rate of false negatives because they often fail to detect the guns due to the low quality and visibility issues of surveillance videos.This research work aims to minimize the rate of false negatives and false positives in weapon detection while keeping the speed of detection as a key parameter.The proposed framework is based on You Only Look Once(YOLO)and Area of Interest(AOI).Initially,themodels take pre-processed frames where the background is removed by the use of the Gaussian blur algorithm.The proposed architecture will be assessed through various performance parameters such as False Negative,False Positive,precision,recall rate,and F1 score.The results of this research work make it clear that due to YOLO-v5s high recall rate and speed of detection are achieved.Speed reached 0.010 s per frame compared to the 0.17 s of the Faster R-CNN.It is promising to be used in the field of security and weapon detection.

UTILITY OPTIMIZATION SCHEDULING FOR MULTI-POINT VIDEO SURVEILLANCE IN UBIQUITOUS NETWORK

Resource allocation is an important problem in ubiquitous network. Most of the existing resource allocation methods considering only wireless networks are not suitable for the ubiquitous network environment, and they will harm the interest of individual users with instable resource requirements. This paper considers the multi-point video surveillance scenarios in a complex network environment with both wired and wireless networks. We introduce the utility estimated by the total costs of an individual network user. The problem is studied through mathematical modeling and we propose an improved problem-specific branch-and-cut algorithm to solve it. The algorithm follows the divide-and-conquer principle and fully considers the duality feature of network selection. The experiment is conducted by simulation through C and Lingo. And it shows that compared with a centralized random allocation scheme and a cost greed allocation scheme, the proposed scheme has better per- formance of reducing the total costs by 13.0% and 30.6% respectively for the user.

Automated Video-Based Face Detection Using Harris Hawks Optimization with Deep Learning

Face recognition technology automatically identifies an individual from image or video sources.The detection process can be done by attaining facial characteristics from the image of a subject face.Recent developments in deep learning(DL)and computer vision(CV)techniques enable the design of automated face recognition and tracking methods.This study presents a novel Harris Hawks Optimization with deep learning-empowered automated face detection and tracking(HHODL-AFDT)method.The proposed HHODL-AFDT model involves a Faster region based convolution neural network(RCNN)-based face detection model and HHO-based hyperparameter opti-mization process.The presented optimal Faster RCNN model precisely rec-ognizes the face and is passed into the face-tracking model using a regression network(REGN).The face tracking using the REGN model uses the fea-tures from neighboring frames and foresees the location of the target face in succeeding frames.The application of the HHO algorithm for optimal hyperparameter selection shows the novelty of the work.The experimental validation of the presented HHODL-AFDT algorithm is conducted using two datasets and the experiment outcomes highlighted the superior performance of the HHODL-AFDT model over current methodologies with maximum accuracy of 90.60%and 88.08%under PICS and VTB datasets,respectively.

Guest Editorial:Intelligent Video Surveillance and Related Technologies

Due to the increasing demand for developing a secure and smart living environment, the intelligent video surveillance technology has attracted considerable attention. Building an automatic, reliable, secure, and intelligent video surveillance system has spawned large research projects and triggered many popular research topics in several international conferences and workshops recently. This special issue of Journal of ElecWonic Science and Technology （JEST） aims to present recent advances in video surveillance systems which address the observation of people in an environment, leading to a real-time description of their actions and interactions.

Human Detection for Video Surveillance in Hospital

This paper presents a human detection system in a vision-based hospital surveillance environment. The system is composed of three subsystems, i.e. background segmentation subsystem （BSS）, human feature extraction subsystem （HFES）, and human recognition subsystem （HRS）. The codebook background model is applied in the BSS, the histogram of oriented gradients （HOG） features are used in the HFES, and the support vector machine （SVM） classification is employed in the HRS. By means of the integration of these subsystems, the human detection in a vision-based hospital surveillance environment is performed. Experimental results show that the proposed system can effectively detect most of the people in hospital surveillance video sequences.

Smart Deep Learning Based Human Behaviour Classification for Video Surveillance

Real-time video surveillance system is commonly employed to aid security professionals in preventing crimes.The use of deep learning(DL)technologies has transformed real-time video surveillance into smart video surveillance systems that automate human behavior classification.The recognition of events in the surveillance videos is considered a hot research topic in the field of computer science and it is gaining significant attention.Human action recognition(HAR)is treated as a crucial issue in several applications areas and smart video surveillance to improve the security level.The advancements of the DL models help to accomplish improved recognition performance.In this view,this paper presents a smart deep-based human behavior classification(SDL-HBC)model for real-time video surveillance.The proposed SDL-HBC model majorly aims to employ an adaptive median filtering(AMF)based pre-processing to reduce the noise content.Also,the capsule network(CapsNet)model is utilized for the extraction of feature vectors and the hyperparameter tuning of the CapsNet model takes place utilizing the Adam optimizer.Finally,the differential evolution(DE)with stacked autoencoder(SAE)model is applied for the classification of human activities in the intelligent video surveillance system.The performance validation of the SDL-HBC technique takes place using two benchmark datasets such as the KTH dataset.The experimental outcomes reported the enhanced recognition performance of the SDL-HBC technique over the recent state of art approaches with maximum accuracy of 0.9922.

Visibility Enhancement of Scene Images Degraded by Foggy Weather Condition: An Application to Video Surveillance

:In recent years,video surveillance application played a significant role in our daily lives.Images taken during foggy and haze weather conditions for video surveillance application lose their authenticity and hence reduces the visibility.The reason behind visibility enhancement of foggy and haze images is to help numerous computer and machine vision applications such as satellite imagery,object detection,target killing,and surveillance.To remove fog and enhance visibility,a number of visibility enhancement algorithms and methods have been proposed in the past.However,these techniques suffer from several limitations that place strong obstacles to the real world outdoor computer vision applications.The existing techniques do not perform well when images contain heavy fog,large white region and strong atmospheric light.This research work proposed a new framework to defog and dehaze the image in order to enhance the visibility of foggy and haze images.The proposed framework is based on a Conditional generative adversarial network(CGAN)with two networks;generator and discriminator,each having distinct properties.The generator network generates fog-free images from foggy images and discriminator network distinguishes between the restored image and the original fog-free image.Experiments are conducted on FRIDA dataset and haze images.To assess the performance of the proposed method on fog dataset,we use PSNR and SSIM,and for Haze dataset use e,r−,andσas performance metrics.Experimental results shows that the proposed method achieved higher values of PSNR and SSIM which is 18.23,0.823 and lower values produced by the compared method which are 13.94,0.791 and so on.Experimental results demonstrated that the proposed framework Has removed fog and enhanced the visibility of foggy and hazy images.

A real-time 5/3 lifting wavelet HD-video de-noising system based on FPGA

In accordance with the application requirements of high definition(HD) video surveillance systems,a real-time 5/3 lifting wavelet HD-video de-noising system is proposed with frame rate conversion(FRC) based on a field-programmable gate array(FPGA),which uses a 3-level pipeline paralleled 5/3 lifting wavelet transformation and reconstruction structure,as well as a fast BayesS hrink adaptive threshold filtering module.The proposed system demonstrates de-noising performance,while also balancing system resources and achieving real-time processing.The experiments show that the proposed system's maximum operating frequency(through logic synthesis and layout using Quartus 13.1 software) can reach 178 MHz,based on the Altera Company's Stratix III EP3SE80 series FPGA.The proposed system can also satisfy real-time de-noising requirements of 1920 × 1080 at60 fps HD-video sources,while also significantly improving the peak signal to noise rate of the denoising images.Compared with similar systems,the system has the advantages of high operating frequency,and the ability to support multiple source formats for real-time processing.

Action Recognition in Surveillance Videos with Combined Deep Network Models

Action recognition is an important topic in computer vision. Recently, deep learning technologies have been successfully used in lots of applications including video data for sloving recognition problems. However, most existing deep learning based recognition frameworks are not optimized for action in the surveillance videos. In this paper, we propose a novel method to deal with the recognition of different types of actions in outdoor surveillance videos. The proposed method first introduces motion compensation to improve the detection of human target. Then, it uses three different types of deep models with single and sequenced images as inputs for the recognition of different types of actions. Finally, predictions from different models are fused with a linear model. Experimental results show that the proposed method works well on the real surveillance videos.

COVAD: Content-oriented video anomaly detection using a self attention-based deep learning model

Background Video anomaly detection has always been a hot topic and has attracted increasing attention.Many of the existing methods for video anomaly detection depend on processing the entire video rather than considering only the significant context. Method This paper proposes a novel video anomaly detection method called COVAD that mainly focuses on the region of interest in the video instead of the entire video. Our proposed COVAD method is based on an autoencoded convolutional neural network and a coordinated attention mechanism,which can effectively capture meaningful objects in the video and dependencies among different objects. Relying on the existing memory-guided video frame prediction network, our algorithm can significantly predict the future motion and appearance of objects in a video more effectively. Result The proposed algorithm obtained better experimental results on multiple datasets and outperformed the baseline models considered in our analysis. Simultaneously, we provide an improved visual test that can provide pixel-level anomaly explanations.

Video-based urban expressway traffic measurement and performance monitoring

This paper presents an urban expressway video surveillance and monitoring system for traffic flow measurement and abnormal performance detection. The proposed flow detection module collects traffic flow statistics in real time by leveraging multi-vehicle tracking information. Based on these online statistics, road operating situations can be easily obtained. Using spatiotemporal trajectories, vehicle motion paths are encoded by hidden Markov models. With path division and parameter matching, abnormal performances containing extra low or high speed driving, illegal stopping and turning are detected in real scenes. The traffic surveillance approach is implemented and evaluated on a DM642 DSP-based embedded platform. Experimental results demonstrate that the proposed system is feasible for the detection of vehicle speed, vehicle counts and road efficiency, and it is effective for the monitoring of the aforementioned anomalies with low computational costs.

Towards Collaborative Robotics in Top View Surveillance:A Framework for Multiple Object Tracking by Detection Using Deep Learning

Collaborative Robotics is one of the high-interest research topics in the area of academia and industry.It has been progressively utilized in numerous applications,particularly in intelligent surveillance systems.It allows the deployment of smart cameras or optical sensors with computer vision techniques,which may serve in several object detection and tracking tasks.These tasks have been considered challenging and high-level perceptual problems,frequently dominated by relative information about the environment,where main concerns such as occlusion,illumination,background,object deformation,and object class variations are commonplace.In order to show the importance of top view surveillance,a collaborative robotics framework has been presented.It can assist in the detection and tracking of multiple objects in top view surveillance.The framework consists of a smart robotic camera embedded with the visual processing unit.The existing pre-trained deep learning models named SSD and YOLO has been adopted for object detection and localization.The detection models are further combined with different tracking algorithms,including GOTURN,MEDIANFLOW,TLD,KCF,MIL,and BOOSTING.These algorithms,along with detection models,help to track and predict the trajectories of detected objects.The pre-trained models are employed;therefore,the generalization performance is also investigated through testing the models on various sequences of top view data set.The detection models achieved maximum True Detection Rate 93%to 90%with a maximum 0.6%False Detection Rate.The tracking results of different algorithms are nearly identical,with tracking accuracy ranging from 90%to 94%.Furthermore,a discussion has been carried out on output results along with future guidelines.