Semantic Segmentation and YOLO Detector over Aerial Vehicle Images

Intelligent vehicle tracking and detection are crucial tasks in the realm of highway management.However,vehicles come in a range of sizes,which is challenging to detect,affecting the traffic monitoring system’s overall accuracy.Deep learning is considered to be an efficient method for object detection in vision-based systems.In this paper,we proposed a vision-based vehicle detection and tracking system based on a You Look Only Once version 5(YOLOv5)detector combined with a segmentation technique.The model consists of six steps.In the first step,all the extracted traffic sequence images are subjected to pre-processing to remove noise and enhance the contrast level of the images.These pre-processed images are segmented by labelling each pixel to extract the uniform regions to aid the detection phase.A single-stage detector YOLOv5 is used to detect and locate vehicles in images.Each detection was exposed to Speeded Up Robust Feature(SURF)feature extraction to track multiple vehicles.Based on this,a unique number is assigned to each vehicle to easily locate them in the succeeding image frames by extracting them using the feature-matching technique.Further,we implemented a Kalman filter to track multiple vehicles.In the end,the vehicle path is estimated by using the centroid points of the rectangular bounding box predicted by the tracking algorithm.The experimental results and comparison reveal that our proposed vehicle detection and tracking system outperformed other state-of-the-art systems.The proposed implemented system provided 94.1%detection precision for Roundabout and 96.1%detection precision for Vehicle Aerial Imaging from Drone(VAID)datasets,respectively.

Sports Events Recognition Using Multi Features and Deep Belief Network

In the modern era of a growing population,it is arduous for humans to monitor every aspect of sports,events occurring around us,and scenarios or conditions.This recognition of different types of sports and events has increasingly incorporated the use of machine learning and artificial intelligence.This research focuses on detecting and recognizing events in sequential photos characterized by several factors,including the size,location,and position of people’s body parts in those pictures,and the influence around those people.Common approaches utilized,here are feature descriptors such as MSER(Maximally Stable Extremal Regions),SIFT(Scale-Invariant Feature Transform),and DOF(degree of freedom)between the joint points are applied to the skeleton points.Moreover,for the same purposes,other features such as BRISK(Binary Robust Invariant Scalable Keypoints),ORB(Oriented FAST and Rotated BRIEF),and HOG(Histogram of Oriented Gradients)are applied on full body or silhouettes.The integration of these techniques increases the discriminative nature of characteristics retrieved in the identification process of the event,hence improving the efficiency and reliability of the entire procedure.These extracted features are passed to the early fusion and DBscan for feature fusion and optimization.Then deep belief,network is employed for recognition.Experimental results demonstrate a separate experiment’s detection average recognition rate of 87%in the HMDB51 video database and 89%in the YouTube database,showing a better perspective than the current methods in sports and event identification.

Robust Human Interaction Recognition Using Extended Kalman Filter

In the field of computer vision and pattern recognition,knowledge based on images of human activity has gained popularity as a research topic.Activity recognition is the process of determining human behavior based on an image.We implemented an Extended Kalman filter to create an activity recognition system here.The proposed method applies an HSI color transformation in its initial stages to improve the clarity of the frame of the image.To minimize noise,we use Gaussian filters.Extraction of silhouette using the statistical method.We use Binary Robust Invariant Scalable Keypoints(BRISK)and SIFT for feature extraction.The next step is to perform feature discrimination using Gray Wolf.After that,the features are input into the Extended Kalman filter and classified into relevant human activities according to their definitive characteristics.The experimental procedure uses the SUB-Interaction and HMDB51 datasets to a 0.88%and 0.86%recognition rate.

Vehicle Detection and Tracking in UAV Imagery via YOLOv3 and Kalman Filter

Unmanned aerial vehicles(UAVs)can be used to monitor traffic in a variety of settings,including security,traffic surveillance,and traffic control.Numerous academics have been drawn to this topic because of the challenges and the large variety of applications.This paper proposes a new and efficient vehicle detection and tracking system that is based on road extraction and identifying objects on it.It is inspired by existing detection systems that comprise stationary data collectors such as induction loops and stationary cameras that have a limited field of view and are not mobile.The goal of this study is to develop a method that first extracts the region of interest(ROI),then finds and tracks the items of interest.The suggested system is divided into six stages.The photos from the obtained dataset are appropriately georeferenced to their actual locations in the first phase,after which they are all co-registered.The ROI,or road and its objects,are retrieved using the GrabCut method in the second phase.The third phase entails data preparation.The segmented images’noise is eliminated using Gaussian blur,after which the images are changed to grayscale and forwarded to the following stage for additional morphological procedures.The YOLOv3 algorithm is used in the fourth step to find any automobiles in the photos.Following that,the Kalman filter and centroid tracking are used to perform the tracking of the detected cars.The Lucas-Kanade method is then used to perform the trajectory analysis on the vehicles.The suggested model is put to the test and assessed using the Vehicle Aerial Imaging from Drone(VAID)dataset.For detection and tracking,the model was able to attain accuracy levels of 96.7%and 91.6%,respectively.

Home Automation-Based Health Assessment Along Gesture Recognition via Inertial Sensors

Hand gesture recognition (HGR) is used in a numerous applications,including medical health-care, industrial purpose and sports detection.We have developed a real-time hand gesture recognition system using inertialsensors for the smart home application. Developing such a model facilitatesthe medical health field (elders or disabled ones). Home automation has alsobeen proven to be a tremendous benefit for the elderly and disabled. Residentsare admitted to smart homes for comfort, luxury, improved quality of life,and protection against intrusion and burglars. This paper proposes a novelsystem that uses principal component analysis, linear discrimination analysisfeature extraction, and random forest as a classifier to improveHGRaccuracy.We have achieved an accuracy of 94% over the publicly benchmarked HGRdataset. The proposed system can be used to detect hand gestures in thehealthcare industry as well as in the industrial and educational sectors.

Exploiting Human Pose and Scene Information for Interaction Detection

Identifying human actions and interactions finds its use in manyareas, such as security, surveillance, assisted living, patient monitoring, rehabilitation,sports, and e-learning. This wide range of applications has attractedmany researchers to this field. Inspired by the existing recognition systems,this paper proposes a new and efficient human-object interaction recognition(HOIR) model which is based on modeling human pose and scene featureinformation. There are different aspects involved in an interaction, includingthe humans, the objects, the various body parts of the human, and the backgroundscene. Themain objectives of this research include critically examiningthe importance of all these elements in determining the interaction, estimatinghuman pose through image foresting transform (IFT), and detecting the performedinteractions based on an optimizedmulti-feature vector. The proposedmethodology has six main phases. The first phase involves preprocessing theimages. During preprocessing stages, the videos are converted into imageframes. Then their contrast is adjusted, and noise is removed. In the secondphase, the human-object pair is detected and extracted from each image frame.The third phase involves the identification of key body parts of the detectedhumans using IFT. The fourth phase relates to three different kinds of featureextraction techniques. Then these features are combined and optimized duringthe fifth phase. The optimized vector is used to classify the interactions in thelast phase. TheMSRDaily Activity 3D dataset has been used to test this modeland to prove its efficiency. The proposed system obtains an average accuracyof 91.7% on this dataset.

Real Objects Understanding Using 3D Haptic Virtual Reality for E-Learning Education

In the past two decades,there has been a lot of work on computer vision technology that incorporates many tasks which implement basic filtering to image classification.Themajor research areas of this field include object detection and object recognition.Moreover,wireless communication technologies are presently adopted and they have impacted the way of education that has been changed.There are different phases of changes in the traditional system.Perception of three-dimensional(3D)from two-dimensional(2D)image is one of the demanding tasks.Because human can easily perceive but making 3D using software will take time manually.Firstly,the blackboard has been replaced by projectors and other digital screens so such that people can understand the concept better through visualization.Secondly,the computer labs in schools are now more common than ever.Thirdly,online classes have become a reality.However,transferring to online education or e-learning is not without challenges.Therefore,we propose a method for improving the efficiency of e-learning.Our proposed system consists of twoand-a-half dimensional(2.5D)features extraction using machine learning and image processing.Then,these features are utilized to generate 3D mesh using ellipsoidal deformation method.After that,3D bounding box estimation is applied.Our results show that there is a need to move to 3D virtual reality(VR)with haptic sensors in the field of e-learning for a better understanding of real-world objects.Thus,people will have more information as compared to the traditional or simple online education tools.We compare our result with the ShapeNet dataset to check the accuracy of our proposed method.Our proposed system achieved an accuracy of 90.77%on plane class,85.72%on chair class,and car class have 72.14%.Mean accuracy of our method is 70.89%.

A Novel Capability of Object Identification and Recognition Based on Integrated mWMM

In the last decade,there has been remarkable progress in the areas of object detection and recognition due to high-quality color images along with their depth maps provided by RGB-D cameras.They enable artificially intelligent machines to easily detect and recognize objects and make real-time decisions according to the given scenarios.Depth cues can improve the quality of object detection and recognition.The main purpose of this research study to find an optimized way of object detection and identification we propose techniques of object detection using two RGB-D datasets.The proposed methodology extracts image normally from depth maps and then performs clustering using the Modified Watson Mixture Model(mWMM).mWMM is challenging to handle when the quality of the image is not good.Hence,the proposed RGB-D-based system uses depth cues for segmentation with the help of mWMM.Then it extracts multiple features from the segmented images.The selected features are fed to the Artificial Neural Network(ANN)and Convolutional Neural Network(CNN)for detecting objects.We achieved 92.13%of mean accuracy over NYUv1 dataset and 90.00%of mean accuracy for the Redweb_v1 dataset.Finally,their results are compared and the proposed model with CNN outperforms other state-of-the-art methods.The proposed architecture can be used in autonomous cars,traffic monitoring,and sports scenes.

Multiple Pedestrian Detection and Tracking in Night Vision Surveillance Systems

Pedestrian detection and tracking are vital elements of today’s surveillance systems,which make daily life safe for humans.Thus,human detection and visualization have become essential inventions in the field of computer vision.Hence,developing a surveillance system with multiple object recognition and tracking,especially in low light and night-time,is still challenging.Therefore,we propose a novel system based on machine learning and image processing to provide an efficient surveillance system for pedestrian detection and tracking at night.In particular,we propose a system that tackles a two-fold problem by detecting multiple pedestrians in infrared(IR)images using machine learning and tracking them using particle filters.Moreover,a random forest classifier is adopted for image segmentation to identify pedestrians in an image.The result of detection is investigated by particle filter to solve pedestrian tracking.Through the extensive experiment,our system shows 93%segmentation accuracy using a random forest algorithm that demonstrates high accuracy for background and roof classes.Moreover,the system achieved a detection accuracy of 90%usingmultiple templatematching techniques and 81%accuracy for pedestrian tracking.Furthermore,our system can identify that the detected object is a human.Hence,our system provided the best results compared to the state-ofart systems,which proves the effectiveness of the techniques used for image segmentation,classification,and tracking.The presented method is applicable for human detection/tracking,crowd analysis,and monitoring pedestrians in IR video surveillance.

A Novel Human Interaction Framework Using Quadratic Discriminant Analysis with HMM

Human-human interaction recognition is crucial in computer vision fields like surveillance,human-computer interaction,and social robotics.It enhances systems’ability to interpret and respond to human behavior precisely.This research focuses on recognizing human interaction behaviors using a static image,which is challenging due to the complexity of diverse actions.The overall purpose of this study is to develop a robust and accurate system for human interaction recognition.This research presents a novel image-based human interaction recognition method using a Hidden Markov Model(HMM).The technique employs hue,saturation,and intensity(HSI)color transformation to enhance colors in video frames,making them more vibrant and visually appealing,especially in low-contrast or washed-out scenes.Gaussian filters reduce noise and smooth imperfections followed by silhouette extraction using a statistical method.Feature extraction uses the features from Accelerated Segment Test(FAST),Oriented FAST,and Rotated BRIEF(ORB)techniques.The application of Quadratic Discriminant Analysis(QDA)for feature fusion and discrimination enables high-dimensional data to be effectively analyzed,thus further enhancing the classification process.It ensures that the final features loaded into the HMM classifier accurately represent the relevant human activities.The impressive accuracy rates of 93%and 94.6%achieved in the BIT-Interaction and UT-Interaction datasets respectively,highlight the success and reliability of the proposed technique.The proposed approach addresses challenges in various domains by focusing on frame improvement,silhouette and feature extraction,feature fusion,and HMM classification.This enhances data quality,accuracy,adaptability,reliability,and reduction of errors.

Tracking and Analysis of Pedestrian’s Behavior in Public Places

Crowd management becomes a global concern due to increased population in urban areas.Better management of pedestrians leads to improved use of public places.Behavior of pedestrian’s is a major factor of crowd management in public places.There are multiple applications available in this area but the challenge is open due to complexity of crowd and depends on the environment.In this paper,we have proposed a new method for pedestrian’s behavior detection.Kalman filter has been used to detect pedestrian’s usingmovement based approach.Next,we have performed occlusion detection and removal using region shrinking method to isolate occluded humans.Human verification is performed on each human silhouette and wavelet analysis and particle gradient motion are extracted for each silhouettes.Gray Wolf Optimizer(GWO)has been utilized to optimize feature set and then behavior classification has been performed using the Extreme Gradient(XG)Boost classifier.Performance has been evaluated using pedestrian’s data from avenue and UBI-Fight datasets,where both have different environment.The mean achieved accuracies are 91.3%and 85.14%over the Avenue and UBI-Fight datasets,respectively.These results are more accurate as compared to other existing methods.

Automatic Anomaly Monitoring in Public Surveillance Areas

With the dramatic increase in video surveillance applications and public safety measures,the need for an accurate and effective system for abnormal/sus-picious activity classification also increases.Although it has multiple applications,the problem is very challenging.In this paper,a novel approach for detecting nor-mal/abnormal activity has been proposed.We used the Gaussian Mixture Model(GMM)and Kalmanfilter to detect and track the objects,respectively.After that,we performed shadow removal to segment an object and its shadow.After object segmentation we performed occlusion detection method to detect occlusion between multiple human silhouettes and we implemented a novel method for region shrinking to isolate occluded humans.Fuzzy c-mean is utilized to verify human silhouettes and motion based features including velocity and opticalflow are extracted for each identified silhouettes.Gray Wolf Optimizer(GWO)is used to optimize feature set followed by abnormal event classification that is performed using the XG-Boost classifier.This system is applicable in any surveillance appli-cation used for event detection or anomaly detection.Performance of proposed system is evaluated using University of Minnesota(UMN)dataset and UBI(Uni-versity of Beira Interior)-Fight dataset,each having different type of anomaly.The mean accuracy for the UMN and UBI-Fight datasets is 90.14%and 76.9%respec-tively.These results are more accurate as compared to other existing methods.

Student’s Health Exercise Recognition Tool for E-Learning Education

Due to the recently increased requirements of e-learning systems,multiple educational institutes such as kindergarten have transformed their learning towards virtual education.Automated student health exercise is a difficult task but an important one due to the physical education needs especially in young learners.The proposed system focuses on the necessary implementation of student health exercise recognition(SHER)using a modified Quaternion-basedfilter for inertial data refining and data fusion as the pre-processing steps.Further,cleansed data has been segmented using an overlapping windowing approach followed by patterns identification in the form of static and kinematic signal patterns.Furthermore,these patterns have been utilized to extract cues for both patterned signals,which are further optimized using Fisher’s linear discriminant analysis(FLDA)technique.Finally,the physical exercise activities have been categorized using extended Kalmanfilter(EKF)-based neural networks.This system can be implemented in multiple educational establishments including intelligent training systems,virtual mentors,smart simulations,and interactive learning management methods.

Human Pose Estimation and Object Interaction for Sports Behaviour

In the new era of technology,daily human activities are becoming more challenging in terms of monitoring complex scenes and backgrounds.To understand the scenes and activities from human life logs,human-object interaction(HOI)is important in terms of visual relationship detection and human pose estimation.Activities understanding and interaction recognition between human and object along with the pose estimation and interaction modeling have been explained.Some existing algorithms and feature extraction procedures are complicated including accurate detection of rare human postures,occluded regions,and unsatisfactory detection of objects,especially small-sized objects.The existing HOI detection techniques are instancecentric(object-based)where interaction is predicted between all the pairs.Such estimation depends on appearance features and spatial information.Therefore,we propose a novel approach to demonstrate that the appearance features alone are not sufficient to predict the HOI.Furthermore,we detect the human body parts by using the Gaussian Matric Model(GMM)followed by object detection using YOLO.We predict the interaction points which directly classify the interaction and pair them with densely predicted HOI vectors by using the interaction algorithm.The interactions are linked with the human and object to predict the actions.The experiments have been performed on two benchmark HOI datasets demonstrating the proposed approach.

CNN Based Multi-Object Segmentation and Feature Fusion for Scene Recognition

Latest advancements in vision technology offer an evident impact on multi-object recognition and scene understanding.Such sceneunderstanding task is a demanding part of several technologies,like augmented reality-based scene integration,robotic navigation,autonomous driving,and tourist guide.Incorporating visual information in contextually unified segments,convolution neural networks-based approaches will significantly mitigate the clutter,which is usual in classical frameworks during scene understanding.In this paper,we propose a convolutional neural network(CNN)based segmentation method for the recognition of multiple objects in an image.Initially,after acquisition and preprocessing,the image is segmented by using CNN.Then,CNN features are extracted from these segmented objects,and discrete cosine transform(DCT)and discrete wavelet transform(DWT)features are computed.After the extraction of CNN features and computation of classical machine learning features,fusion is performed using a fusion technique.Then,to select theminimal set of features,genetic algorithm-based feature selection is used.In order to recognize and understand the multi-objects in the scene,a neuro-fuzzy approach is applied.Once objects in the scene are recognized,the relationship between these objects is examined by employing the object-to-object relation approach.Finally,a decision tree is incorporated to assign the relevant labels to the scenes based on recognized objects in the image.The experimental results over complex scene datasets including SUN Red Green Blue-Depth(RGB-D)and Cityscapes’demonstrated a remarkable performance.

Automated Facial Expression Recognition and Age Estimation Using Deep Learning

With the advancement of computer vision techniques in surveillance systems,the need for more proficient,intelligent,and sustainable facial expressions and age recognition is necessary.The main purpose of this study is to develop accurate facial expressions and an age recognition system that is capable of error-free recognition of human expression and age in both indoor and outdoor environments.The proposed system first takes an input image pre-process it and then detects faces in the entire image.After that landmarks localization helps in the formation of synthetic face mask prediction.A novel set of features are extracted and passed to a classifier for the accurate classification of expressions and age group.The proposed system is tested over two benchmark datasets,namely,the Gallagher collection person dataset and the Images of Groups dataset.The system achieved remarkable results over these benchmark datasets about recognition accuracy and computational time.The proposed system would also be applicable in different consumer application domains such as online business negotiations,consumer behavior analysis,E-learning environments,and emotion robotics.

Intelligent Sign Language Recognition System for E-Learning Context

In this research work,an efficient sign language recognition tool for e-learning has been proposed with a new type of feature set based on angle and lines.This feature set has the ability to increase the overall performance of machine learning algorithms in an efficient way.The hand gesture recognition based on these features has been implemented for usage in real-time.The feature set used hand landmarks,which were generated using media-pipe(MediaPipe)and open computer vision(openCV)on each frame of the incoming video.The overall algorithm has been tested on two well-known ASLalphabet(American Sign Language)and ISL-HS(Irish Sign Language)sign language datasets.Different machine learning classifiers including random forest,decision tree,and naïve Bayesian have been used to classify hand gestures using this unique feature set and their respective results have been compared.Since the random forest classifier performed better,it has been selected as the base classifier for the proposed system.It showed 96.7%accuracy with ISL-HS and 93.7%accuracy with ASL-alphabet dataset using the extracted features.

Sensors-Based Ambient Assistant Living via E-Monitoring Technology

Independent human living systems require smart,intelligent,and sustainable online monitoring so that an individual can be assisted timely.Apart from ambient assisted living,the task of monitoring human activities plays an important role in different fields including virtual reality,surveillance security,and human interaction with robots.Such systems have been developed in the past with the use of various wearable inertial sensors and depth cameras to capture the human actions.In this paper,we propose multiple methods such as random occupancy pattern,spatio temporal cloud,waypoint trajectory,Hilbert transform,Walsh Hadamard transform and bone pair descriptors to extract optimal features corresponding to different human actions.These features sets are then normalized using min-max normalization and optimized using the Fuzzy optimization method.Finally,the Masi entropy classifier is applied for action recognition and classification.Experiments have been performed on three challenging datasets,namely,UTDMHAD,50 Salad,and CMU-MMAC.During experimental evaluation,the proposed novel approach of recognizing human actions has achieved an accuracy rate of 90.1%with UTD-MHAD dataset,90.6%with 50 Salad dataset,and 89.5%with CMU-MMAC dataset.Hence experimental results validated the proposed system.

Pedestrian Physical Education Training Over Visualization Tool

E-learning approaches are one of the most important learning platforms for the learner through electronic equipment.Such study techniques are useful for other groups of learners such as the crowd,pedestrian,sports,transports,communication,emergency services,management systems and education sectors.E-learning is still a challenging domain for researchers and developers to find new trends and advanced tools and methods.Many of them are currently working on this domain to fulfill the requirements of industry and the environment.In this paper,we proposed a method for pedestrian behavior mining of aerial data,using deep flow feature,graph mining technique,and convocational neural network.For input data,the state-of-the-art crowd activity University of Minnesota(UMN)dataset is adopted,which contains the aerial indoor and outdoor view of the pedestrian,for simplification of extra information and computational cost reduction the pre-processing is applied.Deep flow features are extracted to find more accurate information.Furthermore,to deal with repetition in features data and features mining the graph mining algorithm is applied,while Convolution Neural Network(CNN)is applied for pedestrian behavior mining.The proposed method shows 84.50%of mean accuracy and a 15.50%of error rate.Therefore,the achieved results show more accuracy as compared to state-ofthe-art classification algorithms such as decision tree,artificial neural network(ANN).

Improving the Ambient Intelligence Living Using Deep Learning Classifier

Over the last decade,there is a surge of attention in establishing ambient assisted living(AAL)solutions to assist individuals live independently.With a social and economic perspective,the demographic shift toward an elderly population has brought new challenges to today’s society.AAL can offer a variety of solutions for increasing people’s quality of life,allowing them to live healthier and more independently for longer.In this paper,we have proposed a novel AAL solution using a hybrid bidirectional long-term and short-term memory networks(BiLSTM)and convolutional neural network(CNN)classifier.We first pre-processed the signal data,then used timefrequency features such as signal energy,signal variance,signal frequency,empirical mode,and empirical mode decomposition.The convolutional neural network-bidirectional long-term and short-term memory(CNN-biLSTM)classifier with dimensional reduction isomap algorithm was then used to select ideal features.We assessed the performance of our proposed system on the publicly accessible human gait database(HuGaDB)benchmark dataset and achieved an accuracy rates of 93.95 percent,respectively.Experiments reveal that hybrid method gives more accuracy than single classifier in AAL model.The suggested system can assists persons with impairments,assisting carers and medical personnel.