Human Interaction Recognition in Surveillance Videos Using Hybrid Deep Learning and Machine Learning Models

Human Interaction Recognition(HIR)was one of the challenging issues in computer vision research due to the involvement of multiple individuals and their mutual interactions within video frames generated from their movements.HIR requires more sophisticated analysis than Human Action Recognition(HAR)since HAR focuses solely on individual activities like walking or running,while HIR involves the interactions between people.This research aims to develop a robust system for recognizing five common human interactions,such as hugging,kicking,pushing,pointing,and no interaction,from video sequences using multiple cameras.In this study,a hybrid Deep Learning(DL)and Machine Learning(ML)model was employed to improve classification accuracy and generalizability.The dataset was collected in an indoor environment with four-channel cameras capturing the five types of interactions among 13 participants.The data was processed using a DL model with a fine-tuned ResNet(Residual Networks)architecture based on 2D Convolutional Neural Network(CNN)layers for feature extraction.Subsequently,machine learning models were trained and utilized for interaction classification using six commonly used ML algorithms,including SVM,KNN,RF,DT,NB,and XGBoost.The results demonstrate a high accuracy of 95.45%in classifying human interactions.The hybrid approach enabled effective learning,resulting in highly accurate performance across different interaction types.Future work will explore more complex scenarios involving multiple individuals based on the application of this architecture.

Risk identification and safety assessment of human-computer interaction in integrated avionics based on STAMP

To solve the problem of risk identification and quantitative assessment for human-computer interaction(HCI)in complex avionics systems,an HCI safety analysis framework based on system-theoretical process analysis(STPA)and cognitive reliability and error analysis method(CREAM)is proposed.STPACREAM can identify unsafe control actions and find the causal path during the interaction of avionics systems and pilot with the help of formal verification tools automatically.The common performance conditions(CPC)of avionics systems in the aviation environment is established and a quantitative analysis of human failure is carried out.Taking the head-up display(HUD)system interaction process as an example,a case analysis is carried out,the layered safety control structure and formal model of the HUD interaction process are established.For the interactive behavior“Pilots approaching with HUD”,four unsafe control actions and35 causal scenarios are identified and the impact of common performance conditions at different levels on the pilot decision model are analyzed.The results show that HUD's HCI level gradually improves as the scores of CPC increase,and the quality of crew member cooperation and time sufficiency of the task is the key to its HCI.Through case analysis,it is shown that STPACREAM can quantitatively assess the hazards in HCI and identify the key factors that impact safety.

Identification and distribution of 13003 landslides in the northwest margin of Qinghai-Tibet Plateau based on human-computer interaction remote sensing interpretation

The periphery of the Qinghai-Tibet Plateau is renowned for its susceptibility to landslides.However,the northwestern margin of this region,characterised by limited human activities and challenging transportation,remains insufficiently explored concerning landslide occurrence and dispersion.With the planning and construction of the Xinjiang-Tibet Railway,a comprehensive investigation into disastrous landslides in this area is essential for effective disaster preparedness and mitigation strategies.By using the human-computer interaction interpretation approach,the authors established a landslide database encompassing 13003 landslides,collectively spanning an area of 3351.24 km^(2)(36°N-40°N,73°E-78°E).The database incorporates diverse topographical and environmental parameters,including regional elevation,slope angle,slope aspect,distance to faults,distance to roads,distance to rivers,annual precipitation,and stratum.The statistical characteristics of number and area of landslides,landslide number density(LND),and landslide area percentage(LAP)are analyzed.The authors found that a predominant concentration of landslide origins within high slope angle regions,with the highest incidence observed in intervals characterised by average slopes of 20°to 30°,maximum slope angle above 80°,along with orientations towards the north(N),northeast(NE),and southwest(SW).Additionally,elevations above 4.5 km,distance to rivers below 1 km,rainfall between 20-30 mm and 30-40 mm emerge as particularly susceptible to landslide development.The study area’s geological composition primarily comprises Mesozoic and Upper Paleozoic outcrops.Both fault and human engineering activities have different degrees of influence on landslide development.Furthermore,the significance of the landslide database,the relationship between landslide distribution and environmental factors,and the geometric and morphological characteristics of landslides are discussed.The landslide H/L ratios in the study area are mainly concentrated between 0.4 and 0.64.It means the landslides mobility in the region is relatively low,and the authors speculate that landslides in this region more possibly triggered by earthquakes or located in meizoseismal area.

Modulatory role of plant-derived metabolites on host-microbiota interactions:personalized therapeutics outlook

A diverse array of microbes in and on the human body constitute the microbiota.These micro-residents continuously interact with the human host through the language of metabolites to dictate the host’s physiology in health and illnesses.Any biotic and abiotic component ensuring a balanced host-microbiota interaction are potential microbiome therapeutic agents to overcome human diseases.Plant metabolites are continually being used to treat various illnesses.These metabolites target the host’s metabolic machinery and host-gut microbiota interactions to overcome human diseases.Despite the paramount therapeutic significance of the factors affecting host-microbiota interactions,a comprehensive overview of the modulatory role of plant-derived metabolites in host-microbiota interactions is lacking.The current review puts an effort into comprehending the role of medicinal plants in gut microbiota modulation to mitigate various human illnesses.It would develop a holistic understanding of hostmicrobiota interactions and the role of effectors in health and diseases.

Exploiting Human Pose and Scene Information for Interaction Detection

Identifying human actions and interactions finds its use in manyareas, such as security, surveillance, assisted living, patient monitoring, rehabilitation,sports, and e-learning. This wide range of applications has attractedmany researchers to this field. Inspired by the existing recognition systems,this paper proposes a new and efficient human-object interaction recognition(HOIR) model which is based on modeling human pose and scene featureinformation. There are different aspects involved in an interaction, includingthe humans, the objects, the various body parts of the human, and the backgroundscene. Themain objectives of this research include critically examiningthe importance of all these elements in determining the interaction, estimatinghuman pose through image foresting transform (IFT), and detecting the performedinteractions based on an optimizedmulti-feature vector. The proposedmethodology has six main phases. The first phase involves preprocessing theimages. During preprocessing stages, the videos are converted into imageframes. Then their contrast is adjusted, and noise is removed. In the secondphase, the human-object pair is detected and extracted from each image frame.The third phase involves the identification of key body parts of the detectedhumans using IFT. The fourth phase relates to three different kinds of featureextraction techniques. Then these features are combined and optimized duringthe fifth phase. The optimized vector is used to classify the interactions in thelast phase. TheMSRDaily Activity 3D dataset has been used to test this modeland to prove its efficiency. The proposed system obtains an average accuracyof 91.7% on this dataset.

A Novel Human Interaction Framework Using Quadratic Discriminant Analysis with HMM

Human-human interaction recognition is crucial in computer vision fields like surveillance,human-computer interaction,and social robotics.It enhances systems’ability to interpret and respond to human behavior precisely.This research focuses on recognizing human interaction behaviors using a static image,which is challenging due to the complexity of diverse actions.The overall purpose of this study is to develop a robust and accurate system for human interaction recognition.This research presents a novel image-based human interaction recognition method using a Hidden Markov Model(HMM).The technique employs hue,saturation,and intensity(HSI)color transformation to enhance colors in video frames,making them more vibrant and visually appealing,especially in low-contrast or washed-out scenes.Gaussian filters reduce noise and smooth imperfections followed by silhouette extraction using a statistical method.Feature extraction uses the features from Accelerated Segment Test(FAST),Oriented FAST,and Rotated BRIEF(ORB)techniques.The application of Quadratic Discriminant Analysis(QDA)for feature fusion and discrimination enables high-dimensional data to be effectively analyzed,thus further enhancing the classification process.It ensures that the final features loaded into the HMM classifier accurately represent the relevant human activities.The impressive accuracy rates of 93%and 94.6%achieved in the BIT-Interaction and UT-Interaction datasets respectively,highlight the success and reliability of the proposed technique.The proposed approach addresses challenges in various domains by focusing on frame improvement,silhouette and feature extraction,feature fusion,and HMM classification.This enhances data quality,accuracy,adaptability,reliability,and reduction of errors.

Effect of a Thermal Spring Water on Carbohydrate-Protein Interactions in In-Vitro Models Implicating Normal Human Keratinocytes and Recombinant Lectins

Background: Sugar moiety of macromolecules is today very well known for its implications in many biological recognition mechanisms including cell-cell, extracellular matrix-cell and/or bacteria-cell interactions. In this context lectins, which are carbohydrate-binding proteins displaying a high affinity for sugar groups of other molecules, are of a great importance, notably in immune response involving bacteria, viruses and fungi. As protein-carbohydrate interactions are often mediated by ions such as calcium, zinc or magnesium, we were prompted to study the effect of a thermal spring water (which contains this type of component) on interactions existing between: 1) osidic receptors of human normal keratinocytes and 2) two lectins greatly implicated in the immune response mechanisms (i.e. the dectin-1 and the langerin), and their ligands. Materials and Methods: In a first series of experiments, we studied the effect of increasing concentrations of a thermal spring water on interactions existing between glycosylated molecules and the osidic receptors expressed at the normal human keratinocytes surface. In a second step, and in order to better understand the putative effect of our thermal spring water on the immune response, we analyzed its effect on the interactions existing between the dectin-1 (implicated in the recognition of bacteria, viruses and fungi) and the langerin (expressed by Langerhans cells, the immune cells of the cutaneous tissue), and their ligands in a model using recombinant human lectins and appropriate binding molecules. Results: We showed here that our thermal spring water was able to reinforce interactions between keratinocytes osidic receptors and some of their ligands, in a dose-related manner: From 8% to 55% of increase with 10% to 30% (v/v) of thermal spring water. In the second part of our studies, we also showed that our thermal spring water was able to modulate interactions between dectin-1 and langerin and their ligands through a biphasic effect: Interactions were enhanced by more than 40% and 20% respectively with 10% of thermal spring water, and return to their basal level or lower for higher concentrations. Conclusion: The tested thermal spring water, probably due to its ionic composition, could significantly affect interactions of osidic receptors with their ligands. This property could be of a great interest to help immune system to maintain an appropriate “vigilance state” by using the thermal water at up to a concentration of 10%, and by avoiding any runaway reaction in case of aggression, by using concentrations higher than 10%. .

Human Interaction Dynamics for Its Use in Mobile Robotics:Impedance Control for Leader-follower Formation

A complete characterization of the behavior in human-robot interactions(HRI) includes both: the behavioral dynamics and the control laws that characterize how the behavior is regulated with the perception data. In this way, this work proposes a leader-follower coordinate control based on an impedance control that allows to establish a dynamic relation between social forces and motion error. For this, a scheme is presented to identify the impedance based on fictitious social forces, which are described by distance-based potential fields.As part of the validation procedure, we present an experimental comparison to select the better of two different fictitious force structures. The criteria are determined by two qualities: least impedance errors during the validation procedure and least parameter variance during the recursive estimation procedure.Finally, with the best fictitious force and its identified impedance,an impedance control is designed for a mobile robot Pioneer 3AT,which is programmed to follow a human in a structured scenario.According to results, and under the hypothesis that moving like humans will be acceptable by humans, it is believed that the proposed control improves the social acceptance of the robot for this kind of interaction.

Fabrication of an electrochemical sensor for determination of doxorubicin in human plasma and its interaction with DNA

In this work, an electrochemical sensor was fabricated for determination of an anthracycline, doxorubicin(DOX) as a chemotherapy drug in plasma based on multi-walled carbon nanotubes modified platinum electrode(Pt/MWCNTs). DOX was effectively accumulated on the surface of modified electrode and generated a pair of redox peaks at around 0.522 and 0.647 V(vs. Ag/Ag Cl) in Britton Robinson(B-R) buffer(p H 4.0, 0.1 M). The electrochemical parameters including p H, type of buffer, accumulation time, amount of modifier and scan rate were optimized. Under the optimized conditions, there was a linear correlation between cathodic peak current and concentration of DOX in the range of 0.05–4.0 μg/m L with the detection limit of 0.002 μg/m L. The number of electron transfers(n) and electron transfer-coefficient(α) were estimated as 2.0 and 0.25, respectively. The constructed sensor displayed excellent precision, sensitivity, repeatability and selectivity in the determination of DOX in plasma. Moreover, cyclic voltammetry studies of DOX in the presence of DNA showed an intercalation mechanism with binding constant(K_b) of 1.12×10~5L/mol.

BINDING AND INTERACTION OF TRANSCRIPTIONAL REGULATOR HUMAN DAXX WITH ADENOVIRUS12 E1B 55 KILODALTON ONCOPROTEIN

Objective: To study the disruption of co- localization of human Daxx(hDaxx) with promyelocytic leukemia protein(PML) at the PML oncogenic domains (PODs) by the interaction of hDaxx with adenovirus(Ad) 12 E1B 55 Kilodalton Oncoprotein (Ad12 E1B 55kD). Methods: The direct binding reaction of hDaxx and Ad12 E1B 55kD was analyzed by coimmunoprecipitation and Western blotting in vivo or in vitro. The interaction of hDaxx with Ad12 E1B 55kD was studied using yeast two-hybrid assay. Results: hDaxx bounded directly to Ad12 E1B 55kD in vivo and in vitro. hDaxx interacted with full length Ad12 E1B 55kD. Conclusion: Transcriptional regulator hDaxx directly binds to and interacts with Ad12 E1B 55kD.

Human Pose Estimation and Object Interaction for Sports Behaviour

In the new era of technology,daily human activities are becoming more challenging in terms of monitoring complex scenes and backgrounds.To understand the scenes and activities from human life logs,human-object interaction(HOI)is important in terms of visual relationship detection and human pose estimation.Activities understanding and interaction recognition between human and object along with the pose estimation and interaction modeling have been explained.Some existing algorithms and feature extraction procedures are complicated including accurate detection of rare human postures,occluded regions,and unsatisfactory detection of objects,especially small-sized objects.The existing HOI detection techniques are instancecentric(object-based)where interaction is predicted between all the pairs.Such estimation depends on appearance features and spatial information.Therefore,we propose a novel approach to demonstrate that the appearance features alone are not sufficient to predict the HOI.Furthermore,we detect the human body parts by using the Gaussian Matric Model(GMM)followed by object detection using YOLO.We predict the interaction points which directly classify the interaction and pair them with densely predicted HOI vectors by using the interaction algorithm.The interactions are linked with the human and object to predict the actions.The experiments have been performed on two benchmark HOI datasets demonstrating the proposed approach.

Shaping a New Level of Bus Service under a Novel Concept of Bus Interaction: A Meta-Review

Public transport services, particularly bus services, play an important role in a sustainable transportation system. However, despite various efforts, bus ridership has decreased. The appearance of shared and on-demand vehicle services is one of the main reasons for this issue. In addition, bus tourism services have been successfully developed to meet the exigent needs of tourists. Therefore, a new level of daily bus service is necessary to adapt to the changing demands of customers. Bus interaction (BI) plays an important role in bus services. Nevertheless, the conventional concept of BI mainly refers to users, physical interaction, and safety, but it does not address non-users, non-physical interactions, service quality, and other aspects. This study aims to elaborate on a new concept of bus services. Based on this, we developed a theoretical framework for BI. A meta-analysis was then conducted to identify the achievements and untouched aspects. The results of this study provide three main contributions. First, an unprecedented novel concept of BI is defined, including 13 types of interactions. Second, a comprehensive theoretical framework of BI is established based on the relationships between eight sustainable bus system sub-aspects and 13 BI types. Third, based on the theoretical framework and findings of the reviewed studies, a common finding comprehensive framework of BI is completed, which is accompanied by 1) key findings of the 13 BI types, 2) conclusions of traffic conditions affecting BI research, 3) BI research gaps, and 4) 16 main suggestions for future BI research.

Social interactions in periodic urban markets and their contributions to sustainable livelihoods:Evidence from Ghana

Periodic markets are an important aspect of local economies,providing a platform for farmers(producers),wholesalers,retailers,and consumers to interact face-to-face and exchange goods and services.These markets have been increasing in urban areas in Africa,Asia,and South America because of urbanization.The increase of periodic urban markets(PUMs)in urban areas is observed as an index of modernization,reflecting a response to transition process.However,there are limited studies on how social interactions in PUMs contribute to sustainable livelihoods.This study investigated the types of social interactions occurring in PUMs in Ghana,the benefits of social interactions for participants of PUMs,and how social interactions contribute to sustainable livelihoods.This research interviewed 162 participants,comprising 27 farmers(farmers were regarded as producers in this study),61 retailers,47 wholesalers from 9 selected PUMs across Ghana,and 27 officers from government institutions and non-governmental market associations to obtain their opinions.We analyzed the interview data using the NVivo software.The results showed that there are seven kinds of social interactions in PUMs,including(i)producer-wholesaler relationship,(ii)producer-consumer relationship,(iii)wholesaler-retailer relationship,(iv)retailer-consumer relationship,(v)trader-driver relationship,(vi)trader-institution relationship,and(vii)trader-international buyer relationship.We found that these social interactions in PUMs enhance sustainable livelihoods by supporting human,social,financial,natural,and physical assets of traders(traders refer to producers,wholesalers,and retailers in this study).Therefore,we concluded that the development of policies to improve PUMs could strengthen social interactions,enabling the achievement of sustainable livelihoods in developing countries.

Implementation of Level-3 Autonomous Patient-Specific Quality Assurance with Automated Human Interactive Devices

Purpose: Patient-specific quality assurance (PSQA) requires manual operation of different workstations, which is time-consuming and error-prone. Therefore, developing automated solutions to improve efficiency and accuracy is a priority. The purpose of this study was to develop a general software interface with scripting on a human interactive device (HID) for improving the efficiency and accuracy of manual quality assurance (QA) procedures. Methods: As an initial application, we aimed to automate our PSQA workflow that involves Varian Eclipse treatment planning system, Elekta MOSAIQ oncology information system and PTW Verisoft application. A general platform, the AutoFrame interface with two imbedded subsystems—the AutoFlow and the PyFlow, was developed with a scripting language for automating human operations of aforementioned systems. The interface included three functional modules: GUI module, UDF script interpreter and TCP/IP communication module. All workstations in the PSQA process were connected, and most manual operations were automated by AutoFrame sequentially or in parallel. Results: More than 20 PSQA tasks were performed both manually and using the developed AutoFrame interface. On average, 175 (±12) manual operations of the PSQA procedure were eliminated and performed by the automated process. The time to complete a PSQA task was 8.23 (±0.78) minutes for the automated workflow, in comparison to 13.91 (±3.01) minutes needed for manual operations. Conclusion: We have developed the AutoFrame interface framework that successfully automated our PSQA procedure, and significantly reduced the time, human (control/clicking/typing) errors, and operators’ stress. Future work will focus on improving the system’s flexibility and stability and extending its operations to other QA procedures.

RFHUI:an RFID based human-unmanned aerial vehicle interaction system in an indoor environment

In this paper,we present an RFID based human and Unmanned Aerial Vehicle(UAV)Interaction system,termed RFHUI,to provide an intuitive and easy-to-operate method to navigate a UAV in an indoor environment.It relies on the passive Radio-Frequency IDentification(RFID)technology to precisely track the pose of a handheld controller,and then transfer the pose information to navigate the UAV.A prototype of the handheld controller is created by attaching three or more Ultra High Frequency(UHF)RFID tags to a board.A Commercial Off-The-Shelf(COTS)RFID reader with multiple antennas is deployed to collect the observations of the tags.First,the precise positions of all the tags can be obtained by our proposed method,which leverages a Bayesian filter and Channel State Information(CSI)phase measurements collected from the RFID reader.Second,we introduce a Singular Value Decomposition(SVD)based approach to obtain a 6-DoF(Degrees of Freedom)pose of the controller from estimated positions of the tags.Furthermore,the pose of the controller can be precisely tracked in a real-time manner,while the user moves the controller.Finally,control commands will be generated from the controller's pose and sent to the UAV for navigation.The performance of the RFHUI is evaluated by several experiments.The results show that it provides precise poses with 0.045m mean error in position and 2.5∘mean error in orientation for the controller,and enables the controller to precisely and intuitively navigate the UAV in an indoor environment.

Review of human-animal interactions and their impact on animal productivity and welfare

Humans and animals are in regular and at times close contact in modern intensive farming systems.The quality of human-animal interactions can have a profound impact on the productivity and welfare of farm animals.Interactions by humans may be neutral,positive or negative in nature.Regular pleasant contact with humans may result in desirable alterations in the physiology,behaviour,health and productivity of farm animals.On the contrary,animals that were subjected to aversive human contact were highly fearful of humans and their growth and reproductive performance could be compromised.Farm animals are particularly sensitive to human stimulation that occurs early in life,while many systems of the animals are still developing.This may have long-lasting impact and could possibly modify their genetic potential.The question as to how human contact can have a positive impact on responses to stressors,and productivity is not well understood.Recent work in our laboratory suggested that pleasant human contact may alter ability to tolerate various stressors through enhanced heat shock protein（hsp） 70 expression.The induction of hsp is often associated with increased tolerance to environmental stressors and disease resistance in animals.The attitude and consequent behaviour of stockpeople affect the animals＇ fear of human which eventually influence animals＇ productivity and welfare.Other than attitude and behaviour,technical skills,knowledge,job motivation,commitment and job satisfaction are prerequisites for high job performance.

A Facial Expression Emotion Recognition Based Human-robot Interaction System

A facial expression emotion recognition based human-robot interaction(FEER-HRI) system is proposed, for which a four-layer system framework is designed. The FEERHRI system enables the robots not only to recognize human emotions, but also to generate facial expression for adapting to human emotions. A facial emotion recognition method based on2D-Gabor, uniform local binary pattern(LBP) operator, and multiclass extreme learning machine(ELM) classifier is presented,which is applied to real-time facial expression recognition for robots. Facial expressions of robots are represented by simple cartoon symbols and displayed by a LED screen equipped in the robots, which can be easily understood by human. Four scenarios,i.e., guiding, entertainment, home service and scene simulation are performed in the human-robot interaction experiment, in which smooth communication is realized by facial expression recognition of humans and facial expression generation of robots within 2 seconds. As a few prospective applications, the FEERHRI system can be applied in home service, smart home, safe driving, and so on.

Fusion of Convolutional Self-Attention and Cross-Dimensional Feature Transformationfor Human Posture Estimation

Human posture estimation is a prominent research topic in the fields of human-com-puter interaction,motion recognition,and other intelligent applications.However,achieving highaccuracy in key point localization,which is crucial for intelligent applications,contradicts the lowdetection accuracy of human posture detection models in practical scenarios.To address this issue,a human pose estimation network called AT-HRNet has been proposed,which combines convolu-tional self-attention and cross-dimensional feature transformation.AT-HRNet captures significantfeature information from various regions in an adaptive manner,aggregating them through convolu-tional operations within the local receptive domain.The residual structures TripNeck and Trip-Block of the high-resolution network are designed to further refine the key point locations,wherethe attention weight is adjusted by a cross-dimensional interaction to obtain more features.To vali-date the effectiveness of this network,AT-HRNet was evaluated using the COCO2017 dataset.Theresults show that AT-HRNet outperforms HRNet by improving 3.2%in mAP,4.0%in AP75,and3.9%in AP^(M).This suggests that AT-HRNet can offer more beneficial solutions for human posture estimation.

Information perception and feedback mechanism and key techniques of multi-modality human-robot interaction for service robots

With the increasing of the elderly population and the growing hearth care cost, the role of service robots in aiding the disabled and the elderly is becoming important. Many researchers in the world have paid much attention to heaRthcare robots and rehabilitation robots. To get natural and harmonious communication between the user and a service robot, the information perception/feedback ability, and interaction ability for service robots become more important in many key issues.

Human-Object Interaction Recognition Based on Modeling Context

This paper proposes a method to recognize human-object interactions by modeling context between human actions and interacted objects.Human-object interaction recognition is a challenging task due to severe occlusion between human and objects during the interacting process.Since that human actions and interacted objects provide strong context information,i.e.some actions are usually related to some specific objects,the accuracy of recognition is significantly improved for both of them.Through the proposed method,both global and local temporal features from skeleton sequences are extracted to model human actions.In the meantime,kernel features are utilized to describe interacted objects.Finally,all possible solutions from actions and objects are optimized by modeling the context between them.The results of experiments demonstrate the effectiveness of our method.