Risk identification and safety assessment of human-computer interaction in integrated avionics based on STAMP

To solve the problem of risk identification and quantitative assessment for human-computer interaction(HCI)in complex avionics systems,an HCI safety analysis framework based on system-theoretical process analysis(STPA)and cognitive reliability and error analysis method(CREAM)is proposed.STPACREAM can identify unsafe control actions and find the causal path during the interaction of avionics systems and pilot with the help of formal verification tools automatically.The common performance conditions(CPC)of avionics systems in the aviation environment is established and a quantitative analysis of human failure is carried out.Taking the head-up display(HUD)system interaction process as an example,a case analysis is carried out,the layered safety control structure and formal model of the HUD interaction process are established.For the interactive behavior“Pilots approaching with HUD”,four unsafe control actions and35 causal scenarios are identified and the impact of common performance conditions at different levels on the pilot decision model are analyzed.The results show that HUD's HCI level gradually improves as the scores of CPC increase,and the quality of crew member cooperation and time sufficiency of the task is the key to its HCI.Through case analysis,it is shown that STPACREAM can quantitatively assess the hazards in HCI and identify the key factors that impact safety.

Identification and distribution of 13003 landslides in the northwest margin of Qinghai-Tibet Plateau based on human-computer interaction remote sensing interpretation

The periphery of the Qinghai-Tibet Plateau is renowned for its susceptibility to landslides.However,the northwestern margin of this region,characterised by limited human activities and challenging transportation,remains insufficiently explored concerning landslide occurrence and dispersion.With the planning and construction of the Xinjiang-Tibet Railway,a comprehensive investigation into disastrous landslides in this area is essential for effective disaster preparedness and mitigation strategies.By using the human-computer interaction interpretation approach,the authors established a landslide database encompassing 13003 landslides,collectively spanning an area of 3351.24 km^(2)(36°N-40°N,73°E-78°E).The database incorporates diverse topographical and environmental parameters,including regional elevation,slope angle,slope aspect,distance to faults,distance to roads,distance to rivers,annual precipitation,and stratum.The statistical characteristics of number and area of landslides,landslide number density(LND),and landslide area percentage(LAP)are analyzed.The authors found that a predominant concentration of landslide origins within high slope angle regions,with the highest incidence observed in intervals characterised by average slopes of 20°to 30°,maximum slope angle above 80°,along with orientations towards the north(N),northeast(NE),and southwest(SW).Additionally,elevations above 4.5 km,distance to rivers below 1 km,rainfall between 20-30 mm and 30-40 mm emerge as particularly susceptible to landslide development.The study area’s geological composition primarily comprises Mesozoic and Upper Paleozoic outcrops.Both fault and human engineering activities have different degrees of influence on landslide development.Furthermore,the significance of the landslide database,the relationship between landslide distribution and environmental factors,and the geometric and morphological characteristics of landslides are discussed.The landslide H/L ratios in the study area are mainly concentrated between 0.4 and 0.64.It means the landslides mobility in the region is relatively low,and the authors speculate that landslides in this region more possibly triggered by earthquakes or located in meizoseismal area.

Dynamic constraint and objective generation approach for real-time train rescheduling model under human-computer interaction

Real-time train rescheduling plays a vital role in railway transportation as it is crucial for maintaining punctuality and reliability in rail operations.In this paper,we propose a rescheduling model that incorporates constraints and objectives generated through human-computer interaction.This approach ensures that the model is aligned with practical requirements and daily operational tasks while facilitating iterative train rescheduling.The dispatcher’s empirical knowledge is integrated into the train rescheduling process using a human-computer interaction framework.We introduce six interfaces to dynamically construct constraints and objectives that capture human intentions.By summarizing rescheduling rules,we devise a rule-based conflict detection-resolution heuristic algorithm to effectively solve the formulated model.A series of numerical experiments are presented,demonstrating strong performance across the entire system.Furthermore,theflexibility of rescheduling is enhanced through secondary analysis-driven solutions derived from the outcomes of humancomputer interactions in the previous step.This proposed interaction method complements existing literature on rescheduling methods involving human-computer interactions.It serves as a tool to aid dispatchers in identifying more feasible solutions in accordance with their empirical rescheduling strategies.

Reform of Human-Computer Interaction Course based on Mobile Application Interaction Design

With the popularity of new intelligent mobile devices in people’s lives,the development of mobile applications has paid increasing attention to the interactive experience of users.As the content of traditional Human-Computer Interaction(HCI)course and teaching material is out of date,it cannot meet the needs of mobile application interaction design and enterprises for students.Therefore,we need a new generation HCI course based on intelligent mobile devices to study the relationship between users and systems.The HCI course not only teaches students HCI theory and model,but also needs to cultivate students’interaction-oriented design practical ability.This paper proposes a set of HCI teaching material design and teaching methods for improving HCI class quality on mobile application interaction design,so as to make students more suitable for the employment requirements of enterprises.

Design and Implementation of the Image Interactive System Based on Human-Computer Interaction

Based on the traditional Human-Computer Interaction method which is mainly touch input system, the way of capturing the movement of people by using cameras is proposed. This is a convenient technique which can provide users more experience. In the article, a new way of detecting moving things is given on the basis of development of the image processing technique. The system architecture decides that the communication should be used between two different applications. After considered, named pipe is selected from many ways of communication to make sure that video is keeping in step with the movement from the analysis of the people moving. According to a large amount of data and principal knowledge, thinking of the need of actual project, a detailed system design and realization is finished. The system consists of three important modules: detecting of the people's movement, information transition between applications and video showing in step with people's movement. The article introduces the idea of each module and technique.

Woven fabric triboelectric nanogenerators for human-computer interaction and physical health monitoring

Triboelectric nanogenerator(TENG)converts mechanical energy into valuable electrical energy,offering a solution for future energy needs.As an indispensable part of TENG,textile TENG(T-TENG)has incredible advantages in harvesting biomechanical energy and physiological signal monitoring.However,the application of T-TENG is restricted,partly because the fabric structure parameter and structure on T-TENG performance have not been fully exploited.This study comprehensively investigates the effect of weaving structure on fabric TENGs(F-TENGs)for direct-weaving yarn TENGs and post-coating fabric TENGs.For direct-weaving F-TENGs,a single-yarn TENG(Y-TENG)with a core-sheath structure is fabricated using conductive yarn as the core layer yarn and polytetrafluoroethylene(PTFE)filaments as the sheath yarn.Twelve fabrics with five different sets of parameters were designed and investigated.For post-coating F-TENGs,fabrics with weaving structures of plain,twill,satin,and reinforced twill were fabricated and coated with conductive silver paint.Overall,the twill F-TENGs have the best electrical outputs,followed by the satin F-TENGs and plain weave F-TENGs.Besides,the increase of the Y-TENG gap spacing was demonstrated to improve the electrical output performance.Moreover,T-TENGs are demonstrated for human-computer interaction and self-powered real-time monitoring.This systematic work provides guidance for the future T-TENG’s design.

RFHUI:an RFID based human-unmanned aerial vehicle interaction system in an indoor environment

In this paper,we present an RFID based human and Unmanned Aerial Vehicle(UAV)Interaction system,termed RFHUI,to provide an intuitive and easy-to-operate method to navigate a UAV in an indoor environment.It relies on the passive Radio-Frequency IDentification(RFID)technology to precisely track the pose of a handheld controller,and then transfer the pose information to navigate the UAV.A prototype of the handheld controller is created by attaching three or more Ultra High Frequency(UHF)RFID tags to a board.A Commercial Off-The-Shelf(COTS)RFID reader with multiple antennas is deployed to collect the observations of the tags.First,the precise positions of all the tags can be obtained by our proposed method,which leverages a Bayesian filter and Channel State Information(CSI)phase measurements collected from the RFID reader.Second,we introduce a Singular Value Decomposition(SVD)based approach to obtain a 6-DoF(Degrees of Freedom)pose of the controller from estimated positions of the tags.Furthermore,the pose of the controller can be precisely tracked in a real-time manner,while the user moves the controller.Finally,control commands will be generated from the controller's pose and sent to the UAV for navigation.The performance of the RFHUI is evaluated by several experiments.The results show that it provides precise poses with 0.045m mean error in position and 2.5∘mean error in orientation for the controller,and enables the controller to precisely and intuitively navigate the UAV in an indoor environment.

Design of space-centered interaction using invisible and intangible spatial inputs

In this paper,we investigate methodologies to improve direct-touch interaction on invisible and intangible spatial input.We firstly discuss about the motive of looking for a new input method for whole body interaction and how it can be meaningful.We also describe the role that can play spatial interaction to improve the freedom of interaction for a user.We propose a method of spatial centered interaction using invisible and intangible spatial inputs.However,given their lack of tactile feedback and visual representation,direct touch interaction on such input can be confused.In order to make a step toward understanding causes and solutions for such phenomena,we made 2 user experiments.In the first one,we test 5 setups of helper that provide information of the location of the input by constraining the dimension it is located at.The results show that using marker on the ground and a relationship with the height of the user’s body improve significantly the locative task.In the second experiment,we create a dancing game using invisible and intangible spatial inputs and we stress the results obtained in the first experiment within this cognitively demanding context.Results show that the same setup of helper is still providing very good results in that context.

Multi-View Multi-Modal Head-Gaze Estimation for Advanced Indoor User Interaction

Gaze estimation is one of the most promising technologies for supporting indoor monitoring and interaction systems.However,previous gaze estimation techniques generally work only in a controlled laboratory environment because they require a number of high-resolution eye images.This makes them unsuitable for welfare and healthcare facilities with the following challenging characteristics:1)users’continuous movements,2)various lighting conditions,and 3)a limited amount of available data.To address these issues,we introduce a multi-view multi-modal head-gaze estimation system that translates the user’s head orientation into the gaze direction.The proposed system captures the user using multiple cameras with depth and infrared modalities to train more robust gaze estimators under the aforementioned conditions.To this end,we implemented a deep learning pipeline that can handle different types and combinations of data.The proposed system was evaluated using the data collected from 10 volunteer participants to analyze how the use of single/multiple cameras and modalities affect the performance of head-gaze estimators.Through various experiments,we found that 1)an infrared-modality provides more useful features than a depth-modality,2)multi-view multi-modal approaches provide better accuracy than singleview single-modal approaches,and 3)the proposed estimators achieve a high inference efficiency that can be used in real-time applications.

Empathic Responses of Behavioral-Synchronization in Human-Agent Interaction

Artificial entities,such as virtual agents,have become more pervasive.Their long-term presence among humans requires the virtual agent’s ability to express appropriate emotions to elicit the necessary empathy from the users.Affective empathy involves behavioral mimicry,a synchronized co-movement between dyadic pairs.However,the characteristics of such synchrony between humans and virtual agents remain unclear in empathic interactions.Our study evaluates the participant’s behavioral synchronization when a virtual agent exhibits an emotional expression congruent with the emotional context through facial expressions,behavioral gestures,and voice.Participants viewed an emotion-eliciting video stimulus(negative or positive)with a virtual agent.The participants then conversed with the virtual agent about the video,such as how the participant felt about the content.The virtual agent expressed emotions congruent with the video or neutral emotion during the dialog.The participants’facial expressions,such as the facial expressive intensity and facial muscle movement,were measured during the dialog using a camera.The results showed the participants’significant behavioral synchronization(i.e.,cosine similarity≥.05)in both the negative and positive emotion conditions,evident in the participant’s facial mimicry with the virtual agent.Additionally,the participants’facial expressions,both movement and intensity,were significantly stronger in the emotional virtual agent than in the neutral virtual agent.In particular,we found that the facial muscle intensity of AU45(Blink)is an effective index to assess the participant’s synchronization that differs by the individual’s empathic capability(low,mid,high).Based on the results,we suggest an appraisal criterion to provide empirical conditions to validate empathic interaction based on the facial expression measures.

Designing interactive glazing through an engineering psychology approach:Six augmented reality scenarios that envision future car human-machine interface

Background With an increasing number of vehicles becoming autonomous,intelligent,and connected,paying attention to the future usage of car human-machine interface with these vehicles should become more relevant.Several studies have addressed car HMI but were less attentive to designing and implementing interactive glazing for every day(autonomous)driving contexts.Methods Reflecting on the literature,we describe an engineering psychology practice and the design of six novel future user scenarios,which envision the application of a specific set of augmented reality(AR)support user interactions.Additionally,we conduct evaluations on specific scenarios and experiential prototypes,which reveal that these AR scenarios aid the target user groups in experiencing a new type of interaction.The overall evaluation is positive with valuable assessment results and suggestions.Conclusions This study can interest applied psychology educators who aspire to teach how AR can be operationalized in a human-centered design process to students with minimal pre-existing expertise or minimal scientific knowledge in engineering psychology.

Virtual Keyboard:A Real-Time Hand Gesture Recognition-Based Character Input System Using LSTM and Mediapipe Holistic

In the digital age,non-touch communication technologies are reshaping human-device interactions and raising security concerns.A major challenge in current technology is the misinterpretation of gestures by sensors and cameras,often caused by environmental factors.This issue has spurred the need for advanced data processing methods to achieve more accurate gesture recognition and predictions.Our study presents a novel virtual keyboard allowing character input via distinct hand gestures,focusing on two key aspects:hand gesture recognition and character input mechanisms.We developed a novel model with LSTM and fully connected layers for enhanced sequential data processing and hand gesture recognition.We also integrated CNN,max-pooling,and dropout layers for improved spatial feature extraction.This model architecture processes both temporal and spatial aspects of hand gestures,using LSTM to extract complex patterns from frame sequences for a comprehensive understanding of input data.Our unique dataset,essential for training the model,includes 1,662 landmarks from dynamic hand gestures,33 postures,and 468 face landmarks,all captured in real-time using advanced pose estimation.The model demonstrated high accuracy,achieving 98.52%in hand gesture recognition and over 97%in character input across different scenarios.Its excellent performance in real-time testing underlines its practicality and effectiveness,marking a significant advancement in enhancing human-device interactions in the digital age.

A Survey of Gesture Recognition Using Frequency Modulated Continuous Wave Radar

With technology advances and human requirements increasing, human-computer interaction plays an important role in our daily lives. Among these interactions, gesture-based recognition offers a natural and intuitive user experience that does not require physical contact and is becoming increasingly prevalent across various fields. Gesture recognition systems based on Frequency Modulated Continuous Wave (FMCW) millimeter-wave radar are receiving widespread attention due to their ability to operate without wearable sensors, their robustness to environmental factors, and the excellent penetrative ability of radar signals. This paper first reviews the current main gesture recognition applications. Subsequently, we introduce the system of gesture recognition based on FMCW radar and provide a general framework for gesture recognition, including gesture data acquisition, data preprocessing, and classification methods. We then discuss typical applications of gesture recognition systems and summarize the performance of these systems in terms of experimental environment, signal acquisition, signal processing, and classification methods. Specifically, we focus our study on four typical gesture recognition systems, including air-writing recognition, gesture command recognition, sign language recognition, and text input recognition. Finally, this paper addresses the challenges and unresolved problems in FMCW radar-based gesture recognition and provides insights into potential future research directions.

Gesture Recognition Based on Time-of-Flight Sensor and Residual Neural Network

With the advancement of technology and the increase in user demands, gesture recognition played a pivotal role in the field of human-computer interaction. Among various sensing devices, Time-of-Flight (ToF) sensors were widely applied due to their low cost. This paper explored the implementation of a human hand posture recognition system using ToF sensors and residual neural networks. Firstly, this paper reviewed the typical applications of human hand recognition. Secondly, this paper designed a hand gesture recognition system using a ToF sensor VL53L5. Subsequently, data preprocessing was conducted, followed by training the constructed residual neural network. Then, the recognition results were analyzed, indicating that gesture recognition based on the residual neural network achieved an accuracy of 98.5% in a 5-class classification scenario. Finally, the paper discussed existing issues and future research directions.

一种基于设计模式的GIS HCI复用框架

基于特征数据模型的GIS软件开发是一个重要方向。这就需要建立符合其特征的软件平台,以提高开发效率,减少重复应用。该文从软件框架的角度,利用设计模式分析了特征模型的软件结构,提出了GIS HCI框架,并进行了分析。

我国管理信息系统(MIS)领域人机交互(HCI)研究评述

本文通过对我国1990年至2006年MIS领域HCI论文文献进行实证研究,从研究主题、研究方法、参考学科、分析层次、研究情境、个体特征6个角度分析了我国MIS领域HCI研究的现状,并通过与国外相关研究的比较,发现我国MIS领域HCI研究在研究主题、研究方法和研究情景方面存在的差距。最后对今后我国的HCI研究做出展望,指出对我国学者有价值的HCI研究领域。

Mixed Reality-based Interactive Technology for Aircraft Cabin Assembly

Due to the narrowness of space and the complexity of structure, the assembly of aircraft cabin has become one of the major bottlenecks in the whole manufacturing process. To solve the problem, at the beginning of aircraft design, the different stages of the lifecycle of aircraft must be thought about, which include the trial manufacture, assembly, maintenance, recycling and destruction of the product. Recently, thanks to the development of the virtual reality and augmented reality, some low-cost and fast solutions are found for the product assembly. This paper presents a mixed reality-based interactive technology for the aircraft cabin assembly, which can enhance the efficiency of the assemblage in a virtual environment in terms of vision, information and operation. In the mixed reality-based assembly environment, the physical scene can be obtained by a camera and then generated by a computer. The virtual parts, the features of visual assembly, the navigation information, the physical parts and the physical assembly environment will be mixed and presented in the same assembly scene. The mixed or the augmented information will provide some assembling information as a detailed assembly instruction in the mixed reality-based assembly environment. Constraint proxy and its match rules help to reconstruct and visualize the restriction relationship among different parts, and to avoid the complex calculation of constraint＇s match. Finally, a desktop prototype system of virtual assembly has been built to assist the assembly verification and training with the virtual hand.

Vision-Based Hand Gesture Recognition for Human-Computer Interaction——A Survey

Recently,vision-based gesture recognition(VGR)has become a hot research spot in human-computer interaction(HCI).Unlike other gesture recognition methods with data gloves or other wearable sensors,vision-based gesture recognition could lead to more natural and intuitive HCI interactions.This paper reviews the state-of-the-art vision-based gestures recognition methods,from different stages of gesture recognition process,i.e.,(1)image acquisition and pre-processing,(2)gesture segmentation,(3)gesture tracking,(4)feature extraction,and(5)gesture classification.This paper also analyzes the advantages and disadvantages of these various methods in detail.Finally,the challenges of vision-based gesture recognition in haptic rendering and future research directions are discussed.

VR技术在核电厂HCI设计和评价中的应用研究

虚拟现实(VR)技术作为新一代人机交互技术,为核电厂(NPP)人机交互(HCI)设计和评价提供了一种全新的技术手段。利用VR技术,可以有效解决传统HCI设计和评价中的症结问题。通过分析VR技术特点,结合核电厂HCI设计和评价传统方法中存在的问题,提出了VR在HCI设计和评价应用中的四项功能需求:3D可视化沉浸式设计、人因工程设计验证、虚拟仿真评价、培训教学。重点介绍了虚拟人机交互设计和评价系统研发实施过程,阐述了系统研发的技术路线、平台架构和功能分解,并对该系统的研发成果和应用方向作了简要分析。VR技术在核电厂HCI设计和评价中的应用,可以有效解决传统设计和评价方法中的症结问题,实现HCI设计和评价一体化,提升HCI设计质量和设计效率,提高人因可靠性。