An Architecture Supporting Intelligent Mobile Healthcare Using Human-Computer Interaction HCI Principles

Recent advancements in the Internet of Things IoT and cloud computing have paved the way for mobile Healthcare(mHealthcare)services.A patient within the hospital is monitored by several devices.Moreover,upon leaving the hospital,the patient can be remotely monitored whether directly using body wearable sensors or using a smartphone equipped with sensors to monitor different user-health parameters.This raises potential challenges for intelligent monitoring of patient's health.In this paper,an improved architecture for smart mHealthcare is proposed that is supported by HCI design principles.The HCI also provides the support for the User-Centric Design(UCD)for smart mHealthcare models.Furthermore,the HCI along with IoT's(Internet of Things)5-layered architecture has the potential of improving User Experience(UX)in mHealthcare design and help saving lives.The intelligent mHealthcare system is supported by the IoT sensing and communication layers and health care providers are supported by the application layer for the medical,behavioral,and health-related information.Health care providers and users are further supported by an intelligent layer performing critical situation assessment and performing a multi-modal communication using an intelligent assistant.The HCI design focuses on the ease-of-use,including user experience and safety,alarms,and error-resistant displays of the end-user,and improves user's experience and user satisfaction.

Context-Aware Augmented Reality Using Human-Computer Interaction Models

Augmented Reality is a technique that allows users to overlap digital information with their physical world.The Augmented Reality(AR)displays have an exceptional characteristic from the Human–Computer Interaction(HCI)perspective.Due to its increasing popularity and application in diverse domains,increasing user-friendliness and AR usage are critical.Context-aware is one approach since an AR application can adapt to the user,environment,needs and enhance ergonomic principles and functionality.This paper proposes the Intelligent Contextaware Augmented Reality Model(ICAARM)for Human–Computer Interaction systems.This study explores and reduces interaction uncertainty by semantically modeling user-specific interaction with context,allowing personalised interaction.Sensory information is captured from an AR device to understand user interactions and context.These depictions carry semantics to Augmented Reality applications about the user’s intention to interact with a specific device affordance.Thus,this study describes personalised gesture interaction in VR/AR applications for immersive/intelligent environments.

RFHUI:an RFID based human-unmanned aerial vehicle interaction system in an indoor environment

In this paper,we present an RFID based human and Unmanned Aerial Vehicle(UAV)Interaction system,termed RFHUI,to provide an intuitive and easy-to-operate method to navigate a UAV in an indoor environment.It relies on the passive Radio-Frequency IDentification(RFID)technology to precisely track the pose of a handheld controller,and then transfer the pose information to navigate the UAV.A prototype of the handheld controller is created by attaching three or more Ultra High Frequency(UHF)RFID tags to a board.A Commercial Off-The-Shelf(COTS)RFID reader with multiple antennas is deployed to collect the observations of the tags.First,the precise positions of all the tags can be obtained by our proposed method,which leverages a Bayesian filter and Channel State Information(CSI)phase measurements collected from the RFID reader.Second,we introduce a Singular Value Decomposition(SVD)based approach to obtain a 6-DoF(Degrees of Freedom)pose of the controller from estimated positions of the tags.Furthermore,the pose of the controller can be precisely tracked in a real-time manner,while the user moves the controller.Finally,control commands will be generated from the controller's pose and sent to the UAV for navigation.The performance of the RFHUI is evaluated by several experiments.The results show that it provides precise poses with 0.045m mean error in position and 2.5∘mean error in orientation for the controller,and enables the controller to precisely and intuitively navigate the UAV in an indoor environment.

Fire Hawk Optimizer with Deep Learning Enabled Human Activity Recognition

Human-Computer Interaction(HCI)is a sub-area within computer science focused on the study of the communication between people(users)and computers and the evaluation,implementation,and design of user interfaces for computer systems.HCI has accomplished effective incorporation of the human factors and software engineering of computing systems through the methods and concepts of cognitive science.Usability is an aspect of HCI dedicated to guar-anteeing that human–computer communication is,amongst other things,efficient,effective,and sustaining for the user.Simultaneously,Human activity recognition(HAR)aim is to identify actions from a sequence of observations on the activities of subjects and the environmental conditions.The vision-based HAR study is the basis of several applications involving health care,HCI,and video surveillance.This article develops a Fire Hawk Optimizer with Deep Learning Enabled Activ-ity Recognition(FHODL-AR)on HCI driven usability.In the presented FHODL-AR technique,the input images are investigated for the identification of different human activities.For feature extraction,a modified SqueezeNet model is intro-duced by the inclusion of few bypass connections to the SqueezeNet among Fire modules.Besides,the FHO algorithm is utilized as a hyperparameter optimization algorithm,which in turn boosts the classification performance.To detect and cate-gorize different kinds of activities,probabilistic neural network(PNN)classifier is applied.The experimental validation of the FHODL-AR technique is tested using benchmark datasets,and the outcomes reported the improvements of the FHODL-AR technique over other recent approaches.

Home-based Detection and Prediction of Diabetic Foot Ulcers at Early Stage Using Sensor Technology and Supervised Learning

For years,foot ulcers linked with diabetes mellitus and neuropathy have significantly impacted diabetic patients’ health-related quality of life(HRQoL). Diabetes foot ulcers impact15% of all diabetic patients at some point in their lives. The facilities and resources used for DFU detection and treatment are only available at hospitals and clinics,which results in the unavailability of feasible and timely detection at an early stage. This necessitates the development of an at-home DFU detection system that enables timely predictions and seamless communication with users,thereby preventing amputations due to neglect and severity. This paper proposes a feasible system consisting of three major modules:an IoT device that works to sense foot nodes to send vibrations onto a foot sole,a machine learning model based on supervised learning which predicts the level of severity of the DFU using four different classification techniques including XGBoost,K-SVM,Random Forest,and Decision tree,and a mobile application that acts as an interface between the sensors and the patient. Based on the severity levels,necessary steps for prevention,treatment,and medications are recommended via the application.

VR and AR in human performance research―An NUS experience

With the mindset of constant improvement in efficiency and safety in the workspace and training in Singapore,there is a need to explore varying technologies and their capabilities to fulfil this need.The ability of Virtual Reality(VR)and Augmented Reality(AR)to create an immersive experience of tying the virtual and physical environments coupled with information filtering capabilities brings a possibility of introducing this technology into the training process and workspace.This paper surveys current research trends,findings and limitation of VR and AR in its effect on human performance,specifically in Singapore,and our experience in the National University of Singapore(NUS).

COMTIS:Customizable touchless interaction system for large screen visualization

Background Large screen visualization sys tems have been widely utilized in many industries.Such systems can help illustrate the working states of different production systems.However,efficient interaction with such systems is still a focus of related research.Methods In this paper,we propose a touchless interaction system based on RGB-D camera using a novel bone-length constraining method.The proposed method optimizes the joint data collected from RGB-D cameras with more accurate and more stable results on very noisy data.The user can customize the system by modifying the finite-state machine in the system and reuse the gestures in multiple scenarios,reducing the number of gestures that need to be designed and memorized.Results/Conclusions The authors tested the system in two cases.In the first case,we illustrated a process in which we improved the gesture designs on our system and tested the system through user study.In the second case,we utilized the system in the mining industry and conducted a user study,where users say that they think the system is easy to use.

A Robust Approach for Detection and Classification of KOA Based on BILSTM Network

A considerable portion of the population now experiences osteoarthritis of the knee,spine,and hip due to lifestyle changes.Therefore,early treatment,recognition and prevention are essential to reduce damage;nevertheless,this time-consuming activity necessitates a variety of tests and in-depth analysis by physicians.To overcome the existing challenges in the early detection of Knee Osteoarthritis(KOA),an effective automated technique,prompt recognition,and correct categorization are required.This work suggests a method based on an improved deep learning algorithm that makes use of data from the knee images after segmentation to detect KOA and its severity using the Kellgren-Lawrence(KL) classification schemes,such as Class-I,Class-II,Class-III,and Class-IV.Utilizing ResNet to segregate knee pictures,we first collected features from these images before using the Bidirectional Long Short-Term Memory(BiLSTM)architecture to classify them.Given that the technique is a pre-trained network and doesn’t require a large training set,the Mendeley VI dataset has been utilized for the training of the proposed model.To evaluate the effectiveness of the suggested model,cross-validation has also been employed using the Osteoarthritis Initiative(OAI)dataset.Furthermore,our suggested technique is more resilient,which overcomes the challenge of imbalanced training data due to the hybrid architecture of our proposed model.The suggested algorithm is a cuttingedge and successful method for documenting the successful application of the timely identification and severity categorization of KOA.The algorithm showed a cross-validation accuracy of 78.57%and a testing accuracy of 84.09%.Numerous tests have been conducted to show that our suggested algorithm is more reliable and capable than the state-of-the-art at identifying and categorizing KOA disease.

Home Automation-Based Health Assessment Along Gesture Recognition via Inertial Sensors

Hand gesture recognition (HGR) is used in a numerous applications,including medical health-care, industrial purpose and sports detection.We have developed a real-time hand gesture recognition system using inertialsensors for the smart home application. Developing such a model facilitatesthe medical health field (elders or disabled ones). Home automation has alsobeen proven to be a tremendous benefit for the elderly and disabled. Residentsare admitted to smart homes for comfort, luxury, improved quality of life,and protection against intrusion and burglars. This paper proposes a novelsystem that uses principal component analysis, linear discrimination analysisfeature extraction, and random forest as a classifier to improveHGRaccuracy.We have achieved an accuracy of 94% over the publicly benchmarked HGRdataset. The proposed system can be used to detect hand gestures in thehealthcare industry as well as in the industrial and educational sectors.

Eye-shaped keyboard for dual-hand text entry in virtual reality

We propose an eye-shaped keyboard for high-speed text entry in virtual reality (VR), having the shape of dual eyes with characters arranged along the curved eyelids, which ensures low density and short spacing of the keys. The eye-shaped keyboard references the QWERTY key sequence, allowing the users to benefit from their experience using the QWERTY keyboard. The user interacts with an eye-shaped keyboard using rays controlled with both the hands. A character can be entered in one step by moving the rays from the inner eye regions to regions of the characters. A high-speed auto-complete system was designed for the eye-shaped keyboard. We conducted a pilot study to determine the optimal parameters, and a user study to compare our eye-shaped keyboard with the QWERTY and circular keyboards. For beginners, the eye-shaped keyboard performed significantly more efficiently and accurately with less task load and hand movement than the circular keyboard. Compared with the QWERTY keyboard, the eye-shaped keyboard is more accurate and significantly reduces hand translation while maintaining similar efficiency. Finally, to evaluate the potential of eye-shaped keyboards, we conducted another user study. In this study, the participants were asked to type continuously for three days using the proposed eye-shaped keyboard, with two sessions per day. In each session, participants were asked to type for 20min, and then their typing performance was tested. The eye-shaped keyboard was proven to be efficient and promising, with an average speed of 19.89 words per minute (WPM) and mean uncorrected error rate of 1.939%. The maximum speed reached 24.97 WPM after six sessions and continued to increase.

The Research of System Architecture in Expert System

This paper illustrated the software architecture of our concrete safety expert system. Three advanced technologies are proposed and have been applied to our expert system to greatly improve the intelligent level, which are human computer interaction technology (conceptual model, dialogue management, interface entity and interface construct), intelligent agency user interface (IAUI) and component technology. The important character of the system architecture in our expert system is adapting advanced intelligent interface and scientific integration of various components different from common system architecture of expert system. Especially, in the interface\|friendly multimedia system intelligent interface is required.

Robust video foreground segmentation and face recognition

Face recognition provides a natural visual interface for human computer interaction （HCI） applications. The process of face recognition, however, is inhibited by variations in the appearance of face images caused by changes in lighting, expression, viewpoint, aging and introduction of occlusion. Although various algorithms have been presented for face recognition, face recognition is still a very challenging topic. A novel approach of real time face recognition for HCI is proposed in the paper. In view of the limits of the popular approaches to foreground segmentation, wavelet multi-scale transform based background subtraction is developed to extract foreground objects. The optimal selection of the threshold is automatically determined, which does not require any complex supervised training or manual experimental calibration. A robust real time face recognition algorithm is presented, which combines the projection matrixes without iteration and kernel Fisher discriminant analysis （KFDA） to overcome some difficulties existing in the real face recognition. Superior performance of the proposed algorithm is demonstrated by comparing with other algorithms through experiments. The proposed algorithm can also be applied to the video image sequences of natural HCI.

Eye movement prediction by oculomotor plant Kalman filter with brainstem control

Our work addresses one of the core issues related to Human Computer Interaction （HCI） systems that use eye gaze as an input. This issue is the sensor, transmission and other delays that exist in any eye tracker-based system, reducing its performance. A delay effect can be compensated by an accurate prediction of the eye movement trajectories. This paper introduces a mathematical model of the human eye that uses anatomical properties of the Human Visual System to predict eye movement trajectories. The eye mathematical model is transformed into a Kalman filter form to provide continuous eye position signal prediction during all eye movement types. The model presented in this paper uses brainstem control properties employed during transitions between fast （saccade） and slow （fixations, pursuit） eye movements. Results presented in this paper indicate that the proposed eye model in a Kalman filter form improves the accuracy of eye movement prediction and is capable of a real-time performance. In addition to the HCI systems with the direct eye gaze input, the proposed eye model can be immediately applied for a bit-rate/computational reduction in real-time gaze-contingent systems

AR-assisted children book for smart teaching and learning of Turkish alphabets

Background Augmented reality(AR),virtual reality(VR),and remote-controlled devices are driving the need for a better 5 G infrastructure to support faster data transmission.In this study,mobile AR is emphasized as a viable and widespread solution that can be easily scaled to millions of end-users and educators because it is lightweight and low-cost and can be implemented in a cross-platform manner.Low-efficiency smart devices and high latencies for real-time interactions via regular mobile networks are primary barriers to the use of AR in education.New 5 G cellular networks can mitigate some of these issues via network slicing,device-to-device communication,and mobile edge computing.Methods In this study,we use a new technology to solve some of these problems.The proposed software monitors image targets on a printed book and renders 3 D objects and alphabetic models.In addition,the application considers phonetics.The sound(phonetic)and 3 D representation of a letter are played as soon as the image target is detected.3 D models of the Turkish alphabet are created by using Adobe Photoshop with Unity 3 D and Vuforia SDK.Results The proposed application teaches Turkish alphabets and phonetics by using 3 D object models,3 D letters,and 3 D phrases,including letters and sounds.

Review of dynamic gesture recognition

In recent years,gesture recognition has been widely used in the fields of intelligent driving,virtual reality,and human-computer interaction.With the development of artificial intelligence,deep learning has achieved remarkable success in computer vision.To help researchers better understanding the development status of gesture recognition in video,this article provides a detailed survey of the latest developments in gesture recognition technology for videos based on deep learning.The reviewed methods are broadly categorized into three groups based on the type of neural networks used for recognition:two stream convolutional neural networks,3D convolutional neural networks,and Long-short Term Memory(LSTM)networks.In this review,we discuss the advantages and limitations of existing technologies,focusing on the feature extraction method of the spatiotemporal structure information in a video sequence,and consider future research directions.

Mobile Devices Interface Adaptivity Using Ontologies

Currently,many mobile devices provide various interaction styles and modes which create complexity in the usage of interfaces.The context offers the information base for the development of Adaptive user interface(AUI)frameworks to overcome the heterogeneity.For this purpose,the ontological modeling has been made for specific context and environment.This type of philosophy states to the relationship among elements(e.g.,classes,relations,or capacities etc.)with understandable satisfied representation.The contextmechanisms can be examined and understood by anymachine or computational framework with these formal definitions expressed in Web ontology language(WOL)/Resource description frame work(RDF).The Protégéis used to create taxonomy in which system is framed based on four contexts such as user,device,task and environment.Some competency questions and use-cases are utilized for knowledge obtaining while the information is refined through the instances of concerned parts of context tree.The consistency of the model has been verified through the reasoning software while SPARQL querying ensured the data availability in the models for defined use-cases.The semantic context model is focused to bring in the usage of adaptive environment.This exploration has finished up with a versatile,scalable and semantically verified context learning system.This model can be mapped to individual User interface(UI)display through smart calculations for versatile UIs.

启明星UI支持系统应用研究

"企业信息化"是指通过对信息技术的应用、开发和使用企业的信息资源,提高管理水平、开发能力、经营水平的过程。中国从1994年正式接入互联网以来,二十多年的野蛮高速发展,中国的企业信息化建设已经取得了巨大成就。该文将从企业信息化建设发展趋势和信息化产品开发流程角度研究分析"启明星UI支持系统"研发的必要性和重要性。

Training birdsong recognition using virtual reality

Background In mega-biodiverse environments,where different species are more likely to be heard than seen,species monitoring is generally performed using bioacoustics methodologies.Furthermore,since bird vocalizations are reasonable estimators of biodiversity,their monitoring is of great importance in the formulation of conservation policies.However,birdsong recognition is an arduous task that requires dedicated training in order to achieve mastery,which is costly in terms of time and money due to the lack of accessibility of relevant information in field trips or even specialized databases.Immersive technology based on virtual reality(VR)and spatial audio may improve species monitoring by enhancing information accessibility,interaction,and user engagement.Methods This study used spatial audio,a Bluetooth controller,and a head-mounted display(HMD)to conduct an immersive training experience in VR.Participants moved inside a virtual world using a Bluetooth controller,while their task was to recognize targeted birdsongs.We measured the accuracy of recognition and user engagement according to the User Engagement Scale.Results The experimental results revealed significantly higher engagement and accuracy for participants in the VR-based training system than in a traditional computer-based training system.All four dimensions of the user engagement scale received high ratings from the participants,suggesting that VR-based training provides a motivating and attractive environment for learning demanding tasks through appropriate design,exploiting the sensory system,and virtual reality interactivity.Conclusions The accuracy and engagement of the VR-based training system were significantly high when tested against traditional training.Future research will focus on developing a variety of realistic ecosystems and their associated birds to increase the information on newer bird species within the training system.Finally,the proposed VR-based training system must be tested with additional participants and for a longer duration to measure information recall and recognition mastery among users.

A survey of immersive visualization:Focus on perception and interaction

Immersive visualization utilizes virtual reality,mixed reality devices,and other interactive devices to create a novel visual environment that integrates multimodal perception and interaction.This technology has been maturing in recent years and has found broad applications in various fields.Based on the latest research advancements in visualization,this paper summarizes the state-of-theart work in immersive visualization from the perspectives of multimodal perception and interaction in immersive environments,additionally discusses the current hardware foundations of immersive setups.By examining the design patterns and research approaches of previous immersive methods,the paper reveals the design factors for multimodal perception and interaction in current immersive environments.Furthermore,the challenges and development trends of immersive multimodal perception and interaction techniques are discussed,and potential areas of growth in immersive visualization design directions are explored.

Lessons learned from requirements gathering for virtual reality simulators

Background This paper shows how current collaborative virtual environments(VEs)such as Mozilla Hubs and AltspaceVR can aid in the task of requirements gathering in VR for simulation and training.Methods We performed a qualitative study on our use of these technologies in the requirements gathering of two projects.Results Our results show that requirements gathering in virtual reality has an impact on the process of requirements identification.We report advantages and shortcomings that will be of interest to future practitioners.For example,we found that VR sessions for requirements gathering in current VEs could benefit from better pointers and better sound quality.Conclusion Current VEs are useful for the requirements gathering task in the development of VR simulators and VR training environments.