Human-computer interactions constitute an important subject for the development and popularization of information technologies,as they are not only an important frontier technology in computer science but also an impo...Human-computer interactions constitute an important subject for the development and popularization of information technologies,as they are not only an important frontier technology in computer science but also an important auxiliary technology in virtual reality(VR).In recent years,Chinese researchers have made significant advances in human-computer interactions.To systematically display China's latest advances in human-computer interactions and thus provide an impetus for the development of VR and other related fields,we have solicited articles for this special issue from experts in this area to participate in the review process.The following articles have been selected for publication in this special issue.展开更多
With the development of virtual reality(VR)and human-computer interaction technology,how to use natural and efficient interaction methods in the virtual environment has become a hot topic of research.Gesture is one of...With the development of virtual reality(VR)and human-computer interaction technology,how to use natural and efficient interaction methods in the virtual environment has become a hot topic of research.Gesture is one of the most important communication methods of human beings,which can effectively express users'demands.In the past few decades,gesture-based interaction has made significant progress.This article focuses on the gesture interaction technology and discusses the definition and classification of gestures,input devices for gesture interaction,and gesture interaction recognition technology.The application of gesture interaction technology in virtual reality is studied,the existing problems in the current gesture interaction are summarized,and the future development is prospected.展开更多
Background Owing to recent advances in virtual reality(VR)technologies,effective user interaction with dynamic content in 3D scenes has become a research hotspot.Moving target selection is a basic interactive task in ...Background Owing to recent advances in virtual reality(VR)technologies,effective user interaction with dynamic content in 3D scenes has become a research hotspot.Moving target selection is a basic interactive task in which the user performance research in tasks is significant to user interface design in VR.Different from the existing static target selection studies,the moving target selection in VR is affected by the change in target speed,angle and size,and lack of research on some key factors.Methods This study designs an experimental scenario in which the users play badminton under the condition of VR.By adding seven kinds of modal clues such as vision,audio,haptics,and their combinations,five kinds of moving speed and four kinds of serving angles,and the effect of these factors on the performance and subjective feelings in moving target selection in VR,is studied.Results The results show that the moving speed of the shuttlecock has a significant impact on the user performance.The angle of service has a significant impact on hitting rate,but has no significant impact on the hitting distance.The acquisition of the user performance by the moving target is mainly influenced by vision under the combined modalities;adding additional modalities can improve user performance.Although the hitting distance of the target is increased in the trimodal condition,the hitting rate decreases.Conclusion This study analyses the results of user performance and subjective perception,and then provides suggestions on the combination of modality clues in different scenarios.展开更多
Background Crossing-based target selection motion may attain less error rates and higher interactive speed in some cases.Most of the research in target selection fields are focused on the analysis of the interaction r...Background Crossing-based target selection motion may attain less error rates and higher interactive speed in some cases.Most of the research in target selection fields are focused on the analysis of the interaction results.Additionally,as trajectories play a much more important role in crossing-based target selection compared to the other interactive techniques,an ideal model for trajectories can help computer designers make predictions about interaction results during the process of target selection rather than at the end of the whole process.Methods In this paper,a trajectory prediction model for crossing based target selection tasks is proposed by taking the reference of a dynamic model theory.Results Simulation results demonstrate that our model performed well with regard to the prediction of trajectories,endpoints and hitting time for target-selection motion,and the average error of trajectories,endpoints and hitting time values were found to be 17.28%,2.73mm and 11.50%,respectively.展开更多
Activity recognition is a core aspect of ubiquitous computing applications. In order to deploy activity recognition systems in the real world, we need simple sensing systems with lightweight computational modules to a...Activity recognition is a core aspect of ubiquitous computing applications. In order to deploy activity recognition systems in the real world, we need simple sensing systems with lightweight computational modules to accurately analyze sensed data. In this paper, we propose a simple method to recognize human activities using simple object information involved in activities. We apply activity theory for representing complex human activities and propose a penalized naive Bayes classifier for performing activity recognition. Our results show that our method reduces computation up to an order of magnitude in both learning and inference without penalizing accuracy, when compared to hidden Markov models and conditional random fields.展开更多
The challenge of coping with non-frontal head poses during facial expression recognition results in considerable reduction of accuracy and robustness when capturing expressions that occur during natural communications...The challenge of coping with non-frontal head poses during facial expression recognition results in considerable reduction of accuracy and robustness when capturing expressions that occur during natural communications. In this paper, we attempt to recognize facial expressions under poses with large rotation angles from 2D videos. A depth^patch based 4D expression representation model is proposed. It was reconstructed from 2D dynamic images for delineating continuous spatial changes and temporal context under non-frontal cases. Furthermore, we present an effective deep neural network classifier, which can accurately capture pose-variant expression features from the depth patches and recognize non-frontal expressions. Experimental results on the BU-4DFE database show that the proposed method achieves a high recognition accuracy of 86.87% for non-frontal facial expressions within a range of head rotation angle of up to 52%, outperforming existing methods. We also present a quantitative analysis of the components contributing to the performance gain through tests on the BU-4DFE and Multi-PIE datasets.展开更多
Conventional vision-based systems,such as cameras,have demonstrated their enormous versatility in sensing human activities and developing interactive environments.However,these systems have long been criticized for in...Conventional vision-based systems,such as cameras,have demonstrated their enormous versatility in sensing human activities and developing interactive environments.However,these systems have long been criticized for incurring privacy,power,and latency issues due to their underlying structure of pixel-wise analog signal acquisition,computation,and communication.In this research,we overcome these limitations by introducing in-sensor analog computation through the distribution of interconnected photodetectors in space,having a weighted responsivity,to create what we call a computational photodetector.Computational photodetectors can be used to extract mid-level vision features as a single continuous analog signal measured via a two-pin connection.We develop computational photodetectors using thin and flexible low-noise organic photodiode arrays coupled with a self-powered wireless system to demonstrate a set of designs that capture position,orientation,direction,speed,and identification information,in a range of applications from explicit interactions on everyday surfaces to implicit activity detection.展开更多
Emotion plays a crucial role in gratifying users’needs during their experience of movies and TV series,and may be underutilized as a framework for exploring video content and analysis.In this paper,we present Emotion...Emotion plays a crucial role in gratifying users’needs during their experience of movies and TV series,and may be underutilized as a framework for exploring video content and analysis.In this paper,we present EmotionMap,a novel way of presenting emotion for daily users in 2D geography,fusing spatio-temporal information with emotional data.The interface is composed of novel visualization elements interconnected to facilitate video content exploration,understanding,and searching.EmotionMap allows understanding of the overall emotion at a glance while also giving a rapid understanding of the details.Firstly,we develop EmotionDisc which is an effective tool for collecting audiences’emotion based on emotion representation models.We collect audience and character emotional data,and then integrate the metaphor of a map to visualize video content and emotion in a hierarchical structure.EmotionMap combines sketch interaction,providing a natural approach for users’active exploration.The novelty and the effectiveness of EmotionMap have been demonstrated by the user study and experts’feedback.展开更多
文摘Human-computer interactions constitute an important subject for the development and popularization of information technologies,as they are not only an important frontier technology in computer science but also an important auxiliary technology in virtual reality(VR).In recent years,Chinese researchers have made significant advances in human-computer interactions.To systematically display China's latest advances in human-computer interactions and thus provide an impetus for the development of VR and other related fields,we have solicited articles for this special issue from experts in this area to participate in the review process.The following articles have been selected for publication in this special issue.
基金National Key Research and Development(2016YFB1001405)Frontier Subject Key Research(QYZDY-SSW-JSC041)Chinese Academy of Sciences hundred people,National Natural Science Foundation of China(61572479)project support.
文摘With the development of virtual reality(VR)and human-computer interaction technology,how to use natural and efficient interaction methods in the virtual environment has become a hot topic of research.Gesture is one of the most important communication methods of human beings,which can effectively express users'demands.In the past few decades,gesture-based interaction has made significant progress.This article focuses on the gesture interaction technology and discusses the definition and classification of gestures,input devices for gesture interaction,and gesture interaction recognition technology.The application of gesture interaction technology in virtual reality is studied,the existing problems in the current gesture interaction are summarized,and the future development is prospected.
基金National Key Research and Development(2016YFB1001405)Frontier Subject Key Research(QYZDY-SSW JSC041)National Natural Science Foundation of China(61802379).
文摘Background Owing to recent advances in virtual reality(VR)technologies,effective user interaction with dynamic content in 3D scenes has become a research hotspot.Moving target selection is a basic interactive task in which the user performance research in tasks is significant to user interface design in VR.Different from the existing static target selection studies,the moving target selection in VR is affected by the change in target speed,angle and size,and lack of research on some key factors.Methods This study designs an experimental scenario in which the users play badminton under the condition of VR.By adding seven kinds of modal clues such as vision,audio,haptics,and their combinations,five kinds of moving speed and four kinds of serving angles,and the effect of these factors on the performance and subjective feelings in moving target selection in VR,is studied.Results The results show that the moving speed of the shuttlecock has a significant impact on the user performance.The angle of service has a significant impact on hitting rate,but has no significant impact on the hitting distance.The acquisition of the user performance by the moving target is mainly influenced by vision under the combined modalities;adding additional modalities can improve user performance.Although the hitting distance of the target is increased in the trimodal condition,the hitting rate decreases.Conclusion This study analyses the results of user performance and subjective perception,and then provides suggestions on the combination of modality clues in different scenarios.
基金National Key R&D Program of China(2016YFB1001405)the National Natural Science Foundation of China(61802379)Key Research Program of Frontier Sciences,CAS(QYZDY-SSW-JSC041).
文摘Background Crossing-based target selection motion may attain less error rates and higher interactive speed in some cases.Most of the research in target selection fields are focused on the analysis of the interaction results.Additionally,as trajectories play a much more important role in crossing-based target selection compared to the other interactive techniques,an ideal model for trajectories can help computer designers make predictions about interaction results during the process of target selection rather than at the end of the whole process.Methods In this paper,a trajectory prediction model for crossing based target selection tasks is proposed by taking the reference of a dynamic model theory.Results Simulation results demonstrate that our model performed well with regard to the prediction of trajectories,endpoints and hitting time for target-selection motion,and the average error of trajectories,endpoints and hitting time values were found to be 17.28%,2.73mm and 11.50%,respectively.
基金supported by the Korea Research Foundation under Grant No. KRF-2008-357-D00221
文摘Activity recognition is a core aspect of ubiquitous computing applications. In order to deploy activity recognition systems in the real world, we need simple sensing systems with lightweight computational modules to accurately analyze sensed data. In this paper, we propose a simple method to recognize human activities using simple object information involved in activities. We apply activity theory for representing complex human activities and propose a penalized naive Bayes classifier for performing activity recognition. Our results show that our method reduces computation up to an order of magnitude in both learning and inference without penalizing accuracy, when compared to hidden Markov models and conditional random fields.
基金This work was supported by the National Key Research and Development Program of China under Grant No. 2016YFBI001405, and the National Natural Science Foundation of China under Grant Nos. 61232013, 61422212, and 61661146002.
文摘The challenge of coping with non-frontal head poses during facial expression recognition results in considerable reduction of accuracy and robustness when capturing expressions that occur during natural communications. In this paper, we attempt to recognize facial expressions under poses with large rotation angles from 2D videos. A depth^patch based 4D expression representation model is proposed. It was reconstructed from 2D dynamic images for delineating continuous spatial changes and temporal context under non-frontal cases. Furthermore, we present an effective deep neural network classifier, which can accurately capture pose-variant expression features from the depth patches and recognize non-frontal expressions. Experimental results on the BU-4DFE database show that the proposed method achieves a high recognition accuracy of 86.87% for non-frontal facial expressions within a range of head rotation angle of up to 52%, outperforming existing methods. We also present a quantitative analysis of the components contributing to the performance gain through tests on the BU-4DFE and Multi-PIE datasets.
基金supported by the Georgia Tech CRNCH (Center for Research into Novel Computing Hierarchies) Ph.D.Fellowship.
文摘Conventional vision-based systems,such as cameras,have demonstrated their enormous versatility in sensing human activities and developing interactive environments.However,these systems have long been criticized for incurring privacy,power,and latency issues due to their underlying structure of pixel-wise analog signal acquisition,computation,and communication.In this research,we overcome these limitations by introducing in-sensor analog computation through the distribution of interconnected photodetectors in space,having a weighted responsivity,to create what we call a computational photodetector.Computational photodetectors can be used to extract mid-level vision features as a single continuous analog signal measured via a two-pin connection.We develop computational photodetectors using thin and flexible low-noise organic photodiode arrays coupled with a self-powered wireless system to demonstrate a set of designs that capture position,orientation,direction,speed,and identification information,in a range of applications from explicit interactions on everyday surfaces to implicit activity detection.
基金This work was supported by the National Key Research and Development Program of China under Grant No.2016YFB1001200the National Natural Science Foundation of China under Grant No.61872346the Strategic Priority Research Program of the Chinese Academy of Sciences under Grant No.19080102.
文摘Emotion plays a crucial role in gratifying users’needs during their experience of movies and TV series,and may be underutilized as a framework for exploring video content and analysis.In this paper,we present EmotionMap,a novel way of presenting emotion for daily users in 2D geography,fusing spatio-temporal information with emotional data.The interface is composed of novel visualization elements interconnected to facilitate video content exploration,understanding,and searching.EmotionMap allows understanding of the overall emotion at a glance while also giving a rapid understanding of the details.Firstly,we develop EmotionDisc which is an effective tool for collecting audiences’emotion based on emotion representation models.We collect audience and character emotional data,and then integrate the metaphor of a map to visualize video content and emotion in a hierarchical structure.EmotionMap combines sketch interaction,providing a natural approach for users’active exploration.The novelty and the effectiveness of EmotionMap have been demonstrated by the user study and experts’feedback.
