基于Leap Motion手势识别的三维交互系统

随着虚拟交互技术的发展,人们迈入了“体验式经济时代”,消费者越来越关注个性体验,因此,基于Leap Motion手势识别设备,设计了一种三维虚拟室内交互系统。该系统以Unity3D作为开发工具,Leap Motion作为硬件平台,结合C#语言进行脚本的编译,利用3ds Max平台对室内进行场景搭建,通过Unity3D工具将组件整合,设计了七种手势,使用Leap Motion硬件设备对场景中物体进行各种不同的操作。经试验表明,该系统实现了用户与场景中物体的交互能力,可以应用在室内装修和设计等方面,增强人们的体验感与趣味性。

Ensemble learning HMM for motion recognition and retrieval by Isomap dimension reduction

Along with the development of motion capture technique, more and more 3D motion databases become available. In this paper, a novel approach is presented for motion recognition and retrieval based on ensemble HMM (hidden Markov model) learning. Due to the high dimensionality of motion’s features, Isomap nonlinear dimension reduction is used for training data of ensemble HMM learning. For handling new motion data, Isomap is generalized based on the estimation of underlying eigen- functions. Then each action class is learned with one HMM. Since ensemble learning can effectively enhance supervised learning, ensembles of weak HMM learners are built. Experiment results showed that the approaches are effective for motion data recog- nition and retrieval.

Study on automatic recognition of the first motion in a seismic event

In this paper, we have studied the waveforms of background noise in a seismograph and set up an AR model to characterize them. We then complete the modeling and the automatic recognition program. Finally, we provide the results from automatic recognition and the manual recognition of the first motion for 25 underground explosions.

Combining Multi-scale Directed Depth Motion Maps and Log-Gabor Filters for Human Action Recognition

Recognition of the human actions by computer vision has become an active research area in recent years. Due to the speed and the high similarity of the actions, the current algorithms cannot get high recognition rate. A new recognition method of the human action is proposed with the multi-scale directed depth motion maps(MsdDMMs) and Log-Gabor filters. According to the difference between the speed and time order of an action, MsdDMMs is proposed under the energy framework. Meanwhile, Log-Gabor is utilized to describe the texture details of MsdDMMs for the motion characteristics. It can easily satisfy both the texture characterization and the visual features of human eye. Furthermore, the collaborative representation is employed as action recognition by the classification. Experimental results show that the proposed algorithm, which is applied in the MSRAction3 D dataset and MSRGesture3 D dataset, can achieve the accuracy of 95.79% and 96.43% respectively. It also has higher accuracy than the existing algorithms, such as super normal vector(SNV), hierarchical recurrent neural network(Hierarchical RNN).

Human Motion Recognition Based on Incremental Learning and Smartphone Sensors

Batch processing mode is widely used in the training process of human motiun recognition. After training, the motion elassitier usually remains invariable. However, if the classifier is to be expanded, all historical data must be gathered for retraining. This consumes a huge amount of storage space, and the new training process will be more complicated. In this paper, we use an incremental learning method to model the motion classifier. A weighted decision tree is proposed to help illustrate the process, and the probability sampling method is also used. The resuhs show that with continuous learning, the motion classifier is more precise. The average classification precision for the weighted decision tree was 88.43% in a typical test. Incremental learning consumes much less time than the batch processing mode when the input training data comes continuously.

Human Motion Recognition Using Ultra-Wideband Radar and Cameras on Mobile Robot

Cameras can reliably detect human motions in a normal environment,but they are usually affected by sudden illumination changes and complex conditions,which are the major obstacles to the reliability and robustness of the system.To solve this problem,a novel integration method was proposed to combine bi-static ultra-wideband radar and cameras.In this recognition system,two cameras are used to localize the object's region,regions while a radar is used to obtain its 3D motion models on a mobile robot.The recognition results can be matched in the 3D motion library in order to recognize its motions.To confirm the effectiveness of the proposed method,the experimental results of recognition using vision sensors and those of recognition using the integration method were compared in different environments.Higher correct-recognition rate is achieved in the experiment.

Low-Cost Posture Recognition of Moving Hands by Profile-Mold Construction in Cluttered Background and Occlusion

In this paper, we propose a low-cost posture recognition scheme using a single webcam for the signaling hand with nature sways and possible oc-clusions. It goes for developing the untouchable low-complexity utility based on friendly hand-posture signaling. The scheme integrates the dominant temporal-difference detection, skin color detection and morphological filtering for efficient cooperation in constructing the hand profile molds. Those molds provide representative hand profiles for more stable posture recognition than accurate hand shapes with in effect trivial details. The resultant bounding box of tracking the signaling molds can be treated as a regular-type object-matched ROI to facilitate the stable extraction of robust HOG features. With such commonly applied features on hand, the prototype SVM is adequately capable of obtaining fast and stable hand postures recognition under natural hand movement and non-hand object occlusion. Experimental results demonstrate that our scheme can achieve hand-posture recognition with enough accuracy under background clutters that the targeted hand can be allowed with medium movement and palm-grasped object. Hence, the proposed method can be easily embedded in the mobile phone as application software.

基于Leap Motion的手语识别算法优化

Leap Motion设备产生的数据在虚拟环境中可以进行手势识别。通过识别和跟踪用户的手来生成虚拟3D手部模型,从而获取手势信息。本文设计了一种基于隐马尔可夫模型(Hidden Markov Model,HMM)分类算法来学习从Leap Motion中所获取的手势信息的系统,根据手势特征的重要性赋予不同权值,可进一步提高分类准确率,实现手语信息的识别输入。测试结果表明,识别准确率为86.1%,手语打字输入识别速度为每分钟13.09个字符,可显著提高聋哑人与正常人之间沟通的便捷性。

STGNN-LMR:A Spatial–Temporal Graph Neural Network Approach Based on sEMG Lower Limb Motion Recognition

Lower limb motion recognition techniques commonly employ Surface Electromyographic Signal(sEMG)as input and apply a machine learning classifier or Back Propagation Neural Network(BPNN)for classification.However,this artificial feature engineering technique is not generalizable to similar tasks and is heavily reliant on the researcher’s subject expertise.In contrast,neural networks such as Convolutional Neural Network(CNN)and Long Short-term Memory Neural Network(LSTM)can automatically extract features,providing a more generalized and adaptable approach to lower limb motion recognition.Although this approach overcomes the limitations of human feature engineering,it may ignore the potential correlation among the sEMG channels.This paper proposes a spatial–temporal graph neural network model,STGNN-LMR,designed to address the problem of recognizing lower limb motion from multi-channel sEMG.STGNN-LMR transforms multi-channel sEMG into a graph structure and uses graph learning to model spatial–temporal features.An 8-channel sEMG dataset is constructed for the experimental stage,and the results show that the STGNN-LMR model achieves a recognition accuracy of 99.71%.Moreover,this paper simulates two unexpected scenarios,including sEMG sensors affected by sweat noise and sudden failure,and evaluates the testing results using hypothesis testing.According to the experimental results,the STGNN-LMR model exhibits a significant advantage over the control models in noise scenarios and failure scenarios.These experimental results confirm the effectiveness of the STGNN-LMR model for addressing the challenges associated with sEMG-based lower limb motion recognition in practical scenarios.

Multi-Modality Video Representation for Action Recognition

Nowadays,action recognition is widely applied in many fields.However,action is hard to define by single modality information.The difference between image recognition and action recognition is that action recognition needs more modality information to depict one action,such as the appearance,the motion and the dynamic information.Due to the state of action evolves with the change of time,motion information must be considered when representing an action.Most of current methods define an action by spatial information and motion information.There are two key elements of current action recognition methods:spatial information achieved by sampling sparsely on video frames’sequence and the motion content mostly represented by the optical flow which is calculated on consecutive video frames.However,the relevance between them in current methods is weak.Therefore,to strengthen the associativity,this paper presents a new architecture consisted of three streams to obtain multi-modality information.The advantages of our network are:(a)We propose a new sampling approach to sample evenly on the video sequence for acquiring the appearance information;(b)We utilize ResNet101 for gaining high-level and distinguished features;(c)We advance a three-stream architecture to capture temporal,spatial and dynamic information.Experimental results on UCF101 dataset illustrate that our method outperforms other previous methods.

基于Leap Motion的手势识别及在大型结构件虚拟安装中的应用

体感控制器Leap Motion因其追踪精度高、手势交互性好的优点被广泛运用于各类虚拟安装。将Leap Motion手势识别应用于高集成度大型结构件的高精度虚拟安装,可实现虚拟手对安装过程的交互控制。设计了一种基于加权卡方距离的模糊K最近邻结点(KNN)分类方法实现虚拟手势分类,根据手势特征的重要性赋予不同权值,可进一步提高分类准确率,测试结果表明改进分类方法识别准确率达到92.7%,比传统分类算法提高5.3%。使用三种手势进行发动机部件的虚拟安装实验,结果表明手势识别在安装过程中取得了良好的效果,可提升现实安装过程的质量和效率,对于提升大型军品的制造和安装水平具有重要意义。

A Fast Statistical Approach for Human Activity Recognition

An essential part of any activity recognition system claiming be truly real-time is the ability to perform feature extraction in real-time. We present, in this paper, a quite simple and computationally tractable approach for real-time human activity recognition that is based on simple statistical features. These features are simple and relatively small, accordingly they are easy and fast to be calculated, and further form a relatively low-dimensional feature space in which classification can be carried out robustly. On the Weizmann publicly benchmark dataset, promising results (i.e. 97.8%) have been achieved, showing the effectiveness of the proposed approach compared to the-state-of-the-art. Furthermore, the approach is quite fast and thus can provide timing guarantees to real-time applications.

Influence of enhancing dynamic scapular recognition on shoulder disability,and pain in diabetics with frozen shoulder:A case report

BACKGROUND Frozen shoulder(FS)is a familiar disorder.Diabetics with FS have more severe symptoms and a worse prognosis.Thus,this study investigated the influence of enhancing dynamic scapular recognition on shoulder disability and pain in diabetics with FS.CASE SUMMARY A Forty-five years-old male person with diabetes mellitus and a unilateral FS(stage II)for at least 3 mo with shoulder pain and limitation in both passive and active ranges of motion(ROMs)of the glenohumeral joint of≥25%in 2 directions participated in this study.This person received dynamic scapular recognition exercise was applied to a diabetic person with a unilateral FS(stage II).The main outcome measures were upward rotation of the scapula,shoulder pain and disability index,and shoulder range of motion of flexion,abduction,and external rotation.The dynamic scapular exercise was performed for 15 min/session and 3 sessions/wk lasted for 4 wk.After 4 wk of intervention,there were improvements between pre-treatment and post-treatment in shoulder pain,shoulder pain and disability index,shoulder ROM,and upward rotation of the scapula.CONCLUSION This case report suggested that enhancing dynamic scapular recognition may improve shoulder pain and disability;upward rotation of the scapula;and shoulder ROM of shoulder abduction,flexion,and external rotation after 4 wk.

Continuous Arabic Sign Language Recognition in User Dependent Mode

Arabic Sign Language recognition is an emerging field of research. Previous attempts at automatic vision-based recog-nition of Arabic Sign Language mainly focused on finger spelling and recognizing isolated gestures. In this paper we report the first continuous Arabic Sign Language by building on existing research in feature extraction and pattern recognition. The development of the presented work required collecting a continuous Arabic Sign Language database which we designed and recorded in cooperation with a sign language expert. We intend to make the collected database available for the research community. Our system which we based on spatio-temporal feature extraction and hidden Markov models has resulted in an average word recognition rate of 94%, keeping in the mind the use of a high perplex-ity vocabulary and unrestrictive grammar. We compare our proposed work against existing sign language techniques based on accumulated image difference and motion estimation. The experimental results section shows that the pro-posed work outperforms existing solutions in terms of recognition accuracy.

Behaviour recognition based on the integration of multigranular motion features in the Internet of Things

With the adoption of cutting-edge communication technologies such as 5G/6G systems and the extensive development of devices,crowdsensing systems in the Internet of Things(IoT)are now conducting complicated video analysis tasks such as behaviour recognition.These applications have dramatically increased the diversity of IoT systems.Specifically,behaviour recognition in videos usually requires a combinatorial analysis of the spatial information about objects and information about their dynamic actions in the temporal dimension.Behaviour recognition may even rely more on the modeling of temporal information containing short-range and long-range motions,in contrast to computer vision tasks involving images that focus on understanding spatial information.However,current solutions fail to jointly and comprehensively analyse short-range motions between adjacent frames and long-range temporal aggregations at large scales in videos.In this paper,we propose a novel behaviour recognition method based on the integration of multigranular(IMG)motion features,which can provide support for deploying video analysis in multimedia IoT crowdsensing systems.In particular,we achieve reliable motion information modeling by integrating a channel attention-based short-term motion feature enhancement module(CSEM)and a cascaded long-term motion feature integration module(CLIM).We evaluate our model on several action recognition benchmarks,such as HMDB51,Something-Something and UCF101.The experimental results demonstrate that our approach outperforms the previous state-of-the-art methods,which confirms its effective-ness and efficiency.

基于Leap Motion的工业智能装配系统设计

针对常规人机交互方式(手动编程、示教器等)难以实现装配动作高自由度输入的问题,本文提出基于Leap Motion手势交互传感器的智能装配系统构建方案,用以指导机械臂装配动作的自动实现。首先对Leap Motion传感器进行性能测试实验,构建最佳工作空间与运动轨迹描述,随后搭建ResNet-50架构的深度卷积神经网络,对预定的动态手势进行分类识别并映射到装配动作。实验结果证明,该系统满足了轻巧、便携与非接触式测量的需求,实现了复杂灵活的装配动作复现,装配效率大幅提升。

CPG Human Motion Phase Recognition Algorithm for a Hip Exoskeleton with VSA Actuator

Due to the dynamic stiffness characteristics of human joints, it is easy to cause impact and disturbance on normal movements during exoskeleton assistance. This not only brings strict requirements for exoskeleton control design, but also makes it difficult to improve assistive level. The Variable Stiffness Actuator (VSA), as a physical variable stiffness mechanism, has the characteristics of dynamic stiffness adjustment and high stiffness control bandwidth, which is in line with the stiffness matching experiment. However, there are still few works exploring the assistive human stiffness matching experiment based on VSA. Therefore, this paper designs a hip exoskeleton based on VSA actuator and studies CPG human motion phase recognition algorithm. Firstly, this paper puts forward the requirements of variable stiffness experimental design and the output torque and variable stiffness dynamic response standards based on human lower limb motion parameters. Plate springs are used as elastic elements to establish the mechanical principle of variable stiffness, and a small variable stiffness actuator is designed based on the plate spring. Then the corresponding theoretical dynamic model is established and analyzed. Starting from the CPG phase recognition algorithm, this paper uses perturbation theory to expand the first-order CPG unit, obtains the phase convergence equation and verifies the phase convergence when using hip joint angle as the input signal with the same frequency, and then expands the second-order CPG unit under the premise of circular limit cycle and analyzes the frequency convergence criterion. Afterwards, this paper extracts the plate spring modal from Abaqus and generates the neutral file of the flexible body model to import into Adams, and conducts torque-stiffness one-way loading and reciprocating loading experiments on the variable stiffness mechanism. After that, Simulink is used to verify the validity of the criterion. Finally, based on the above criterions, the signal mean value is removed using feedback structure to complete the phase recognition algorithm for the human hip joint angle signal, and the convergence is verified using actual human walking data on flat ground.

表面肌电与三轴信息融合的运动判断实验

为了提高基于表面肌电与三轴加速度信号的运动识别准确率,提出了一套多源信息融合处理的实验流程与方法。该方法利用5层离散小波变换对表面肌电信号进行分解,充分提取不同运动产生的肌电信号中各频域的特征信息;再将分解后的表面肌电信号与三轴加速度信号通过滑动窗口的方法进行特征融合,构造融合肌电与空间运动特征的特征图;最后用融合特征图对深度学习模型进行训练,并结合自动状态机进行最终运动状态的识别。实验结果表明,多源信息融合处理方法可以提高运动识别的准确性,总体识别精度分别达到了95.4%和89.2%。该方法在实时性与准确性上均有良好表现。

基于改进粒子群算法的UWB雷达人体动作识别研究

针对雷达信号中的杂波干扰及样本数量对人体动作识别精度的限制,提出一种基于改进粒子群算法(particle swarm optimization,PSO)优化支持向量机(support vector machine,SVM)模型的超宽带(ultra-wideband,UWB)雷达人体动作识别算法。利用动态目标指示(moving target indication,MTI)与小波阈值滤波对接收到的UWB回波信号进行预处理,消除回波信号中的杂波和噪声对人体动作识别的影响;结合二维离散小波包分解(two dimensional discrete wavelet packet decomposition,2D-DWPD)与奇异值分解(singular value decomposition,SVD),对预处理后的雷达信号进行特征提取和降维;提出一种改进粒子群算法,优化SVM模型的相关参数进行识别和分类。实验结果表明,提出的算法准确率可达到96.25%,具有良好的识别性能。

基于运动特征的骨骼行为识别方法

针对现有的骨骼行为识别方法对人体行为的运动信息利用不足的问题,提出一种基于运动特征的时空注意力图卷积(STA-GCN)行为识别模型。对动作捕捉设备采集到的关节点运动轨迹和速度信息进行建模,在时间和空间构建注意力权重矩阵,结合图卷积网络进行特征提取,能够关注到具有判别力的关节点和时间帧。通过在自建动作捕捉数据集和NTU-RGB+D数据集的CS和CV标准上进行实验,其结果表明,该模型增强了对人体骨骼行为信息的理解能力,验证了模型对行为识别的有效性。